Resume Experiment Analysis¶
In this exercise, we will be using a data set from a randomized experiment conducted by Marianne Bertrand and Sendhil Mullainathan, who sent 4,870 fictious resumes out to employers in response to job adverts in Boston and Chicago in 2001. The resumes differ in various attributes including the names of the applicants, and different resumes were randomly allocated to job openings. To manipulate perceived race, resumes are randomly assigned Black- or White-sounding names. The researchers collecting these data were interested to learning more about how racial bias impacts job market outcomes by testing whether Black sounding names obtain fewer callbacks for interviews than White names.
You can get access to original article here.
Download the data set
resume_experiment.dta
from github here, or by doing towww.github.com/nickeubank/MIDS_Data
and opening theresume_experiment
folder.For
python
users, useread_stata
inpandas
to load the data set; ForR
users, useread_dta
inhaven
to load the data setblack
is the treatment variable in the data set (whether the resume has a Black-sounding name).call
is the dependent variable of interest (did the employer call the fictitious applicant for an interview)
In addition, the data includes a number of variables the describe the other features in each fictitious resume, including applicants education level (education
), years of experience (yearsexp
), gender (female
), computer skills (computerskills
), and number of previous jobs (ofjobs
). Each resume has a random selection of these attributes, so on average the Black-named fictitious applicant resumes have the same qualifications as the White-named applicant resumes.
Checking for Balance¶
Exercise 1¶
Check for balance in terms of applicant gender (female
), computer skills (computerskills
), and years of experience (yearsexp
) across the two arms of the experiment (i.e. by black
). Calculate both the differences across treatment arms and test for statistical significance of these differences. Do gender and computer skills look balanced across race groups? (1 point)
Exercise 2¶
Do a similar tabulation for education (education
). Education is a categorical variable coded as follows:
0: Education not reported
1: High school dropout
2: High school graduate
3: Some college
4: College graduate or higher
Because these are categorical, you shouldn’t just calculate and compare means – you should compare share of observations with each value separately using a ttest, or do a chi-squared test (technically chi-squared is the correct test, but I’m ok with either).
Does education and the number of previous jobs look balanced across racial groups? (2 points)
Exercise 3¶
What do you make of the overall results on resume characteristics? Why do we care about whether these variables look similar across the race groups?
..
Estimating Effect of Race¶
Exercise 4¶
The variable of interest in the data set is the variable call
, which indicates a call back for an interview. Perform a two-sample t-test comparing applicants with black sounding names and white sounding names.
Exercise 5¶
Now, use a regression model to estimate the differential likelihood of being called back by applicant race (i.e. the racial discrimination by employers). Is the difference statistically significant?
NOTE: When using a linear probability model (i.e. fitting a linear regression with a limited (0-1) dependent variable), you need to use heteroskedastic robust standard errors, not the default standard errors for homoskedastic data. To do so, use get_robustcov_results(cov_type="HC3")
on your fit model, like this:
import statsmodels.formula.api as smf
model = smf.ols("outcome ~ treatment", data).fit()
model.get_robustcov_results(cov_type="HC3").summary()
Exercise 6¶
Now let’s see if we can improve our estimates by adding in other variables as controls. Add in education
, yearsexp
, female
, and computerskills
– be sure to treat education as a categorical variable!
Estimating Heterogeneous Effects¶
Exercise 7¶
These effects are the average effects. Now let’s look for heterogeneous treatment effects.
Look only at candidates with high educations. Is there more or less racial discrimination among these highly educated candidates? Is the difference statistically significant?
Exercise 8¶
Now let’s compare men and women – is discrimination greater for Black men or Black women? Is the difference statistically significant?
Exercise 9¶
Calculate and/or lookup the following online:
What is the share of applicants in our dataset with college degrees?
What share of Black adult Americans have college degrees (i.e. have completed a bachelors degree)?
Exercise 10¶
What are the implications of your answers to Exercise 7 and to Exercise 9 to how you interpret the Average Treatment Effect you estimated in Exercise 6?
Absolutely positively need the solutions?¶
Don’t use this link until you’ve really, really spent time struggling with your code! Doing so only results in you cheating yourself.