Resume Experiment Analysis

In this exercise, we will be using a data set from a randomized experiment conducted by Marianne Bertrand and Sendhil Mullainathan, who sent 4,870 fictious resumes out to employers in response to job adverts in Boston and Chicago in 2001. The resumes differ in various attributes including the names of the applicants, and different resumes were randomly allocated to job openings. To manipulate perceived race, resumes are randomly assigned Black- or White-sounding names. The researchers collecting these data were interested to learning more about how racial bias impacts job market outcomes by testing whether Black sounding names obtain fewer callbacks for interviews than White names.

You can get access to original article here.

  • Download the data set resume_experiment.dta from github here, or by doing to www.github.com/nickeubank/MIDS_Data and opening the resume_experiment folder.

  • For python users, use read_stata in pandas to load the data set; For R users, use read_dta in haven to load the data set

  • black is the treatment variable in the data set (whether the resume has a Black-sounding name).

  • call is the dependent variable of interest (did the employer call the fictitious applicant for an interview)

In addition, the data includes a number of variables the describe the other features in each fictitious resume, including applicants education level (education), years of experience (yearsexp), gender (female), computer skills (computerskills), and number of previous jobs (ofjobs). Each resume has a random selection of these attributes, so on average the Black-named fictitious applicant resumes have the same qualifications as the White-named applicant resumes.

Checking for Balance

Exercise 1

Check for balance in terms of applicant gender (female), computer skills (computerskills), and years of experience (yearsexp) across the two arms of the experiment (i.e. by black). Calculate both the differences across treatment arms and test for statistical significance of these differences. Do gender and computer skills look balanced across race groups? (1 point)

Exercise 2

Do a similar tabulation for education (education). Education is a categorical variable coded as follows:

  • 0: Education not reported

  • 1: High school dropout

  • 2: High school graduate

  • 3: Some college

  • 4: College graduate or higher

Because these are categorical, you shouldn’t just calculate and compare means – you should compare share of observations with each value separately using a ttest, or do a chi-squared test (technically chi-squared is the correct test, but I’m ok with either).

Does education and the number of previous jobs look balanced across racial groups? (2 points)

Exercise 3

What do you make of the overall results on resume characteristics? Why do we care about whether these variables look similar across the race groups?

..

Estimating Effect of Race

Exercise 4

The variable of interest in the data set is the variable call, which indicates a call back for an interview. Perform a two-sample t-test comparing applicants with black sounding names and white sounding names.

Exercise 5

Now, use a regression model to estimate the differential likelihood of being called back by applicant race (i.e. the racial discrimination by employers). Is the difference statistically significant?

NOTE: When using a linear probability model (i.e. fitting a linear regression with a limited (0-1) dependent variable), you need to use heteroskedastic robust standard errors, not the default standard errors for homoskedastic data. To do so, use get_robustcov_results(cov_type="HC3") on your fit model, like this:

import statsmodels.formula.api as smf

model = smf.ols("outcome ~ treatment", data).fit()
model.get_robustcov_results(cov_type="HC3").summary()

Exercise 6

Now let’s see if we can improve our estimates by adding in other variables as controls. Add in education, yearsexp, female, and computerskills – be sure to treat education as a categorical variable!

Estimating Heterogeneous Effects

Exercise 7

These effects are the average effects. Now let’s look for heterogeneous treatment effects.

Look only at candidates with high educations. Is there more or less racial discrimination among these highly educated candidates? Is the difference statistically significant?

Exercise 8

Now let’s compare men and women – is discrimination greater for Black men or Black women? Is the difference statistically significant?

Exercise 9

Calculate and/or lookup the following online:

  • What is the share of applicants in our dataset with college degrees?

  • What share of Black adult Americans have college degrees (i.e. have completed a bachelors degree)?

Exercise 10

What are the implications of your answers to Exercise 7 and to Exercise 9 to how you interpret the Average Treatment Effect you estimated in Exercise 6?

Absolutely positively need the solutions?

Don’t use this link until you’ve really, really spent time struggling with your code! Doing so only results in you cheating yourself.

Link