Class Schedule

Class: Tuesday / Thursday, 1:25-2:40pm

Office Hours:

• Nick: Thursday, 9:00-10:00am, https://duke.zoom.us/my/nickeubank or Gross Hall 231

• Echo: Friday 2-3pm

• Kashaf: Tuesday, 3-4pm

• Pragya: Wednesday 3-4pm

Date	Topic	Do Before Class	In Class
Thrs, Jan 11	Course Overview		Acme Proposal
Tues, Jan 16	Solving Problems by Answering Questions	Read, sign, submit syllabus on gradescope. How to Read Solving Problems	Generating Questions
Thrs, Jan 18	Stakeholder Management	Solving the Right Problem History of Data Science Why It’s Important To Get Your Question Right (30 min video)	Stakeholder Management
Tues, Jan 23	Proscriptive v. Descriptive Questions	Descriptive versus Prescriptive Questions
Thrs, Jan 25	Exploratory Questions	EDA Using Exploratory Questions Answering Exploratory Questions	Exercise
Tues, Jan 30	Exploratory Questions	Internal v. External Validity Writing Good Notebooks Start Team Project	Explor Assign
Thrs, Feb 1	Teams 1	What Project Aristotle Learned Edmondson, The Fearless Organization, Chpt 1 (on Canvas)
Tues, Feb 6	Teams 2	Review Team Charter Assignment Edmondson, Teaming, Chpt 2 from “Cognitive Barriers to Teaming” (p. 82) to end of Chapter (on Canvas) Optional: Fostering Psychological Safety Tips
Thrs, Feb 8	Passive Prediction Questions	Using Passive Prediction Answering Passive Prediction	Ex
Tues, Feb 13	Passive Prediction Questions	Stop Using Black Boxes Black box models require trusting data too Optional: Contraining Artificial Stupidity Combining explicit models and black boxes in Scientific ML
Thrs, Feb 15	Passive Prediction Questions	GAM: The Predictive Silver Bullet A Tour of pyGAM	Ex
Tues, Feb 20	Causal Questions	Using Causal Questions Answering Causal Questions
Thrs, Feb 22	Causal Questions	Potential Outcomes Framework	Exercise
Tues, Feb 27	Causal Questions	Cunningham, Chpt 4, pp 135 (“Independence Assumption”) - 148 (Stop at “Randomization Inference”) Indicator Variables (Note SDO (Simple Difference in Outcomes) in Cunningham same as “widehat{ATE} from class) Optional: Read Chpt 4 in Cunningham from start (different presentation of potential outcomes)	Exercise
Thrs, Feb 29	Causal Questions: Experiments	Experiments: Internal Validity (In Practice): Kohvani, Tang and Xu: Chpt 2 (End to End Example) Kohavi, Tang and Xu: Chpt 3, “Threats to Internal Validity” (p. 42-47) Kohvani, Tang and Xu: Chpt 19 (A/A Testing)
Tues, Mar 5	Causal Questions: Experiments	Overall Evaluation Criteria: Kohavi, Tang, and Xu Chpt 7. Finishing Internal Validity: Different Randomizations: Kohavi, Tang and Xu Chpt 22 Experiments: External Validity (In Practice): Kohavi, Tang and Xu, Chpt 3, “Threats to External Validity” to end (p. 47-54) Kohvai, Tang and Xu, Chpt 23 (Primacy Effects / Long Term Decay)	Exercise
Thrs, Mar 7	Optional Class	Optional Class
Tues, Mar 12	NO CLASS
Thrs, Mar 14	NO CLASS
Tues, Mar 19	Causal Questions: Experiments	Designing Experiments Power Calculations: Georgiev, Chpt 4 (Canvas) Don’t stop experiments early!	Power Ex
Thrs, Mar 21	Review Day	AB Testing Review
Tues, Mar 26	Causal Questions: Experiments	Statistical Decision Theory (on Canvas). 550-556 (This is same as IDS 705 Lecture 8 Reading)
Thrs, Mar 28	MIDTERM	MIDTERM
Tues, Apr 2	Causal Questions: Regression	Causal Beyond Experiments	Link
Thrs, Apr 4	Exploratory Questions Feedback	Bring comments on other teams’ report
Tues, Apr 9	Causal Questions: Matching	The Why of Matching The How of Matching Summary Methods for Matching Watch the video above from about 15 minutes in (where link starts) till at least 45 minutes in, keep going if you want to learn about propensity score matching problems.	Exercise
Thrs, Apr 11	Causal Questions: Differences and Differences / Panels	Cunningham, Chpt 9 pp 406 (Difference in Differences) - pp 433 (Stop at “Importance of Placebos in Diff-in-Diff”) -(Book page numbers at bottom of PDF on canvas. Full chapter is in PDF, more than you need to read.) Optional but encouraged: (dont need to follow everything, but here’s a real diff-in-diff) Enfranchisement and Incarceration Diff-in-Diffs at Netflix
Tues, Apr 16	Interpretation of Results	Revised Exploratory Due along with Causal Proposal	Exercise
Wed, April 24		Optional Due Date for Causal Rough Draft (11pm)
Wed, May 1		Final Causal Report Due

Texts Referenced in Schedule:

• Kohvani, Tang and Xu: Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing