Difference-in-Differences for Programme Impact Evaluation Training Course

Course overview

Development and humanitarian programmes are often implemented without the option to randomise beneficiaries. Managers still need credible evidence on whether cash transfers, livelihoods support, health outreach, education initiatives or protection interventions changed outcomes beyond wider trends. Difference-in-differences (DiD) provides a practical quasi-experimental approach when a programme group and a defensible comparison group can be observed before and after implementation. This course helps monitoring, evaluation and research staff move from routine before-and-after reporting to impact estimates that can withstand technical review, donor scrutiny and internal challenge.

Participants learn to formulate a DiD evaluation question, define treatment and comparison groups, construct panel and repeated cross-sectional datasets, and estimate programme effects using regression models. The course examines the parallel-trends assumption, event-study graphs, staggered programme rollout, fixed effects, clustered standard errors, covariate adjustment, matching combined with DiD, and common sources of bias such as spillovers, selective attrition and changing outcome measurement. Participants interpret coefficients, confidence intervals and subgroup effects in language suitable for programme decision-makers.

Delivery combines instructor-led explanation with guided analysis of a realistic development programme dataset in Stata and R, with Excel used for data checking and reporting. Participants diagnose whether DiD is appropriate for a proposed intervention, build and test a model, produce visual and tabular results, and write a concise impact-evaluation findings note. They leave with an evaluation design template, reproducible analysis workflow, annotated code, a parallel-trends diagnostic checklist and a decision-ready results brief that can be adapted to their own programme.

The course is designed for professionals who already work with monitoring, survey, administrative or evaluation data and need a defensible non-randomised impact-evaluation method. It is especially relevant where phased implementation, geographic targeting or eligibility thresholds create useful comparison opportunities but randomised controlled trials are impractical or unethical.

Course objectives

By the end of this course, participants will be able to:

• Formulate a difference-in-differences evaluation question with defined treatment, comparison, outcome and time dimensions
• Construct panel and repeated cross-sectional datasets suitable for difference-in-differences estimation
• Assess the parallel-trends assumption using pre-intervention trend plots, placebo tests and event-study specifications
• Estimate two-way fixed-effects difference-in-differences models in Stata and R
• Apply clustered standard errors and interpret confidence intervals for programme impact estimates
• Diagnose threats from staggered rollout, spillovers, compositional change and differential attrition
• Produce coefficient plots, event-study charts and results tables for an impact-evaluation report
• Write a decision-focused difference-in-differences findings note with limitations and recommendations

Benefits of attending

Target competencies

Who should attend

• Monitoring and Evaluation Managers — who need credible programme-effect estimates for management and donor reporting
• Impact Evaluation Specialists — who design quasi-experimental studies where randomisation is not feasible
• Programme Managers — who must judge whether observed outcome changes can reasonably be attributed to an intervention
• Research and Learning Officers — who analyse survey and administrative data to generate actionable evidence
• Data Analysts — who need to implement fixed-effects and event-study models for development programme datasets
• Donor Programme Officers — who assess the methodological credibility of grantee evaluation plans and findings

Requirements and prerequisites

Participants should be comfortable reading tables and charts, working with row-and-column datasets, and interpreting basic descriptive statistics such as means, percentages and sample sizes. Prior exposure to regression is helpful: participants should recognise the ideas of an outcome variable, explanatory variable, coefficient and confidence interval. Bring a laptop capable of running Stata or R/RStudio; installation guidance and course files are provided before the course. Participants do not need prior difference-in-differences experience, advanced econometrics, calculus, coding expertise or experience conducting a randomised controlled trial.

Training methodology

Each day combines short instructor-led technical sessions with guided work on a development programme dataset containing baseline and follow-up observations for intervention and comparison areas. Participants clean and reshape data, draw pre-trend graphs, estimate models in Stata and R, and compare interpretations in small groups. Case discussions examine realistic threats including phased rollout, spillovers and changing beneficiary composition. Facilitated peer review is used to challenge assumptions and reporting language. The final session is an application-planning workshop in which participants adapt the DiD design template and analysis checklist to a live or anticipated programme evaluation.

Course outline

Day 1: Designing a credible difference-in-differences evaluation

• Causal attribution questions in development and humanitarian programmes
• The 2x2 difference-in-differences estimator
• Treatment, comparison, pre-intervention and post-intervention periods
• Theory of change and outcome selection for DiD
• Panel data versus repeated cross-sectional data
• Eligibility, targeting and phased rollout as sources of comparison
• Causal diagrams for confounding and selection risks

Workshop: Participants map a livelihoods programme into a DiD design matrix and produce a one-page evaluation question, comparison-group rationale and outcome definition.

Day 2: Preparing data and examining assumptions

• Data structure requirements for DiD estimation
• Merging baseline, endline and administrative datasets
• Wide-to-long reshaping for panel analysis
• Treatment timing and intervention exposure coding
• Outcome construction and consistent indicator definitions
• Pre-intervention trend plots and visual diagnostics
• Parallel-trends assumption, plausibility arguments and limitations

Workshop: Participants prepare a longitudinal analysis file, create treatment and time variables, and produce an annotated pre-trends chart.

Day 3: Estimating and interpreting DiD models

• Regression formulation of the DiD interaction term
• Two-way fixed effects for unit and time differences
• Individual, household, facility and geographic fixed effects
• Clustered standard errors and inference levels
• Covariate adjustment and precision improvement
• Binary, continuous and count outcome considerations
• Coefficient interpretation, confidence intervals and practical significance

Workshop: Participants estimate and interpret a fixed-effects DiD model in Stata and R, then produce a formatted results table for a programme brief.

Day 4: Testing robustness and handling complex implementation

• Event-study specifications and leads-and-lags graphs
• Placebo timing tests and falsification outcomes
• Staggered adoption and heterogeneous treatment timing
• Modern DiD estimators for staggered rollout
• Spillovers, contamination and displacement effects
• Attrition, migration and changing sample composition
• Matching and weighting combined with difference-in-differences

Workshop: Participants run an event-study and placebo test for a phased cash-transfer rollout, then document the robustness evidence and remaining risks.

Day 5: Reporting evidence for decisions

• Subgroup effects by gender, geography and vulnerability status
• Translating model output into programme effect statements
• Coefficient plots and donor-ready evidence tables
• Limitations statements and claims that DiD cannot support
• Reproducible scripts, file structures and analysis logs
• Difference-in-differences evaluation protocol template
• Management recommendations linked to impact evidence

Workshop: Participants complete a findings note and application plan for their own programme, including a DiD design, diagnostic tests, reporting outputs and next actions.

Tools & standards covered

Stata, R, Microsoft Excel, DAGitty

A typical training day

Live online deliveries follow the same structure in the East Africa Time zone, with shorter screen blocks and longer breaks.

What the fee includes

How you can take this course

Certification

Participants who complete the full five days receive the Skillset Development Certificate of Completion, stating the course title, course code, dates and delivery format — suitable for professional-development records and employer reimbursement.

Frequently asked questions