Impact Evaluation and Evidence-Based Decision Making Training Course

Participants can build a testable Theory of Change with explicit causal pathways and assumptions.

• Theory of Change links inputs, activities, outputs, outcomes and impacts
• Causal pathways specify mechanisms producing expected behavioural or institutional change
• Assumptions identified at household, market, service and policy levels
• External factors mapped through context analysis and risk registers
• Contribution claims separated from attribution claims at each outcome level
• Results chains converted into testable causal hypotheses and indicators

Participants can translate programme decisions into evaluation questions, criteria and evidence requirements.

• Primary evaluation questions framed using population, intervention, comparison and outcome
• Evaluation matrices connect questions, indicators, methods, sources and responsibilities
• Causal questions distinguished from descriptive, normative and explanatory questions
• Subgroup questions specify equity dimensions, vulnerabilities and distributional effects
• Prioritisation methods balance decision value, feasibility, cost and ethical risk
• Data needs documented before selecting methods or commissioning fieldwork

Participants can specify valid outcome indicators and establish an appropriate baseline strategy.

• Indicator definitions specify constructs, units, disaggregation and calculation rules
• Validity, reliability and responsiveness assessed for outcome measurement instruments
• Baseline studies establish pre-intervention conditions and comparison group characteristics
• Proxy indicators assessed against direct measures of intended programme outcomes
• Outcome measurement timing aligned with causal pathways and expected lags
• Data collection protocols reduce interviewer effects, recall bias and social desirability

Participants can explain counterfactual reasoning and identify major threats to causal attribution.

• Counterfactual outcomes define what would occur without programme exposure
• Selection bias arises when participants differ systematically from non-participants
• Confounding variables distort estimated relationships between intervention and outcome
• History, maturation and regression effects threaten before-and-after comparisons
• Spillover effects alter outcomes for households outside formal programme targeting
• Attrition bias assessed through dropout patterns and missing outcome data

Participants can assess when randomised designs are ethical, feasible and decision-useful.

• Randomised controlled trials allocate eligible units through transparent random assignment
• Individual, cluster and phased randomisation compared for delivery settings
• Lottery procedures documented to protect fairness, transparency and auditability
• Power calculations estimate sample sizes for detectable programme effects
• Treatment fidelity checks confirm interventions were delivered as designed
• Intention-to-treat estimates retain participants according to original assignment

Participants can select and justify quasi-experimental designs when randomisation is unavailable.

• Difference-in-differences compares outcome changes across treatment and comparison groups
• Parallel trends assumption tested using pre-intervention outcome observations
• Propensity score matching balances observed characteristics between comparison groups
• Regression discontinuity exploits eligibility thresholds and assignment cut-off rules
• Interrupted time series examines outcome shifts before and after intervention
• Design choice reflects programme rules, data availability and plausible assumptions

Participants can design theory-based evaluations for complex programmes without strong counterfactuals.

• Contribution analysis tests causal contribution through evidence and alternative explanations
• Process tracing assesses causal mechanisms using diagnostic evidence tests
• Outcome harvesting identifies significant changes and verifies contribution claims
• Realist evaluation frames context-mechanism-outcome configurations for programme learning
• Causal process observations strengthen explanation beyond outcome correlations alone
• Rival explanations documented and assessed through structured evidence collection

Participants can combine quantitative and qualitative methods into a coherent evaluation design.

• Mixed-methods designs integrate numerical estimates with explanatory qualitative evidence
• Sequential explanatory designs use interviews to interpret quantitative impact findings
• Triangulation compares findings across surveys, records, observations and interviews
• Qualitative sampling selects information-rich cases, outliers and relevant stakeholder groups
• Joint displays connect quantitative results with themes, mechanisms and contextual factors
• Method integration planned at design, analysis and reporting stages

Participants can develop a defensible sampling approach and fieldwork plan for an evaluation.

• Probability sampling methods support representative estimates and inferential statistical analysis
• Stratified sampling improves precision across regions, genders and participant categories
• Cluster sampling accounts for service sites, villages and implementation areas
• Sample size calculations consider effect size, power, confidence and design effects
• Non-response protocols protect sample quality and reduce systematic missingness
• Fieldwork plans assign enumerators, supervision, schedules and respondent safeguarding

Participants can establish ethical and quality assurance procedures for evaluation data collection.

• Informed consent procedures explain purpose, risks, confidentiality and voluntary participation
• Do-no-harm assessments address protection risks, stigma and respondent burden
• Data protection plans define access controls, retention periods and anonymisation
• Enumerator training standardises question delivery, probing and response recording
• Back-checks, spot-checks and logic checks identify fieldwork quality problems
• Research ethics review requirements assessed before collecting sensitive participant data

Participants can interpret common impact estimates and communicate their statistical meaning accurately.

• Average treatment effects estimate mean outcome differences attributable to intervention
• Confidence intervals express uncertainty around estimated programme effect sizes
• Statistical significance distinguished from practical significance and decision relevance
• Regression models adjust estimates for observed covariates and baseline differences
• Subgroup analysis examines heterogeneous effects across predefined participant characteristics
• Missing data treatments document assumptions, exclusions and sensitivity checks

Participants can analyse qualitative evidence to explain mechanisms, implementation and contextual variation.

• Codebooks define themes, subthemes, inclusion criteria and exclusion criteria
• Thematic analysis identifies recurring patterns across interviews, groups and observations
• Framework analysis compares evidence against Theory of Change assumptions
• Negative case analysis examines evidence that challenges emerging explanations
• Memo writing records analytic decisions, interpretations and evidentiary limitations
• Participant quotations selected ethically to substantiate findings without disclosure

Participants can incorporate cost evidence into impact findings and programme scale decisions.

• Cost analysis separates financial, economic, fixed and variable programme costs
• Cost-effectiveness ratios compare costs per outcome achieved across alternatives
• Cost-benefit analysis values benefits against programme and opportunity costs
• Value for money assesses economy, efficiency, effectiveness and equity
• Sensitivity analysis tests conclusions against uncertain costs and outcome assumptions
• Scale decisions consider fidelity, delivery capacity, affordability and contextual fit

Participants can prepare terms of reference and manage evaluators through quality assurance milestones.

• Terms of reference specify purpose, questions, scope, methods and deliverables
• Technical proposals assessed against design quality, feasibility and evaluator independence
• Inception reports test evaluator assumptions, timelines, sampling and analysis plans
• Governance arrangements define decision rights, review points and escalation routes
• Evaluation budgets distinguish design, fieldwork, analysis, dissemination and contingency costs
• Quality assurance reviews assess protocols, instruments, datasets and draft findings

Participants can appraise evaluation quality and convert findings into decision-ready reporting.

• Evidence appraisal examines design validity, measurement quality and analytical transparency
• Limitations statements distinguish supported findings from uncertain or unsupported claims
• Evidence briefs present findings, confidence, implications and recommended management actions
• Data visualisation selects charts that show comparisons, trends and uncertainty clearly
• Management responses assign actions, owners, deadlines and follow-up evidence
• Learning agendas connect evaluation findings to future programme and policy questions

Participants can build, review and present a complete impact evaluation design and decision brief for a selected programme.

• Programme problem, Theory of Change and evaluation purpose consolidated coherently
• Evaluation questions prioritised against decisions, risks and available resources
• Design rationale defends counterfactual or theory-based causal inference approach
• Evaluation matrix specifies indicators, sources, methods, responsibilities and timing
• Sampling, ethics, quality assurance and analysis plans reviewed systematically
• Decision brief presents findings pathways, limitations, costs and management actions