Impact Evaluation of Projects and Programmes Training Course

Participants can frame evaluation purposes around decisions, users and recognised evaluation criteria.

• OECD DAC criteria applied to impact-focused programme decision questions
• Primary intended users mapped against decisions, timing and evidence needs
• Evaluation questions narrowed using relevance, feasibility and decision value
• Accountability and learning purposes balanced within a single evaluation
• Decision calendars aligned with baseline, follow-up and reporting milestones
• Stakeholder interests analysed for influence, incentives and information needs

Participants can build a theory of change that supports testable causal evaluation questions.

• Results chains connect inputs, activities, outputs, outcomes and impacts
• Causal pathways specify mechanisms linking programme actions to change
• Assumptions surfaced through backward mapping from intended impacts
• Contextual factors separated from intervention-controlled implementation conditions
• Rival explanations documented alongside each proposed causal pathway
• Theory of change diagrams converted into measurable evaluation propositions

Participants can define the intervention and population required for a credible impact study.

• Treatment components specified by dosage, timing, quality and delivery channel
• Eligibility rules documented before participant recruitment or programme rollout
• Target populations segmented by geography, vulnerability and service need
• Exposure measured where participants receive varied intervention intensity
• Contamination risks assessed between treatment, comparison and control groups
• Implementation fidelity tracked alongside outcome measurement and causal analysis

Participants can specify outcome measures that answer impact evaluation questions reliably.

• Outcome indicators defined with numerator, denominator, unit and disaggregation
• SMART criteria used to test indicator usefulness and measurability
• Primary and secondary outcomes prioritised before collecting any data
• Proxy indicators assessed against direct measures of intended change
• Measurement validity checked through construct, content and criterion evidence
• Indicator reference sheets record sources, frequency, responsibility and limitations

Participants can plan measurement points that capture change and support attribution analysis.

• Baseline timing assessed against programme start dates and participant selection
• Endline schedules reflect expected outcome maturation and decision deadlines
• Panel, repeated cross-section and cohort designs compared for follow-up
• Recall periods selected to reduce memory error and seasonal distortion
• Seasonality mapped across livelihoods, health, education and protection outcomes
• Attrition risks anticipated through tracking protocols and replacement rules

Participants can assess when randomised assignment offers a feasible evaluation design.

• Randomised controlled trials assign treatment through transparent allocation procedures
• Individual, cluster and phased randomisation compared against delivery constraints
• Randomisation protocols protect fairness, documentation and allocation concealment
• Power calculations link sample size, effect size and statistical precision
• Spillover effects considered when participants interact across assigned groups
• Intention-to-treat analysis preserves original random assignment in estimates

Participants can select quasi-experimental approaches that address realistic sources of bias.

• Selection bias diagnosed where participation reflects need, choice or eligibility
• Quasi-experimental designs matched to programme rollout and available data
• Comparison groups assessed for similarity before intervention exposure begins
• Design trade-offs documented across credibility, cost, ethics and timing
• Triangulation plans address residual uncertainty in non-randomised evaluations
• Sensitivity analysis planned for assumptions that cannot be directly tested

Participants can apply time-based comparison logic to estimate programme effects.

• Difference-in-differences estimates changes between treated and comparison groups
• Parallel trends assumption assessed using pre-intervention outcome observations
• Interrupted time series tracks outcome patterns before and after intervention
• Multiple pre-intervention observations strengthen trend-based causal claims
• Policy shocks and concurrent events recorded as threats to interpretation
• Graphical displays communicate level changes, slope changes and uncertainty

Participants can judge when matching or eligibility thresholds support causal comparisons.

• Propensity score matching balances observed characteristics across comparison groups
• Matching variables selected before outcome analysis and programme exposure
• Common support checked where treated and untreated cases overlap
• Regression discontinuity exploits eligibility thresholds and assignment scores
• Continuity assumptions examined around the programme eligibility cutoff
• Manipulation tests identify strategic movement around threshold values

Participants can develop a sampling plan aligned with design, precision and field constraints.

• Sampling frames assessed for coverage, duplication and current contact information
• Probability sampling methods selected for defined target populations
• Stratification improves representation of priority groups and programme areas
• Cluster sampling accounts for service points, villages and schools
• Sample size assumptions document power, design effect and expected attrition
• Non-response procedures protect sample integrity and response-rate reporting

Participants can plan reliable quantitative fieldwork and data quality controls.

• Questionnaire items aligned with indicators, hypotheses and respondent knowledge
• Cognitive testing identifies ambiguous wording and culturally unsuitable questions
• Digital data collection forms apply skips, constraints and validation rules
• Enumerator training covers consent, neutrality, probing and safeguarding duties
• Field supervision uses spot checks, back-checks and daily error reviews
• Data cleaning protocols document missing values, outliers and corrections

Participants can integrate qualitative evidence to explain mechanisms, context and variation.

• Key informant interviews investigate implementation decisions and causal mechanisms
• Focus group discussions explore shared experience, norms and unintended effects
• Process tracing tests alternative explanations using evidence from multiple sources
• Contribution analysis examines whether evidence supports a contribution claim
• Mixed-methods matrices connect qualitative questions with quantitative outcome measures
• Joint displays integrate numerical patterns with explanatory participant evidence

Participants can apply ethical safeguards to impact evaluation design and fieldwork.

• Informed consent procedures explain purpose, risks, voluntary participation and withdrawal
• Do-no-harm assessment identifies possible consequences of data collection
• Safeguarding protocols address disclosures, referrals and researcher conduct
• Data minimisation limits collection to information required for evaluation
• Anonymisation and pseudonymisation protect identifiable respondent information
• Ethics review requirements mapped to institutional, national and donor rules

Participants can design evaluations that examine differential effects among population groups.

• Equity questions identify who benefits, who misses out and why
• Disaggregation plans specify sex, age, disability, location and socioeconomic status
• Intersectional analysis examines overlapping sources of exclusion and disadvantage
• Gender-responsive methods address power, access, safety and decision-making
• Washington Group Short Set considered for disability-related disaggregation
• Subgroup analysis planned with adequate sample sizes and interpretation caution

Participants can prepare an analysis plan and interpret common impact estimates responsibly.

• Pre-analysis plans specify hypotheses, outcomes, models and subgroup tests
• Descriptive statistics profile sample characteristics and baseline balance
• Effect sizes express magnitude beyond statistical significance alone
• Confidence intervals communicate estimate precision and plausible value ranges
• P-values interpreted without treating arbitrary thresholds as proof
• Missing data approaches distinguish complete-case, imputation and sensitivity analyses

Participants can set an evaluation scope and budget proportionate to decision stakes.

• Evaluation budgets separate design, fieldwork, analysis, reporting and dissemination costs
• Cost-effectiveness questions compare outcomes achieved per resource unit
• Cost-benefit analysis distinguishes monetised benefits from implementation costs
• Value-for-money considerations combine economy, efficiency, effectiveness and equity
• Proportionality criteria link methodological ambition to available resources
• Budget contingencies address access constraints, security and respondent tracking

Participants can commission external evaluators through clear technical and management arrangements.

• Terms of reference specify purpose, questions, scope, methods and deliverables
• Evaluator selection criteria assess technical expertise, independence and contextual knowledge
• Inception reports tested against agreed evaluation questions and methodological requirements
• Governance structures define roles for commissioners, implementers and advisory groups
• Quality assurance reviews assess evidence trails, analysis and conclusion logic
• Conflict-of-interest declarations protect evaluator independence and public confidence

Participants can draw conclusions that accurately reflect evidence strength and uncertainty.

• Findings separated from interpretations, conclusions and recommendations
• Causal claims calibrated to design strength, data quality and assumptions
• Limitations reported with likely direction and significance of bias
• Negative and null findings examined before recommending programme continuation
• Unexpected effects traced through implementation evidence and contextual changes
• Evidence-to-decision tables connect findings with feasible management actions

Participants can communicate impact findings for technical, operational and senior audiences.

• Evaluation reports follow clear questions, methods, findings, conclusions and recommendations
• Executive summaries state decisions, headline findings, caveats and actions
• Results tables present denominators, comparison groups, estimates and uncertainty
• Data visualisation selects charts that avoid misleading scales and comparisons
• Learning workshops convert findings into owners, actions and review dates
• Management responses document acceptance, adaptation or rejection of recommendations

Participants can build, review and present a complete impact evaluation design for a real project or programme.

• Theory of change finalised with causal pathways, assumptions and rival explanations
• Evaluation questions prioritised against decisions, users and available resources
• Design choice justified through counterfactual strategy and validity risk assessment
• Sampling, measurement and fieldwork plans assembled into an evaluation protocol
• Ethics, governance, budget and quality assurance arrangements documented clearly
• Peer review tests feasibility, methodological logic and evidence-use planning