Educational Programme Impact Evaluation Training Course

Participants can analyse contextual factors that affect educational programme results and evaluability.

• Education sector analysis maps policy, financing, governance and service delivery
• Problem trees identify root causes of exclusion and poor learning
• Stakeholder mapping identifies decision-makers, implementers, communities and affected learners
• Conflict sensitivity examines risks affecting schools, learners and data collection
• Political economy analysis tests feasibility of proposed programme changes
• Context assumptions recorded within the evaluation design and risk register

Participants can construct a testable theory of change for an educational intervention.

• Theory of Change maps causal pathways from intervention to learner outcomes
• Backward mapping starts with intended learning, retention or transition results
• Assumptions specify conditions required for activities to produce outcomes
• Causal pathways identify mechanisms such as teacher practice and attendance
• External factors include household poverty, displacement and education policy
• Testable hypotheses convert programme assumptions into evaluation questions

Participants can develop focused evaluation questions and a usable evaluation matrix.

• Evaluation questions distinguish relevance, effectiveness, impact and sustainability
• Question prioritisation balances decision value, feasibility, cost and timing
• Evaluation matrices link questions, indicators, sources, methods and responsibilities
• Judgement criteria define what counts as acceptable programme performance
• Evidence gaps identified before fieldwork plans and data collection begin
• Evaluation scope sets populations, geography, intervention periods and exclusions

Participants can specify valid indicators for education access, quality, learning and equity.

• Indicator reference sheets define numerator, denominator, source and frequency
• Access indicators measure enrolment, attendance, completion and transition rates
• Learning indicators measure foundational literacy, numeracy and subject attainment
• Teacher practice indicators capture pedagogy, feedback and classroom time
• Equity indicators reveal disparities between learner groups and locations
• Proxy indicators assessed for validity before replacing direct outcome measures

Participants can establish credible baselines, targets and counterfactual requirements.

• Baseline studies establish pre-intervention conditions and outcome distributions
• Target setting uses historical performance, benchmarks and implementation capacity
• Counterfactuals estimate outcomes participants would experience without intervention exposure
• Baseline equivalence checks compare treatment and comparison group characteristics
• Seasonality affects attendance, nutrition, migration and learning measurement schedules
• Recall periods aligned with school calendars and programme implementation cycles

Participants can assess when randomised designs are ethical, feasible and informative.

• Randomised controlled trials allocate eligible units through documented random assignment
• Cluster randomisation assigns schools, classrooms or communities rather than learners
• Randomisation protocols protect allocation integrity and prevent selection bias
• Power calculations estimate sample sizes for detectable education programme effects
• Spillover effects considered when comparison schools receive indirect programme exposure
• Ethical review addresses withholding, delayed treatment and informed consent

Participants can select quasi-experimental designs where random assignment is unavailable.

• Difference-in-differences compares outcome changes between intervention and comparison groups
• Propensity score matching balances observable characteristics across participant groups
• Regression discontinuity exploits eligibility thresholds for programme participation decisions
• Interrupted time series assesses outcome trends before and after implementation
• Matching diagnostics assess covariate balance and common support
• Design selection considers data availability, bias risks and operational constraints

Participants can develop a defensible sampling strategy for an education evaluation.

• Sampling frames define eligible learners, schools, teachers and households
• Probability sampling supports population estimates with known selection probabilities
• Stratified sampling protects representation across sex, location and school type
• Cluster sampling accounts for learner grouping within classrooms and schools
• Sample size calculations consider effect size, clustering and anticipated attrition
• Replacement procedures prevent field teams from making undocumented substitutions

Participants can assess the quality and suitability of education outcome measures.

• Learning assessments align items with curriculum and intended competencies
• EGRA and EGMA assess foundational reading and mathematics skills
• Reliability examines consistency of scores across items, raters and occasions
• Validity assesses whether instruments measure intended learning constructs
• Translation and adaptation protect meaning across languages and contexts
• Data quality assessments review completeness, accuracy, timeliness and integrity

Participants can design qualitative inquiry that explains educational programme results.

• Key informant interviews examine implementation decisions and causal mechanisms
• Focus group discussions capture learner, caregiver and teacher perspectives
• Classroom observation protocols document pedagogy, participation and instructional time
• Purposive sampling selects information-rich participants and contrasting cases
• Thematic coding organises transcripts using deductive and inductive codebooks
• Reflexivity records researcher influence, positionality and interpretation limits

Participants can integrate quantitative and qualitative evidence into coherent findings.

• Convergent mixed-methods designs collect quantitative and qualitative evidence concurrently
• Explanatory sequential designs use interviews to interpret statistical findings
• Triangulation compares evidence across methods, sources, researchers and time
• Joint displays present numerical results alongside explanatory qualitative themes
• Divergent findings trigger further inquiry rather than selective reporting
• Inference quality assessed through credibility, transferability and causal plausibility

Participants can prepare fieldwork systems that protect data quality and participant welfare.

• Data collection protocols define recruitment, consent, administration and escalation procedures
• Enumerator training standardises instrument delivery, probing and neutral respondent treatment
• Pilot testing identifies confusing questions, timing problems and skip errors
• Digital forms apply validation rules, skip logic and timestamp controls
• Field supervision checks consent records, interview quality and daily completion rates
• Safeguarding referral pathways address disclosures involving children and vulnerable adults

Participants can interpret core statistical outputs used in education impact evaluations.

• Descriptive statistics summarise participant characteristics, outcomes and programme exposure
• Cross-tabulations identify disparities by sex, disability, grade and location
• Mean comparisons estimate differences between intervention and comparison groups
• Regression models adjust effect estimates for baseline covariates and clustering
• Confidence intervals communicate uncertainty around estimated programme effects
• Attrition analysis tests whether missing observations threaten result validity

Participants can make proportionate causal claims when attribution evidence is limited.

• Contribution analysis tests whether programme theory explains observed outcomes
• Contribution stories assemble evidence on mechanisms, assumptions and external influences
• Alternative explanations identified before judging the programme contribution claim
• Process tracing examines causal mechanisms through evidence tests and sequences
• Strength of evidence assessed against transparency, consistency and plausibility criteria
• Causal language calibrated to design strength and evidence limitations

Participants can integrate equity and protection considerations into evaluation design and interpretation.

• Equity-focused evaluation examines differential outcomes among excluded learner groups
• Washington Group questions support disability disaggregation in household surveys
• Gender analysis examines barriers, norms, safety and participation patterns
• Language-of-instruction analysis assesses access and learning for multilingual learners
• Education in emergencies considers displacement, protection risks and interrupted schooling
• Do-no-harm procedures protect participants during sensitive data collection

Participants can compare education programme costs with measured outcomes and alternatives.

• Cost analysis distinguishes financial, economic, fixed and variable programme costs
• Ingredient costing identifies inputs required to deliver educational interventions
• Cost-effectiveness ratios compare costs per learner outcome achieved
• Sensitivity analysis tests how assumptions affect cost-effectiveness conclusions
• Cost data aligned with programme components, delivery channels and beneficiaries
• Value-for-money findings combine economy, efficiency, effectiveness and equity evidence

Participants can communicate findings responsibly and establish arrangements for evaluation use.

• Evaluation reports distinguish findings, conclusions, recommendations and management responses
• Results tables present disaggregated effects, confidence intervals and sample sizes
• Data visualisation selects charts suited to education outcomes and audiences
• Recommendation prioritisation assigns actions, owners, resources and completion dates
• Learning workshops test findings with implementers, communities and decision-makers
• Evaluation governance defines independence, quality assurance and conflict-of-interest controls

Participants can commission and oversee an external education impact evaluation effectively.

• Terms of reference specify purpose, questions, methods, deliverables and governance
• Evaluator selection assesses technical competence, independence and contextual knowledge
• Inception reports test design feasibility before fieldwork resources are committed
• Quality assurance reviews evaluation matrices, instruments, datasets and draft reports
• Data ownership agreements define access, retention, anonymisation and publication rights
• Management responses convert recommendations into tracked improvement actions

Participants can build, review and present a complete impact evaluation plan for an educational programme.

• Programme brief converted into a theory of change and evaluation scope
• Evaluation questions prioritised using decision value, feasibility and ethical criteria
• Design choice justified against counterfactual requirements and implementation constraints
• Indicator matrix specifies measures, sources, disaggregation and collection timing
• Sampling, fieldwork, analysis and safeguarding plans integrated into one package
• Peer review tests causal claims, risks, costs and reporting usefulness