Education Policy Evaluation Training Course

Participants can construct a theory of change that specifies causal pathways, assumptions and risks.

• Theory of Change components: inputs, activities, mechanisms and outcomes
• Causal pathways from teacher training to classroom practice
• Assumptions tested through evidence, stakeholder consultation and field observation
• External factors affecting attendance, learning and school retention
• Results chains for national reforms and local implementation models
• Policy logic diagrams with feedback loops and unintended effects

Participants can frame answerable evaluation questions aligned with intended users and decisions.

• OECD DAC criteria adapted for education policy evaluation
• Question hierarchy covering relevance, effectiveness, impact and sustainability
• Defining primary questions, secondary questions and analytic subgroups
• Evaluation matrices linking questions, indicators, sources and methods
• Prioritising questions against time, budget and data constraints
• Formative, summative and developmental evaluation purposes compared

Participants can develop an indicator framework for education access, learning, equity and delivery quality.

• Indicator reference sheets defining numerator, denominator and disaggregation
• Learning outcomes measured through assessments, proficiency thresholds and progression
• Access indicators for enrolment, attendance, completion and transition
• Equity disaggregation by gender, disability, location and income
• Implementation indicators covering teacher attendance, materials and instructional time
• SMART indicator tests applied to education programme results

Participants can select an evaluation design that supports appropriate causal claims.

• Counterfactual reasoning and the problem of missing outcomes
• Experimental, quasi-experimental and non-experimental designs compared
• Internal validity threats: selection, history, maturation and attrition
• External validity when scaling pilots across districts
• Causal diagrams identifying confounders, mediators and selection mechanisms
• Matching design strength to policy stakes and feasibility

Participants can prepare a baseline and sampling plan with credible comparison strategies.

• Baseline timing relative to policy rollout and seasonal effects
• Target populations, sampling frames and unit-of-analysis decisions
• Probability sampling: simple random, stratified and cluster approaches
• Sample size drivers: effect size, power and intracluster correlation
• Comparison group selection using eligibility thresholds and rollout schedules
• Attrition prevention plans for learners, teachers and schools

Participants can specify fit-for-purpose data collection instruments and fieldwork controls.

• Survey questionnaires aligned with indicators and respondent burden
• Classroom observation protocols measuring instructional practice and engagement
• Student assessment items matched to curriculum and grade level
• Key informant interviews with officials, headteachers and implementers
• Focus group guides for learners, parents and community members
• Enumerator manuals, piloting procedures and field supervision controls

Participants can establish ethical, safeguarding and data quality requirements for an education study.

• Informed consent and assent procedures for child participants
• Safeguarding protocols for disclosures, referrals and fieldworker conduct
• Data protection controls for student records and identifiers
• Reliability checks for observation, survey and assessment data
• Data quality assurance through range checks and back-checks
• Ethics review requirements, risk registers and incident escalation

Participants can convert an evaluation design into a resourced workplan and procurement brief.

• Evaluation work breakdown structure from inception through dissemination
• Critical path scheduling for baseline, endline and reporting
• Budget categories for personnel, travel, tools and data collection
• Terms of reference specifying scope, methods and deliverables
• Evaluator selection criteria covering independence, expertise and feasibility
• Risk registers addressing access, security and implementation delays

Participants can interpret core quantitative analyses used in education policy evaluations.

• Descriptive statistics for enrolment, attendance and learning outcomes
• Cross-tabulations comparing results by treatment and subgroup
• Confidence intervals, p-values and practical significance interpreted
• Regression models controlling for baseline differences and covariates
• Standardised mean differences for learning outcome comparisons
• Missing-data patterns, imputation choices and reporting limitations

Participants can design qualitative inquiry and integrate findings with quantitative evidence.

• Purposive sampling for diverse schools, respondents and implementation contexts
• Thematic coding using deductive and inductive codebooks
• Process tracing for policy mechanisms and implementation decisions
• Triangulation across administrative records, interviews and observations
• Joint displays connecting quantitative patterns with qualitative explanations
• Negative case analysis challenging initial interpretations and recommendations

Participants can assess when randomisation is ethical, feasible and useful for an education intervention.

• Randomised controlled trial logic for education interventions
• Individual, classroom, school and cluster randomisation compared
• Random assignment procedures, allocation concealment and balance checks
• Waitlist designs for phased education programme implementation
• Spillover effects between schools, teachers and communities
• Ethical justification where scarce services are allocated randomly

Participants can specify and interpret quasi-experimental approaches for education policy impact assessment.

• Difference-in-Differences using baseline and follow-up comparison data
• Parallel trends assessment before applying Difference-in-Differences estimates
• Regression discontinuity around scholarship and eligibility thresholds
• Propensity score matching for non-random programme participation
• Interrupted time series for system-wide policy changes
• Sensitivity analysis testing alternative model specifications and assumptions

Participants can evaluate whether a policy was delivered as designed and explain variation.

• Implementation fidelity dimensions: adherence, dose, quality and reach
• Bottleneck analysis across procurement, training and school support
• Service delivery pathways from central policy to classrooms
• Implementation dashboards combining milestones, risks and field evidence
• Adaptation tracking during pilots, scale-up and local delivery
• Explaining outcome variation through implementation and contextual factors

Participants can assess who benefits, who is excluded and why results differ across groups.

• Equity analysis by gender, disability, poverty and geography
• Intersectional subgroup analysis without misleading small-sample conclusions
• Barrier analysis for marginalised learners and families
• Accessibility review of assessment, surveys and consultation methods
• Distributional impact questions for targeted education subsidies
• Equity-focused recommendations linked to delivery and financing decisions

Participants can compare education intervention costs, outputs and outcomes for resource allocation decisions.

• Cost analysis separating start-up, recurrent and opportunity costs
• Unit-cost calculations per learner, school and outcome
• Cost-effectiveness ratios for literacy, attendance and completion gains
• Cost-benefit analysis assumptions, discounting and sensitivity testing
• Value-for-money dimensions: economy, efficiency, effectiveness and equity
• Interpreting economic evidence alongside feasibility and implementation risk

Participants can communicate evaluation findings and recommendations for realistic policy decisions.

• Evidence grading based on design strength and data limitations
• Findings statements separating observation, inference and interpretation
• Recommendation design specifying action, owner, timeframe and resources
• Policy briefs structured for ministers, boards and funding committees
• Data visualisation choices for learning, equity and cost findings
• Management responses and follow-up trackers for evaluation recommendations

Participants can build, review and present a complete evaluation and impact assessment plan for an education policy or programme.

• Policy problem statement supported by decision-use requirements
• Theory of Change with assumptions, risks and measurable pathways
• Evaluation matrix linking questions, indicators, methods and sources
• Design rationale covering comparison strategy, sampling and ethics
• Workplan, budget, governance arrangements and quality assurance controls
• Peer review against feasibility, credibility, equity and usefulness