Governance and Justice Impact Evaluation Training Course

Participants can build a testable theory of change and prioritise evaluation questions.

• Theory of change diagrams specifying actors, mechanisms and assumptions
• Problem-driven iterative adaptation and implications for evaluation design
• Articulating causal pathways from legal aid to case resolution
• Identifying behavioural, institutional and normative change mechanisms
• Evaluation matrices connecting questions, indicators, methods and evidence sources
• Prioritising questions using decision relevance, feasibility and ethical risk
• Testing assumptions through contribution analysis and stakeholder evidence

Participants can assess whether a governance or justice intervention is ready for impact evaluation.

• Political economy analysis of interests, incentives, power and reform coalitions
• Stakeholder mapping for state institutions, civil society and affected groups
• Context mapping of conflict, exclusion, corruption and institutional legitimacy
• Evaluability assessments covering intervention clarity, data availability and timing
• Identifying contamination, spillovers and concurrent reform initiatives
• Assessing implementation fidelity before attributing observed outcome changes
• Documenting contextual assumptions within an evaluation inception report

Participants can assess and outline a randomised evaluation design where conditions permit.

• Randomised controlled trials and credible counterfactual group construction
• Individual, cluster and phased-in randomisation for public services
• Eligibility criteria, lotteries and transparent assignment procedures
• Intention-to-treat estimates under incomplete programme participation
• Managing treatment contamination, non-compliance and intervention spillovers
• Ethical conditions for withholding, delaying or varying programme access
• Pre-analysis plans for limiting selective outcome reporting

Participants can select feasible alternatives when random assignment is unavailable.

• Difference-in-differences using baseline and follow-up comparison group data
• Regression discontinuity around legal eligibility or programme thresholds
• Propensity score matching and limitations of observed covariates
• Interrupted time series for policy, court or policing reforms
• Synthetic control methods for jurisdiction-level reform interventions
• Contribution analysis for complex reforms lacking a valid counterfactual
• Process tracing tests for causal mechanisms and alternative explanations

Participants can prepare a sampling and measurement approach for mixed-methods evaluation.

• Sampling frames for service users, institutions, communities and officials
• Power calculations using minimum detectable effects and intracluster correlation
• Stratified sampling for gender, disability, ethnicity and geographic exclusion
• Constructing indicators for trust, procedural justice and perceived safety
• Validated survey scales, cognitive testing and questionnaire translation
• Key informant interviews, focus groups and structured observation protocols
• Mixed-methods integration through joint displays and convergence assessment

Participants can apply proportionate ethical safeguards to sensitive governance and justice research.

• Informed consent procedures for low-literacy and vulnerable participant groups
• Do No Harm principles for politically sensitive research activities
• Risk assessments addressing retaliation, surveillance and disclosure harms
• Confidentiality protocols for survivors, detainees, whistleblowers and complainants
• Data minimisation, access controls and secure identifier management
• Referral pathways for safeguarding disclosures during fieldwork interviews
• Ethics review documentation and incident escalation procedures

Participants can establish quality assurance procedures for credible evaluation data.

• Data quality assurance plans covering validity, reliability and completeness
• Enumerator recruitment, training and supervision for sensitive interviews
• Field piloting to identify comprehension, recall and response biases
• Skip logic, range checks and duplicate detection in digital surveys
• Back-checks, spot-checks and audio audits for enumerator verification
• Managing missing data, attrition and respondent non-response patterns
• Fieldwork dashboards tracking sample progress, risks and protocol deviations

Participants can interpret impact findings and communicate appropriately qualified causal conclusions.

• Estimating average treatment effects and confidence intervals from evaluation data
• Disaggregating results by gender, age, disability and marginalised status
• Testing parallel trends and balance assumptions before causal interpretation
• Sensitivity analysis for attrition, hidden bias and model specification
• Triangulating quantitative estimates with qualitative mechanism evidence
• Interpreting null findings, heterogeneous effects and unintended consequences
• Writing causal claims proportionate to design strength and data limitations

Participants can prepare an actionable evaluation design pack and findings-use plan.

• Terms of reference specifying questions, methods, governance and deliverables
• Evaluation budgets covering design, fieldwork, analysis and dissemination
• Assessing evaluator proposals using technical, ethical and contextual criteria
• Findings-use plans identifying decisions, users, products and timing
• Evidence communication through policy briefs, learning sessions and dashboards
• Management responses linking findings to adaptation and accountability actions
• Application plan for transferring course methods to active programmes