Randomized Controlled Trials for Programme Impact Evaluation Training Course

Course overview

Programme teams are frequently asked to demonstrate whether a cash transfer, livelihoods package, education intervention, protection service or health campaign caused measurable change—not simply whether participants improved over time. Randomized controlled trials (RCTs) provide a defensible way to establish attribution, but weak treatment allocation, underpowered samples, contamination, missing data and poorly specified outcomes can undermine findings and expose organisations to ethical, operational and reputational risk. This course equips M&E professionals to judge when an RCT is feasible and to commission, manage or conduct one to decision-grade standards.

Participants work through the complete RCT lifecycle for development and humanitarian programmes: developing a theory of change and estimand, defining eligibility and outcomes, selecting individual, cluster or phased randomisation designs, calculating sample size and designing field-ready allocation procedures. They learn intention-to-treat analysis, balance checks, treatment-on-the-treated estimation, attrition analysis, subgroup analysis and transparent reporting. Sessions also address informed consent, safeguarding, trial registration, data protection, spillovers, non-compliance and the practical constraints of evaluating interventions in dynamic field settings.

Teaching combines short technical briefings with worked examples in R, Stata and Excel, using realistic programme datasets and implementation scenarios. Participants design an RCT protocol for a live or proposed intervention, build a randomisation plan, specify a minimum analysis plan and interpret model output for a management audience. They leave with a structured RCT evaluation pack containing a theory of change, outcome matrix, sample-size assumptions, allocation procedure, analysis plan and implementation risk register that can be adapted for their organisation’s next evaluation.

The course is designed for professionals who already work with programme data, evaluations or research partners and need stronger technical control over causal impact evidence.

Course objectives

By the end of this course, participants will be able to:

• Formulate an RCT evaluation question, estimand and testable theory of change for a programme intervention
• Select individual, cluster, stratified or phased randomisation designs based on operational and ethical constraints
• Construct an outcome matrix with primary outcomes, secondary outcomes, measurement timing and data sources
• Calculate sample-size requirements using effect size, power, intra-cluster correlation and anticipated attrition assumptions
• Create a reproducible treatment-allocation procedure using random-number generation, blocking and stratification
• Analyse RCT data using baseline balance checks, intention-to-treat estimates and cluster-robust standard errors
• Diagnose attrition, non-compliance, spillovers and contamination and specify appropriate mitigation or sensitivity analyses
• Produce an RCT protocol and minimum analysis plan suitable for ethics review, implementation teams and decision-makers

Benefits of attending

Target competencies

Who should attend

• Monitoring and Evaluation Managers — who need to design or oversee credible impact evaluations across programme portfolios
• Impact Evaluation Specialists — who require stronger command of trial design, analysis and reporting choices
• Programme Managers — who must align delivery operations, beneficiary selection and evaluation requirements
• Research and Learning Officers — who turn programme evidence into usable learning and management decisions
• Data and Evidence Analysts — who prepare, analyse and quality-assure trial datasets and results
• Donor-funded Project Directors — who need to assess whether proposed RCTs are feasible, ethical and value for money

Requirements and prerequisites

Participants should have practical experience with programme monitoring, survey data, evaluation studies or research commissioning. The course assumes familiarity with basic statistical concepts: averages, proportions, confidence intervals, hypothesis testing and regression as an idea, though not advanced mathematics. Participants should be able to work with spreadsheets and bring a laptop capable of running R or Stata; guided code and templates are provided. Prior RCT experience, programming fluency, econometrics, power-analysis software and a postgraduate research qualification are not required. Those with no prior exposure to evaluation terminology should review basic results-framework and indicator concepts before attending.

Training methodology

The instructor uses a single development-programme case throughout the week, moving from a livelihoods intervention question to a completed trial design and analysis plan. Short lectures establish the statistical and ethical rationale; guided labs use Excel, R and Stata to randomise units, inspect baseline balance, estimate impacts and test robustness. Small groups resolve field constraints such as village-level delivery, delayed rollout and participant spillovers. Daily peer review links technical choices to implementation realities, and the final session converts each participant’s work into an application plan for a current or forthcoming evaluation.

Course outline

Day 1: Causal questions and RCT design choices

• Causal attribution, counterfactuals and potential outcomes
• Theory of change and intervention logic for trial design
• Estimands: intention-to-treat, treatment-on-the-treated and average treatment effects
• Eligibility criteria, target populations and unit-of-randomisation decisions
• Individual, cluster and phased-rollout RCT designs
• Primary outcomes, secondary outcomes and pre-specification
• Ethical equipoise, informed consent and safeguarding in programme trials

Workshop: Participants convert a programme theory of change into an RCT evaluation question, estimand, eligibility definition and outcome matrix.

Day 2: Sampling, power and treatment allocation

• Sampling frames, recruitment pathways and representativeness
• Minimum detectable effects and statistical power
• Sample-size calculations for individual randomised trials
• Design effects and intra-cluster correlation in cluster trials
• Attrition assumptions and sample inflation strategies
• Simple, blocked and stratified randomisation
• Allocation concealment, randomisation logs and audit trails

Workshop: Participants calculate a trial sample size and produce a reproducible blocked, stratified allocation list for a simulated beneficiary roster.

Day 3: Field implementation and data quality

• Baseline, endline and follow-up measurement schedules
• Survey instrument design and outcome measurement validity
• Tracking systems for mobile and displaced populations
• Treatment fidelity monitoring and implementation indicators
• Contamination, spillovers and information leakage
• Data-management plans, identifiers and secure data access
• Protocol deviations, adverse events and field decision rules

Workshop: Participants develop a field implementation plan that maps enrolment, consent, assignment, service delivery, measurement and risk controls.

Day 4: Estimating and testing programme effects

• Trial dataset structure, codebooks and analysis-ready files
• Baseline balance tables and randomisation checks
• Intention-to-treat estimation using difference in means
• Regression adjustment with baseline covariates
• Cluster-robust standard errors and cluster-level analysis
• Missing outcomes, attrition diagnostics and bounds analysis
• Non-compliance, instrumental variables and treatment-on-the-treated estimation

Workshop: Participants analyse a simulated RCT dataset in R, Stata or Excel and prepare an impact-results table with interpretation notes.

Day 5: Interpretation, reporting and evaluation governance

• Subgroup analysis, heterogeneity and multiple-testing risks
• Effect sizes, confidence intervals and practical significance
• Sensitivity analysis for alternative specifications
• Cost and cost-effectiveness measures alongside impact estimates
• Pre-analysis plans, trial registration and version control
• CONSORT reporting standards and transparent result communication
• Using RCT findings for adaptation, scale-up and donor decisions

Workshop: Participants complete and peer-review an RCT evaluation pack comprising a protocol summary, randomisation plan, minimum analysis plan and management briefing.

Tools & standards covered

R, Stata, Microsoft Excel, CONSORT 2010 Statement

A typical training day

Live online deliveries follow the same structure in the East Africa Time zone, with shorter screen blocks and longer breaks.

What the fee includes

How you can take this course

Certification

Participants who complete the full five days receive the Skillset Development Certificate of Completion, stating the course title, course code, dates and delivery format — suitable for professional-development records and employer reimbursement.

Frequently asked questions