Quality controls for evidence-based interventions

The present attention to evidence-based interventions follows the World Development Report 2018 and requires a careful distinction between public commitment, institutional practice and demonstrated result. For the corrective action, effective improvement requires ownership, a time-bound intervention and independent confirmation that the intended result has been achieved. The effect on learner access and reliable decision-making should inform the scale of control applied.

Improvement objective and baseline

The reference basis—the World Development Report 2018—is evidential rather than self-executing. Patterns in the material may justify enquiry, although they do not by themselves determine legal position or cause. In applying it to evidence-based interventions, users should review the source definitions, population coverage, reference period and stated limitations before transferring a system-level finding to an individual provider or learner group.

For evidence-based interventions, the World Development Report 2018, Learning to Realize Education’s Promise, distinguishes schooling from learning and describes poor outcomes, immediate barriers within teaching and learning, and deeper system-level causes. It proposes three connected directions: assess learning, act on evidence and align actors around learning. Attendance and completion remain important, but they do not provide direct evidence that learners have acquired the intended knowledge and skills.

In the context of evidence-based interventions, a proper review of corrective action should establish the intended outcome before selecting controls or indicators. Follow-up should determine whether the change is embedded in ordinary operations and whether it has created new risks or unequal effects. A chosen approach should be justified against its context, with departures and review points under documented control.

The principal risks in relation to the intended improvement are responsibility distributed without clear ownership, variation across sites or programmes, corrective action closed on activity rather than effect, and learner experience omitted from review. For decisions concerning evidence-based interventions, the relationship between the risks is material: one failed safeguard may remove the evidence needed to activate another.

For the relevant practice, an improvement plan should connect a verified problem with a specific intervention, accountable ownership, resources, milestones and a measure of effect. In work concerning evidence-based interventions, broad intentions should be converted into decisions capable of review. Any condition preventing complete assurance should appear with the evidence on which the judgement relies.

The principal risks associated with corrective action should be assessed as connected conditions. As regards evidence-based interventions, the conclusion should rely on evidence whose date, source and coverage are sufficient for the decision. Within the scope under review, contradictory evidence should be investigated and resolved, not omitted from the record.

Controls and accountable action

Care is required in drawing conclusions about evidence-based interventions. For corrective action, quality cannot be inferred from reputation, intention or documentation alone. The record should show how the arrangement operates and what outcome follows in the affected scope. Improvement data should not be selected only because it is readily available. Material limitations should be stated with the finding presented to decision-makers and affected learners.

Records relating to the relevant practice should preserve both the conclusion and its limits. In the context of evidence-based interventions, if further evidence changes the position, the correction should identify its scope and any earlier decision requiring reconsideration.

• Define the intended outcome.
• Identify the affected scope.
• Act on adverse findings.
• Assign accountable ownership.
• Test material variation.

Evidence of effect

The review method for evidence-based interventions should be reproducible. Corrective action should be proportionate to the identified condition and tested where risk permits. Wider implementation should follow evidence of benefit and acceptable unintended effects. Amend the plan where evidence does not support the original causal assumption. The retained analysis should be reproducible from the selected evidence, decision rule and recorded reasons for accepted exceptions.

Improvement of evidence-based interventions should proceed through controlled tests where risk permits.

• Who confirms sustained effectiveness?
• Does each action address a stated cause?
• What measure will establish success?
• Is the problem defined by evidence?
• Are dependencies and resources identified?

Sustaining improvement

For evidence-based interventions, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions.

In work concerning evidence-based interventions, the intended substantive result should remain the starting point for review. Quality assurance should connect stated educational purposes with implemented controls, reliable evidence and action where outcomes fall below expectation. Within the scope under review, inputs and formal commitments should be distinguished from demonstrated operation and outcome. Assurance should address actual effect and provide a means of correcting disadvantage that the arrangement did not intend.

Progress on quality controls for evidence-based interventions under review is not the amount of policy or documentation produced. Progress is demonstrated when the intended educational result is achieved, adverse variation is identified and responsible bodies act where it is not.