Provides a disciplined basis for interpreting evidence on completion and retention, including material variation, missing information and revision risk.
The 2015 education target monitoring provides the immediate reference point for consideration of completion and retention in 2008. In reviewing the reported measure, the value of the present data lies in the questions it can answer reliably and in the limits it makes visible. The proportionality test should consider both the identified risk and the consequences of the control for affected learners.
The present position
The historical reference basis is the 2015 education target monitoring. Its relevance to completion and retention should be assessed against the affected jurisdiction, learner population and form of provision. International developments provide context; decisions affecting learners require evidence that is current and representative of the setting concerned.
A proper review of the analytical question should establish the intended outcome before selecting controls or indicators. In reviewing the comparison, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. Suitability, authorised variation and the date for reconsideration should be established when the arrangement is approved.
Risk assessment of the analytical question should give particular attention to aggregate gains concealing excluded groups, admission without adequate support, and attrition detected too late. A provider should also consider completion measures detached from learning and support allocated without evidence of need. Stronger controls are required where learners may not detect an error or where later correction cannot restore the lost opportunity.
The analysis of the comparison should make its decision rule explicit. Oversight of the reported measure should reflect the principle that an estimate should be reported with the uncertainty arising from sampling, measurement, missing information and model choice. A precise number may still support only a cautious conclusion if the underlying assumptions are weak. The method should prevent an unfavourable result from being dismissed through an unrecorded change in interpretation.
Each source should have a stated purpose in supporting or limiting the conclusion. For the evidence under review, the most relevant material is likely to include evaluation of interventions, clearly defined access and completion indicators, resource allocation by need, and progression and early-warning information. Confidence is strengthened by corroboration, not by the volume of records drawn from the same underlying source.
Responsibilities and material risks
The analysis of completion and retention should remain within the limits of the evidence. The analysis of the matter examined proceeds on the basis that a single indicator rarely provides an adequate account of quality. Quantitative evidence should be considered with implementation records and the experience of affected learners. Oversight of the matter examined should reflect the principle that equal treatment does not necessarily produce equitable access or outcomes. Different support may be justified where it addresses a documented barrier without changing the required educational outcome. If uncertainty could change a consequential decision, additional evidence or a narrower conclusion is required.
Records relating to the evidence under review should preserve both the conclusion and its limits. The correction record should state what the new evidence changes and which earlier conclusions or decisions require review. This is material where learners, authorities or institutions relied on information that cannot be corrected by replacing the current text alone.
- Analyse barriers across the learner journey, and retain the basis, responsible function and affected scope.
- Report limitations in available data before it informs a consequential decision.
- Evaluate differential outcomes within a defined period and review the result.
- Monitor progression at useful intervals, identifying the accountable function and affected scope.
- Identify groups not represented in aggregate results, recording who is responsible and which provision or learners are affected.
Testing implementation and effect
For operational review of completion and retention, authorities and providers should proceed in a defined sequence. For the evidence under review, the reviewer should report intervals or sensitivity ranges where appropriate, test whether the conclusion changes under plausible assumptions, and distinguish statistical detectability from educational significance. Record unresolved uncertainty beside the principal finding. Observations may inform further enquiry, but only supported findings should determine conformity or effectiveness.
Decision-makers using evidence on the analytical question should be told what the data cannot establish as clearly as what it can. Users should be able to distinguish descriptive, comparative and evaluative findings and understand their proper level of application. Application in another setting depends on a separate examination of context and comparability.
- What are the principal sources of uncertainty?
- Would a reasonable alternative assumption change the conclusion?
- Are small groups adequately represented?
- How should uncertainty affect the decision?
- Is the apparent difference larger than the measurement error?
Matters requiring continuing review
Public reporting on completion and retention should distinguish established fact, analytical judgement and planned action. The record should preserve every revision capable of affecting a prior decision. If definitions, coverage or evidence alter an earlier conclusion, the reason should be stated so that revision is not mistaken for changed performance.
The quality significance of the evidence under review follows from a basic distinction between availability and effective provision. Oversight of the comparison should reflect the principle that education systems should examine not only who enters education, but who can participate effectively, progress and complete with the intended learning outcomes. Oversight should examine implementation throughout the learner journey, not only at entry or through one reported outcome.
A complete conclusion on the matter examined requires evidence extending beyond an individual measure or safeguard. The final judgement should connect the applicable expectation to implementation and outcomes while identifying unresolved risk.