Considers the controls required to improve post-PISA improvement planning and to distinguish completed activity from demonstrated change.
The present attention to post-PISA improvement planning follows the PISA 2022 follow-up during 2024 and requires a careful distinction between public commitment, institutional practice and demonstrated result. In reviewing the improvement priority, the purpose of an improvement method is not to produce an action plan; it is to change a material condition and verify that the change is sustained. Review should cover the complete affected scope and preserve material differences between locations, programmes, delivery modes and learner groups. A policy approved at the centre is insufficient where local implementation has not been tested.
The quality significance of the intervention follows from a basic distinction between availability and effective provision. In reviewing the intervention, education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation. A single entry control or reported outcome cannot demonstrate consistent operation across the learner journey.
Public-interest context
The evidential record for post-PISA improvement planning should permit a reviewer to trace the matter from decision to outcome. This may require coverage and missingness analysis, population and sampling information, disaggregated results, and uncertainty estimates where relevant, supported by triangulation with administrative and qualitative evidence and indicator definitions and metadata. Sampling remains insufficient where it excludes a material group or cannot resolve contradictory evidence or recurrence.
The reference point is the PISA 2022 follow-up during 2024. Intervention in the intervention should be proportionate to the identified condition and tested where risk permits. Wider implementation should follow evidence of benefit and acceptable unintended effects. Decision-makers should state which matters are evidenced, which express policy and which require authorised judgement. Decisions and public statements should preserve the distinction, including when the matter is reconsidered.
The first PISA 2022 results were released on 5 December 2023, with mathematics as the principal assessment domain and reading and science also reported. The cycle was conducted after major disruption to education systems. Comparisons require attention to participation, coverage and the exceptional context; changes from earlier cycles should not be attributed to a single cause without further evidence.
A focused examination of the intervention requires a clear analytical discipline. The analysis of the matter under review proceeds on the basis that effectiveness is the demonstrated change in the condition the action was intended to address. Completion of training, publication of guidance or installation of a system is an output and should not be reported as an outcome without further evidence. The distinction matters because evidence may appear sufficient while addressing a different population, period or outcome.
A narrow control over the corrective programme may create false assurance. In the present context, proxy measures treated as direct outcomes, changes in definition presented as changes in performance and small differences overstated may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. Adverse cases should form part of the sample wherever they may reveal a material control weakness.
Responsibilities and material risks
Responsibility for post-PISA improvement planning should be visible at the point where consequential decisions are made. A decision concerning the intervention should recognise that the intervention should be tested on a scale proportionate to the risk before wider implementation, unless immediate system-wide action is necessary to protect learners. The matter should be escalated when evidence is incomplete, a conflict is present, affected learners are not represented or the likely effect is material.
Where the corrective programme involves partners, suppliers or several public bodies, responsibility should be mapped across the complete service. Contractual or inter-agency arrangements should identify who holds records, informs learners and acts on incidents. Multiple delivery partners do not justify fragmented accountability or remedy.
Decisions concerning the matter under review should remain traceable to the information available for the stated reference period. A revision should state whether the change concerns the underlying condition, the evidence, the method or the interpretation. Users should not be left to infer a change in performance where the observed movement results from revised reporting.
- Report uncertainty and revisions, including material exceptions and unequal effects.
- Document numerator and denominator and retain evidence sufficient for independent review.
- Test comparability and retain evidence sufficient for independent review.
- Analyse missing information before using it to determine a learner or provider outcome.
- Avoid causal claims unsupported by the design and retain evidence sufficient for independent review.
What should be examined
A proportionate method is available for post-PISA improvement planning. A competent review of the affected practice should set a baseline and success measure before intervention, define the review period, compare the result with the intended outcome and examine adverse or unequal effects. Continue monitoring long enough to determine whether the improvement is sustained. Contrary evidence should not be removed merely because aggregate performance appears acceptable.
A decision to close improvement work on the intervention should be made by a person with authority and sufficient independence from implementation. The closure evidence should cover the relevant period and scope, include adverse cases and show whether the change is sustained. Recurrence or unequal effect should trigger renewed analysis rather than automatic repetition of the same intervention.
The analysis of the improvement priority should remain within the limits of the evidence. For the affected practice, methods should be proportionate to the significance and recurrence of the problem; low-risk local issues and systemic learner-protection failures require different levels of control. A decision concerning the intervention should recognise that measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work. Material uncertainty should result in further enquiry or an expressly limited finding.
The measure of progress on the matter under review is not the amount of policy or documentation produced. The relevant measure is demonstrated public benefit, including detection and correction of material variation.