Data governance in monitoring post-PISA improvement planning

Its significance for post-PISA improvement planning lies in the quality of implementation rather than in formal acknowledgement alone. Evidence concerning post-PISA improvement planning should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support.

Application of the evidence to data governance in monitoring post-PISA improvement planning

For post-PISA improvement planning, the first PISA 2022 results were released on 5 December 2023, with mathematics as the principal assessment domain and reading and science also reported. The cycle was conducted after major disruption to education systems. Comparisons require attention to participation, coverage and the exceptional context; changes from earlier cycles should not be attributed to a single cause without further evidence.

In examining data governance in monitoring post-PISA improvement planning, for post-PISA improvement planning, education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation.

• Define the decision the indicator will inform before using it to determine a learner or provider outcome.
• Document numerator and denominator.
• Avoid causal claims unsupported by the design.
• Test comparability.
• Analyse missing information.

Controls relevant to data governance in monitoring post-PISA improvement planning

Its relevance to post-PISA improvement planning should be assessed against the affected jurisdiction, learner population and form of provision.

For decisions concerning post-PISA improvement planning, analysis should make its decision rule explicit. Broad intentions should be converted into decisions capable of review.

Review criteria for data governance in monitoring post-PISA improvement planning

Implementation of post-PISA improvement planning should be organised around a decision that can be tested.

Risk assessment of the available evidence should give particular attention to averages concealing distribution, proxy measures treated as direct outcomes, and small differences overstated. A provider should also consider incomplete coverage and changes in definition presented as changes in performance.

• Who confirms sustained effectiveness?
• What measure will establish success?
• Does each action address a stated cause?
• Is the problem defined by evidence?
• Are dependencies and resources identified?

Implications for data governance in monitoring post-PISA improvement planning

Evidence concerning post-PISA improvement planning should be selected against a clearly defined question. The most relevant material is likely to include uncertainty estimates where relevant, triangulation with administrative and qualitative evidence, indicator definitions and metadata, and population and sampling information.

For operational review, authorities and providers should proceed in a defined sequence. Responsible bodies should prioritise actions by learner impact and control weakness, establish dependencies, test implementation at suitable intervals and retain unresolved items until effectiveness is verified. In reviewing post-PISA improvement planning, amend the plan where evidence does not support the original causal assumption.

Evidence considered for data governance in monitoring post-PISA improvement planning

Decision-makers using evidence on post-PISA improvement planning should be told what the data cannot establish as clearly as what it can. The basis for applying the result elsewhere should be established rather than assumed.

Proportionality in relation to the issue does not mean reduced protection for learners exposed to greater risk. For post-PISA improvement planning, measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work.

The assurance record for post-PISA improvement planning should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied.

Public reporting on the measure should distinguish established fact, analytical judgement and planned action. For decisions concerning post-PISA improvement planning, revision history should remain available where users have relied on the earlier conclusion.