Public performance reporting: definitions and comparability

In examining public performance reporting: definitions and comparability, for the available evidence, the available evidence should be interpreted with close attention to definitions, population coverage, collection methods and the limits of comparison.

Evidence base for public performance reporting

Application to public performance reporting depends on evidence from the relevant jurisdiction or institution.

In the context of public performance reporting, education indicators should support decisions by describing outcomes and variation with definitions and limitations that permit responsible interpretation.

• Define the decision the indicator will inform.
• Analyse missing information.
• Report uncertainty and revisions before using it to determine a learner or provider outcome.
• Avoid causal claims unsupported by the design.
• Test comparability.

Controls for public performance reporting

When examining public performance reporting, definitions, reference periods, population coverage, institutional boundaries and collection practices must be sufficiently aligned for the observed difference to have a stable meaning.

The principal risks in relation to the analysis are averages concealing distribution, incomplete coverage, small differences overstated, and data revisions not carried through to published conclusions. For public performance reporting, a weakness in one part of the control environment may obscure a related failure elsewhere. Across the defined scope, review should follow the sequence of decisions and records rather than assess documents in isolation.

In the context of public performance reporting, readily available material should not define the enquiry if it cannot answer the relevant decision question. For the available evidence, the most relevant material is likely to include disaggregated results, revision and comparability records, triangulation with administrative and qualitative evidence, and coverage and missingness analysis.

• Do the reference periods align?
• Is the remaining difference educationally material?
• Has a classification changed?
• Are the populations defined on the same basis?
• Are exclusions and missing records comparable?

Review of public performance reporting

A competent review of the comparison should prepare a comparability table before analysing results. Record common elements, material differences, breaks in series and the direction in which each limitation may affect the conclusion; do not rank systems where those limitations remain material.

Interpretation of the measure should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. For decisions concerning public performance reporting, measurement can reveal where outcomes differ; it does not by itself establish why they differ or which intervention will work.

The assurance record for the comparison should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. In the context of public performance reporting, traceable source and version information allow genuine improvement to be distinguished from administrative revision.

For public performance reporting, no individual measure is sufficient to establish effective operation of the available evidence across the affected scope.