Examines the evidential basis for public performance reporting, with attention to definitions, coverage, reference periods and responsible use of findings.
In 2017, consideration of public performance reporting must take account of the 2017 accountability agenda and the responsibilities it places before education systems. For the evidence under review, the principal analytical task is to separate an observed difference from a conclusion about its cause. Assessment should consider learner impact, institutional accountability and stewardship of entrusted resources. Uniform administrative form is not required where equivalent public outcomes can be demonstrated.
Responsibility for the analytical question should be visible at the point where consequential decisions are made. For the comparison, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review.
Public-interest context
The historical reference basis is the 2017 accountability agenda. Its relevance to public performance reporting should be assessed against the affected jurisdiction, learner population and form of provision. International developments provide context; decisions affecting learners require evidence that is current and representative of the setting concerned.
The quality significance of the comparison follows from a basic distinction between availability and effective provision. The analysis of the analytical question proceeds on the basis that governing bodies should receive sufficient, reliable and timely information to oversee education quality, learner protection and material institutional risk. Review should cover the stages at which learners receive information, provision, assessment, support and remedy.
- Verify corrective action independently within a defined period and review the result.
- Define information required for oversight and retain evidence sufficient for independent review.
- Escalate material exceptions within a defined period and review the result.
- Preserve a traceable decision record within a defined period and review the result.
- Separate incompatible responsibilities within a defined period and review the result.
Implications for institutional governance and accountability
The technical issue within public performance reporting concerns the basis on which a conclusion is reached. In reviewing the analytical question, public information should be accurate, current, complete in relation to material matters and presented before a learner is required to make a consequential commitment. Qualifications and limitations should receive comparable prominence to the principal claim. The judgement should state its supporting evidence and any condition limiting application to the declared scope.
Failure in relation to the matter examined may arise even where the stated policy is reasonable. Material concerns include management assurance accepted without testing, material risks omitted from reporting, authority assigned without accountability, and conflicts not identified. An exception should be assessed by effect, duration, recurrence and reach, including possible exposure beyond the initial sample.
Assurance of the analytical question should draw on more than one form of evidence. Useful records include corrective-action verification, public reports reconciled with controlled records, independent review records, governing-body papers and decisions, and risk and assurance plans. Policy and records should be tested against actual practice, including evidence from learners where appropriate. A positive example may illustrate operation, but it cannot demonstrate coverage or consistency.
- Can it be reconciled with the controlled source?
- Are corrections prompt and traceable?
- Who approves changes?
- Is the information available before commitment?
- Does it identify material conditions and limitations?
Testing implementation and effect
Implementation of public performance reporting can be tested without imposing unnecessary reporting. In reviewing the analytical question, responsible bodies should identify material information across the learner journey, assign source ownership, reconcile public statements with controlled records and retain corrections. Test whether a reasonable user can understand status, cost, obligations, support and routes for redress. Existing records may be used if reliable and relevant, but data collected for another purpose may not answer the assurance question.
The analytical record for the matter examined should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations. Results should be reproducible from the retained data and method. Any causal explanation should be identified separately from descriptive findings and supported by an appropriate design.
The analysis of the comparison should remain within the limits of the evidence. The analysis of the analytical question proceeds on the basis that association should not be presented as causation, and statistical significance should not be treated as evidence of educational importance without further analysis. A decision concerning the analytical question should recognise that governance structures do not provide assurance merely because committees exist. Membership, information quality, challenge, decisions and follow-through determine whether oversight is effective. Decision-makers should not extend assurance beyond the point supported by the available evidence.
The assurance record for the comparison should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. Traceable source and version information allow genuine improvement to be distinguished from administrative revision. A superseded conclusion should be retained where it formed the basis of a material decision.
Public reporting on the matter examined should distinguish established fact, analytical judgement and planned action. A material change should not remove the earlier position from the evidential trail. Changes to definitions or evidence should be recorded separately from changes in educational performance.
Neither one indicator nor one control can establish the complete position on the analytical question. Assurance should be based on the combined legal or policy basis, operating evidence and learner effect, not on one element alone.