Examines public performance reporting, addressing cross-system measurement and the evidential limits relevant to responsible interpretation and decision-making.
In 2017, consideration of public performance reporting must take account of the 2017 accountability agenda and the responsibilities it places before education systems. For the available evidence, the principal analytical task is to separate an observed difference from a conclusion about its cause. Assessment should consider learner impact, institutional accountability and stewardship of entrusted resources. Uniform administrative form is not required where equivalent public outcomes can be demonstrated.
For public performance reporting, responsibility should be identifiable at the point where consequential decisions are made. For comparative analysis, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review.
Evidence and method
This analysis is informed by 2017 accountability agenda. Its relevance to public performance reporting should be assessed against the affected jurisdiction, learner population and form of provision.
The analysis proceeds on the basis that governing bodies should receive sufficient, reliable and timely information to oversee education quality, learner protection and material institutional risk. When examining public performance reporting, review should cover the stages at which learners receive information, provision, assessment, support and remedy.
- Verify corrective action independently.
- Define information required for oversight.
- Escalate material exceptions.
- Preserve a traceable decision record.
- Separate incompatible responsibilities.
Patterns requiring examination
Public information should be accurate, current, complete in relation to material matters and presented before a learner is required to make a consequential commitment. As regards public performance reporting, qualifications and limitations should receive comparable prominence to the principal claim. The judgement should state its supporting evidence and any condition limiting application to the declared scope.
Failure in relation to the issue may arise even where the stated policy is reasonable. Material concerns include management assurance accepted without testing, material risks omitted from reporting, authority assigned without accountability, and conflicts not identified. In work concerning public performance reporting, an exception should be assessed by effect, duration, recurrence and reach, including possible exposure beyond the initial sample.
Assurance of the analysis should draw on more than one form of evidence. Useful records include corrective-action verification, public reports reconciled with controlled records, independent review records, governing-body papers and decisions, and risk and assurance plans. As regards public performance reporting, policy and records should be tested against actual practice, including evidence from learners where appropriate. Within the scope under review, a positive example may illustrate operation, but it cannot demonstrate coverage or consistency.
- Can it be reconciled with the controlled source?
- Are corrections prompt and traceable?
- Who approves changes?
- Is the information available before commitment?
- Does it identify material conditions and limitations?
Implications for decision-makers
Implementation of public performance reporting can be tested without imposing unnecessary reporting. Responsible bodies should identify material information across the learner journey, assign source ownership, reconcile public statements with controlled records and retain corrections. Test whether a reasonable user can understand status, cost, obligations, support and routes for redress. Existing records may be used if reliable and relevant, but data collected for another purpose may not answer the assurance conclusion.
The analytical record for public performance reporting should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations.
In the context of public performance reporting, analysis should remain within the limits of the evidence. The analysis proceeds on the basis that association should not be presented as causation, and statistical significance should not be treated as evidence of educational importance without further analysis. Governance structures do not provide assurance merely because committees exist. Membership, information quality, challenge, decisions and follow-through determine whether oversight is effective. Decision-makers should not extend assurance beyond the point supported by the available evidence.
The assurance record for the comparison should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. As regards public performance reporting, traceable source and version information allow genuine improvement to be distinguished from administrative revision. Within the scope under review, a superseded conclusion should be retained where it formed the basis of a material decision.
Public reporting on public performance reporting should distinguish established fact, analytical judgement and planned action. Changes to definitions or evidence should be recorded separately from changes in educational performance.
Neither one indicator nor one control can establish the complete position on the analysis.