Education at a Glance 2016: using attainment and expenditure indicators with due regard to context

The Education at a Glance 2016 provides the immediate context for the comparison. A decision concerning the evidence under review should recognise that comparable indicators can support public decision-making, but they do not remove the need to examine variation within systems and institutions. The relevant concern is the effect of consequential decisions on learners, institutions and resources entrusted for education. The appropriate administrative form will depend on the jurisdiction and the allocation of lawful responsibility.

Public-interest context

For Education at a Glance 2016, the public interest is not confined to institutional compliance. Oversight of the comparison should reflect the principle that public and institutional resources should be directed to defined educational needs, with decisions transparent enough to identify unequal effects and protect essential provision. Material arrangements should be communicated clearly, with an accessible route to correct error or unfair treatment.

• Review whether savings transfer costs to learners within a defined period and review the result.
• Link expenditure to an intended result, and retain the basis, responsible function and affected scope.
• Assess distributional effects before reallocating funds before it informs a consequential decision.
• Report limitations in expenditure comparisons within a defined period and review the result.
• Protect essential learning and safeguarding functions within a defined period and review the result.

Implications for education finance and resource stewardship

The position at publication is informed by Education at a Glance 2016; evidence from the affected setting remains necessary before reaching a conclusion on the analytical question. Verified fact, policy expectation and discretionary institutional choice should remain distinct in the record. That distinction should remain visible in the decision record, public reporting and later review.

The analysis of the analytical question should make its decision rule explicit. For The matter examined, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Transfer points should be tested because responsibility and information may be lost between otherwise sound functions. Comparable evidence should be assessed against criteria settled before the result is known.

Basis for a reliable conclusion

Responsibility for Education at a Glance 2016 should be visible at the point where consequential decisions are made. Oversight of the matter examined should reflect the principle that reported averages should be accompanied by sufficient distributional information to identify material differences between learner groups, locations and forms of provision. A decision should not be closed at the operating level where material impact, conflict or a significant evidential gap remains unresolved.

Failure in relation to the matter examined may arise even where the stated policy is reasonable. Material concerns include unclear cross-subsidy between activities, reporting expenditure without evidence of effect, across-the-board reductions with unequal consequences, and short-term savings that weaken completion or safety. Materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.

• Who controls each stage?
• Where do exceptions occur?
• What action is required by the finding?
• What outcome is intended?
• Which evidence establishes operation?

Proportionality and exceptions

Relevant evidence for Education at a Glance 2016 will normally include documented decisions on material reallocations, controls over restricted or public funds, service and outcome measures, distributional analysis across learner groups and locations, and unit-cost and workload information. Currency, provenance and representativeness should be established before evidence is used for assurance. An unresolved contradiction is a limitation on the conclusion and should be reported as such.

A proportionate method is available for the evidence under review. A competent review of the comparison should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Recurrence, common cause or wider exposure requires systemic action in addition to correction of individual cases. Contrary evidence should not be removed merely because aggregate performance appears acceptable.

Maintaining effective oversight

The analytical record for Education at a Glance 2016 should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations. Results should be reproducible from the retained data and method. Any causal explanation should be identified separately from descriptive findings and supported by an appropriate design.

Proportionality in relation to the matter examined does not mean reduced protection for learners exposed to greater risk. Oversight of the reported measure should reflect the principle that higher expenditure is not, by itself, evidence of higher quality, and lower unit cost is not evidence of efficiency where access, learning or completion has deteriorated. Oversight of the reported measure should reflect the principle that a single indicator rarely provides an adequate account of quality. Quantitative evidence should be considered with implementation records and the experience of affected learners. Each exception should record its basis, authorisation, duration and review date.

The assurance record for the reported measure should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. The retained record should show whether later movement reflects changed conditions or a change in the information reported. The evidential history should preserve conclusions that were operative when a material decision was made.

Public reporting on the analytical question should distinguish established fact, analytical judgement and planned action. Revision history should remain available where users have relied on the earlier conclusion. Changes to definitions or evidence should be recorded separately from changes in educational performance.

The measure of progress on the reported measure is not the amount of policy or documentation produced. Performance should be judged by outcomes and timely response to shortfalls, not by the volume of administrative activity.