Participation and expenditure data: identifying the learners concealed by national averages

The policy and evidence context for participation and expenditure data has been materially shaped by Education at a Glance 2011. For the analysis, evidence should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support. The relevant concern is the effect of consequential decisions on learners, institutions and resources entrusted for education. Uniform administrative form is not required where equivalent public outcomes can be demonstrated.

Implementation of the available evidence should be organised around a decision that can be tested. The analysis proceeds on the basis that reported averages should be accompanied by sufficient distributional information to identify material differences between learner groups, locations and forms of provision. For participation and expenditure data, resources and activity should be reconciled with the operating evidence and result for which the responsible function is accountable.

Evidence base for participation and expenditure data

Education at a Glance 2011 provides the reference point for this analysis. Its relevance to participation and expenditure data should be assessed against the affected jurisdiction, learner population and form of provision. The international development warrants attention, but a consequential conclusion still requires current, attributable and representative evidence for the affected scope.

For decisions concerning participation and expenditure data, public and institutional resources should be directed to defined educational needs, with decisions transparent enough to identify unequal effects and protect essential provision.

• Report limitations in expenditure comparisons.
• Review whether savings transfer costs to learners.
• Record material judgements and conflicts, identifying the accountable function and affected scope.
• Assess distributional effects before reallocating funds.
• Protect essential learning and safeguarding functions before it is relied on for a decision with material effect.

Coverage and comparability

In work concerning participation and expenditure data, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. A control framework may fail at its interfaces even where each component appears satisfactory in isolation. An imprecise scope or measure may produce a credible-looking record that does not answer the relevant decision question.

Failure in relation to the measure may arise even where the stated policy is reasonable. Material concerns include funding disconnected from learner need, reporting expenditure without evidence of effect, unclear cross-subsidy between activities, and delayed detection of financial stress. Within the scope under review, the assessment of an exception should address severity, persistence and the likelihood that the condition is more widely present.

Assurance of the issue should draw on more than one form of evidence. Useful records include service and outcome measures, documented decisions on material reallocations, controls over restricted or public funds, unit-cost and workload information, and distributional analysis across learner groups and locations. When examining participation and expenditure data, documents should be reconciled with observed practice and, where relevant, the experience of affected learners.

• Which evidence establishes operation?
• Where do exceptions occur?
• What action is required by the finding?
• Who controls each stage?
• What outcome is intended?

Responsible interpretation

Implementation of participation and expenditure data can be tested without imposing unnecessary reporting. The review should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Recurrence, common cause or wider exposure requires systemic action in addition to correction of individual cases. Reuse of existing information is appropriate only where its purpose, scope and reliability correspond to the decision under review.

The analytical record for participation and expenditure data should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations.

Proportionality in relation to the issue does not mean reduced protection for learners exposed to greater risk. In the context of participation and expenditure data, higher expenditure is not, by itself, evidence of higher quality, and lower unit cost is not evidence of efficiency where access, learning or completion has deteriorated. International comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another. Each exception should record its basis, authorisation, duration and review date.

Records relating to the measure should preserve both the conclusion and its limits. In work concerning participation and expenditure data, a changed evidential position should be applied to the affected scope, including prior decisions that may no longer be reliable.

Public reporting on the issue should distinguish established fact, analytical judgement and planned action. As regards participation and expenditure data, revision history should remain available where users have relied on the earlier conclusion. Within the scope under review, changes to definitions or evidence should be recorded separately from changes in educational performance.

The decision record for participation and expenditure data should state the unsupported element and the further work required.