The use of proxy measures in analysing assessment moderation

Evidence concerning assessment moderation should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support.

Application to use of proxy measures in analysing assessment

The position at publication is informed by the cross-system comparability and academic standards; evidence from the affected setting remains necessary before reaching a conclusion on assessment moderation. The decision basis should identify what is evidenced, what reflects policy and what depends on authorised discretion.

For the available evidence, the public interest is not confined to institutional compliance. When examining assessment moderation, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.

• Moderate material variation.
• Retain evidence sufficient for review.
• Define the decision each assessment must support.
• Align tasks and criteria with learning outcomes.
• Review differential and anomalous results before it informs a consequential decision.

Controls for use of proxy measures in analysing assessment

Review of assessment moderation should follow a stated and reproducible method. A proxy is useful only where its relationship with the intended outcome is sufficiently understood. Participation, activity and expenditure may support learning, but none constitutes direct evidence of learning without an explicit and tested connection.

Relevant evidence for the comparison will normally include authorship and identity controls proportionate to risk, appeal and correction records, assessment maps to learning outcomes, analysis of results and differential outcomes, and marking criteria and calibrated judgement. For assessment moderation, evidence outside the relevant period or scope should be identified and given no more weight than its limitations permit.

Across the defined scope, in this case, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. For assessment moderation, management should assign each material action to an accountable owner and completion date.

For assessment moderation, decision-makers using evidence on the measure should be told what the data cannot establish as clearly as what it can. The finding should identify its analytical character and the system, institution, programme or learner population to which it applies.

Review of use of proxy measures in analysing assessment

Authorities and providers reviewing assessment moderation should proceed in a defined sequence. A competent review of the comparison should state the construct to be measured, explain why the proxy is expected to represent it, test that relationship against direct evidence and identify circumstances in which the proxy may fail. Do not allow convenience to determine the measure.

Risk assessment of the issue should give particular attention to tasks that do not assess the stated outcome, inconsistent judgement between markers or locations, and results used beyond the evidence they support. A provider should also consider weak assurance of authorship or performance and uncontrolled changes to assessment.

The assurance record for assessment moderation should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied.

Particular care is required when interpreting evidence about the issue. In reviewing assessment moderation, reliability without validity produces consistent but potentially irrelevant results. For the measure, association should not be presented as causation, and statistical significance should not be treated as evidence of educational importance without further analysis.

In examining the use of proxy measures in analysing assessment moderation, the current development provides a basis for examining whether the issue is supported by responsible action and demonstrable result.