Consistency in the assessment of student progression data

In examining consistency in the assessment of student progression data, for the applicable requirement, the requirement should be read as an assurance obligation: the provider must be able to explain the control, show its operation and account for material exceptions.

In reviewing student progression data, interpretation should preserve the unit and population represented in the data collection. A national or international pattern may justify closer review of the applicable expectation, but provider-level action requires evidence relating to the affected provision. The comparability record should identify material variation in coverage, period and classification.

Scope and application of consistency in the assessment of student progression data

For student progression data, the intended substantive result should remain the starting point for review. Assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.

For decisions concerning student progression data, consistency does not require identical decisions regardless of context. It requires comparable matters to be treated on the same principles, with material differences explained by relevant evidence and recorded criteria.

A failure concerning consistency in the assessment of student progression data may arise even where the stated policy is reasonable. Material concerns include uncontrolled changes to assessment, weak assurance of authorship or performance, reasonable adjustment altering the assessed outcome, and results used beyond the evidence they support. For student progression data, materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.

Controls relevant to consistency in the assessment of student progression data

Useful records include appeal and correction records, moderation and exception records, assessment maps to learning outcomes, analysis of results and differential outcomes, and authorship and identity controls proportionate to risk.

Records relating to the control should preserve both the conclusion and its limits. Across the defined scope, new evidence should trigger a traceable correction and review of decisions materially affected by the earlier conclusion. For student progression data, the correction process should identify prior users and decisions where published information has had material effect.

• Review differential and anomalous results.
• Align tasks and criteria with learning outcomes before using it to determine a learner or provider outcome.
• Moderate material variation before using it to determine a learner or provider outcome.
• Control changes.
• Retain evidence sufficient for review before using it to determine a learner or provider outcome.

Review criteria for consistency in the assessment of student progression data

The review method for the conclusion should be reproducible. A competent review of the conclusion should use common definitions and decision criteria, calibrate responsible staff, review outliers and compare outcomes across locations and groups. For decisions concerning student progression data, where variation is justified, retain the reason and verify that it is applied without arbitrary disadvantage.

Assurance concerning student progression data should be expressed at the level established by the evidence.

Accountability for student progression data should follow decision-making authority.

Implications for consistency in the assessment of student progression data

For the applicable requirement, reliability without validity produces consistent but potentially irrelevant results. Across the defined scope, relevance, integrity and coverage are more important than the number of records produced.

Performance in relation to student progression data should be judged by outcomes and timely response to shortfalls, not by the volume of administrative activity.