Examines online assessment through policy and regulatory implications, clarifying legal effect, institutional responsibility, learner safeguards and public-interest risk.
The digital delivery and assessment growth provides the immediate reference point for consideration of online assessment in 2014. The relevant policy question is how the stated public objective is translated into responsibilities that can be applied, supervised and reviewed. The proportionality test should consider both the identified risk and the consequences of the control for affected learners.
The reference point is the digital delivery and assessment growth. Data used for implementation should be interpreted against stable definitions and an identifiable population. For online assessment, changes in method, definition or series should remain separate from changes in the underlying result. Authorities and providers should distinguish established fact, policy expectation and matters left to institutional judgement. That distinction should remain visible in the decision record, public reporting and later review.
Status and scope
The system and institutional dimensions of online assessment should be considered together. For implementation, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably. Neither public oversight nor provider control removes the responsibilities assigned to the other level.
The review should be based on a stated method rather than general assurance. As regards online assessment, cross-jurisdiction interpretation should distinguish international commitment, regional instrument, national law, regulatory direction and provider policy. Each has a different source of authority and may apply to a different object or person. The method, assumptions and limitations should be stated in terms suitable for responsible decision-making.
Failure in relation to the issue may arise even where the stated policy is reasonable. Material concerns include results used beyond the evidence they support, inconsistent judgement between markers or locations, weak assurance of authorship or performance, and uncontrolled changes to assessment. Within the scope under review, an exception should be assessed by effect, duration, recurrence and reach, including possible exposure beyond the initial sample.
Public-interest implications
Relevant evidence for online assessment will normally include authorship and identity controls proportionate to risk, assessment maps to learning outcomes, approval and change-control records, moderation and exception records, and appeal and correction records. Contradictory evidence should be investigated and resolved, not omitted from the record.
For online assessment, accountability and effective correction both depend on a record that can be followed from evidence to decision. In this case, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Material changes require a traceable effective date and explanation so that prior reliance can be reviewed fairly.
- Control changes before it informs a consequential decision.
- Calibrate assessors.
- Align tasks and criteria with learning outcomes.
- Moderate material variation.
- Review differential and anomalous results.
Institutional responsibilities
Implementation of online assessment should be organised around a decision that can be tested. A credible response should identify the applicable jurisdiction, the affected learners and providers, the authority responsible for implementation, and the evidence by which performance will be judged. Resources and activity should be reconciled with the operating evidence and result for which the responsible function is accountable.
Authorities and providers reviewing the arrangements should proceed in a defined sequence. The method for implementation is to prepare a jurisdictional register identifying the service, learner location, provider location, responsible authority, applicable instrument and conflict rule. In the context of online assessment, obtain competent interpretation where the legal position is uncertain and do not resolve uncertainty through promotional wording.
The implementation record for the arrangements should identify the instrument being applied, its status, the competent authority, the affected jurisdiction and the action expected of each responsible body. The record should differentiate legal duties, public policy commitments and institutional action. As regards online assessment, a staged implementation record should set out transition dates, interim safeguards and the readiness review point.
Public reporting on online assessment should distinguish established fact, analytical judgement and planned action. If definitions, coverage or evidence alter an earlier conclusion, the reason should be stated so that revision is not mistaken for changed performance.
Continuing review
The analysis of online assessment should remain within the limits of the evidence. For the measure, a policy direction should not be presented as a uniform legal obligation where national implementation differs. Within the scope under review, providers remain responsible for identifying the requirements that apply to their own activities. Reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Decision-makers should not extend assurance beyond the point supported by the available evidence.
Governance of the measure requires a clear allocation of authority, information and follow-through. In reviewing online assessment, the responsible body should receive matters requiring resources, policy change or formal risk acceptance.