Explains risk and assurance controls in relation to online assessment, covering scope, evidence, decision authority, material exceptions and continuing assurance.
The policy and evidence context for online assessment has been materially shaped by the digital delivery and assessment growth. Interpretation should begin with the intended outcome, then identify the controls and evidence needed to show that the outcome is achieved across the declared scope. The materiality of the issue depends on its consequences for learners, responsible institutions and educational resources. Different administrative structures may support the same public-interest outcome.
This analysis is informed by digital delivery and assessment growth. Its relevance to the applicable requirement should be assessed against the affected jurisdiction, learner population and form of provision.
Responsibility for the control should be identifiable at each consequential decision point. In reviewing online assessment, delegating operational work does not transfer accountability for its effect on learners. A provider should be able to trace the expectation from approved policy through implementation, monitoring, identified exceptions and corrective action. Escalation should follow whenever the available record cannot support a safe conclusion for the affected learners.
Meaning in practice
The system and institutional dimensions of online assessment should be considered together. Assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably. Neither public oversight nor provider control removes the responsibilities assigned to the other level.
Review of the control should be based on a stated method rather than general assurance. In work concerning online assessment, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. The assessment should follow authority and information across functional boundaries and verify completion of required action. Those required to act should be able to understand the method and its material limitations.
Failure in relation to the applicable expectation may arise even where the stated policy is reasonable. Material concerns include uncontrolled changes to assessment, reasonable adjustment altering the assessed outcome, results used beyond the evidence they support, and tasks that do not assess the stated outcome. Within the scope under review, materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.
As regards online assessment, assurance of the matter should draw on more than one form of evidence. Useful records include moderation and exception records, analysis of results and differential outcomes, approval and change-control records, assessment maps to learning outcomes, and marking criteria and calibrated judgement. Evidence of effectiveness should represent the declared scope, including adverse and exceptional cases.
Responsibilities and material risks
Implementation of online assessment can be tested without imposing unnecessary reporting. The method for the applicable requirement is to map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. A case-by-case response is insufficient when evidence shows recurrence or a population beyond the cases already identified. Information should not be treated as sufficient merely because it is already available; its relevance to the present question must be established.
Interpretation of online assessment should produce a test that another competent reviewer can apply to comparable evidence.
The assurance record the control should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. When examining online assessment, traceable source and version information allow genuine improvement to be distinguished from administrative revision. Earlier conclusions should remain traceable if they affected a learner, provider or public decision.
Basis for a reliable conclusion
Care is required in drawing conclusions about online assessment. Reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. The volume of documentation is not a measure of conformity. Within the scope under review, relevance, integrity and coverage are more important than the number of records produced. Limitations should be prominent wherever the finding may influence a consequential decision.
Accountability for online assessment should follow decision-making authority.
In the context of online assessment, progress should not be assessed by the amount of policy or documentation produced. Progress is demonstrated when the intended educational result is achieved, adverse variation is identified and responsible bodies act where it is not.