{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-1ecd9bbd7c9b0858","slug":"2014-06-02-risk-and-assurance-controls-for-online-assessment","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2014-06-02","last_modified_date":"2014-06-02","title":"Risk and assurance controls for online assessment","summary":"Sets out the scope, decision criteria and evidential basis relevant to risk and assurance controls for online assessment, including the treatment of material limitations.","category":{"code":"STANDARDS_INTERPRETATION","label":"Standards Interpretation"},"article_type":"Standards interpretation","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Digital delivery and assessment growth","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The policy and evidence context for online assessment has been materially shaped by the digital delivery and assessment growth. In reviewing the control, interpretation should begin with the intended outcome, then identify the controls and evidence needed to show that the outcome is achieved across the declared scope. The materiality of the issue depends on its consequences for learners, responsible institutions and educational resources. Different administrative structures may support the same public-interest outcome.","The historical reference basis is the digital delivery and assessment growth. Its relevance to the relevant requirement should be assessed against the affected jurisdiction, learner population and form of provision.","Responsibility for the control should be identifiable at each consequential decision point. Delegating operational work does not transfer accountability for its effect on learners. Oversight of the stated expectation should reflect the principle that a provider should be able to trace the expectation from approved policy through implementation, monitoring, identified exceptions and corrective action. In relation to risk and assurance controls for online assessment, escalation should follow whenever the available record cannot support a safe conclusion for the affected learners."]},{"heading":"Meaning in practice","paragraphs":["The system and institutional dimensions of online assessment should be considered together. Oversight of risk and assurance controls for online assessment should reflect the principle that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably. Neither public oversight nor provider control removes the responsibilities assigned to the other level.","In practical terms, the control should be reviewed against a stated method rather than general assurance. A decision concerning the control should recognise that the subject should be examined as a connected system of policy, people, resources, decisions and evidence. The assessment should follow authority and information across functional boundaries and verify completion of required action. Those required to act should be able to understand the method and its material limitations.","Failure in relation to the stated expectation may arise even where the stated policy is reasonable. Material concerns include uncontrolled changes to assessment, reasonable adjustment altering the assessed outcome, results used beyond the evidence they support, and tasks that do not assess the stated outcome. Materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records.","Assurance of the matter should draw on more than one form of evidence. Useful records include moderation and exception records, analysis of results and differential outcomes, approval and change-control records, assessment maps to learning outcomes, and marking criteria and calibrated judgement. Evidence of effectiveness should represent the declared scope, including adverse and exceptional cases."]},{"heading":"Responsibilities and material risks","paragraphs":["Implementation of online assessment can be tested without imposing unnecessary reporting. The method for the relevant requirement is to map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. A case-by-case response is insufficient when evidence shows recurrence or a population beyond the cases already identified. In relation to risk and assurance controls for online assessment, information should not be treated as sufficient merely because it is already available; its relevance to the present question must be established.","Interpretation of the stated expectation should produce a test that another competent reviewer can apply to comparable evidence.","The assurance record the control should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. Traceable source and version information allow genuine improvement to be distinguished from administrative revision. Earlier conclusions should remain traceable if they affected a learner, provider or public decision."]},{"heading":"Basis for a reliable conclusion","paragraphs":["Care is required in drawing conclusions about online assessment. The analysis of the control proceeds on the basis that reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. The analysis of the control proceeds on the basis that the volume of documentation is not a measure of conformity. Relevance, integrity and coverage are more important than the number of records produced. Limitations should be prominent wherever the finding may influence a consequential decision.","Accountability for the issue under review should follow decision-making authority.","The measure of progress on the assurance question is not the amount of policy or documentation produced. Progress is demonstrated when the intended educational result is achieved, adverse variation is identified and responsible bodies act where it is not."]}],"word_count":749,"content_hash":"sha256-802920ecb253350883ccc2c7b4d5f090b7fa1ce2ba552f5915395507ebfc9b08","seo_keywords":["risk and assurance controls for online assessment","education quality standards","education quality","ICEQC"],"schema_type":"TechArticle"}