{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-2bff5b4a2d206b3b","slug":"2014-06-19-public-interest-oversight-of-online-assessment","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2014-06-19","last_modified_date":"2014-06-19","title":"Public-interest oversight of online assessment","summary":"Explains the regulatory context for public-interest oversight of online assessment and the responsibilities, evidence and safeguards relevant to implementation.","category":{"code":"POLICY_AND_REGULATORY_INTERPRETATION","label":"Industry Policy and Regional Regulatory Interpretation"},"article_type":"Policy and regulatory analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Digital delivery and assessment growth","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The present attention to online assessment follows the digital delivery and assessment growth and requires a careful distinction between public commitment, institutional practice and demonstrated result. Oversight of the relevant measure should reflect the principle that the immediate task for education authorities is to distinguish the policy objective from the legal and operational measures needed to give it effect. The effect on learner access and reliable decision-making should inform the scale of control applied."]},{"heading":"Regulatory context","paragraphs":["The stated reference is the digital delivery and assessment growth. Application to online assessment depends on evidence from the relevant jurisdiction or institution. Authorities and providers should distinguish established fact, policy expectation and matters left to institutional judgement.","The governing expectation for the issue should be capable of consistent application. The analysis of the affected arrangements proceeds on the basis that implementation should be assessed against observable effects on access, learning, safety and fair treatment, rather than against the existence of a policy statement alone. In relation to public-interest oversight of online assessment, criteria affecting learners should not permit materially different interpretation without an evidenced reason.","The technical issue within the affected arrangements concerns the basis on which a conclusion is reached. In reviewing the relevant measure, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Transfer points should be tested because responsibility and information may be lost between otherwise sound functions. Any condition preventing complete assurance should appear with the evidence on which the judgement relies.","Readily available material should not define the enquiry if it cannot answer the relevant decision question. In this case, the most relevant material is likely to include moderation and exception records, assessment maps to learning outcomes, authorship and identity controls proportionate to risk, and approval and change-control records. In relation to public-interest oversight of online assessment, independent records should be reconciled, with disagreement and uncertainty reported alongside the finding."]},{"heading":"Operational effect","paragraphs":["The quality significance of online assessment follows from a basic distinction between availability and effective provision. In reviewing the relevant measure, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result.","Failure in relation to the relevant measure may arise even where the stated policy is reasonable. Material concerns include reasonable adjustment altering the assessed outcome, results used beyond the evidence they support, uncontrolled changes to assessment, and weak assurance of authorship or performance. An exception should be assessed by effect, duration, recurrence and reach, including possible exposure beyond the initial sample."],"bullets":["Define the decision each assessment must support.","Control changes, with responsibility, scope and timing recorded.","Align tasks and criteria with learning outcomes, with responsibility, scope and timing recorded.","Calibrate assessors.","Review differential and anomalous results."]},{"heading":"Required governance attention","paragraphs":["A proportionate method is available for online assessment. A competent review of the relevant measure should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. The finding should state whether the condition is isolated, recurring or potentially systemic. The review record should preserve exceptions capable of showing a weakness in design, implementation or coverage.","Oversight of the affected arrangements should be based on an implementation map linking the public objective to domestic measures, provider controls and learner remedies.","Any conclusion on the policy matter should remain within the scope supported by the evidence. A decision concerning the issue should recognise that reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Oversight of the policy matter should reflect the principle that the existence of an international commitment does not remove the need for jurisdiction-specific interpretation, consultation and proportionate transition arrangements. Material limitations should be stated with the finding presented to decision-makers and affected learners.","The assurance record for the affected arrangements should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. This enables later review to separate substantive change from correction, reclassification or expanded coverage. The evidential history should preserve conclusions that were operative when a material decision was made."]},{"heading":"Evidence and accountability","paragraphs":["Accountability for online assessment should follow decision-making authority. Relevant evidence should reach the body authorised to commit resources, amend policy or accept residual risk, and its judgement should be recorded. Operational tasks may be delegated, but accountability for material effects on learners must remain identifiable.","The measure of progress on the policy matter is not the amount of policy or documentation produced. Progress is demonstrated when the intended educational result is achieved, adverse variation is identified and responsible bodies act where it is not."]}],"word_count":775,"content_hash":"sha256-b2da92fc3473ac91734cebb8f28569a1bfd3bbe0ba55a6630e53f0e40f4703ef","seo_keywords":["public-interest oversight of online assessment","education policy and regulation","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}