{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-fac7a27d3b182f3e","slug":"2020-09-14-measuring-remote-assessment-integrity-across-education-systems","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2020-09-14","last_modified_date":"2020-09-14","title":"Measuring remote assessment integrity across education systems","summary":"Reviews measuring remote assessment integrity across education systems and identifies the conditions required for sound interpretation, responsible comparison and defensible.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Emergency remote assessment","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The policy and evidence context for remote assessment integrity has been materially shaped by the emergency remote assessment. For the comparison, the principal analytical task is to separate an observed difference from a conclusion about its cause. A proportionate arrangement protects educational outcomes and fair treatment without creating avoidable barriers.","Responsibility for the comparison should be visible at the point where consequential decisions are made. In reviewing the comparison, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. In relation to measuring remote assessment integrity across education systems, incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review."]},{"heading":"Analytical scope","paragraphs":["The circumstances described by the emergency remote assessment are developing and may differ materially between locations. Decisions on remote assessment integrity should therefore be based on verified information available for the affected community and should be reviewed as conditions change. Temporary measures require recorded authority, learner communication and an end or review point; urgency does not remove the need to preserve safety, fair treatment and reliable records.","The system and institutional dimensions of the issue should be considered together. Oversight of the analysis should reflect the principle that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. Public authorities establish the legal and policy setting; providers remain accountable for the quality and integrity of provision within their control."],"bullets":["Control changes.","Calibrate assessors.","Define the decision each assessment must support.","Retain evidence sufficient for review.","Moderate material variation."]},{"heading":"Definitions and data coverage","paragraphs":["In practical terms, remote assessment integrity should be reviewed against a stated method rather than general assurance. For the comparison, the subject should be examined as a connected system of policy, people, resources, decisions and evidence. Assurance should not overlook failures arising at the boundary between otherwise adequate controls.","Risk assessment of the issue should give particular attention to inconsistent judgement between markers or locations, uncontrolled changes to assessment, and tasks that do not assess the stated outcome. A provider should also consider results used beyond the evidence they support and reasonable adjustment altering the assessed outcome. Stronger controls are required where learners may not detect an error or where later correction cannot restore the lost opportunity.","Assurance of the analysis should draw on more than one form of evidence. Useful records include assessment maps to learning outcomes, appeal and correction records, approval and change-control records, authorship and identity controls proportionate to risk, and moderation and exception records. In relation to measuring remote assessment integrity across education systems, policy and records should be tested against actual practice, including evidence from learners where appropriate. A positive example may illustrate operation, but it cannot demonstrate coverage or consistency."],"bullets":["Where do exceptions occur?","Which evidence establishes operation?","What action is required by the finding?","Who controls each stage?","What outcome is intended?"]},{"heading":"Use of the findings","paragraphs":["Implementation of remote assessment integrity can be tested without imposing unnecessary reporting. In reviewing the analysis, responsible bodies should map the complete process, identify the intended result and responsible authority at each stage, and test normal cases together with exceptions. Reuse of existing information is appropriate only where its purpose, scope and reliability correspond to the decision under review.","The analytical record for the analysis should state the research question, data source, unit of analysis, reference period, coverage, exclusions, treatment of missing values and principal limitations.","The analysis of the available evidence should remain within the limits of the evidence. A decision concerning the issue should recognise that missing or delayed information may be patterned rather than random. For the available evidence, reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Material uncertainty should result in further enquiry or an expressly limited finding.","The assurance record for the available evidence should retain the date of the evidence, the source responsible for it, the scope examined and the version of any instrument or definition applied. Earlier conclusions should remain traceable if they affected a learner, provider or public decision.","Where the measure involves partners, suppliers or several public bodies, responsibility should be mapped across the complete service. Contractual or inter-agency arrangements should identify who holds records, informs learners and acts on incidents. Learner safeguards should remain continuous where provision is delivered by several bodies.","Assessment of the available evidence should reconcile more than one source of evidence and control. The final judgement should connect the applicable expectation to implementation and outcomes while identifying unresolved risk."]}],"word_count":762,"content_hash":"sha256-93028ba0e4f715613736c196d62312aafe7206036bef1a0eb4f4a40bf4200edd","seo_keywords":["measuring remote assessment integrity across education systems","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}