{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-8efa10ff8bef36fc","slug":"2021-10-11-remote-assessment-evidence-coverage-and-limitations","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2021-10-11","last_modified_date":"2021-10-11","title":"Remote assessment: evidence, coverage and limitations","summary":"Reviews remote assessment: evidence, coverage and limitations and identifies the conditions required for sound interpretation, responsible comparison and defensible conclusions.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Continued online and hybrid assessment","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["Current consideration of remote assessment is informed by the continued online and hybrid assessment, with consequences for governance, evidence and the treatment of affected learners. The analysis of the comparison proceeds on the basis that the available evidence should be interpreted with close attention to definitions, population coverage, collection methods and the limits of comparison.","The reference point is the continued online and hybrid assessment. Evidence concerning the comparison should be current, attributable and representative of the affected scope. Material gaps or contradictions should remain visible in the conclusion. Authorities and providers should distinguish established fact, policy expectation and matters left to institutional judgement.","The required public outcome should be stated in operational terms. A decision concerning the comparison should recognise that assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result. In relation to remote assessment, assurance should not stop at adoption, resourcing or completion of administrative tasks."]},{"heading":"Evidence base for remote assessment","paragraphs":["In practical terms, remote assessment should be reviewed against a stated method rather than general assurance. The analysis of the available evidence proceeds on the basis that data quality comprises accuracy, completeness, timeliness, consistency and traceability. Strength in one dimension does not compensate automatically for weakness in another, particularly where the information informs a consequential learner decision. Those required to act should be able to understand the method and its material limitations.","Implementation of the available evidence should be organised around a decision that can be tested. For the comparison, where an indicator is used as a proxy, the relationship between the proxy and the underlying educational outcome should be stated and tested. In practice, the stated objective should connect to responsibility, committed resources, operating evidence and the outcome reported for oversight."]},{"heading":"Coverage and comparability","paragraphs":["A narrow control over remote assessment may create false assurance. In the present context, weak assurance of authorship or performance, tasks that do not assess the stated outcome and inconsistent judgement between markers or locations may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. A sample confined to compliant cases cannot establish the reliability of the control.","Relevant evidence for the issue will normally include approval and change-control records, authorship and identity controls proportionate to risk, appeal and correction records, marking criteria and calibrated judgement, and moderation and exception records. In relation to remote assessment, the conclusion should rely on evidence whose date, source and coverage are sufficient for the decision. An unresolved contradiction is a limitation on the conclusion and should be reported as such.","Implementation of the issue can be tested without imposing unnecessary reporting. Review of the available evidence should trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. Escalate discrepancies that could alter a published conclusion or individual outcome. Existing records may be used if reliable and relevant, but data collected for another purpose may not answer the assurance question."]},{"heading":"Responsible interpretation","paragraphs":["Publication of findings on remote assessment should distinguish observed values, estimates and interpretation.","The analysis of the available evidence should remain within the limits of the evidence. The analysis of the issue proceeds on the basis that international comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another. Oversight of the available evidence should reflect the principle that reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. Decision-makers should not extend assurance beyond the point supported by the available evidence.","Decisions concerning the available evidence should remain traceable to the information available for the stated reference period. Changes in condition, evidence, method and interpretation should be recorded separately when a conclusion is revised. Without this distinction, a reporting change may be mistaken for improvement or deterioration in educational practice.","Accountability for the analysis should follow decision-making authority.","Authorities and providers should use the current development to test whether the measure connects public commitment with effective operation and evidence of result."]}],"word_count":675,"content_hash":"sha256-c56f0f7602d26d26915431cb11c4812096925cffcc3cf4be45d214108964448f","seo_keywords":["remote assessment","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}