{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-7420870fb5d69993","slug":"2012-09-18-timeliness-and-completeness-of-data-on-online-assessment-authenticity","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2012-09-18","last_modified_date":"2012-09-18","title":"Timeliness and completeness of data on online assessment authenticity","summary":"Examines timeliness and completeness of data on online assessment authenticity, defining the evidence base, coverage and limits that should govern comparison and use of.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Expansion of large-scale online learning","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The present attention to online assessment authenticity follows the expansion of large-scale online learning and requires a careful distinction between public commitment, institutional practice and demonstrated result. The analysis of the issue proceeds on the basis that evidence should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support. In relation to timeliness and completeness of data on online assessment authenticity, the appropriate administrative form will depend on the jurisdiction and the allocation of lawful responsibility.","The position at publication is informed by the expansion of large-scale online learning; evidence from the affected setting remains necessary before reaching a conclusion on the measure. Authorities and providers should distinguish established fact, policy expectation and matters left to institutional judgement. Later review should not obscure whether the earlier position rested on fact, policy or judgement.","The system and institutional dimensions of the comparison should be considered together. For the measure, assessment should provide valid and sufficiently consistent evidence that the stated learning outcomes have been achieved by the learner receiving the result."]},{"heading":"Evidence base","paragraphs":["The analysis of online assessment authenticity should make its decision rule explicit. A decision concerning the measure should recognise that data quality comprises accuracy, completeness, timeliness, consistency and traceability. Strength in one dimension does not compensate automatically for weakness in another, particularly where the information informs a consequential learner decision. This supports consistent review and reduces the risk of redefining the basis of judgement after an adverse result appears.","A proper review of the measure should establish the intended outcome before selecting controls or indicators. In reviewing the analysis, trend claims require comparable observations over time and a documented account of revisions, breaks in series and changes in coverage. A chosen approach should be justified against its context, with departures and review points under documented control."]},{"heading":"Coverage and comparability","paragraphs":["The principal risks in relation to online assessment authenticity are tasks that do not assess the stated outcome, inconsistent judgement between markers or locations, results used beyond the evidence they support, and uncontrolled changes to assessment. The control environment should be assessed as a connected system rather than as unrelated individual risks.","Relevant evidence for the comparison will normally include analysis of results and differential outcomes, assessment maps to learning outcomes, approval and change-control records, moderation and exception records, and authorship and identity controls proportionate to risk. In relation to timeliness and completeness of data on online assessment authenticity, the conclusion should rely on evidence whose date, source and coverage are sufficient for the decision. Conflicting records require reconciliation before a complete assurance conclusion is reached.","A proportionate method is available for the measure. The method for the available evidence is to trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. Escalate discrepancies that could alter a published conclusion or individual outcome. The review record should preserve exceptions capable of showing a weakness in design, implementation or coverage."]},{"heading":"Responsible interpretation","paragraphs":["Publication of findings on online assessment authenticity should distinguish observed values, estimates and interpretation.","Proportionality in relation to the analysis does not mean reduced protection for learners exposed to greater risk. Oversight of the available evidence should reflect the principle that reliability without validity produces consistent but potentially irrelevant results. Validity without adequate consistency may expose learners to unequal judgement. A decision concerning the comparison should recognise that missing or delayed information may be patterned rather than random. No exception should continue without a documented basis, accountable approval and scheduled review.","A traceable record enables responsibility to be established and errors to be corrected fairly. For the analysis, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Historical decisions should be assessed against the information then available, with later amendments separately dated and explained.","In this case, governing bodies should receive a concise account of the intended result, affected scope, principal risks, evidence limitations and unresolved exceptions. Evidence of outcome, rather than completion of tasks, should determine whether corrective work can close.","Any response to the present development should test the evidential connection between the issue, its implementation and the outcome claimed. Institutional improvement and public confidence both depend on transparent responsibility and credible evidence."]}],"word_count":718,"content_hash":"sha256-be9f54ac49a6ac31bce01b16279e836b0e202757bd108d44c2c5cf215edcfac0","seo_keywords":["timeliness and completeness of data on online assessment authenticity","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}