{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-e9cfc8f96ed0ff7b","slug":"2013-06-24-reporting-online-delivery-coverage-and-revision-risk","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2013-06-24","last_modified_date":"2013-06-24","title":"Reporting online delivery: coverage and revision risk","summary":"Assesses evidence on reporting online delivery: coverage and revision risk, with particular attention to definitions, population coverage, comparability and uncertainty.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Continued expansion of digital provision","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The present attention to online delivery follows the continued expansion of digital provision and requires a careful distinction between public commitment, institutional practice and demonstrated result. In reviewing the measure, evidence should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support. The control response should be sufficient to protect learners while avoiding burdens not justified by the evidence.","The stated reference is the continued expansion of digital provision. Application to the measure depends on evidence from the relevant jurisdiction or institution. In relation to reporting online delivery, later review should not obscure whether the earlier position rested on fact, policy or judgement.","Responsibility for the issue should be visible at the point where consequential decisions are made. The analysis of the issue proceeds on the basis that a sound interpretation should identify the unit of analysis, reference period, denominator, exclusions, missing values and any change in definition or collection practice. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review."]},{"heading":"Evidence and method","paragraphs":["The system and institutional dimensions of online delivery should be considered together. the analysis proceeds on the basis that a change in delivery mode should not weaken the defined learning outcomes, learner protection, accessibility or reliability of assessment. Authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably. The allocation of responsibility should prevent gaps between system oversight and institutional operation.","In practical terms, the comparison should be reviewed against a stated method rather than general assurance. The analysis of the issue proceeds on the basis that data quality comprises accuracy, completeness, timeliness, consistency and traceability. Strength in one dimension does not compensate automatically for weakness in another, particularly where the information informs a consequential learner decision. Decision-makers should receive an intelligible account of how the result was reached and where it should not be applied.","A narrow control over the comparison may create false assurance. In the present context, technology access determining educational access, reduced opportunities for timely support and supplier dependency without continuity controls may produce acceptable aggregate reporting while individual learners remain exposed to material disadvantage. The test should deliberately include exceptions and cases in which the expected outcome was not achieved.","Assurance of the available evidence should draw on more than one form of evidence. Useful records include service availability and incident records, teacher capability and workload information, assessment validity and integrity reviews, accessibility and usability testing, and supplier performance and exit arrangements. Evidence of effectiveness should represent the declared scope, including adverse and exceptional cases."]},{"heading":"Patterns requiring examination","paragraphs":["For operational review of online delivery, authorities and providers should proceed in a defined sequence. A competent review of the issue should trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. Escalate discrepancies that could alter a published conclusion or individual outcome. Findings should state the affected scope and required action; an observation should not be represented as evidence of conformity or effectiveness.","Publication of findings on the measure should distinguish observed values, estimates and interpretation.","In relation to reporting online delivery, the evidential trail should allow an affected decision to be identified, examined and corrected. For the analysis, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. Historical decisions should be assessed against the information then available, with later amendments separately dated and explained."]},{"heading":"Implications for decision-makers","paragraphs":["Interpretation of online delivery should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. A decision concerning the measure should recognise that digital participation data should not be treated as a direct measure of learning. Log-ins, connection time and activity counts require interpretation alongside assessment and learner experience. The analysis of the issue proceeds on the basis that missing or delayed information may be patterned rather than random.","Public reporting on the measure should distinguish established fact, analytical judgement and planned action.","Assurance concerning the analysis requires corroborating evidence across the material scope. A conclusion should be revised when stronger evidence materially changes the assessment of implementation, outcome or risk."]}],"word_count":717,"content_hash":"sha256-00b992ecc379fd7ee70c430828ac53cf45a588ba0654d6bd5c75849b7ebb24b4","seo_keywords":["reporting online delivery","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}