{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-f76a4933f5981f77","slug":"2023-08-23-cross-system-comparability-in-technology-in-education","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2023-08-23","last_modified_date":"2023-08-23","title":"Cross-system comparability in technology in education","summary":"Reviews cross-system comparability in technology in education and identifies the conditions required for sound interpretation, responsible comparison and defensible conclusions.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"2023 Global Education Monitoring Report","reference_authority":"United Nations Educational, Scientific and Cultural Organization","sections":[{"heading":null,"paragraphs":["In 2023, consideration of technology in education must take account of the 2023 Global Education Monitoring Report and the responsibilities it places before education systems. A decision concerning the available evidence should recognise that the principal analytical task is to separate an observed difference from a conclusion about its cause."]},{"heading":"Evidence and method","paragraphs":["The reference basis—the 2023 Global Education Monitoring Report—is evidential rather than self-executing. Patterns in the material may justify enquiry, although they do not by themselves determine legal position or cause. In applying it to technology in education, users should review the source definitions, population coverage, reference period and stated limitations before transferring a system-level finding to an individual provider or learner group."]},{"heading":"Patterns requiring examination","paragraphs":["The quality significance of technology in education follows from a basic distinction between availability and effective provision. Oversight of the available evidence should reflect the principle that a change in delivery mode should not weaken the defined learning outcomes, learner protection, accessibility or reliability of assessment. In relation to cross-system comparability in technology in education, review should cover the stages at which learners receive information, provision, assessment, support and remedy.","In practical terms, the analysis should be reviewed against a stated method rather than general assurance. In reviewing the analysis, comparison requires more than the use of a common label. Definitions, reference periods, population coverage, institutional boundaries and collection practices must be sufficiently aligned for the observed difference to have a stable meaning. The method, assumptions and limitations should be stated in terms suitable for responsible decision-making."],"bullets":["Do the reference periods align?","Has a classification changed?","Are the populations defined on the same basis?","Is the remaining difference educationally material?","Are exclusions and missing records comparable?"]},{"heading":"Implications for decision-makers","paragraphs":["The governing expectation for technology in education should be capable of consistent application. A decision concerning the analysis should recognise that a sound interpretation should identify the unit of analysis, reference period, denominator, exclusions, missing values and any change in definition or collection practice. Operational definitions should be precise enough to support consistent consequential decisions and explain justified variation.","Failure in relation to the comparison may arise even where the stated policy is reasonable. Material concerns include assessment methods that do not support valid judgements, unclear identity and participation records, technology access determining educational access, and inaccessible content or interaction. Materiality depends on the consequence and extent of an exception, not only on how often it appears in sampled records."]},{"heading":"Limits of inference","paragraphs":["Assurance of technology in education should draw on more than one form of evidence. Useful records include service availability and incident records, delivery-mode design and approval records, assessment validity and integrity reviews, supplier performance and exit arrangements, and learner access and participation information. Documents should be reconciled with observed practice and, where relevant, the experience of affected learners. A positive example may illustrate operation, but it cannot demonstrate coverage or consistency.","Implementation of the measure can be tested without imposing unnecessary reporting. A competent review of the analysis should prepare a comparability table before analysing results. Record common elements, material differences, breaks in series and the direction in which each limitation may affect the conclusion; do not rank systems where those limitations remain material. Reuse of existing information is appropriate only where its purpose, scope and reliability correspond to the decision under review.","Decision-makers using evidence on the issue should be told what the data cannot establish as clearly as what it can. Reporting should state whether a result describes, compares or evaluates, together with the level at which it is valid. The basis for applying the result elsewhere should be established rather than assumed."]},{"heading":"Limits of inference","paragraphs":["Interpretation of technology in education should avoid two errors: treating a formal commitment as proof of effect, and treating one adverse case as proof that every part of the system has failed. For the analysis, digital participation data should not be treated as a direct measure of learning. Log-ins, connection time and activity counts require interpretation alongside assessment and learner experience. In reviewing the available evidence, international comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another.","Decisions concerning the comparison should remain traceable to the information available for the stated reference period. In relation to cross-system comparability in technology in education, a revision should state whether the change concerns the underlying condition, the evidence, the method or the interpretation.","Accountability for the analysis should follow decision-making authority. Where work is delegated, the record should continue to identify who is accountable for material consequences to learners.","No individual measure is sufficient to establish effective operation of the measure across the affected scope. A reasoned conclusion should reconcile the governing requirement, evidence of operation, learner outcomes and residual risk, and remain open to better evidence."]}],"word_count":790,"content_hash":"sha256-d2a7c47d8e100bdc5b2a1049b828c58d224b57d65a9af6542ed235b9b69822c9","seo_keywords":["cross-system comparability in technology in education","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}