{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-705335caf67b8f9e","slug":"2019-09-13-longitudinal-evidence-on-artificial-intelligence-in-education-policy","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2019-09-13","last_modified_date":"2019-09-13","title":"Longitudinal evidence on artificial intelligence in education policy","summary":"Examines longitudinal evidence on artificial intelligence in education policy, defining the evidence base, coverage and limits that should govern comparison and use of.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Beijing Consensus adopted in May 2019","reference_authority":"United Nations Educational, Scientific and Cultural Organization","sections":[{"heading":null,"paragraphs":["The Beijing Consensus adopted in May 2019 provides the immediate context for artificial intelligence in education policy. Oversight of the analysis should reflect the principle that evidence should inform action without implying a level of precision, coverage or causal certainty that the underlying data cannot support."]},{"heading":"Evidence base","paragraphs":["The Beijing Consensus on Artificial Intelligence and Education was adopted in May 2019. It addresses policy planning, management, teaching, learning, skills, lifelong learning, inclusion, gender equality, data and research. It promotes human-centred and equitable use rather than technology adoption as an end in itself. Authorities and providers should therefore connect each proposed use to an educational purpose, governance responsibility and evidence of benefit and risk.","The system and institutional dimensions of artificial intelligence in education policy should be considered together. For the analysis, technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review. In relation to longitudinal evidence on artificial intelligence in education policy, authorities and providers hold different responsibilities, both of which must be discharged for the arrangement to operate reliably. Each level should be able to demonstrate the decisions and controls for which it is accountable."],"bullets":["Classify uses by effect on learners.","Retain accountable human decision-makers.","Prohibit uses for which evidence or authority is insufficient.","Control personal and confidential information.","Review incidents and supplier changes."]},{"heading":"Coverage and comparability","paragraphs":["The instrument identified by the Beijing Consensus adopted in May 2019 provides a formal policy reference for artificial intelligence in education policy. This distinction protects learners from overstated claims and enables providers to plan against a defined obligation.","In practical terms, the measure should be reviewed against a stated method rather than general assurance. The analysis of the issue proceeds on the basis that trend analysis depends on stable definitions and repeated observation of comparable populations. A change in policy, coverage or recording practice can create an apparent movement that is not a change in the underlying educational condition."]},{"heading":"Responsible interpretation","paragraphs":["The governing expectation for artificial intelligence in education policy should be capable of consistent application. In reviewing the available evidence, reported averages should be accompanied by sufficient distributional information to identify material differences between learner groups, locations and forms of provision. Operational definitions should be precise enough to support consistent consequential decisions and explain justified variation.","The principal risks in relation to the available evidence are automation bias in consequential decisions, opaque use of personal or inferred data, unverified outputs entering teaching or assessment, and unequal performance across learner groups. Risk assessment should account for dependencies between controls and the possibility that one failure masks the next."],"bullets":["Were earlier values revised?","Is the period long enough to show sustained movement?","What external condition may explain the change?","Is the baseline still comparable?","Have coverage or definitions changed?"]},{"heading":"Limitations and reporting","paragraphs":["Relevant evidence for artificial intelligence in education policy will normally include data provenance and access controls, records of human review and overrides, documented authority for each consequential use, pre-deployment and periodic performance testing, and an inventory of systems and their intended uses. In relation to longitudinal evidence on artificial intelligence in education policy, conflicting records require reconciliation before a complete assurance conclusion is reached.","For operational review of the measure, authorities and providers should proceed in a defined sequence. Review of the issue should establish a baseline, annotate every material change in definition or collection, compare like periods and retain revised series. Where comparability is interrupted, begin a new series or present the break clearly rather than joining unlike observations. The record should distinguish a finding that requires action from an observation that supports no formal conclusion."]},{"heading":"Limitations and reporting","paragraphs":["Publication of findings on artificial intelligence in education policy should distinguish observed values, estimates and interpretation.","Conclusions concerning the comparison require careful treatment of scope and evidential limits. For the comparison, a technical capability is not evidence that a use is educationally justified. Accuracy measured in one setting may not transfer to another population, language, curriculum or decision context. A decision concerning the analysis should recognise that a single indicator rarely provides an adequate account of quality. Quantitative evidence should be considered with implementation records and the experience of affected learners. Material limitations should be stated with the finding presented to decision-makers and affected learners.","Records relating to the comparison should preserve both the conclusion and its limits. A changed evidential position should be applied to the affected scope, including prior decisions that may no longer be reliable. Where reliance has occurred, correction may require review of affected decisions as well as amendment of published information.","Accountability for the comparison should follow decision-making authority.","Governance of the available evidence requires a clear allocation of authority, information and follow-through. The responsible body should receive matters requiring resources, policy change or formal risk acceptance. Public confidence cannot be separated from an institution's ability to identify responsibility and substantiate its conclusions."]}],"word_count":804,"content_hash":"sha256-869308c135238fc048859dcff37caab3402a33d839e1ba0cab95b1262457d860","seo_keywords":["longitudinal evidence on artificial intelligence in education policy","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}