{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-73a58faaf5d499f6","slug":"2024-08-09-implementation-of-risk-based-artificial-intelligence-regulation-evidence-coverage-and-limitations","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2024-08-09","last_modified_date":"2024-08-09","title":"Implementation of risk-based artificial intelligence regulation: evidence, coverage and limitations","summary":"Reviews implementation of risk-based artificial intelligence regulation: evidence, coverage and limitations and identifies the conditions required for sound interpretation.","category":{"code":"DATA_RESEARCH","label":"Data Research"},"article_type":"Data and research analysis","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"Artificial Intelligence Act entered into force in August 2024","reference_authority":"European Union institutions and relevant national authorities","sections":[{"heading":null,"paragraphs":["The artificial Intelligence Act entered into force in August 2024 provides the immediate reference point for consideration of implementation of risk-based artificial intelligence regulation in 2024. For the available evidence, the value of the present data lies in the questions it can answer reliably and in the limits it makes visible. The decision should address both public impact and the responsibilities attached to entrusted educational resources. Suitability should be judged within the relevant system rather than against a presumed universal administrative model.","The applicability described by the artificial Intelligence Act entered into force in August 2024 changes the implementation context for the measure. Entry into force or applicability establishes an operative reference point, but the resulting duties must still be traced to the persons, services and jurisdictions covered. Authorities should distinguish immediate duties from staged provisions, and providers should retain the legal and operational basis for any conclusion about application.","The European Union Artificial Intelligence Act entered into force on 1 August 2024. It applies a risk-based framework and includes provisions relevant to certain education and vocational-training uses, particularly systems capable of influencing access, evaluation or progression. Requirements apply according to the Act’s staged timetable. Providers should classify intended uses, identify their role in the supply chain and preserve human oversight, data governance and incident controls.","The quality significance of the analysis follows from a basic distinction between availability and effective provision. In this case, technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review."]},{"heading":"Evidence base for implementation of risk-based artificial intelligence regulation","paragraphs":["The technical issue within implementation of risk-based artificial intelligence regulation concerns the basis on which a conclusion is reached. In this case, data quality comprises accuracy, completeness, timeliness, consistency and traceability. Strength in one dimension does not compensate automatically for weakness in another, particularly where the information informs a consequential learner decision. In relation to implementation of risk-based artificial intelligence regulation, any condition preventing complete assurance should appear with the evidence on which the judgement relies.","Responsibility for the analysis should be visible at the point where consequential decisions are made. A decision concerning the analysis should recognise that trend claims require comparable observations over time and a documented account of revisions, breaks in series and changes in coverage. Incomplete evidence, unmanaged conflict, absent learner groups or material learner impact require a higher level of review."]},{"heading":"Coverage and comparability","paragraphs":["The principal risks in relation to implementation of risk-based artificial intelligence regulation are automation bias in consequential decisions, unverified outputs entering teaching or assessment, loss of meaningful human review, and opaque use of personal or inferred data. The risks are interdependent; failure of one control may conceal or disable another."],"bullets":["Control personal and confidential information before it informs a consequential decision.","Notify users of material limitations.","Prohibit uses for which evidence or authority is insufficient.","Classify uses by effect on learners.","Review incidents and supplier changes."]},{"heading":"Responsible interpretation","paragraphs":["Evidence collection should be designed around the decision question rather than administrative convenience. For implementation of risk-based artificial intelligence regulation, the most relevant material is likely to include records of human review and overrides, documented authority for each consequential use, pre-deployment and periodic performance testing, and an inventory of systems and their intended uses.","The review method for the measure should be reproducible. Review of the analysis should trace selected records to source, reconcile totals across systems, quantify missing and late submissions, review manual adjustments and retain a revision history. Escalate discrepancies that could alter a published conclusion or individual outcome. Documentation should be sufficient to reconstruct the judgement without relying on unrecorded explanation.","Decision-makers using evidence on the available evidence should be told what the data cannot establish as clearly as what it can. The nature of the result and its applicable unit—system, institution, programme or learner group—should be explicit. Use in a different context requires an independent judgement that the settings are materially comparable."]},{"heading":"Limitations and reporting","paragraphs":["The analysis of implementation of risk-based artificial intelligence regulation should remain within the limits of the evidence. A decision concerning the comparison should recognise that international comparison can identify variation, but institutional and policy context remains necessary before a practice is transferred from one setting to another. Oversight of the available evidence should reflect the principle that a technical capability is not evidence that a use is educationally justified. Accuracy measured in one setting may not transfer to another population, language, curriculum or decision context. In relation to implementation of risk-based artificial intelligence regulation, decision-makers should not extend assurance beyond the point supported by the available evidence.","Accountability and effective correction both depend on a record that can be followed from evidence to decision. For the available evidence, the responsible body should be able to identify the evidence considered, the judgement made, the person or body authorised to make it and the action that followed. The record should prevent a later amendment from being treated as if it applied when an earlier decision was made.","Accountability for the analysis should follow decision-making authority. Operational tasks may be delegated, but accountability for material effects on learners must remain identifiable.","The record for the comparison should identify the responsible function, decision authority and escalation route. Improvement should be supported by evidence and an accountable decision record capable of public scrutiny."]}],"word_count":873,"content_hash":"sha256-b335c7e642a2453bb9886aecf1bb3ccde91e242c4c8cb34487269a2e3146f14f","seo_keywords":["implementation of risk-based artificial intelligence regulation","international education data research","education quality","ICEQC"],"schema_type":"AnalysisNewsArticle"}