{"schema_version":"ICEQC_NEWS_ARTICLE_V1","id":"iceqc-news-9b5d6071e8f0b80c","slug":"2023-10-24-testing-whether-improvements-in-governance-of-generative-artificial-intelligence-are-sustained","language":"en","publication_status":"READY_FOR_IMPORT","publication_date":"2023-10-24","last_modified_date":"2023-10-24","title":"Testing whether improvements in governance of generative artificial intelligence are sustained","summary":"Sets out a controlled approach to testing whether improvements in governance of generative artificial intelligence are sustained, linking diagnosis, accountable action, outcome.","category":{"code":"QUALITY_IMPROVEMENT_METHODS","label":"Quality Improvement Methods"},"article_type":"Quality improvement method","publisher":"International Council for Education Quality Certification (ICEQC)","jurisdictional_scope":"International","historical_reference_basis":"International guidance released in September 2023","reference_authority":"Relevant public authorities and official international sources","sections":[{"heading":null,"paragraphs":["The immediate international context is the international guidance released in September 2023. Its significance for governance of generative artificial intelligence lies in the quality of implementation rather than in formal acknowledgement alone. In reviewing the practice, the purpose of an improvement method is not to produce an action plan; it is to change a material condition and verify that the change is sustained. Learner effect, institutional duty and proper resource use should inform the judgement. Different administrative structures may support the same public-interest outcome.","International guidance on generative artificial intelligence in education and research was released in September 2023. It calls for a human-centred approach, protection of data privacy, age-appropriate use, validation and institutional capacity. Immediate provider controls should address authorised uses, assessment, disclosure, information security, unequal access and human review while evidence on educational benefit and risk continues to develop.","Implementation of the improvement objective should be organised around a decision that can be tested. The analysis of the intervention proceeds on the basis that the intervention should be tested on a scale proportionate to the risk before wider implementation, unless immediate system-wide action is necessary to protect learners. In relation to testing whether improvements in governance of generative artificial intelligence are sustained, oversight requires a traceable line from the approved objective through responsible action to evidence of outcome."]},{"heading":"Scope of the improvement","paragraphs":["The relevance of the international guidance released in September 2023 is contextual. Consequential findings on governance of generative artificial intelligence require current, attributable evidence for the scope concerned.","The quality significance of the practice follows from a basic distinction between availability and effective provision. Oversight of the corrective programme should reflect the principle that technology may support teaching, administration and access, but consequential educational decisions must remain accountable, explainable and open to effective review."],"bullets":["Prohibit uses for which evidence or authority is insufficient.","Classify uses by effect on learners.","Test performance across relevant groups.","Notify users of material limitations.","Control personal and confidential information."]},{"heading":"Implementation responsibilities","paragraphs":["The analysis of governance of generative artificial intelligence should make its decision rule explicit. A decision concerning the intervention should recognise that effectiveness is the demonstrated change in the condition the action was intended to address. Completion of training, publication of guidance or installation of a system is an output and should not be reported as an outcome without further evidence. A stated decision rule enables comparable examination and limits retrospective explanations of adverse evidence.","Failure in relation to the improvement objective may arise even where the stated policy is reasonable. Material concerns include unequal performance across learner groups, loss of meaningful human review, opaque use of personal or inferred data, and automation bias in consequential decisions. Review should consider whether an exception is prolonged, recurring or capable of affecting learners outside the cases examined.","Each source should have a stated purpose in supporting or limiting the conclusion. For the improvement objective, the most relevant material is likely to include learner information and accessible challenge routes, records of human review and overrides, an inventory of systems and their intended uses, and documented authority for each consequential use. In relation to testing whether improvements in governance of generative artificial intelligence are sustained, confidence is strengthened by corroboration, not by the volume of records drawn from the same underlying source."],"bullets":["What condition should change?","Has the improvement been sustained?","What was the baseline?","When should an effect be visible?","Did the effect reach the intended group?"]},{"heading":"Testing effectiveness","paragraphs":["The review method for governance of generative artificial intelligence should be reproducible. In reviewing the improvement objective, responsible bodies should set a baseline and success measure before intervention, define the review period, compare the result with the intended outcome and examine adverse or unequal effects. Continue monitoring long enough to determine whether the improvement is sustained. The retained analysis should be reproducible from the selected evidence, decision rule and recorded reasons for accepted exceptions.","Improvement of the intervention should proceed through controlled tests where risk permits.","The analysis of the practice should remain within the limits of the evidence. Oversight of the intervention should reflect the principle that correcting an individual record does not establish that the process which produced the error has been corrected. For the issue under review, a technical capability is not evidence that a use is educationally justified. Accuracy measured in one setting may not transfer to another population, language, curriculum or decision context. Material uncertainty should result in further enquiry or an expressly limited finding.","Public reporting on the intervention should distinguish established fact, analytical judgement and planned action. Material revisions should be traceable to their reason and effective date. If definitions, coverage or evidence alter an earlier conclusion, the reason should be stated so that revision is not mistaken for changed performance.","Any response to the present development should test the evidential connection between the practice, its implementation and the outcome claimed. Improvement should be supported by evidence and an accountable decision record capable of public scrutiny."]}],"word_count":821,"content_hash":"sha256-598a12c1b9198ceec6f52d7f91b1a9d1e6c38a71799c3b53d9d7779a268b5ed9","seo_keywords":["testing whether improvements in governance of generative artificial intelligence are sustained","education quality improvement","education quality","ICEQC"],"schema_type":"TechArticle"}