{"id":214,"date":"2025-09-15T17:17:35","date_gmt":"2025-09-15T17:17:35","guid":{"rendered":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/?post_type=chapter&#038;p=214"},"modified":"2025-10-20T14:01:34","modified_gmt":"2025-10-20T14:01:34","slug":"chapter-3-4-governance-and-human-oversight","status":"publish","type":"chapter","link":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/chapter\/chapter-3-4-governance-and-human-oversight\/","title":{"raw":"3.4. Governance and Human Oversight","rendered":"3.4. Governance and Human Oversight"},"content":{"raw":"Even the most carefully designed ethical principles will fail without governance. Governance means having structures, policies, and oversight mechanisms that ensure AI use aligns with human values. It also means recognizing the irreplaceable role of human judgment in guiding, supervising, and correcting AI systems. This chapter explores what governance looks like in practice and why \u201chumans in the loop\u201d remain essential.\r\n<h2>What Is Governance?<\/h2>\r\nGovernance refers to the systems and processes that hold AI accountable. Just as universities have policies, committees, and accreditation bodies, AI requires oversight mechanisms to prevent harm and promote trust.\r\n\r\nCommon governance practices include:\r\n<ul>\r\n \t<li><strong>Audits and evaluations:<\/strong> independent checks of algorithms and outcomes to detect bias or unintended effects (<a href=\"https:\/\/ai.google\/responsibility\/responsible-ai-practices\/\" target=\"_blank\" rel=\"noopener\">Google AI Responsible Practices<\/a>).<\/li>\r\n \t<li><strong>Documentation:<\/strong> transparency tools like <a href=\"https:\/\/arxiv.org\/abs\/1810.03993\" target=\"_blank\" rel=\"noopener\">Model Cards<\/a> or <a href=\"https:\/\/arxiv.org\/abs\/1803.09010\" target=\"_blank\" rel=\"noopener\">Datasheets for Datasets<\/a> that record how systems were trained and tested.<\/li>\r\n \t<li><strong>Institutional review boards (IRBs):<\/strong> committees that oversee research involving human participants, now increasingly evaluating AI applications.<\/li>\r\n \t<li><strong>Feedback loops:<\/strong> systems that let users challenge or appeal AI-driven decisions (e.g., admissions, hiring, grading).<\/li>\r\n<\/ul>\r\n<h2>Human-in-the-Loop<\/h2>\r\nGovernance also requires humans to remain actively involved in AI processes. While machines can process data and identify patterns, they lack judgment, empathy, and contextual awareness. Human oversight is needed to:\r\n<ul>\r\n \t<li>Interpret results in context.<\/li>\r\n \t<li>Correct errors or biases.<\/li>\r\n \t<li>Make value-laden decisions (e.g., admissions, hiring, grading).<\/li>\r\n \t<li>Take responsibility for outcomes.<\/li>\r\n<\/ul>\r\n<details><summary>\ud83d\udcd6 Analogy: The Airline Autopilot (click to expand)<\/summary>\r\n<div style=\"border: 1px solid #ddd;padding: 0.8em;margin-top: 0.5em;background: #fafafa\">\r\n\r\nModern airplanes use autopilot systems to handle much of the flying. Yet no passenger would board a plane without a human pilot in the cockpit. Why? Because unexpected conditions\u2014storms, emergencies, equipment failures\u2014require human judgment. AI governance works the same way. Automation can assist, but human oversight ensures safety and accountability when the unexpected arises.\r\n\r\n<\/div>\r\n<\/details>\r\n<h2>Global and Institutional Frameworks<\/h2>\r\nSeveral frameworks provide guidance for building governance structures:\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.nist.gov\/itl\/ai-risk-management-framework\" target=\"_blank\" rel=\"noopener\">NIST AI Risk Management Framework (U.S.)<\/a>: emphasizes risk identification, measurement, and mitigation.<\/li>\r\n \t<li><a href=\"https:\/\/digital-strategy.ec.europa.eu\/en\/policies\/european-approach-artificial-intelligence\" target=\"_blank\" rel=\"noopener\">European Union AI Act<\/a>: classifies AI systems by risk level and imposes stricter requirements on high-risk systems.<\/li>\r\n \t<li><a href=\"https:\/\/library.educause.edu\/resources\/2025\/9\/2025-educause-horizon-action-plan-building-skills-and-literacy-for-teaching-with-genai\" target=\"_blank\" rel=\"noopener\">EDUCAUSE AI Action Plan<\/a>: provides higher education with strategies for readiness, policy development, and oversight.<\/li>\r\n<\/ul>\r\nAt the institutional level, universities are beginning to adapt these frameworks to their contexts, developing governance structures that balance innovation with responsibility.\r\n<div class=\"pb-embed\">\r\n\r\n[embed]https:\/\/www.youtube.com\/embed\/Suo0qFs9Dwo[\/embed]\r\n\r\n<\/div>\r\n<h2>\ud83d\udcda Weekly Reflection Journal<\/h2>\r\n<div style=\"border: 2px solid #4CAF50;padding: 1em;margin: 1em 0;background-color: #f9fff9\">\r\n\r\n<strong>Reflection Prompt:<\/strong>\r\n<ul>\r\n \t<li>What are the risks of relying too heavily on automation without adequate human oversight?<\/li>\r\n \t<li>How can institutions balance the need for innovation with the need for accountability?<\/li>\r\n \t<li>When AI outputs conflict with human judgment, whose decision should prevail?<\/li>\r\n<\/ul>\r\n<\/div>\r\n<h2>Looking Ahead<\/h2>\r\nGovernance and oversight create the structures for ethical AI use. Next, we will examine how these ideas play out specifically in higher education contexts, where academic integrity, equity, and accessibility are central concerns.","rendered":"<p>Even the most carefully designed ethical principles will fail without governance. Governance means having structures, policies, and oversight mechanisms that ensure AI use aligns with human values. It also means recognizing the irreplaceable role of human judgment in guiding, supervising, and correcting AI systems. This chapter explores what governance looks like in practice and why \u201chumans in the loop\u201d remain essential.<\/p>\n<h2>What Is Governance?<\/h2>\n<p>Governance refers to the systems and processes that hold AI accountable. Just as universities have policies, committees, and accreditation bodies, AI requires oversight mechanisms to prevent harm and promote trust.<\/p>\n<p>Common governance practices include:<\/p>\n<ul>\n<li><strong>Audits and evaluations:<\/strong> independent checks of algorithms and outcomes to detect bias or unintended effects (<a href=\"https:\/\/ai.google\/responsibility\/responsible-ai-practices\/\" target=\"_blank\" rel=\"noopener\">Google AI Responsible Practices<\/a>).<\/li>\n<li><strong>Documentation:<\/strong> transparency tools like <a href=\"https:\/\/arxiv.org\/abs\/1810.03993\" target=\"_blank\" rel=\"noopener\">Model Cards<\/a> or <a href=\"https:\/\/arxiv.org\/abs\/1803.09010\" target=\"_blank\" rel=\"noopener\">Datasheets for Datasets<\/a> that record how systems were trained and tested.<\/li>\n<li><strong>Institutional review boards (IRBs):<\/strong> committees that oversee research involving human participants, now increasingly evaluating AI applications.<\/li>\n<li><strong>Feedback loops:<\/strong> systems that let users challenge or appeal AI-driven decisions (e.g., admissions, hiring, grading).<\/li>\n<\/ul>\n<h2>Human-in-the-Loop<\/h2>\n<p>Governance also requires humans to remain actively involved in AI processes. While machines can process data and identify patterns, they lack judgment, empathy, and contextual awareness. Human oversight is needed to:<\/p>\n<ul>\n<li>Interpret results in context.<\/li>\n<li>Correct errors or biases.<\/li>\n<li>Make value-laden decisions (e.g., admissions, hiring, grading).<\/li>\n<li>Take responsibility for outcomes.<\/li>\n<\/ul>\n<details>\n<summary>\ud83d\udcd6 Analogy: The Airline Autopilot (click to expand)<\/summary>\n<div style=\"border: 1px solid #ddd;padding: 0.8em;margin-top: 0.5em;background: #fafafa\">\n<p>Modern airplanes use autopilot systems to handle much of the flying. Yet no passenger would board a plane without a human pilot in the cockpit. Why? Because unexpected conditions\u2014storms, emergencies, equipment failures\u2014require human judgment. AI governance works the same way. Automation can assist, but human oversight ensures safety and accountability when the unexpected arises.<\/p>\n<\/div>\n<\/details>\n<h2>Global and Institutional Frameworks<\/h2>\n<p>Several frameworks provide guidance for building governance structures:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.nist.gov\/itl\/ai-risk-management-framework\" target=\"_blank\" rel=\"noopener\">NIST AI Risk Management Framework (U.S.)<\/a>: emphasizes risk identification, measurement, and mitigation.<\/li>\n<li><a href=\"https:\/\/digital-strategy.ec.europa.eu\/en\/policies\/european-approach-artificial-intelligence\" target=\"_blank\" rel=\"noopener\">European Union AI Act<\/a>: classifies AI systems by risk level and imposes stricter requirements on high-risk systems.<\/li>\n<li><a href=\"https:\/\/library.educause.edu\/resources\/2025\/9\/2025-educause-horizon-action-plan-building-skills-and-literacy-for-teaching-with-genai\" target=\"_blank\" rel=\"noopener\">EDUCAUSE AI Action Plan<\/a>: provides higher education with strategies for readiness, policy development, and oversight.<\/li>\n<\/ul>\n<p>At the institutional level, universities are beginning to adapt these frameworks to their contexts, developing governance structures that balance innovation with responsibility.<\/p>\n<div class=\"pb-embed\">\n<p><iframe loading=\"lazy\" id=\"oembed-1\" title=\"NIST AI Risk Management Framework\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/Suo0qFs9Dwo?feature=oembed&#38;rel=0\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<\/div>\n<h2>\ud83d\udcda Weekly Reflection Journal<\/h2>\n<div style=\"border: 2px solid #4CAF50;padding: 1em;margin: 1em 0;background-color: #f9fff9\">\n<p><strong>Reflection Prompt:<\/strong><\/p>\n<ul>\n<li>What are the risks of relying too heavily on automation without adequate human oversight?<\/li>\n<li>How can institutions balance the need for innovation with the need for accountability?<\/li>\n<li>When AI outputs conflict with human judgment, whose decision should prevail?<\/li>\n<\/ul>\n<\/div>\n<h2>Looking Ahead<\/h2>\n<p>Governance and oversight create the structures for ethical AI use. Next, we will examine how these ideas play out specifically in higher education contexts, where academic integrity, equity, and accessibility are central concerns.<\/p>\n","protected":false},"author":6,"menu_order":4,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":["jhargis"],"pb_section_license":"","_links_to":"","_links_to_target":""},"chapter-type":[],"contributor":[66],"license":[],"class_list":["post-214","chapter","type-chapter","status-publish","hentry","contributor-jhargis"],"part":32,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/chapters\/214","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/wp\/v2\/users\/6"}],"version-history":[{"count":14,"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/chapters\/214\/revisions"}],"predecessor-version":[{"id":788,"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/chapters\/214\/revisions\/788"}],"part":[{"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/parts\/32"}],"metadata":[{"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/chapters\/214\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/wp\/v2\/media?parent=214"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/pressbooks\/v2\/chapter-type?post=214"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/wp\/v2\/contributor?post=214"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/books.nbsplabs.com\/ai-lit-intro\/wp-json\/wp\/v2\/license?post=214"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}