Responsible AI Governance

Your release assistant now rejects the poisoned eval summary from the previous lesson. The attack said to bypass approval and promote a model candidate. A server-side gate blocked the effect.

That's a security control. It isn't yet a governance program.

A model owner disputes a denied promotion. A reviewer asks which policy version ran. An auditor asks who owned the prompt-injection risk and whether the fix was tested before release. The answer can't be "the model behaved better in our demo." The platform needs evidence.

Responsible AI governance is the engineering loop that answers five questions:

1. What workflow is being deployed, and who can it affect?
2. What harm is plausible, and who owns that risk?
3. What control reduces it?
4. What evidence proves the control ran and was tested?
5. What review, escalation, or appeal happens when the system is wrong?

Build that evidence package for a governed model workflow. This is engineering practice, not legal advice. Legal review still determines which duties apply to a real deployment.

What makes a technical control part of governance?

Answer

The control is mapped to an owned risk, versioned, tested, recorded in review evidence, and revisited after failures or material changes.

Move from a blocked attack to an evidence loop

The NIST AI Risk Management Framework describes four connected functions: Govern, Map, Measure, and Manage.^{[1]Reference 1Artificial Intelligence Risk Management Framework (AI RMF 1.0)https://www.nist.gov/itl/ai-risk-management-framework} NIST's Generative AI Profile applies that loop to risks specific to generative systems, including harmful outputs, information integrity, privacy, and security concerns.^{[2]Reference 2Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profilehttps://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence}

For a model-promotion assistant, those functions become concrete:

Don't begin with a large policy document. Begin with the workflow that can cause an effect.

Inventory workflows before classifying them

A platform uses language models in several places. The model family isn't the risk classification unit. An assistant that summarizes public documentation and an assistant that proposes a credit decision can use the same model while requiring very different review.

Inventory should record intended use, affected stakeholder, possible effect, and first review route. This is an engineering triage label. It flags legal and product review; it doesn't make the final legal determination.

The inventory deliberately routes the model-promotion assistant to review even when it isn't making a credit decision. It interacts with people and can change production traffic. Those facts determine controls such as disclosure, approval, audit evidence, and appeals.

Map the EU AI Act with a date attached

The EU AI Act uses a risk-based framework. Its official explanation separates prohibited uses, high-risk systems, transparency-risk systems, and minimal or no-risk systems. High-risk examples include employment and worker management and access to essential services such as creditworthiness assessment for natural persons. Conversational systems and generated content can create transparency duties. A minimal or no-risk tier doesn't add tier-specific rules, but teams still need to check cross-cutting AI Act duties, existing law, and contractual obligations.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689}

That means legal review must examine the intended workflow:

• A system assessing creditworthiness for a consumer loan applicant requires high-risk review.
• A system allocating employee shifts based on individual performance requires high-risk review because it concerns worker management.
• A model-promotion assistant requires transparency and effects review: it interacts with people, and tool permissions determine whether it can shift production traffic.
• A public-document summarizer still needs ordinary security, privacy, and quality controls, even if it isn't routed into the higher categories.

Record regulatory status, not a timeless claim

Regulatory timelines move. The enacted AI Act originally staged general application on August 2, 2026 and product-related high-risk rules on August 2, 2027. The AI Omnibus entered into force on July 27, 2026, extending Annex III high-risk rules to December 2, 2027 and high-risk AI embedded in products to August 2, 2028.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689[4]Reference 4AI Acthttps://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai}

The safe engineering practice is simple: include a legal_basis_checked_on date and a required legal signoff in the classification memo. Recheck Official Journal status, Commission guidance, and local counsel before launch or a material feature change.

Why classify the workflow rather than saying a model is "high-risk"?

Answer

Risk and duties depend on intended use and impact: credit assessment, worker allocation, production-effect automation, or ordinary summarization can use the same model with different consequences.

Create a risk register row a reviewer can use

A classification memo says where review starts. A risk register says what can go wrong, how the platform reduces that harm, who owns the decision, and what evidence exists.

Use the poisoned eval-summary incident as the first row:

Governance also owns evaluation integrity. A second row class covers contaminated evidence that can make a release look safer than reality:

Risk scoring isn't a substitute for judgment. It makes prioritization and escalation reproducible.

The residual impact remains high because an unauthorized production promotion is still serious. The control lowers likelihood. That difference matters: it prevents a team from declaring the risk solved merely because a filter was added.

Reject ceremonial risk registers

A row without an owner, control evidence, or review date is a description of a worry. It can't block release or drive follow-up work.

Document the system and its data separately

A model card describes intended use, out-of-scope use, limitations, evaluations, and known risks for a model or system. A datasheet describes the origin, composition, collection, processing, and recommended uses of a dataset. Model cards and datasheets were introduced as practical documentation patterns for accountability and reproducibility.^{[5]Reference 5Model Cards for Model Reportinghttps://arxiv.org/abs/1810.03993[6]Reference 6Datasheets for Datasetshttps://arxiv.org/abs/1803.09010}

For an LLM application, the release evidence package should reference both:

• The system card identifies model version, prompt or policy version, enabled tools, effect gates, safety tests, limitations, and rollback owner.
• The dataset record identifies the attack fixtures, ordinary release-request examples, provenance, labeling instructions, sensitive fields, evaluation split, and retention rule.

A model card isn't automatically statutory technical documentation. It's a useful engineering artifact that can support a wider documentation obligation when it's accurate, versioned, and reviewed.

Why keep a dataset record next to the system card?

Answer

An evaluation score is only interpretable if reviewers can identify the examples, provenance, labeling instructions, sensitive-data treatment, and version that produced it.

Preserve an audit trail without collecting everything

The prompt-injection lesson separated untrusted retrieved text from trusted policy and gated promotion effects in code. Governance requires a replayable record of that decision:

• Workflow, system, policy, and tool versions.
• A privacy-minimized request identifier and actor role.
• Source trust labels, not a dump of every private document.
• The proposed effect and the deterministic gate result.
• Required approval, final effect status, and appeal identifier if one exists.

An audit trail shouldn't store hidden reasoning or unlimited private evaluation data. Store the business facts needed to reproduce an effect decision, apply access restrictions and retention rules, and keep sensitive content out unless it's necessary and authorized.

This example hardcodes the key only to stay runnable locally. In production, keep a versioned HMAC key in a secret manager, restrict access, and document rotation. Pseudonyms reduce direct exposure, but they remain linkable and still require privacy controls.

Make evidence tampering observable

Restricted storage and access controls matter first. A chained digest plus a protected anchor can also show that a stored sequence was rewritten after the review record was created. Store that anchor outside the mutable event log. This is an integrity signal, not a complete audit-storage design.

For EU high-risk deployments, logging, documentation, record-keeping, and human oversight can be regulated duties, and the obligations depend on the provider or deployer role and the system classification.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689} The design lesson is durable even when specific law changes: preserve enough trustworthy evidence to inspect consequential behavior.

Turn red-team results into approval decisions

Red teaming means deliberately attempting to elicit harmful or disallowed behavior before attackers, users, or staff encounter it. Research on language-model red teaming shows that adversarial testing can discover harmful behaviors and create reusable evaluation data.^{[7]Reference 7Red Teaming Language Models with Language Models.https://arxiv.org/abs/2202.03286}

For a governed model workflow, don't report only that an attack prompt was rejected. Evaluate observable effects:

• Attack success rate (ASR): fraction of attack traces that cause a forbidden effect or disclosure.
• False rejection rate (FRR): fraction of ordinary approval requests that are blocked unnecessarily.
• Finding owner: person responsible for remediation or accepted residual risk.
• Evidence link: trace fixture, gate version, reproduction, and retest result.

A single forbidden promotion blocks this release even if the aggregate rate looks small. An excessive FRR also needs work: a safety control that strands legitimate model owners creates a different harm and increases appeal volume.

Measure who bears errors and friction

A secure effect gate isn't sufficient if legitimate model owners are consistently blocked on a supported interaction path or can't reach an appeal. Ethics becomes engineering work when the team measures outcomes, investigates disparities, and repairs barriers.

For a release assistant, include ordinary eligible promotion scenarios through each supported path: web console, CLI workflow, keyboard-only navigation, screen-reader-assisted interaction, and escalation to a person. Test the interface with users and accessibility specialists where possible. A small fixture set can reveal a release blocker; it can't establish that a product is fair for every affected population.

This diagnostic test makes a broken accessible path visible before release:

Store this report with the dataset version, test limitations, accessibility review, and remediation owner. It complements oversight: the escalation route must itself be usable by the people who need it.

Make human oversight an executable path

"Human in the loop" is too vague for a release review. Define which effects require approval, what information the reviewer sees, how conflicting interests are handled, and how an affected actor challenges an outcome.

For model promotions:

1. The assistant may answer eval and release-policy questions.
2. It may propose a promotion with cited eval evidence.
3. Every production promotion requires approval. Untrusted retrieved instructions route the proposal to security review, while high-traffic targets route to exception review.
4. The reviewer approves or rejects the effect with a reason code.
5. A model owner may appeal; a second reviewer receives the original record and decision basis.

For a legally high-risk workflow, human oversight requirements need careful mapping to the applicable duties and roles.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689} For any workflow with production or people-facing effects, a clear review and appeal mechanism is also sound product engineering.

When is "human oversight" meaningful?

Answer

When a named reviewer can see the relevant evidence before a consequential effect, has authority to stop or change it, records a reason, and an affected person has an escalation path.

Gate releases on an evidence package

Governance fails when documents live in separate folders while the deployment pipeline ignores them. A release candidate should fail if required evidence is missing or if a red-team finding remains open.

For the model-promotion assistant, the minimal package contains:

Notice that dataset_record is release evidence, not administrative decoration. If nobody can identify where evaluation examples came from or how feedback labels were assigned, the result can't be reliably reproduced. The next lesson develops that data pipeline.