Responsible AI Governance

Your release assistant now rejects the poisoned eval summary from the previous lesson. The attack said to bypass approval and promote a model candidate. A server-side gate blocked the effect.

That's a security control. It isn't yet a governance program.

A model owner disputes a denied promotion. A reviewer asks which policy version ran. An auditor asks who owned the prompt-injection risk and whether the fix was tested before release. The answer can't be "the model behaved better in our demo." The platform needs evidence.

Responsible AI governance is the engineering loop that answers five questions:

1. What workflow is being deployed, and who can it affect?
2. What harm is plausible, and who owns that risk?
3. What control reduces it?
4. What evidence proves the control ran and was tested?
5. What review, escalation, or appeal happens when the system is wrong?

Build that evidence package for a governed model workflow. This is engineering practice, not legal advice. Legal review still determines which duties apply to a real deployment.

Move from a blocked attack to an evidence loop

The NIST AI Risk Management Framework describes four connected functions: Govern, Map, Measure, and Manage.^{[1]Reference 1Artificial Intelligence Risk Management Framework (AI RMF 1.0)https://www.nist.gov/itl/ai-risk-management-framework} NIST's Generative AI Profile applies that loop to risks specific to generative systems, including harmful outputs, information integrity, privacy, and security concerns.^{[2]Reference 2Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profilehttps://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence}

For a model-promotion assistant, those functions become concrete:

Don't begin with a large policy document. Begin with the workflow that can cause an effect.

Inventory workflows before classifying them

A platform uses language models in several places. The model family isn't the risk classification unit. An assistant that summarizes public documentation and an assistant that proposes a credit decision can use the same model while requiring very different review.

Inventory should record intended use, affected stakeholder, possible effect, and first review route. This is an engineering triage label. It flags legal and product review; it doesn't make the final legal determination.

The inventory deliberately routes the model-promotion assistant to review even when it isn't making a credit decision. It interacts with people and can change production traffic. Those facts determine controls such as disclosure, approval, audit evidence, and appeals.

Map the EU AI Act with a date attached

The EU AI Act uses a risk-based framework. Its official explanation separates prohibited uses, high-risk systems, transparency-risk systems, and minimal or no-risk systems. High-risk examples include employment and worker management and access to essential services such as creditworthiness assessment for natural persons. Conversational systems and generated content can create transparency duties. A minimal or no-risk tier doesn't add tier-specific rules, but teams still need to check cross-cutting AI Act duties, existing law, and contractual obligations.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689}

That means legal review must examine the intended workflow:

• A system assessing creditworthiness for a consumer loan applicant requires high-risk review.
• A system allocating employee shifts based on individual performance requires high-risk review because it concerns worker management.
• A model-promotion assistant requires transparency and effects review: it interacts with people, and tool permissions determine whether it can shift production traffic.
• A public-document summarizer still needs ordinary security, privacy, and quality controls, even if it isn't routed into the higher categories.

Record regulatory status, not a timeless claim

Regulatory timelines move. The enacted AI Act originally staged high-risk rules for August 2, 2026 and August 2, 2027. As checked on June 9, 2026, Council and Parliament negotiators had reached a May 7 provisional agreement under which rules for stand-alone high-risk systems in areas such as employment, education, and critical infrastructure would apply from December 2, 2027, while rules for high-risk systems integrated into regulated products would apply from August 2, 2028. Parliament and Council still needed to adopt the provisional agreement formally.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689[4]Reference 4EU agrees to simplify AI rules to boost innovation and ban nudification apps to protect citizenshttps://digital-strategy.ec.europa.eu/en/news/eu-agrees-simplify-ai-rules-boost-innovation-and-ban-nudification-apps-protect-citizens}

The safe engineering practice is simple: include a legal_basis_checked_on date and a required legal signoff in the classification memo. Recheck official status before launch or a material feature change.

Create a risk register row a reviewer can use

A classification memo says where review starts. A risk register says what can go wrong, how the platform reduces that harm, who owns the decision, and what evidence exists.

Use the poisoned eval-summary incident as the first row:

Risk scoring isn't a substitute for judgment. It makes prioritization and escalation reproducible.

The residual impact remains high because an unauthorized production promotion is still serious. The control lowers likelihood. That difference matters: it prevents a team from declaring the risk solved merely because a filter was added.

Reject ceremonial risk registers

A row without an owner, control evidence, or review date is a description of a worry. It can't block release or drive follow-up work.

Document the system and its data separately

A model card describes intended use, out-of-scope use, limitations, evaluations, and known risks for a model or system. A datasheet describes the origin, composition, collection, processing, and recommended uses of a dataset. Model cards and datasheets were introduced as practical documentation patterns for accountability and reproducibility.^{[5]Reference 5Model Cards for Model Reportinghttps://arxiv.org/abs/1810.03993[6]Reference 6Datasheets for Datasetshttps://arxiv.org/abs/1803.09010}

For an LLM application, the release evidence package should reference both:

• The system card identifies model version, prompt or policy version, enabled tools, effect gates, safety tests, limitations, and rollback owner.
• The dataset record identifies the attack fixtures, ordinary release-request examples, provenance, labeling instructions, sensitive fields, evaluation split, and retention rule.

A model card isn't automatically statutory technical documentation. It's a useful engineering artifact that can support a wider documentation obligation when it's accurate, versioned, and reviewed.

Preserve an audit trail without collecting everything

The prompt-injection lesson separated untrusted retrieved text from trusted policy and gated promotion effects in code. Governance requires a replayable record of that decision:

• Workflow, system, policy, and tool versions.
• A privacy-minimized request identifier and actor role.
• Source trust labels, not a dump of every private document.
• The proposed effect and the deterministic gate result.
• Required approval, final effect status, and appeal identifier if one exists.

An audit trail shouldn't store hidden reasoning or unlimited private evaluation data. Store the business facts needed to reproduce an effect decision, apply access restrictions and retention rules, and keep sensitive content out unless it's necessary and authorized.

This example hardcodes the key only to stay runnable locally. In production, keep a versioned HMAC key in a secret manager, restrict access, and document rotation. Pseudonyms reduce direct exposure, but they remain linkable and still require privacy controls.

Make evidence tampering observable

Restricted storage and access controls matter first. A chained digest plus a protected anchor can also show that a stored sequence was rewritten after the review record was created. Store that anchor outside the mutable event log. This is an integrity signal, not a complete audit-storage design.

For EU high-risk deployments, logging, documentation, record-keeping, and human oversight can be regulated duties, and the obligations depend on the provider or deployer role and the system classification.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689} The design lesson is durable even when specific law changes: preserve enough trustworthy evidence to inspect consequential behavior.

Turn red-team results into approval decisions

Red teaming means deliberately attempting to elicit harmful or disallowed behavior before attackers, users, or staff encounter it. Research on language-model red teaming shows that adversarial testing can discover harmful behaviors and create reusable evaluation data.^{[7]Reference 7Red Teaming Language Models with Language Models.https://arxiv.org/abs/2202.03286}

For a governed model workflow, don't report only that an attack prompt was rejected. Evaluate observable effects:

• Attack success rate (ASR): fraction of attack traces that cause a forbidden effect or disclosure.
• False rejection rate (FRR): fraction of ordinary approval requests that are blocked unnecessarily.
• Finding owner: person responsible for remediation or accepted residual risk.
• Evidence link: trace fixture, gate version, reproduction, and retest result.

A single forbidden promotion blocks this release even if the aggregate rate looks small. An excessive FRR also needs work: a safety control that strands legitimate model owners creates a different harm and increases appeal volume.

Measure who bears errors and friction

A secure effect gate isn't sufficient if legitimate model owners are consistently blocked on a supported interaction path or can't reach an appeal. Ethics becomes engineering work when the team measures outcomes, investigates disparities, and repairs barriers.

For a release assistant, include ordinary eligible promotion scenarios through each supported path: web console, CLI workflow, keyboard-only navigation, screen-reader-assisted interaction, and escalation to a person. Test the interface with users and accessibility specialists where possible. A small fixture set can reveal a release blocker; it can't establish that a product is fair for every affected population.

This diagnostic test makes a broken accessible path visible before release:

Store this report with the dataset version, test limitations, accessibility review, and remediation owner. It complements oversight: the escalation route must itself be usable by the people who need it.

Make human oversight an executable path

"Human in the loop" is too vague for a release review. Define which effects require approval, what information the reviewer sees, how conflicting interests are handled, and how an affected actor challenges an outcome.

For model promotions:

1. The assistant may answer eval and release-policy questions.
2. It may propose a promotion with cited eval evidence.
3. Every production promotion requires approval. Untrusted retrieved instructions route the proposal to security review, while high-traffic targets route to exception review.
4. The reviewer approves or rejects the effect with a reason code.
5. A model owner may appeal; a second reviewer receives the original record and decision basis.

For a legally high-risk workflow, human oversight requirements need careful mapping to the applicable duties and roles.^{[3]Reference 3EU AI Act: Regulation laying down harmonised rules on artificial intelligencehttps://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32024R1689} For any workflow with production or people-facing effects, a clear review and appeal mechanism is also sound product engineering.

Gate releases on an evidence package

Governance fails when documents live in separate folders while the deployment pipeline ignores them. A release candidate should fail if required evidence is missing or if a red-team finding remains open.

For the model-promotion assistant, the minimal package contains:

Notice that dataset_record is release evidence, not administrative decoration. If nobody can identify where evaluation examples came from or how feedback labels were assigned, the result can't be reliably reproduced. The next lesson develops that data pipeline.

Build the model-release evidence package

Take the poisoned eval-summary trace from the prompt-injection chapter and produce a reviewable package:

1. Write a workflow memo for promotion-assistant-7.2: intended use, stakeholders, tool effects, review route, and legal_basis_checked_on.
2. Add a risk row for indirect injection leading to an unauthorized promotion. Name an owner, control, evidence, residual risk, and next review trigger.
3. Create a system-card entry that names the policy gate version, enabled tools, effect threshold, and known limitation.
4. Create a dataset-record entry for attack and benign traces. Record provenance, labels, removed sensitive fields, and version.
5. Run attacks and ordinary release requests. Record ASR and FRR, with a hard block for any unauthorized promotion.
6. Slice valid-request friction by supported interaction path and fix accessibility barriers before release.
7. Capture an audit replay showing the malicious retrieved instruction, proposed effect, gate decision, reviewer action, and final outcome without unnecessary private eval text.
8. Write the approval and appeal route and run one denied-promotion replay through it.
9. Evaluate the release gate. It must name every missing artifact, unresolved unsafe effect, or open accessibility finding.

A sufficient answer

A strong submission doesn't claim the workflow is compliant because a Markdown file exists. It states the review route and date, connects a plausible harm to owned controls, produces reproducible test and trace evidence, and blocks release when one forbidden effect survives.

A revealing failure case

Deliberately remove the dataset record, set unsafe_effects=1, or leave open_accessibility_findings=1 in the release gate. If the candidate still releases, your governance process is only descriptive. The gate must be connected to the deployment decision.

Mastery check

You're ready to operate this workflow when you can do all of the following:

• Explain why intended use, not the foundation model name, drives risk triage.
• Write a dated classification memo without pretending an engineering label is legal signoff.
• Convert one harm into a risk row with a control, owner, evidence, residual risk, and review trigger.
• Distinguish a system card from a dataset record and show why both affect reproducibility.
• Record an agent effect decision without storing unrestricted private data or hidden reasoning.
• Convert ASR and FRR results into a release-blocking decision.
• Detect when a supported interaction path blocks legitimate model owners and require remediation.
• Specify where human approval and appeal happen in the effect path.
• Reject a release candidate whose evidence package is missing or whose unsafe effects remain.

Evaluation rubric

• Foundational: Identifies affected stakeholders, possible effects, and an appropriate review route.
• Intermediate: Produces owned risk rows, versioned documentation, and privacy-conscious decision traces.
• Applied: Measures adversarial and benign behavior, then gates releases on evidence and unsafe effects.
• Advanced: Connects dated regulatory review, technical controls, human oversight, appeals, and data provenance into one operating loop.

Common pitfalls

• Symptom: A team calls a foundation model "high-risk" without naming a use case. Cause: Model capability was confused with deployed impact. Fix: Classify each workflow and affected stakeholder.
• Symptom: Legal timeline text becomes stale. Cause: Documentation recorded a claim without a verification date. Fix: Store legal_basis_checked_on and require review before release changes.
• Symptom: A risk register grows but never blocks a deployment. Cause: Rows lack owners, evidence, or release integration. Fix: Validate rows and fail the release gate.
• Symptom: Audit logging creates new privacy exposure. Cause: Raw content was stored instead of decision facts. Fix: Store minimized, access-controlled effect evidence with an explicit retention rule.
• Symptom: Attack tests pass while ordinary model owners can't obtain valid promotions. Cause: ASR was tracked without FRR and appeals. Fix: Evaluate attacks and benign flows together.
• Symptom: Overall FRR looks acceptable while an accessible path fails. Cause: Aggregate metrics hid who bears friction. Fix: Slice supported paths, investigate barriers, and block release until repaired.