Governance starts with purpose and authority
Define what the system is allowed to do, who it serves, which outcomes it supports, and which decisions remain outside its authority. A system that answers questions may have different risk than a system that recommends actions, changes records, sends messages, or triggers external tools.
Assign owners for policy, knowledge, model behavior, technical operation, human review, incidents, and change approval. Governance cannot work when responsibility disappears between teams.
Control the information boundary
Identify approved sources, source priority, freshness, ownership, conflicts, permissions, and unsupported topics. Define when the system must cite a source, disclose uncertainty, ask for more context, refuse to answer, or route the question to a person.
Retrieval quality is not only a search problem. It is also a governance problem. The system needs rules for what counts as authoritative and what happens when authoritative sources disagree.
Define behavior and tool permissions
Specify tone, scope, required questions, prohibited claims, decision limits, output formats, and escalation triggers. If the system uses tools, define which tools are available, which parameters are allowed, what confirmation is required, and which actions need approval.
Separate read, recommend, draft, and execute authority. A system may be allowed to retrieve an account status, draft a response, and recommend a next step without being allowed to change the account or send the message.
Make human review operational
“Human in the loop” is incomplete unless the loop is designed. Define the event that triggers review, the evidence shown to the reviewer, the reviewer’s qualifications and authority, response time, possible decisions, recordkeeping, and what the system does while waiting.
High-risk review should be proportionate to consequence. Low-risk routine work may need sampling and monitoring. High-consequence outputs may require pre-release approval or a qualified professional.
Control the output
An output contract defines what every response or artifact must contain, how it is formatted, which source or evidence supports it, how uncertainty appears, what quality checks occur, and whether release requires approval.
Output contracts improve consistency and make evaluation possible. They also give downstream workflows a stable structure instead of forcing people or software to interpret free-form responses.
Test, monitor, and govern change
Test normal scenarios, edge cases, unsupported requests, source conflicts, malicious input, tool failure, missing context, permission boundaries, escalation, and recovery. Record the expected behavior before deciding that the system passed.
After release, monitor quality, refusals, escalations, incidents, source freshness, drift, and user feedback. Changes to models, prompts, tools, knowledge, rules, or thresholds should have ownership, impact review, test requirements, approval, versioning, and rollback.
Governance is not a restriction added after the intelligent system works. It is part of the definition of what working means.