How to Build an AI Governance Framework in Six Stages

Most AI governance programs produce a policy document that engineering never reads. The document is usually fine. The failure is structural: the policy was written before anyone knew what AI the company was actually running, and it was never wired into the path a feature takes to production. Here is the build sequence that avoids both problems.

Stage 1: Inventory what you already have

Start with discovery, not drafting. Every model, agent, retrieval pipeline, vendor feature with AI switched on, and browser extension someone installed. Include the AI features your existing SaaS vendors enabled by default, because those are in scope whether you chose them or not.

Expect the inventory to be worse than leadership believes. That gap is the most useful artifact of stage one, because it converts governance from an abstract initiative into a specific list. You cannot govern an estate you have not enumerated, and every later stage depends on this list being real.

Stage 2: Classify by risk, not by technology

Tier each use case on three axes: consequence if the output is wrong, how much autonomy the system has, and how sensitive the data it touches is. A meeting summarizer and an agent that issues refunds are both language models, and treating them alike guarantees you either smother the first or under-control the second.

Three tiers is usually enough. Keep the definitions concrete enough that a product manager can self-classify correctly on the first try, because a tiering scheme that requires a committee will be bypassed.

Stage 3: Write policy that maps to the tiers

One policy, differentiated obligations. Low tier gets acceptable-use rules and logging. Middle tier adds review before launch and named ownership. High tier adds impact assessment, human checkpoints, evaluation gates, and executive approval.

The part most policies omit is decision rights. Name who approves a high-tier launch and, more importantly, who has authority to stop one that is already running and what triggers that. A policy without a documented stop authority is a statement of intent.

Stage 4: Wire the controls into the build path

This is where governance becomes real or becomes paperwork, and it is the stage that gets skipped. The test is simple: could a team ship a high-tier AI feature without governance noticing? If yes, you do not have a program.

Concretely, that means intake lives in the ticketing system engineers already use rather than a separate form; evaluation gates run in CI so a model change fails the build rather than a review meeting; logging of prompts, outputs, and tool calls is on by default rather than requested; and human checkpoints exist as code paths rather than as sentences in a document. Everything in stages 1 through 3 is preparation for this.

Stage 5: Assess impact before launch, and after

Higher tiers need a repeatable impact assessment covering who could be harmed, how the system fails, what the failure costs, and what mitigations are in place, re-run when something significant changes. Keep it short enough that teams complete it honestly. A twelve-page template gets copy-pasted from the last project.

This is where several obligations converge, which is an argument for building it once and reusing it: ISO/IEC 42001 asks for AI system impact assessments, the EU AI Act imposes obligations on high-risk systems, and privacy DPIAs cover overlapping ground where personal data is involved. Whether you anchor to 42001 or the NIST AI RMF affects the paperwork more than the substance.

Stage 6: Monitor, audit, and report upward

Production monitoring for drift, incidents, and override rates, plus periodic internal audit against your own policy. Then a board-level view: how many systems in each tier, how many high-tier assessments are open versus closed, incidents this period, and where the program is behind.

Keep that view to one page. AI governance programs rarely die from a scandal; they die from being invisible during budget season.

The three failure modes to watch for

Policy without inventory, which produces rules aimed at imagined systems. Governance as a gate rather than a service, which teaches teams to route around it and creates the shadow AI you were trying to prevent. And governance that never tests anything, which is the most common of all. A framework tells you what should be true about your AI systems; only adversarial testing tells you what is actually true, and prompt injection reaching real tools is a finding no policy review surfaces.

Where BD Emerson fits

Our AI governance practice runs this sequence end to end, with ISO 42001 consulting where certification is the goal and EU AI Act advisory where the regulation applies. Because we also run enterprise AI implementations and offensive testing of AI systems, stage four gets built by people who have shipped these systems, and stage six gets evidence from people who have broken them.