AI Governance for Companies That Already Shipped: A Retrofit Playbook

Oso Data Privacy
May 17
5 min read

Most AI governance content assumes you're building a program before you launch AI. Almost nobody is.

Walk into any U.S. company with more than fifty employees today and you'll find something interesting: there's already AI in production. A copilot in customer support. An agent that triages incoming tickets. A model summarizing legal documents. A vendor whose product quietly added an LLM to a feature you thought was rules-based.

Most of it got there without a privacy review, without a security review, without a procurement review. It got there because someone needed it to work last quarter and it did.

Now the company is being asked — by a board, by a regulator, by an enterprise customer's vendor questionnaire — whether they have AI governance in place. And the honest answer is: they have an AI use policy that got drafted last year, and almost nothing underneath it.

This is the situation we walk into most often. It's also the one the published roadmaps don't cover. So here's the retrofit playbook.

The retrofit problem is not the same as the greenfield problem

When a company is building AI governance from scratch — before any models are in production — the work is conceptual. You're picking frameworks, drafting policies, defining acceptable use, mapping risk tiers, and lining up approvals. It can be done thoughtfully because nothing is shipping yet.

Retrofit is the opposite. Models are already running. Agents are already taking actions. The work is forensic before it's strategic. You can't just write a policy and expect things to comply with it — you have to first figure out what's already happening, who owns it, and what to do about the parts that shouldn't be happening.

Most of the public AI governance content skips this and goes straight to the policy-writing step. That's why so many programs look great on paper and fall apart the first time a regulator asks a real question.

Four things the retrofit playbook actually has to cover

1. Discovery: find every agent

You can't govern what you can't see. The first job in a retrofit is to build a current-state inventory of every AI system the company is actually using.

The hard part is that the inventory isn't sitting anywhere. Marketing has a tool with an LLM bolted onto it. Engineering wrote some internal automation. Customer support is using a vendor that recently added agentic features. Finance ran a pilot last quarter. HR has a resume screener. None of these are on the same list.

The cheapest version of this is a structured intake survey across functions. The good version pairs that with network egress monitoring (where is data leaving your environment, and to what AI services?) and a sweep of your SaaS contracts for "AI feature" disclosures.

Plan for the inventory to take longer than you expect and to produce findings you didn't want. That's the work.

2. Triage: not every agent is equal

Once you have the inventory, the next job is to sort it. Three buckets work:

High-risk, high-impact: agents that take actions affecting customers, employees, or regulated decisions. Anything that touches a hiring decision, a credit decision, a clinical decision, a benefits decision, or a customer-facing communication.
Medium-risk: agents that handle sensitive internal data but don't make decisions with external consequences. Internal summarization tools, code assistants on proprietary repos, etc.
Low-risk: general-purpose productivity tools used inside the company with no sensitive data or decisions attached.

You don't have time to govern everything to the same standard. Triage lets you put your governance effort where the actual exposure is.

3. Operational ownership: the part most companies skip

The biggest gap we see in retrofit programs is not policy. It's ownership. Companies have an "AI use policy" approved by legal. What they don't have is anyone who can answer:

Who decides whether a new use case is allowed?
Who has authority to pause an agent if something looks off?
Who fields the first regulator inquiry when one comes?
Who owns the harm-and-redress process when an agent makes a bad decision?
Who reviews vendor changes when a SaaS product adds new AI features?

A policy without those four or five named owners isn't a governance program. It's a PDF. And the gap between "we have an AI policy" and "we have AI governance" is exactly the gap between that PDF and the operational machinery underneath it.

The fix isn't complicated, but it's specific. Each high-risk agent needs a named operational owner. Each medium-risk agent needs a clear escalation path. Each category needs a defined review cadence.

4. Guardrails before policy

Most retrofit programs try to draft the perfect policy first and then enforce it. That order is wrong. By the time the policy is approved, the world has moved.

The order that works better: identify the smallest set of guardrails that prevent the worst outcomes, get those in place immediately (even informally), and then iterate the policy around what's actually working. Most of the high-risk patterns are knowable in advance — uncontrolled data sharing, agents acting without human review on consequential decisions, vendors silently adding model features without disclosure. Address those first. Refine the framework later.

What "done" looks like

A retrofit AI governance program doesn't look like a 40-page document. It looks like:

A living inventory of every AI system in the company, updated quarterly
A risk tier assigned to each one
A named operational owner for every high-risk system
A defined process for new use case approval that takes days, not weeks
A clear path to pause or roll back any agent if something goes wrong
A documented response process for regulator inquiries or customer complaints

None of those are exotic. They're just the things that don't exist when a company has only a policy and no infrastructure underneath it.

The compounding cost of waiting

The longer a retrofit gets delayed, the harder it becomes. Vendors keep adding AI features. Employees keep adopting tools. The inventory gap widens. The number of agents you need to retrospectively govern grows. The cost of triage rises.

Companies that started AI governance retrofit work eighteen months ago are now far ahead of companies that are starting today. Companies starting today will be far ahead of companies starting eighteen months from now. The window for catching up cheaply is closing.

Oso Data Privacy is a privacy compliance and AI governance consultancy focused on U.S. organizations, with full coverage across Latin America. We help companies that already have AI in production figure out what they actually have, who owns it, and what to do next. If a retrofit conversation would be useful, we'd be happy to have one.