Introduction: Why Enterprise AI Needs an Operating Model
A team can add AI to customer support, marketing, or engineering in a few weeks. Making that AI deliver dependable value across an enterprise takes much longer.
McKinsey's 2026 survey of 1,719 respondents found that 44% report enterprise scaling, while only 37% report any EBIT impact.
Technology alone doesn't close that gap. New tools can't fix unclear ownership, weak processes, or messy data.
An AI operating model defines how people, processes, data, technology, funding, decision rights, and risk controls work together. That matters more as agents move into business workflows.
This article covers the main design choices, the common archetypes, governance requirements, and a practical path to accountable AI execution.
Key Takeaways
- An AI operating model is an accountability system, not an org chart or a tool stack.
- Maturity, data quality, talent, and regulatory exposure should drive the choice of model.
- Centralized, federated, and embedded models trade speed against control differently.
- Boards should demand named owners, escalation thresholds, and outcome-based metrics.
What Is an AI Operating Model?
An AI operating model is how an enterprise organizes people, processes, data, platforms, funding, governance, and accountability to identify, build, deploy, monitor, and retire AI capabilities. McKinsey describes it as the decisions governing staffing, structure, technology development, compliance, and how AI is used and measured.
How It Differs from Strategy and an "AI Operating System"
Strategy says where the organization wants to create value. The operating model says who decides and how work gets done.
"AI-based operating system" is a separate term. McKinsey uses "AI operating system" for a management layer that maintains context and recommends or executes decisions. No universal definition exists, and an enterprise operating model is primarily an organizational and governance design.
Why Traditional IT Models Fall Short
AI changes how work is assigned, decisions are made, and outcomes are owned. That requires:
- Continuous product ownership instead of project handoffs
- Ongoing monitoring, feedback, and improvement
- Domain experts, technology, security, legal, risk, and operations working together rather than passing tickets
The Core Design Choices in an Enterprise AI Operating Model
Define the Mandate First
Leaders should state whether the near-term emphasis is efficiency, growth, customer experience, resilience, risk reduction, or a deliberate mix. Without that, priorities compete, pilots stay disconnected, and nobody can say which use cases deserve funding.
A useful start: pick three to five business goals for the next 12 to 18 months and give each an executive owner.
Decision Rights and Roles
Spell out who approves use cases, selects models or vendors, owns production outcomes, accepts risk, and authorizes exceptions. A short decision-rights map does this well when it covers:
- Risk acceptance
- Exceptions
- Budget tradeoffs
- Incident severity
- Critical-supplier go/no-go decisions
Those rights must stay clear even when development is spread across functions.
Roles to define:
- Executive sponsors and use-case owners
- Data and technology teams
- Security, privacy, legal, and compliance reviewers
- Domain experts and operators who monitor outputs
Employees will increasingly supervise AI agents, so validation, exception handling, and escalation belong in role design. Microsoft's 2025 survey of 31,000 workers found leaders expect teams to train agents (41%) and manage agents (36%) within five years. Those are expectations, not outcomes. Onboarding and career paths will need to change as routine execution is automated.
Shared Foundations and Guardrails
Common foundations include:
- Data ownership and access controls
- Model inventory and reusable components
- Evaluation and logging
- Incident response and retirement
Self-service tools need approved patterns and observable controls, not unrestricted experimentation. "Teams use what they want" is a red flag.
Microsoft's internal approach is instructive: employee-built agents follow risk-based policies, enter a registry, and are monitored through telemetry.
Funding and Measurement
Separate budgets for:
- Experimentation
- Production operations
- Platform capabilities
- Risk controls
- Workforce enablement
Judge investments on business outcomes, adoption, reliability, risk exposure, and total cost of ownership, not the number of pilots launched.
BCG's 2026 CEO survey shows planned AI spending rising from 0.8% to about 1.7% of revenue. Its 10-20-70 principle puts 10% on algorithms, 20% on technology, and 70% on people and processes. Treat that as a transformation principle, not a universal budget.
AI Operating Model Archetypes: Centralized, Federated, or Embedded?
No single five-part taxonomy is widely accepted as authoritative. The labels below are practical shorthand drawn from common patterns enterprises already use.
| Archetype | Structure and strength | Main risk | Fits when |
|---|---|---|---|
| Siloed experimentation | Teams explore freely; fast learning | Fragmented, ungoverned adoption | Very early maturity, low-risk use only |
| Centralized center of excellence | One team sets standards and prioritizes | Queues, weak business ownership | Scarce talent, high regulatory exposure |
| Hub-and-spoke (federated) | Central standards, domain-owned delivery | Needs strong monitoring and access controls | Talent exists in business units |
| Center for acceleration | Central team speeds up business delivery | Overlaps with hub-and-spoke roles | Treat as a variant of the hub |
| Embedded business ownership | Domain teams own capabilities | Highest governance burden | Mature data, controls, and accountability |
These aren't permanent stages. One enterprise can use different models by risk tier, business unit, or use case.
Control Versus Proximity
Centralized models concentrate talent and standardize controls but create queues. Distributed models improve relevance and speed but demand stronger standards, identity and access controls, and accountability.
McKinsey's 2025 survey shows the mix in practice: risk and compliance is fully centralized at 57% of organizations, while AI adoption is hybrid at 54%. Those are function-level arrangements, not overall archetype choices.
Risk should set the level of central review. AI that helps staff write faster differs from AI that touches customers, makes decisions, or changes pricing. A simple guardrail scheme: green-zone ideas move fast, yellow-zone ideas need a brief consultation, and red-zone ideas get formal review.
In banking, the Federal Reserve's SR 26-2 guidance ties greater model materiality to more rigorous oversight. It principally covers traditional quantitative models, so don't apply it wholesale to generative or agentic AI.
Selection Guide
| Signal | Leans toward |
|---|---|
| Scarce talent, weak data, high regulatory exposure | Centralize |
| Capable domain teams, shared platform, defined standards | Federate |
| Mature controls, strong monitoring, accountable business leaders | Embed |
Treat the choice as a hypothesis to test and refresh, not an irreversible reorganization.
Governance, Decision Rights, and Accountability for Enterprise AI
Make Governance Operational
Principles and policy documents don't govern anything on their own. Controls must run across the full lifecycle:
- Intake and risk classification
- Approval, testing, and deployment
- Monitoring and incident response
- Change management and retirement
Useful anchors include the voluntary NIST AI RMF (Govern, Map, Measure, Manage) and ISO/IEC 42001 for management systems. Laws vary by state and sector, so none of these replaces legal advice. Colorado's automated decision-making requirements, for example, take effect January 1, 2027.
Board and Management Roles
Boards and committees should get plain-English reporting on material AI risks, significant changes, unresolved decisions, and whether management is within risk appetite. Management owns execution, controls, remediation, and escalation, with a named owner for each material AI capability.
The evidence on boards is mixed. Deloitte's 2025 sample found 20% of boards assign oversight to the full board only, and 16% have no express responsibility.
A stable dashboard should show trend, exposure, control health, and decisions required, not technical trivia.
Human Oversight and Escalation
Set rules in advance:
- Mandatory human approval for irreversible or high-impact actions
- Sampling-based review for lower-risk outputs
- Stop, defer, or escalate triggers when confidence, data, or authority falls short
Keep decision logs, a system inventory, audit trails, and defined override authority. OWASP's agent guidance calls for least-privilege tools, approval for high-impact actions, and rollback.
Third-Party AI
Cover these areas in contracts and reviews:
- Due diligence and data-use rights
- Confidentiality and model-change terms
- Service dependencies and concentration risk
- Incident notification and exit planning
A "free" AI tool can carry serious risk outside your walls.
In July 2025, Fortune reported that Replit's coding agent deleted production data during a code freeze. It's one documented authorization-boundary failure, not proof that every agent is unsafe, but it shows why permissions must be enforced.
Metrics and Review
Track metrics that force real choices:
- Time from approved use case to controlled deployment
- Adoption and outcome realization
- Reliability, exception volume, and control completion
- Incidents, unresolved risks, and reuse
Use them to drive decisions and escalation, not to reward launch volume.
A Practical Roadmap for Building and Refreshing the Model
Baseline and Priorities
Start with a full inventory:
- AI applications and shadow AI
- Vendors, data sources, models, and agents
- Owners and control gaps
IBM's 2025 breach study found 20% of breached organizations reported shadow-AI incidents.
Then assess each priority use case for value, risk, data readiness, dependencies, workforce impact, and failure consequences. MIT CISR's four stages offer a diagnostic path:
- Experiment and Prepare
- Build Pilots and Capabilities
- Develop AI Ways of Working
- Become AI Future Ready
No framework guarantees you advance on a fixed schedule.
Minimum Viable Governance
For each priority capability, assign clear roles:
- Executive sponsor
- Accountable business owner
- Technical owner
- Security or risk reviewer
- Escalation authority
Set approval criteria, risk tiers, oversight rules, reporting, and incident pathways before you scale.
A 90-Day Plan
- Days 1-30: confirm priorities, decision rights, inventory, risk classes, and baseline measures.
- Days 31-60: pilot one or two controlled workflows, validate human-AI handoffs, test monitoring, and log exceptions.
- Days 61-90: review outcomes with executives and the relevant board or risk stakeholders, fix gaps, and decide to scale, pause, or redesign.
Treat this as a practical sequence. Outcomes depend on your starting point and how tightly you run the pilots.
When Temporary Leadership Helps
New leadership, M&A, an incident, modernization, or a fast-growing AI portfolio can leave ownership unclear. A board advisor or interim leader can stabilize things. BTG's data shows interim-leader demand up 310% since 2020.
Tyson Martin offers this kind of support: board advisory and interim or fractional CISO leadership focused on clarifying decision rights, improving executive reporting, and building a measurable plan with owners. His AI Governance Starter Pack, for example, is a 30-day sprint covering risk assessment, decision-rights mapping, and a one-page policy.
Refresh Cadence
Review the model when AI capabilities, regulations, vendors, strategy, risk appetite, workforce design, or incidents change. A model may move from centralized experimentation to federated ownership, or tighten temporarily after a major incident.
Executive Checklist
Ask of every material AI capability:
- Is it owned?
- Is it monitored and governed?
- Is it funded?
- Is it tied to a business outcome?
The goal is not maximum deployment. Aim for fewer surprises, faster escalation, cleaner communication, stable metrics, and decisions leaders can defend.
Frequently Asked Questions
What is an AI-based operating system?
It usually means a software or workflow layer, or in McKinsey's usage a management layer that supports decisions. An enterprise AI operating model is different: it governs people, processes, technology, data, and accountability.
What is the 30% rule of AI?
No standard definition exists. People cite McKinsey's modeled 30% of U.S. work hours by 2030, Nadella's 20-30% of Microsoft code, or a Forbes 70/30 split. Ask which claim they mean.
What are the 7 types of AI?
IBM groups AI by capability (narrow, general, super) and functionality (reactive machines, limited memory, theory of mind, self-aware). Several are theoretical. For operating model design, use-case risk and accountability matter more.
How do you choose the right AI operating model for an enterprise?
Weigh AI maturity, talent distribution, data readiness, use-case risk, governance capability, and business-unit ownership. Then treat the choice as a hypothesis and revisit it as conditions change.
Who should own AI governance in an enterprise?
Governance is shared across the board, executives, business owners, technology, security, legal, risk, and operations. Each material AI capability still needs one clearly accountable owner.
