AI has moved beyond experimentation. For most organisations, the question is no longer whether to invest in AI, but how to organise for it. And that question is more consequential than it first appears.
Many teams assume that hiring a few data scientists or deploying a model is the hard part. In reality, the operating model, how AI capabilities are structured, integrated, and governed, often determines whether initiatives scale or stall.
Two dominant patterns have emerged: centralised AI teams and embedded AI squads. Both can work. Both can fail. The difference lies in how well they align with organisational maturity, product complexity, and decision-making structures.
This is not a purely technical choice. It is an organisational design decision that shapes speed, quality, ownership, and ultimately business impact.
The Centralised AI Model
In a centralised model, AI capabilities sit within a dedicated team, often referred to as a Centre of Excellence (CoE). This team owns model development, tooling, standards, and sometimes even delivery.
At first glance, this structure offers clarity. It consolidates scarce expertise, ensures consistency, and creates a focal point for innovation.
There are good reasons why many organisations start here.
A centralised team can:
- Establish shared infrastructure and tooling
- Define best practices for model development and deployment
- Ensure governance, compliance, and risk controls
- Build reusable components across business units
For organisations early in their AI journey, this structure reduces fragmentation. Instead of multiple teams experimenting in isolation, the CoE provides direction and coherence.
Research from McKinsey suggests that organisations with stronger governance, operating discipline, and adoption practices are more likely to capture measurable value from AI.
However, centralisation introduces its own constraints.
As demand grows, the central team often becomes a bottleneck. Business units queue for access. Prioritisation becomes political. Delivery slows.
More importantly, distance from domain context becomes a limiting factor. AI solutions are rarely generic. They depend heavily on local knowledge, workflows, and user behaviour. A central team, removed from day-to-day operations, may struggle to capture this nuance.
This often leads to a familiar pattern: technically sound models that fail to integrate into real workflows. IBM’s AI adoption research shows that scaling AI depends not only on the technology itself, but also on deployment, operational execution, and broader organisational adoption.
Embedded AI Squads
The embedded model takes the opposite approach. Instead of a single central team, AI capabilities are distributed across cross-functional squads, aligned to products or business domains.
In this structure, data scientists, ML engineers, and AI specialists sit alongside product managers, engineers, and domain experts.
The impact is immediate.
AI becomes part of the product, not a separate function.
Embedded squads:
- Work directly within business contexts
- Iterate faster due to proximity to users and stakeholders
- Align AI development with product roadmaps
- Take end-to-end ownership of outcomes
This model mirrors broader shifts in software engineering, where cross-functional teams replaced siloed departments to improve delivery speed and accountability.
When applied to AI, the benefits are similar, but amplified.
AI systems are inherently iterative. They require constant feedback, tuning, and adaptation. Being close to the problem space allows teams to refine models based on real-world signals, not abstract assumptions.
BCG argues that organisations create stronger AI outcomes when AI is tied closely to business transformation and critical functions, rather than treated as a disconnected specialist activity.
However, embedded models introduce new risks. Without coordination, fragmentation emerges quickly:
- Multiple teams may duplicate work
- Tooling and infrastructure become inconsistent
- Governance and compliance controls weaken
- Knowledge sharing becomes informal and unreliable
There is also a talent challenge. AI expertise is still relatively scarce. Distributing it across many squads can dilute impact, especially in organisations without a strong baseline capability.
In other words, embedded teams optimise for speed and relevance, but can struggle with consistency and scale.
Hybrid Operating Models
Most organisations that move beyond early experimentation eventually converge on a hybrid model. This is not a compromise. It is a recognition that both centralisation and embedding solve different problems.
In a hybrid model:
- A central AI function defines standards, tooling, and governance
- Embedded squads own and apply AI within specific domains
- Shared platforms enable reuse without enforcing rigidity
The central team evolves from being a delivery function to a capability enabler.
Its responsibilities typically include:
- Maintaining core infrastructure (data platforms, model serving, monitoring)
- Defining best practices and architectural patterns
- Ensuring regulatory compliance and risk management
- Supporting knowledge sharing across teams
Meanwhile, embedded squads:
- Own use cases and business outcomes
- Build and iterate on models within their domain
- Integrate AI into products and workflows
- Provide feedback to improve shared systems
This model reflects a broader shift in how organisations treat AI, not as a standalone function, but as a distributed capability supported by central foundations.
Accenture’s research on AI maturity suggests that leading organisations combine strong foundations in strategy, talent, governance, and technology, rather than relying on isolated AI efforts.
The success of this model depends on clarity.
If the boundary between central and embedded teams is unclear, friction emerges. If it is too rigid, innovation slows. The balance must be intentional and continuously adjusted.
Choosing Based on Maturity
There is no universally “correct” structure. The right model depends on where an organisation stands in its AI journey. A useful way to think about this is through stages of maturity.
Early Stage (Exploration)
At this stage, organisations are experimenting with AI, often with limited expertise and unclear use cases.
A centralised model is typically the most effective starting point. It allows:
- Concentration of scarce skills
- Faster capability building
- Controlled experimentation
- Establishment of foundational infrastructure
The goal here is not scale, but learning.
Mid Stage (Expansion)
As successful use cases emerge, demand for AI grows across the organisation.
This is where centralised models begin to strain. The shift towards embedded squads starts to make sense.
Organisations often:
- Embed AI specialists into key product or business teams
- Maintain a central function for governance and support
- Begin building reusable components and platforms
The focus shifts from experimentation to repeatability.
Advanced Stage (Scale)
At higher levels of maturity, AI is no longer a separate initiative. It becomes part of how the organisation operates. Since everyone is efficient in AI, ownership gets de-centralised.
Here, hybrid models are essential.
Organisations:
- Treat AI as a core capability embedded across functions
- Invest heavily in shared infrastructure and platforms
- Establish strong governance integrated with existing risk frameworks
- Enable autonomous teams while maintaining alignment
At this stage, the challenge is not adoption, but coordination and optimisation.
Common Pitfalls Across Models
Regardless of structure, several recurring issues tend to undermine AI team effectiveness.
1. Treating AI as a standalone function
When AI is isolated from product and business teams, it struggles to deliver meaningful impact.
2. Over-centralisation
Central teams becoming bottlenecks, slowing down delivery and disconnecting from real-world needs.
3. Under-governed decentralisation
Embedded teams operating without shared standards, leading to fragmentation and risk exposure.
4. Lack of platform thinking
Rebuilding infrastructure repeatedly instead of investing in reusable systems.
5. Misalignment with business priorities
AI initiatives driven by technical curiosity rather than clear business outcomes.
These issues are not structural alone. They reflect deeper organisational dynamics, incentives, communication, and leadership alignment.
From Structure to Capability
It is tempting to frame this discussion as a binary choice between centralised and embedded models. In reality, the more important question is different:
How does AI become a sustainable organisational capability?
Structure is only one part of the answer. Successful organisations:
- Align AI initiatives with strategic priorities
- Invest in both talent and infrastructure
- Integrate AI governance into existing frameworks
- Foster collaboration between technical and domain teams
The operating model should support these goals, not constrain them.
What becomes clear over time is that AI does not scale through isolated excellence. It scales through coordinated systems, shared understanding, and continuous adaptation.
From Structure to Scalable AI Capability
The debate between centralised AI teams and embedded squads is often framed as a choice. In practice, it is a progression.
Centralisation builds foundations.
Embedding drives relevance and speed.
Hybrid models enable scale.
The real challenge is not choosing one model, but knowing when and how to evolve.
Organisations that get this right treat AI not as a project, but as an organisational design problem. They recognise that structure shapes behaviour, and behaviour determines outcomes.
In the end, the question is simple, but demanding: Is your AI capability organised to deliver value where it matters most?