Organization Design in AI Transformation: Centralized or Distributed?

An AI operating model is the structure that defines how an organization distributes its AI capability, decision rights, and responsibilities across which teams and how. In its shortest form this is a governance question: do you gather AI work in a single center, spread it into business units, or build a middle path that balances the two? This decision has far more lasting consequences than the technical choices.

Organizations usually begin their AI transformation by asking "which model, which platform, which use case should we pick." Yet the decision that silently carries these projects to success or failure is often overlooked: who will do AI, where, and with what authority? The right AI operating model answers exactly this question. In this guide we compare the three approaches — centralized, distributed, and federated; we examine the centralized model's bottleneck, the distributed model's duplication cost, the federated middle path, the role of the center of excellence (CoE), the role split, and the maturity-based right answer, with a consultant's rigor. The aim is not to impose a single "correct" model on you, but to give you the decision guide that lets you make a conscious choice for your own organization.

Definition

AI Operating Model

The structure that defines how an organization distributes its AI capability, decision rights, and responsibilities across which teams and how. It has three basic forms: centralized (gathers all work in a single center of excellence), distributed (spreads authority into business units), and federated (pairs a small central core with local teams). The right form varies with the organization's maturity, scale, and culture; what is decisive is not the model's name but the clarity of the role split and decision rights.

Also known as: AI organization model, AI operating model, centralized distributed federated

An AI operating model is the structure that defines how AI work is organized inside a company — where authority, budget, and responsibility sit. More than an org chart, it is an operating model: it is the answer to who sets strategy, who builds the platform, who runs projects, who defines standards and security rules, who measures success. With the same technology and the same budget, two organizations that answer these questions differently get completely different outcomes.

This structure has three basic forms, and they form the backbone of this guide. In the centralized model all AI work is gathered in a single center of excellence. In the distributed model authority is spread into business units; each unit builds its own solution. The federated model is the balance of the two: a small, strong central core holds the standards and the platform, while local teams inside the business units run execution. This trio shows that "centralized or distributed" is actually not a dilemma but a spectrum.

The critical distinction is this: an AI operating model is not a technology decision but a governance decision. Which vector database you pick can change in six months; but who makes AI decisions, how the budget flows, and where responsibility sits determine the organization's transformation speed for years. So the model choice should be seen not as "a detail IT will handle" but as a strategic design that senior leadership must own. Reading this design alongside the framework we cover in what is digital maturity makes it easier to see which stage your organization is at.

AI transformation is mostly presented as a technology project; yet most failures stem not from technology but from organization. When the right AI operating model is not built, even the best model and the highest budget produce no value; because decisions clog, responsibility disperses, and projects stall at the pilot stage. Organization design is the invisible infrastructure of transformation: built right, everything flows; built wrong, every step catches on friction.

The first reason is decision speed. AI projects require many intersecting decisions: which data will we use, which risk will we accept, who will approve, where will the budget come from. If it is unclear who has the authority for these decisions, every project slows in an approval maze. A clear AI operating model removes this friction by defining decision rights in advance; an ambiguous structure turns every decision into a battle that is renegotiated each time.

The second reason is duplication and consistency. Without organization design, different teams redo the same work unknowingly; one builds a data pipeline while another writes the same thing from scratch, and a third adopts an entirely different standard. In the end the organization accumulates dozens of islands that do not talk to each other and sit at different quality and security levels. The third reason is talent utilization: AI expertise is scarce and expensive; how this scarce resource is positioned — in a single center, dispersed, or federated — determines how many times over that expertise produces value. In short, the AI operating model is the multiplier that produces very different outputs from the same inputs.

Now we come to the heart of the guide. Placing the three models side by side turns the "centralized or distributed" debate into a concrete choice. The table below compares the three AI operating models along the dimensions of advantage, risk, and suitable maturity; this is the reference frame that helps you make the first cut for your own organization.

Three AI operating models: advantage, risk, and suitable maturity

Model	Advantage	Risk	Suitable maturity
Centralized (single center of excellence)	Fast standards, consistency, focus of scarce expertise	Bottleneck, detachment from units, shadow solutions	Early / low maturity
Distributed (spread into business units)	Proximity, speed, fit to business context	Duplication cost, inconsistency, scattered governance	High maturity + strong coordination
Federated (central core + local teams)	Balance of standards and speed, scalable governance	Double work and coordination load if role limits are unclear	Mid / maturing organization

The most important point when reading this table is the last column: suitable maturity. Models are not "good" or "bad"; they are suitable or unsuitable for the stage an organization is at. While the centralized model is right for an early-stage organization, the same model suffocates a mature one; the distribution that is natural for a mature organization drives an early-stage one into chaos. So the right AI operating model is not an absolute but a contextual choice.

The second point to note is that these three models are not separated by sharp boundaries. Real organizations usually sit somewhere on a spectrum and slide along it over time. An organization starts with a centralized core, devolves authority to a few business units as they mature, and evolves into a federated structure without noticing. So managing the transition between models consciously is as important as choosing the model — we will return to this in later sections. Now let us deepen each model one by one, with its strengths and weaknesses.

In the centralized model all AI work is gathered under one roof, usually in a center of excellence. This team sets strategy, runs projects, defines standards, and operates the platform; business units convey their needs to this center and receive the output from it. For an organization at the start of its AI journey, this is often the most correct starting point — and the reasons are strong.

The advantages of the centralized model are clear in the early stage. First, the focus of scarce expertise: AI capability is expensive and rare; gathering it in a single team is far more effective than spreading and diluting it. Second, fast standards and consistency: a single center sets security, data, and quality rules quickly; the same approach is applied everywhere. Third, visibility and learning: because all projects are in one place, lessons are shared quickly, duplication is prevented, and senior leadership can track progress from a single point. So the centralized model is the concrete counterpart of the "first build a core" advice.

But the same strength turns into weakness as the organization grows. The biggest risk of the centralized model is the bottleneck. When all AI demand flows into a single center of excellence, the business units' demand inevitably exceeds the center's capacity; projects queue up, waiting times lengthen, and business units tire of the slowness. At this point two dangerous behaviors begin: either business units bypass the center and build their own shadow solutions (uncontrolled distribution), or they abandon AI initiatives altogether due to the loss of speed. Both poison the transformation.

The second risk is detachment. The central team is, by nature, far from the business unit's daily reality; it may not sufficiently know the context of a production line, a call center, or a legal unit. The result is solutions that are technically elegant but useless in the field. The centralized model therefore develops a scaling problem over time: to produce value it must get close to the business context, but its very centralization prevents that proximity. These two risks — bottleneck and detachment — explain why the centralized model is not enough in a maturing organization and why there is a need to move to a federated structure.

In the distributed model authority is taken from the center and spread into business units; each unit sets its own AI team, solutions, and priorities. The appeal of this model is strong: the business unit is the party closest to its own problem; when it applies AI with its own hands it gains speed, proximity, and fit to context. A marketing team builds its own campaign model, and a production team its own quality-control solution, knowing their own context and without waiting for anyone. This is a real advantage in mature organizations.

But the distributed model has a hidden cost, and its name is duplication cost. Without central coordination, each business unit redoes the same basic work unknowingly. One unit builds a data pipeline while another writes the same thing from scratch; one develops a security solution while another reinvents the wheel; each team develops its own model evaluation method, its own monitoring infrastructure, its own prompts separately. In the end the organization spends the same money repeatedly and accumulates scattered solution islands that do not talk to each other.

Duplication cost is not only wasted money; it also creates inconsistency and a governance gap. When different units adopt different security standards, different data-handling approaches, and different quality thresholds, an organization-wide consistency and auditability crisis begins. This is especially dangerous in terms of KVKK and risk: a personal-data rule loosely applied by one unit puts the whole organization at risk. We cover how to systematically assess risk in how to prepare an AI risk assessment document; in the distributed model, doing this assessment separately and consistently in each unit is particularly hard.

That is why the pure distributed model works only under very special conditions: when the AI culture is very mature, each unit's own data and engineering capacity is strong, and — critically — a strong central coordination mechanism exists. Moving to the distributed model without these conditions means buying speed while losing consistency and cost discipline. It is exactly at this point that the third path enters — the one that solves both the centralized model's bottleneck and the distributed model's duplication cost: the federated model.

The federated model combines the best sides of the centralized and distributed approaches and is today the most balanced choice in most mature organizations. Its basic idea is simple: a small, strong central core holds the standards, the platform, and governance; local teams in the business units run execution, solution development, and the business context. This way the consistency of centralization and the speed of distribution meet in the same structure. This structure is often also called "hub-and-spoke."

A metaphor helps explain how the federated model works. Think of the central core as a municipality that builds a city's infrastructure: it provides the roads, electricity, water, and common rules. The local teams in the business units are those who erect their own buildings on this infrastructure; each builds according to its own need, but all share the same common infrastructure and rules. No one builds their own power grid (duplication cost is prevented), but everyone designs their own building (proximity and speed are preserved). The federated model establishes exactly this balance.

In this model the center's role changes critically: it becomes not a "doer" but an "enabler." The center no longer runs every project itself; instead it provides the shared platform, operates the common component library, sets the standards and security rules, runs governance, and advises the business units. The business units, in turn, build their own solutions to their own problems on this backbone. When the federated model succeeds, the center shrinks but its impact grows: it stops being a bottleneck and turns into an accelerator.

Role split between the center and business units in the federated model

Responsibility	Central core	Business unit local team
Standards and security	Sets and audits	Applies
Platform and shared tools	Builds and operates	Uses and gives feedback
Business problem and priority	Advises	Owns and defines
Solution development	Provides scaffolding and templates	Builds and operates
Talent development	Builds program and curriculum	Trains its team
Quality measurement	Defines the framework	Applies and reports

The federated model has its risk too, and it must not be ignored: if role boundaries are unclear, the center and business units do the same work twice or responsibility gaps arise. The ambiguity of "is this my job or the center's?" is the federated structure's biggest enemy. So the federated model requires that the role split be defined clearly on paper and reviewed regularly. Built well, it offers a scalable balance that avoids both the centralized model's bottleneck and the distributed model's duplication cost.

At the heart of all three models stands, in different weights, the concept of a center of excellence (CoE). This term is used often but frequently misunderstood: a center of excellence is not "the team that does all AI work." A mature center of excellence does not replace the business units; it is a backbone that accelerates them, keeps them consistent, and makes them capable. This distinction is the most important line separating a well-built center of excellence from a poorly built one.

The core functions of a center of excellence fall under a few headings. First, standards and governance: setting and auditing organization-wide rules for data, security, model quality, and KVKK compliance. Second, platform and shared assets: providing the shared infrastructure, tools, and reusable components that no one has to build over and over. Third, talent and culture: building training programs, developing the teams in the business units, and spreading AI literacy. Fourth, advisory and acceleration: guiding business units at the start of projects so they avoid the traps.

The role of the center of excellence changes with the chosen AI operating model. In the centralized model the center of excellence is both the "doer" and the "definer"; all projects pass through it. In the federated model its role shifts: it no longer does most projects itself, but instead "enables" the business units to do them better. This shift is the sign of the center of excellence maturing — it starts to measure its success not by the number of projects it produces but by how capable it makes the business units. We detail how to build this enablement function systematically with an enterprise AI academy in building an enterprise AI academy.

The most common trap a center of excellence falls into is becoming an "ivory tower": a structure detached from the business unit, technically excellent but useless in the field. The way to prevent this is to keep the central team in constant contact with the business units and close to the business context through mechanisms like rotation and embedded work. A good center of excellence is one that serves rather than commands; that opens doors rather than guards them. Thinking about this human-centered way of working alongside the principles in our human-AI collaboration guide clarifies how both the team and the individual should be positioned.

Now we come to the most practical but most neglected dimension of the model debate: the role split. Seen with an experienced eye, what determines an AI organization's success is not the name of the chosen model but the clarity of the role split. The question "centralized or federated" is meaningless until the question "who owns what" is clearly answered. A good AI operating model is, more than a diagram of boxes and arrows, a clear map of responsibilities.

In a sound AI organization the following roles and responsibilities are clearly defined. Governance owner: the central authority that sets organization-wide standards, security and KVKK rules, and model quality thresholds. Platform owner: the engineering that builds and operates the shared infrastructure, tools, and reusable components. Business unit owner: the party that defines the real business problem AI will solve, the priority, and the success metric. Solution team: the team that actually builds the solution with product, data, and engineering capability. Compliance and security: the unit that assesses in terms of legal, KVKK, and risk. And the critical role forgotten in most organizations — the evaluation owner: the person continuously responsible for ensuring quality does not degrade over time.

Core roles in an AI organization, the decision each owns, and the gap when unassigned

Role	Decision it owns	Gap if unassigned
Governance owner	Standards, security, KVKK, quality threshold	Each unit applies different rules, risk grows
Platform owner	Shared infrastructure and tools	Everyone builds their own (duplication cost)
Business unit owner	Problem, priority, success metric	Technically correct but useless solution
Solution team	Building and operating the solution	Project never goes live
Compliance and security	Legal, KVKK, risk assessment	Hidden compliance breach, costly later fix
Evaluation owner	Continuously measuring quality	System silently degrades, no one notices

The most critical row in this table is the last. Assigning the evaluation responsibility to no one is the most common and most insidious mistake of AI organizations. The "everyone's job is no one's job" trap appears especially in governance, data currency, and quality measurement; these areas concern everyone, so no one owns them and they silently rot. The right role split binds each responsibility consciously to a name.

It should be noted that the role split can be applied in different forms depending on the size of the organization. In a small organization these roles can merge into a single person; in a large organization they are separate teams. What matters is not the number of roles but that each responsibility is consciously assigned to someone. The role split is not a document written once and forgotten but a living structure reorganized as maturity grows — which brings us to the next section, the maturity-based right answer.

The most important message of this guide is this: the right AI operating model is not single and fixed; it changes with the organization's maturity. Declaring a model absolutely "the best" is the most common mistake in consulting language. The truth is that the same organization needs different models at different maturity stages; and the real skill is managing not the model but the transition between models.

In the early stage — when the organization is new to AI, expertise is scarce, and the first wins are not yet shown — the centralized model is right. At this stage the priority is to gather scarce capability under one roof, set standards quickly, and produce visible first successes. Distribution is a disaster at this stage: spreading authority while no one yet fully knows what they are doing produces uncoordinated chaos. So nearly every organization should start with a centralized core.

In the maturing stage — when some business units have reached the capability to run their own projects and the demand coming to the center exceeds its capacity — the time to move to the federated model arrives. This transition is triggered by the first signs of the centralized model's bottleneck: lengthening wait times, business units' impatience, the appearance of shadow solutions. The federated model preserves both speed and consistency by devolving authority to capable business units while keeping standards at the center. For most organizations this is the longest and most productive stage.

Recommended AI operating model by maturity stage

Maturity stage	Symptom	Recommended model	Center's role
Beginning	Scarce expertise, first pilots, no standards	Centralized	Doer and definer
Growth	Demand rising, center bottlenecking	Centralized to federated transition	Doer + enabler
Mature	Business units capable, demand high	Federated	Enabler and governor
Advanced mature	Strong AI culture, every unit capable	Federated (light center) / controlled distributed	Standard-setter and coordinator

Moving to a pure distributed model should only be considered at the most advanced maturity and very carefully. Organizations with a very strong AI culture, where each business unit has its own data and engineering capacity, and which can build a strong coordination mechanism, can move to this model; but for most organizations a federated structure supported by a light center is safer and more productive than pure distribution. To see which stage your organization is at, you can use the assessment dimensions in what is digital maturity and then match them to this table.

"Should the AI team be centralized?" is the question organizations ask most often and are most confused about. The short answer: yes at the start, no permanently. But framing this question as a "yes/no" dilemma is wrong; the right question should be "which function should we keep centralized and which distributed." Because in a mature AI operating model some functions naturally stay centralized while others spread into business units.

As a general rule, functions that bring standardization and economies of scale should stay centralized: security and KVKK rules, the shared platform and infrastructure, model quality standards, the governance and talent-development framework. Having each unit do these separately is both waste and risk. In contrast, functions that require proximity to the business context and speed should spread into business units: problem definition, solution development, prioritization, and field application. A business unit knows its own problem better than a central team.

To clarify the centralized-or-distributed decision, several concrete criteria are examined. Scale: how many business units, how many concurrent projects are there? With few projects the center suffices; with many the center clogs. Maturity: are the business units capable enough to run their own solutions? If not, keep it centralized. Homogeneity: are the business units' needs similar or very different? Similar needs point to a centralized solution, very different needs to a distributed one. Risk profile: in high-risk (regulated) areas central control is safer. Assessing these criteria together turns "centralized or distributed" from an emotional preference into a data-based decision.

So far we have examined the three models, their strengths and weaknesses, the role split, and the maturity dimension. Now let us turn these into an applicable decision guide. The steps below offer a practical path you can follow when choosing the right AI operating model for your own organization; each step builds on the previous one.

How to

AI operating model selection guide

A step-by-step decision process to consciously choose the centralized, distributed, or federated model for your organization.

1. 1

   Assess your maturity honestly

   How widespread is expertise, how many business units can run their own project, are your standards settled? This sets your starting point.

2. 2

   Split functions into layers

   Separate the layers that should stay centralized (standards, security, platform) from those that should be distributed (problem definition, solution).

3. 3

   Write the role split

   Name the governance, platform, business unit, solution, compliance, and evaluation owners; consciously assign each responsibility to someone.

4. 4

   Clarify the funding model

   Will the center be funded by a central budget or by chargeback from the business units? This directly shapes behavior.

5. 5

   Choose the starting model and test with a pilot

   If you are early stage, start with a centralized core; if mature, build federated. Test on a small scope.

6. 6

   Define transition triggers in advance

   Write down from the start the signs (wait time, demand, shadow solutions) that determine when you move from centralized to federated.

7. 7

   Re-evaluate regularly

   As maturity grows, review the model and the role split; organization design is a living structure.

The most frequently skipped step when applying this guide is the fourth: the funding model. How the center is funded directly determines its behavior. If the center is funded entirely by a central budget, business units see it as "free" and demand grows uncontrolled; this feeds the bottleneck. In contrast, if business units repay the center's service through an internal charge, demand naturally gets prioritized and the center stays customer-focused. Funding is the invisible but decisive part of organization design.

The spirit of the decision guide is this: build not a one-off choice but an evolving design. The model that is right today may be wrong two years later; so defining transition triggers from the start and reviewing the model regularly is more valuable than searching for a single "perfect" model. For an organization design and transition plan tailored to your own organization, you can work with us within AI consulting and review corporate training options for your teams' competency.

Real organizations do not pick a model and freeze there; they evolve between models as maturity grows. Managing this evolution consciously is as important as choosing the right model; because transitions are the moments of the most friction and the most value loss. A poorly managed transition can regress the transformation by breaking a working structure and replacing it with one that has not yet settled.

The most common evolution path is the move from centralized to federated. This transition begins the moment the centralized model shows bottleneck signs. The critical thing is to build the transition not as a "break" but as a gradual "devolution of authority." The center first selects one or two of the most mature business units, teaches them the platform and the standards, then lets those units run their own solutions; when successful, it spreads this devolution to the other units. This gradual approach prevents both the center losing control and the business units being caught unprepared.

The most common mistake in transition is leaving behind the standard and the platform while devolving authority; this means unintentionally sliding from the federated model to a pure distributed one. In a correct transition the center shrinks but does not disappear: it moves from the "doer" role to the "enabler" role and continues to hold the standard and the backbone. Another common mistake is transitioning too early: devolving authority to business units that have not yet become capable leaves them alone with a responsibility they are not ready for, and quality drops.

A reverse evolution is also possible and sometimes necessary: re-centralizing an over-dispersed, uncontrolled structure. When an organization notices the duplication cost and inconsistency of uncoordinated distribution, it can regather scattered teams under a federated roof. This "re-centralization" is not a failure but a maturation move; the organization sees the limits of distribution and pulls back to a more balanced point. In both directions the point is the same: an AI operating model is not a static choice but a design that breathes with the organization.

Choosing the right model in theory is easy; the hard part is keeping it alive consistently in the field. Seen with an experienced eye, failed AI organizations collapse with similar mistakes. The most common ones can be listed as follows:

• Choosing the model independent of maturity: Building an early-stage organization as federated by saying "the federated model is best" means leaning on business unit capability that does not yet exist; the result is uncoordinated chaos. The model must be chosen to fit maturity.
• Starting without writing the role split: Drawing boxes and arrows and setting out without naming responsibilities creates the "everyone's job is no one's job" trap. Evaluation and governance in particular are left ownerless.
• Clinging to the centralized model: Continuing the centralized model — correct at the start — after the organization has grown makes the bottleneck permanent. The center is a core, not a fortress.
• Moving to pure distribution early: Spreading authority without building central coordination invites duplication cost and inconsistency; each unit reinvents the wheel.
• Neglecting the funding model: Presenting the center as an entirely free resource grows demand uncontrolled and feeds the bottleneck. Funding shapes behavior.
• Making the center of excellence an ivory tower: A center detached from the business unit, technically excellent but useless in the field, produces no value. Proximity must be preserved with a mechanism.
• Not managing the transition: Leaving the evolution between models to its own devices breaks a working structure. Transition triggers and gradual devolution must be planned from the start.

The most practical way to avoid these mistakes is to start small and grow by measuring. Instead of trying to reorganize the whole organization at once, testing the right model with one or two business units, learning, and then spreading lowers the risk. Organization design is not a perfect plan but a learning process.

No matter how well designed, an AI operating model does not stand without two invisible scaffolds: governance and funding. These two are invisible on the org chart but determine the model's real behavior. The same box-and-arrow chart, with a different governance and funding setup, produces completely different outcomes.

Governance is the institutionalized answer to "who decides what." Good governance clarifies decision rights, sets standards, and defines an escalation path: when a disagreement arises, who resolves it? AI-specific governance also defines clear gates for model approval, risk acceptance, and KVKK compliance. A governance gap is most dangerous in federated and distributed models; because as authority spreads, the question "who decides" blurs. So the more authority is distributed, the clearer governance must be — a counterintuitive but critical balance.

The funding model, in turn, silently shapes behavior. There are three basic approaches. Central budget: the center is funded entirely by the corporate budget; simple, but grows demand uncontrolled and weakens prioritization. Chargeback: business units pay for the center's service through an internal charge; demand naturally gets prioritized and the center stays customer-focused, but it can add bureaucracy. Hybrid: the core platform and governance are funded by the central budget, and project-based services by chargeback; for most mature organizations this is the most balanced option.

How governance and funding are set up directly determines the success of the chosen model. In regulated sectors this is even more critical; we cover how the approval layers and governance work with real observations in the field note on AI approval processes in regulated sectors. When choosing a model, designing not only the boxes but also this invisible scaffold is what saves the transformation from staying on paper.

The chosen AI operating model cannot be managed without being measured. Most organizations set up the model once and assume it "works"; yet an organization design, just like a product, must be tracked with indicators and improved. So what concretely measures the success of the operating model? Several dimensions stand out.

The first dimension is speed: the time it takes a business unit to get from idea to a working solution (lead time). When the centralized model bottlenecks this time lengthens; if the transition to the federated model is successful it shortens. The second dimension is reuse: how much the shared components, platform, and standards are shared across the organization. Low reuse is a sign of duplication cost and indicates that distribution is uncontrolled. The third dimension is consistency: whether different units' solutions meet the same security, quality, and KVKK standard.

The fourth dimension is capability spread: how many business units can produce value on their own, without depending on the center. This is the best indicator of the federated model's success in particular; because the federated model's aim is precisely to make the business units capable. The fifth dimension is satisfaction and demand fulfillment: are the business units satisfied with the service they get from the center, or are they bypassing it and building shadow solutions? The proliferation of shadow solutions is the clearest sign that the current model does not meet the business units' needs.

Indicators that measure the success of the AI operating model

Dimension	What it measures	Bad signal
Speed (lead time)	Time from idea to solution	Lengthening waits, queue
Reuse	Sharing of common components and platform	Every team builds from scratch
Consistency	Common security/quality/KVKK standard	Standard gaps between units
Capability spread	Number of units producing value independently	Everything still depends on the center
Satisfaction	Business unit's satisfaction with the center	Rise of shadow solutions

Tracking these indicators regularly answers the question "is our model working, or should it change" with evidence rather than guesswork. We detail the place of user adoption in this measurement in the field note on the factors that determine user adoption; because even the best organization model produces no value if the solutions are not adopted in the field. Measurement turns organization design from a one-off decision into a continuously improving structure.

To make the concepts concrete, let us follow a typical journey; this is not a single organization but the composite of a pattern seen many times in the field. A mid-to-large organization begins its AI work in the classic way: a few curious engineers, scattered pilots, no common standard. The first wins are exciting, but because each team works with its own method, quality and security are uneven. Management makes the right call and moves to the centralized model by building a center of excellence.

The centralized model works great in the first year. The center of excellence settles the standards, builds the shared platform, produces a few visible successes, and AI gains legitimacy in the organization. But in the second year demand explodes: every business unit wants AI, all of them come to the center of excellence. The center, with its team of fifteen, faces a queue of forty projects. Wait times stretch to months; impatient business units start building their own shadow solutions. The centralized model's bottleneck emerges precisely as a result of its own success.

Management reads this sign correctly and starts a gradual transition to the federated model. The two most mature business units are selected; the center of excellence teaches them the platform and the standards, places an embedded expert in each, and lets those units run their own solutions. The center shifts from the "doer" role to the "enabler" role: it keeps holding the standard, the security, and the platform but leaves execution to the business units. The role split is rewritten; each responsibility is bound to a name. Funding moves to a hybrid model: the core platform is funded by the central budget, project services by chargeback.

The result becomes visible at the end of a year. The time from idea to solution shortens; reuse rises because everyone shares the same platform; consistency is preserved because the standard is still at the center; and business units now produce their own value. The center of excellence shrinks but its impact grows. The lesson of this journey is clear: the right AI operating model is not a single choice but a design that evolves with maturity; and managing this evolution consciously is more decisive than choosing the model "right." When designing similar transformations, reading how careers and roles change alongside career and skill transformation in the age of AI completes the human dimension.

So far we have discussed the model mostly along the maturity axis; but the right AI operating model depends not only on maturity but also on the organization's scale and sector. Two organizations at the same maturity may need different models when they are of different sizes and in different sectors. So when applying the decision guide, you must read your own context together with these two additional dimensions.

Let us look from the scale angle. In a small or medium organization (SME), there are already few business units and scarce resources; here the centralized model is almost the only sensible option, because there are not enough business units and local capability to feed a federated structure. For a federated structure to be meaningful, there must be enough — and sufficiently mature — business units to devolve authority to. In a large organization, the centralized model quickly bottlenecks; the demand of dozens of business units does not fit into a single center, so the federated model is almost inevitable. In a multi-business holding the situation is even more layered: each company can build a small federated structure within itself, while the holding center becomes an upper-standard and sharing layer above these structures.

The sector also affects the decision markedly. In highly regulated sectors — banking, insurance, healthcare, public — central control and consistency are more valuable than speed; because a single compliance breach puts the whole organization at risk. In these sectors the center is kept stronger and more authoritative even in a federated model; the standard and approval layers are centralized. In contrast, in lightly regulated, fast-moving sectors — retail, media, technology — more room can be given to distribution; speed and freedom to experiment are more precious than tight central control. So the centralized-distributed balance shifts with the sector's risk appetite.

A caveat is needed: scale and sector do not replace maturity, they are assessed together with it. A highly regulated but low-maturity organization must first build its centralized core; a large but weak-culture organization must make its business units capable before moving to federated. The right model is found at the intersection of these three dimensions — maturity, scale, sector; a decision made by looking at a single dimension is usually incomplete.

An AI operating model is made not so much of boxes and arrows as of people. Even the most elegant org chart stays on paper without the culture and change management to keep it alive. So managing the human dynamics around the model matters as much as choosing it, and it determines the fate of the transformation. Experience shows that most failed organization designs collapse for cultural, not technical, reasons.

The most common cultural obstacle is the ownership conflict. When a central team is built, business units sometimes perceive it as a "takeover": "AI is now their job, not ours." In the move to federated, a resistance in the opposite direction arises: the center does not want to devolve authority; the fear of losing control delays the necessary devolution. In both cases the issue is not technical but about power and a sense of ownership. Good change management addresses these feelings openly: it concretely explains and shows that the center's role is not to "take over" but to "enable," and that devolving authority is not a "loss" but a "growth."

The second cultural dimension is how shadow solutions are met. When business units bypass the center and build their own solutions, instead of punishing this as a "violation" it should be read as a "signal": this behavior tells you the current model does not meet the business unit's need. Mature organizations do not ban shadow solutions; they make them visible, bring the good ones to the center, and use them to understand where the model is clogging. This is a learning rather than a punishing culture, and it fits the spirit of the federated model.

The third dimension is the visible ownership of senior leadership. An AI operating model is left in an authority vacuum when senior leadership does not openly own it; decisions are suspended, conflicts go unresolved. The model's success depends on an executive — in most organizations a digital or AI leader — visibly owning it, allocating resources, and arbitrating conflicts. We detail the effect of executive behavior on adoption in the field note on the factors that determine user adoption; the same principle holds for organization design: the executive's visible attention is the strongest signal that legitimizes the model.

The chosen model directly determines the talent the organization needs and where that talent is positioned. So the AI operating model and the talent strategy are inseparably bound; building one independent of the other is a common and costly mistake. Different models demand different skill profiles in different places.

In the centralized model talent concentrates at a single point: deep expertise is gathered in a strong center of excellence. This makes it easier to attract and retain scarce experts, because a strong technical community that learns from each other forms. But it has a risk: AI literacy does not develop in the business units; everyone waits for "the center to handle it" and the organization's broad base does not become capable. In the federated model, talent splits in two: platform and standard experts at the center, and application and solution capability in the business units. This model requires a broader talent base but at the same time makes the whole organization capable.

In the distributed model talent spreads most broadly; each unit builds its own team. This is the model that creates the highest total talent need, and exactly for this reason it works only in mature organizations where talent is abundant. Spreading a scarce talent pool across a distributed model leaves each unit alone with a semi-capable team and quality drops. This is why talent scarcity naturally steers most organizations toward the centralized or federated model.

A critical part of the talent strategy is internal development. Because AI talent is scarce and expensive in the market, organizations increasingly turn to developing their own talent from within. This becomes especially critical in the federated model: the capability to be spread into business units often cannot be hired one by one from outside; it must be developed systematically inside. We cover how to build this development function with an enterprise academy in building an enterprise AI academy and individual skill development in career and skill transformation in the age of AI. Without a right talent strategy, even the best organization model remains an empty shell.

Organization design and technology decisions are not independent of each other; on the contrary, one deeply shapes the other. Which platform, which tools, and which infrastructure you choose directly affect which AI operating model is possible. Thinking about these two dimensions separately often leads to decisions that undermine each other.

The most basic connection is established through the shared platform. The federated model works only if there is a shared platform the business units can use in common; without this platform, the structure you call "federated" turns in practice into distributed chaos. So if you have decided to move to a federated model, you must first build the shared platform that makes it possible — shared data infrastructure, a shared model-serving layer, shared security and monitoring tools. The platform is the physical backbone of the federated model; without it the model remains only a diagram.

The second connection is established through the technology choices between centralization and distribution. A central cloud platform is naturally inclined toward central control and standards; everything is managed from a single place. In contrast, a structure that lets each unit choose its own tools feeds distribution but makes consistency harder. So technology decisions are actually implicit organization decisions: when you choose a platform, you have unknowingly also chosen an organization model. Thinking about how on-premise or sovereign infrastructure choices affect this balance alongside the AI risk assessment document framework is important, especially in organizations sensitive to data residency.

The third connection is reusability and component libraries. The technical counterpart of preventing duplication cost is building a shared component library and template repository: ready pieces the business units can reuse instead of writing from scratch. This library is the most concrete mechanism that lowers the federated model's duplication cost. Designing technology and organization together — building the platform for the model and the model for the platform — is one of the most important disciplines that saves the transformation from staying on paper.

Theory is nice; but most organizations struggle with the question "where to start." It is helpful to describe the concrete first steps of building a new AI operating model from scratch through a typical first 90-day plan. This plan aims not to build a perfect structure but to lay the right foundation and grow by learning.

The first 30 days are devoted to listening and mapping. In this period, instead of hastily drawing an org chart, the priority is to understand the current state: what kind of AI work exists in which business units, which shadow solutions exist, where expertise is concentrated, what the most urgent business problems are? This mapping reveals the organization's real maturity and distribution; without it the chosen model rests not on reality but on an assumption. In the same period expectations are clarified with senior leadership and sponsorship is solidified.

The second 30 days are the period of building the core and setting the first standards. In an early-stage organization this means forming a small but strong central core; writing the most basic standards (security, data, KVKK); and choosing one or two high-value, low-risk pilots. The aim in this period is not to build a large structure but to secure the model's legitimacy by producing a quick win. The first visible success produces the political capital that opens the way for all subsequent steps.

The third 30 days lay the foundations of measurement and spread. In this period the first pilot's results are measured, lessons are drawn, and the role split is clarified for the first time: who will own what? Transition triggers are also defined from the start — the signs that determine when to move from centralized to federated. At the end of 90 days the organization has not a perfect organization but a solid start and a learning structure. This gradual, measuring, and learning approach is always more successful than a large and ambitious restructuring that stays on paper. For a 90-day organization plan tailored to your organization, you can work with us within AI consulting, review corporate training options for your teams, and deepen all concepts in the learning center.

Designing the right AI operating model is not enough; presenting it convincingly to senior leadership and the board to secure resources and authority is an inseparable part of the work. Because organization design changes budget, authority, and reporting lines, it necessarily requires high-level approval. Framing this presentation well determines whether even the best model comes to life.

The most common mistake when presenting to the board is explaining the topic in technical language. The board is interested not in which platform you chose but in what this model will bring the organization: faster value creation, controlled risk, scalable capability. So the presentation should be built on outcomes rather than model names. The sentence "we will start with a centralized core and evolve to federated" may look like a technical choice, but what the board actually needs to hear is this: "This structure will produce fast and consistent wins in the first year, then let us scale by making the business units capable."

An effective presentation has three basic elements. First, an honest picture of the current state: scattered pilots, duplication cost, inconsistent security — that is, a clear answer to "why must we change." Second, a concrete narrative of how the chosen model will solve these problems: which role sits with whom, how decisions will flow, how risk will be controlled. Third, how success will be measured: concrete indicators like time from idea to solution, reuse, capability spread. The board approves a measurable promise far more easily than a vague "transformation" rhetoric.

Another critical point is laying out the funding and authority request clearly. How the center will be funded (central budget, chargeback, or hybrid), who will have decision authority, and how conflicts will be resolved must be defined from the start. When the board approves an organization model it is actually approving a distribution of authority and budget; leaving this blurry creates authority gaps that are hard to fix later. In regulated sectors this presentation must also add a compliance and risk dimension; we cover how the approval layers work in the field note on AI approval processes in regulated sectors. A well-framed presentation turns organization design from a technical detail into a strategic management decision.

The core message we have stressed throughout this guide is that the AI operating model is not static but a living design. The concrete way to put this principle into practice is to build a ritual that reviews the model regularly — typically once a year. Just as a strategy or a budget is reviewed annually, the organization model must be put on the table periodically; otherwise the organization keeps advancing with a structure that was right two years ago but is now a drag.

The annual review answers a few basic questions. First: has our maturity changed? If the business units have become more capable than last year, the time for a shift from centralized to federated may have come. Second: is the current model showing bottleneck signs? Lengthening wait times, rising shadow solutions, and business units' dissatisfaction are signs that the model does not meet the need. Third: is the role split still valid? New roles may have arisen and some responsibilities may have been left ownerless; evaluation and governance ownership in particular must be checked regularly.

The most valuable input in the review is measurement data. If you have tracked the indicators we discussed earlier — speed, reuse, consistency, capability spread, satisfaction — over a year, the review rests on evidence rather than guesswork. This data gives an objective answer to "should we change the model." A review done without measurement turns into a debate the loudest person wins; a review done with measurement reveals the organization's real need.

Another function of the annual review is to consciously readjust the centralized-distributed balance. Perhaps some functions need to be kept more centralized and others distributed more; this fine-tuning can only be done through a regular review. What matters is to do this review not in a crisis moment but in a regular and predictable rhythm — because organization decisions made in a crisis are usually hasty and reactive. We cover how to tie capability development to this rhythm with an enterprise academy in building an enterprise AI academy. Regular review is the discipline that turns the AI operating model from a one-off decision into an asset that matures with the organization.

A practical question often asked when building an AI operating model is: should we build this structure entirely in-house, or draw on external consulting? The answer, like most good answers, is a matter of balance; neither full external dependence nor rediscovering everything internally is right. The right balance changes with the organization's maturity and which stage of the model it is in.

The moment external consulting is most valuable is the beginning. An early-stage organization often thinks for the first time about which model to choose, which traps to avoid, and what to do in which order; here an outside view that has seen similar journeys before shortens months of trial and error. External support is an accelerator especially in designing the role split, defining transition triggers, and setting the first standards. But there is a critical limit: external consulting should help build the model, not live it in the organization's place. Entrusting the model entirely to an outside team produces a structure that collapses when that team leaves.

Internal capability, in turn, is the only real source of sustainability. An organization model becomes lasting only when the organization's own people own and run it. So the aim of external support should be not to create dependence but to accelerate internal capability: the consultant should develop the organization's own team, transfer knowledge, and withdraw over time. The healthiest model is the structure where experience brought from outside meets ownership developed inside. This approach aligns with the core philosophy of the center of excellence too: speed from outside, continuity from inside.

As a practical rule, strategic and organization-specific decisions — which model we choose, how we manage risk, our priorities — should stay inside; while accelerating experience, a map of traps, and external benchmarking can be taken from consulting. When this balance is set well, the organization both saves time by learning from others' mistakes and builds a lasting structure that can stand on its own feet. To build an organization design tailored to your organization in a way that makes your internal team capable, you can consider AI consulting and corporate training options together.

Should the AI team be centralized or distributed?

There is no single right answer; it depends on the organization's maturity. In an organization at the start of its AI journey, the centralized model is more correct: it gathers scarce expertise under one roof, sets standards quickly, and makes early wins visible. As the organization matures, when the business units' demand exceeds the center's capacity, the centralized model turns into a bottleneck; at that point moving to a federated model is the most balanced choice. A pure distributed model only works in very mature organizations and only with strong central coordination. So the question should not be "centralized or distributed" but "which balance at which maturity."

What is a center of excellence (CoE)?

A center of excellence (CoE) is the central team that gathers an organization's AI expertise, standards, and shared assets under one roof. Its job is not to do all AI projects alone; it is to set standards, provide the shared platform, define governance and security rules, develop talent, and advise the business units. A mature center of excellence does not replace the business units; it acts as a backbone that accelerates them. In the federated model this center shrinks and shifts from a "doer" to an "enabler" role.

Which AI operating model is right for us?

The choice depends on three variables: maturity, scale, and culture. If you are new to AI, expertise is scarce, and you need to show early wins, the centralized model fits. If the business units have matured enough to run their own projects and demand exceeds the center's capacity, move to the federated model: standards at the center, execution in the units. Consider a pure distributed model only in organizations with a very mature AI culture and with strong coordination. Practical advice: start with a centralized core, deliberately evolve toward a federated structure as maturity grows, and redefine the role split at each stage.

What is the biggest risk of the centralized model?

The biggest risk of the centralized model is the bottleneck. When all AI demand flows into a single center of excellence, the center becomes unable to meet demand as the organization grows; projects queue up, business units tire of waiting, and over time they start bypassing the center and building their own shadow solutions. The second risk is detachment: when the center is not close enough to the business unit's real problem, it can produce technically correct but useless solutions. These two risks together turn a centralized model that was valuable at the start into a drag in a maturing organization; the solution is to devolve part of the authority to the business units in a federated way.

How is duplication cost prevented in the distributed model?

In the distributed model, when each business unit builds its own AI solution, the same infrastructure, the same integration, and the same governance work are redone repeatedly; this is called duplication cost. The way to prevent it is to move from pure distribution to a federated model: a shared platform, a common component library, shared standards, and a central knowledge-sharing mechanism. This way the business units keep the advantage of proximity and speed without reinventing the wheel each time. Building a community of practice and making successful solutions reusable are the two most practical mechanisms that lower duplication cost.

How is the role split defined in an AI organization?

The role split is the clear answer to "who owns what" and is more decisive than the model's name. In a clear AI operating model the following roles are defined: a central core that sets governance and standards; engineering that provides the platform and shared tools; the business unit owner that defines the business problem and priority; the product/data teams that build the solution; legal/security that assesses compliance, KVKK, and risk; and an evaluation owner that continuously measures quality. What matters is that each responsibility is consciously assigned to someone; the "everyone's job is no one's job" trap appears especially in the areas of governance and measurement.

In short, an AI operating model is the structure that defines how an organization distributes its AI capability, decision rights, and responsibilities, and it has three basic forms. The centralized model gathers all work in a single center of excellence; it gives fast standards but becomes a bottleneck as it grows. The distributed model spreads authority into business units; it brings proximity and speed but, without coordination, creates duplication cost and inconsistency. The federated model is the balance of the two: the center holds the standards, the platform, and governance, while the business units run execution; it is the most balanced choice for most mature organizations.

The most important message is this: the right AI operating model is not single and fixed; it changes with the organization's maturity. Most organizations should start with a centralized core, evolve consciously into a federated structure as they mature, and redefine the role split at each stage. More critical than the model's name is a clear definition — and regular measurement — of the role split, decision rights, governance, and funding. For the basic concepts, the what is digital maturity and building an enterprise AI academy guides offer good ground; for an organization design and transition plan tailored to your organization you can work with us within AI consulting, review corporate training options for your teams' competency, and deepen all concepts in the learning center.