Gartner has given a name to something production teams worked out the hard way. Here is what composite AI actually is, why it fails more often than it succeeds, and the one layer that decides which way it goes.
There is a particular kind of relief that comes when the industry finally names the thing you have been building for two years without a label for it.
Composite AI is that name. The idea is simple enough: instead of routing every problem through one large language model and hoping it generalises, you combine several techniques into a single architecture and put each one where it actually performs. A language model for interpretation. A machine learning model for prediction. A rules engine for the things that must return the same answer every time. A knowledge graph for the relationships. Deterministic software for the repetitive mechanical work that never needed intelligence in the first place.
Gartner projects that annual AI services spending will reach $516 billion by 2029, with composite AI accounting for roughly two thirds of it. That is a striking forecast for an approach that barely registers as a line item in most 2026 budget breakdowns.
We think the forecast is directionally right and slightly beside the point. The interesting question is not whether enterprises will move to composite architectures. They will, because the alternative does not survive contact with a regulated business process. The interesting question is what happens next, once an organisation has twelve specialised components, four agents, three model vendors and a rules engine, and nobody can answer the question "what did this system decide last Tuesday, and why".
That question is where most composite AI programmes quietly fall over. It is also the layer we build.
What composite AI actually is, in plain terms
A large language model is an extraordinary language interface and a mediocre decision engine.
It is genuinely good at interpretation, summarisation, drafting, classification and conversation. It is genuinely bad at following a rule the same way ten thousand times, at holding state across a long process, at guaranteeing a numeric value matches its source, and at explaining, in a form an auditor will accept, exactly why it did what it did.
Composite AI keeps the model where it is strong and surrounds it with components that cover the gaps. In practice, a composite system draws on some combination of the following:
- Machine learning for pattern detection and prediction from historical data, such as fraud scoring, demand forecasting or churn.
- Natural language processing so people can interact with the system in ordinary words rather than forms and dropdowns.
- Knowledge graphs to hold facts and the relationships between them, so the system reasons from verified structure rather than inference.
- Rule-based systems for fixed, human-defined logic that returns an identical result every time and can be audited line by line.
- Optimisation algorithms for routing, scheduling, pricing and anything else that means choosing the best option against real constraints.
- Symbolic reasoning over explicitly encoded logic, where traceability matters more than flexibility.
- Computer vision for reading images, documents and video, from a defect on a line to a signature on a scanned form.
- Deterministic computer use for the unglamorous mechanical work of moving data between systems that were never meant to talk to each other.
What comes out of the far end is not a chat window. It is operational output: dashboards, agents acting inside other systems, recommendations for a human to approve, automated decisions inside defined boundaries, and alerts for the cases that need a person.
We have a phrase for this internally. We call it the determinism spectrum. Production-grade AI does not sit at one end of that spectrum, it places each task at the right point along it. Some work should be non-deterministic because judgement is the point. Some work should be rigidly deterministic because consistency is the point. Most real business processes contain both, tangled together, and the design work is separating them.
That is the whole of composite AI, honestly. The naming is new. The practice is not.
Our first opinion: this is a fit decision, not a maturity level
Here is where we part company with a lot of the commentary.
Composite AI is being sold in some quarters as a rung on a ladder. Single model at the bottom, multi-model in the middle, sprawling composite architecture with eight technique types at the top, and a strong implication that if you are not climbing you are falling behind.
We do not accept that framing, and we say so to clients regularly, including when it costs us scope.
Architecture is a match, not a maturity score. A single well-designed agent running one process end to end, with a clean data foundation underneath it and a human approval point where accountability actually sits, is not a less mature system than a nine-agent mesh. It is often the correct answer. We have seen more value destroyed by premature architectural ambition than by conservatism.
The list of eight techniques above is not a shopping list. Most mid-market organisations we work with need three of them. Adding the other five because a reference architecture said so buys you cost, latency, integration surface and failure modes, and it buys you them permanently.
The useful progression is not technique count. It is scope. Task, then process, then function, then across the business, then genuinely orchestrated at enterprise level. You earn each step by proving the last one in production, not by drawing it.
The problem composition creates
Composition solves the capability problem and immediately creates a control problem.
Once you have several specialised components, multiple agents, a couple of rule layers and more than one model vendor, you have built something that no single person understands end to end. Each part may be individually excellent. Collectively, they are a set of disconnected decisions with no shared memory and no common record.
This is agent sprawl, and it arrives faster than anyone expects. It usually starts with success. A team ships something that works, another team copies the pattern, a third buys a point solution with an agent baked into it, and within two quarters the organisation has AI running in eleven places with no consolidated view of what any of it is doing, what it costs, or what happens when a model provider changes a default.
The symptoms are consistent. Nobody can produce an audit trail across the whole flow. Cost is visible per vendor invoice but not per process. Two components hold contradictory versions of the same customer fact. A model upgrade changes behaviour in a workflow three teams away from anyone who noticed. And when something goes wrong, the investigation is archaeology rather than a query.
IBM's research puts a number on the other side of this: organisations that build a genuine orchestration layer are around thirteen times more likely to scale AI successfully, and see materially fewer AI-related incidents. We would treat the precise multiple with the usual caution applied to vendor-adjacent research. The direction is not in doubt, because it matches everything we see in delivery.
The control plane is the product
The layer that fixes this is what we call the AI Control Plane, and it is the single most important architectural decision an organisation will make about AI in the next three years.
The AICP sits between people, systems, data, workflows, agents and models. It is not a model. It is not a chatbot builder. It is the operating layer above the models, and it is what turns a collection of clever components into a controllable operating capability.
Concretely, the control plane decides:
- Who is asking, and what they are permitted to access.
- Whether non-deterministic AI is needed at all for this step.
- Whether a deterministic rule, a lookup or a template would do the job better and cheaper.
- Which agent should handle the work.
- What context that agent needs, and only that context.
- Which model is the right tool for this specific task.
- What runs automatically, and what requires a human.
Every one of those is a governance decision disguised as an engineering decision. Made deliberately, in one place, they compose into a system you can stand behind in front of a regulator. Made implicitly, scattered across eleven codebases and four vendor consoles, they compose into an incident report.
We build the AICP on OneReach.ai's Generative Studio X. It is the platform that fits the work: agent design, workflow control, model routing, integration and governance in one place, recognised as a leader simultaneously by Gartner, Forrester and IDC. We are not commercially obliged to use it, and where a client's environment requires a different orchestration engine, the same architecture and the same standards apply. The platform is a choice. The layer is not.
For UK organisations there is a second consideration that rarely appears in American commentary on this subject. Composition multiplies the number of places your data can go. The control plane is where you constrain that, which is why we run client systems in a Private Dedicated Environment: single tenant, UK data residency, dedicated controls, with the boundary defined at the architecture rather than promised in a contract.
Our second opinion: composition without a data foundation is a more expensive way to be wrong
Agents are only as good as the data they reason on.
That sentence is unfashionable because it is not new, and because it points at work that is slow, political and impossible to demo. It is also the reason most composite AI programmes underdeliver. You can assemble a beautiful architecture of specialised components, orchestrate it immaculately, and produce confident, fast, well-governed nonsense, because the customer record it reasons from has been wrong since a migration in 2019.
Trusted, governed, contextual data is the precondition for trusted AI. Not a parallel workstream. A precondition.
In practice this means treating data products with the same discipline as agents: a named owner, a quality standard, a freshness tier set deliberately per use case rather than defaulted to real time, and a kill criterion defined before anything ships. It means identity and access at the data layer, lineage across pipelines, and quality fixed at source rather than patched downstream by a prompt.
It also means saying no occasionally. If a use case depends on a dataset nobody owns and nobody trusts, the honest answer is that the data work comes first. We would rather have that conversation in week two than in month seven.
The cost argument, which arrives later than it should
There is one point in the current composite AI guidance that we think is badly underweighted, and it is the financial one.
In a composite architecture, tokens become a unit of operating cost. Not a line on an innovation budget. A running cost that scales with usage, and therefore scales with success. The better your adoption, the larger your bill, which is a very uncomfortable discovery to make eighteen months in, at the exact moment you are trying to justify expansion.
Gartner's own guidance for architects includes establishing AI-specific financial controls early, with adaptive spending guardrails embedded in the architecture itself, so that components exceeding consumption thresholds trigger alerts or get blocked automatically. We agree, and we would go further.
Cost attribution has to be architectural. If you cannot answer "what did this process cost us to run last month, and what did it return", you do not have an AI investment, you have an AI subscription. That means telemetry at the control plane, attribution mapped to processes rather than vendors, and the numbers visible to the people who sign for them. We build this into the RiverAI Portal for exactly this reason: leadership should be able to see what is running, what it costs, and what it returns, without asking anyone to build a report.
Model control and cost discipline are what will separate the organisations that scale AI from the ones that stall on the cost of deployment. That gap is going to open up publicly over the next two years and it will be uncomfortable to watch.
Our third opinion: adoption decides all of it, and almost nobody plans for it
You will read a great deal about composite AI architecture. You will read very little about whether anyone uses it.
We think this is the biggest blind spot in the entire category. If an AI agent does not get used, it failed. Architecture, governance, model selection, ROI on a slide: none of it survives the moment people quietly route around the thing you built.
Adoption is the single biggest predictor of whether AI delivers value. Bigger than model choice. Bigger than architecture. Bigger than budget. A use case running at eighty per cent adoption with mediocre design will comfortably outperform one running at twenty per cent adoption with brilliant design, and we have watched both happen.
Three things drive it, and all three are design problems rather than communications problems:
- The work has to get easier. Not different. Easier. One extra click, one moment of additional cognitive load, one reason to second-guess the output, and people go back to the old way permanently. The bar is not "better than nothing". The bar is "easier than what they did yesterday".
- Trust has to be earned in use. People trust systems they can see, challenge and override. Black boxes do not survive contact with a sceptical workforce, and a composite architecture gives you a real advantage here, because it has seams you can expose. Show the working. Accept the correction. Feed it back.
- Change has to be run, not announced. Comms, training, named champions, feedback loops, active course correction through the first weeks of live use, owned by somebody accountable. Adoption is not the final phase of a project. It is what the project is for.
This is also where human-in-the-loop belongs in the conversation. It is a design choice, placed deliberately where accountability sits, not sprinkled defensively across a workflow to make a governance slide look responsible. Approval points in the wrong places are the fastest way to make an AI system slower than the process it replaced.
What it looks like when it is real
We would rather show than argue, so here is one.
Mortgage 1st is a national award-winning mortgage and protection brokerage. Compliance-heavy, process-heavy, high volume, exactly the profile where a single general-purpose model on its own is not fit for the work.
We ran an AI opportunity workshop in November. The resulting agentic system was live and demonstrated publicly on stage at their own industry conference in February. Under ninety days from first conversation to a working system in front of their market.
What it produced: roughly a tenfold increase in qualified leads, first contact with a new enquiry in under thirty seconds, approximately ninety per cent reduction in associated administrative work, and a record month for the business. Not a pilot. Not a proof of concept parked in a sandbox. Production, running, measured.
The architecture underneath it is composite in exactly the sense described above, and deliberately unremarkable in its ambition. Language models where interpretation and conversation matter. Deterministic logic where compliance and consistency matter. Integration into the systems people already used, so the new way was easier than the old way on day one. A control plane holding it together, so there is a record of what happened and a route to change it safely.
The interesting part is not the technology. It is that it took ninety days, and that the business could see what it was returning from the first month.
Where we would tell you to start
If you are working out what to do with any of this, five things, in this order.
Start with a process that costs money, not with an architecture diagram. Pick something with a measurable baseline: hours, cycle time, error rate, conversion, cost per transaction. Capture that baseline before you build anything, because you will not be able to reconstruct it afterwards and you will need it.
Establish the control plane before you need it. The right time to introduce an orchestration layer is before agent sprawl, not after. Retrofitting governance across eleven live systems is an order of magnitude more expensive than designing it in at the start, and considerably more political.
Be honest about the data. Name the owner. Set the freshness tier deliberately. Define the kill criteria before it ships. If the data underneath a use case is not trustworthy, fix that first or pick a different use case.
Instrument cost and value from day one. Attribution per process, visible to the people funding it. We anchor at ten units of value for every unit of cost across a portfolio, with a floor of four to one below which we will not take work on. Individual cases land above and below. The portfolio is what matters, and honesty about what is not working is what keeps the average real.
Design for adoption before you design for scale. Sit with the people who will use it. Watch the current process. Make the new path the easier path. Everything else is downstream of that.
The position, plainly
Models are not the operating model. AI Operations is.
The next phase of enterprise AI will not be won by buying access to the most powerful model, and it will not be won by assembling the largest number of components either. It will be won by building the control layer that makes AI usable, safe, observable, governable, reversible and commercially viable inside a real business with real constraints and real people who have real jobs to get on with.
Composite AI supplies the techniques. The control plane is what turns them into something you can operate, audit, afford and expand. That is the difference between a collection of capable parts and an AI capability you can put your name to.
We are builders and operators, not advisers. We do not sell discovery phases, and we do not hand over a strategy document and wish you luck. We design the architecture, build the system, run it in production and transfer the capability when you want to own it.
If you are somewhere in this problem, whether that is a first process worth automating, a growing collection of agents nobody is governing, or an AI spend nobody can attribute, we would be glad to have the conversation. No pitch, no discovery invoice. Bring a process that costs you money and we will tell you honestly whether we can make it worth doing, and what it would take.
Talk to us at riverai.co.uk.