Strategy·November 2026·10 min read

Buying AI vs Building It: What Retail CTOs Get Wrong About Both

Buy versus build is the wrong question, because an AI feature is 5 components with different economics, and the right answer splits down the middle of the stack. Buy the model, rent the plumbing if it helps, and own the evals, the data contracts, and the integration into your systems, because those 3 are where switching costs and differentiation actually live.

Two documents land on a retail CTO's desk in the same November week. One is a vendor proposal for an AI merchandising platform, priced per SKU per month, live in 6 weeks. The other is an internal pitch: 2 engineers, 2 quarters, we build it ourselves and own it forever.

Both documents share a wrong assumption, which is that the thing being bought or built is one thing. Pull an AI feature apart and it is 5 components, and the buy-or-build answer differs for each of them.

The 5 components of any production AI feature

The model. The thing that predicts, classifies, or generates. Foundation models made this the most commoditized layer in the stack, which is the opposite of where most executives place it.
The orchestration. Prompts, retrieval, tool wiring, retries, fallbacks. The workflow logic that turns a model into a feature.
The integration. The joins into your OMS, PIM, catalog and ticketing, where your data's actual shape lives.
The evals. The golden set and graders that define correct for your business and catch regressions before customers do.
The data contracts. The schemas, feeds and quality guarantees the feature consumes and emits.

Vendors sell all 5 as a bundle. Internal teams pitch all 5 as a project. The mistake is symmetrical, and so is the fix: decide per component.

What buyers get wrong: the model is the replaceable part

The instinct when buying is to evaluate the vendor's model quality above all else. Model quality matters, but it is the component with the shortest shelf life and the least loyalty to your decision.

Look at the vendors' own retirement pages. Anthropic publishes a deprecation policy and a table of retired models with their shutdown dates, and OpenAI maintains the same: dated announcements, deprecation windows, and hard retirement dates after which the API version stops answering. Model turnover is not a risk scenario. It is the published operating model of the industry.

That reframes the buying question. The model you evaluate in the demo will be retired within a couple of years, so the durable asset is whatever lets you re-validate the feature when the swap comes.

That asset is the eval suite, and it is exactly the component vendor bundles rarely hand over in a portable form. A team that owns a golden set and graders can migrate models in a sprint, scored, with evidence. A team that owns a vendor dashboard re-runs the original pilot from scratch, on the new model, under deadline.

Lock-in in AI is rarely the API contract, which takes an afternoon to swap behind an interface. Lock-in is not having the eval suite that proves the swap did no harm, and not owning the labeled data the vendor accumulated on your traffic.

So the 2 questions that matter most in vendor due diligence are unglamorous. Can we export the labeled examples, corrections and verdicts our usage generates in a documented format, and can we run our own eval set against your system on demand? A no on either is the real price tag.

What builders get wrong: the model is the small part

The internal pitch has the mirror-image blind spot. It scopes the interesting work, the model and the prompts, and treats the rest as detail. The rest is the project.

This is the oldest documented lesson in production ML. The NeurIPS paper on hidden technical debt in machine learning systems argued in 2015 that only a small fraction of a real-world ML system is the ML code, with the surrounding infrastructure, glue code, configuration, data dependencies and monitoring dwarfing it. It also named the maintenance traps by type: entanglement, where changing anything changes everything, undeclared consumers of model outputs, and data dependencies that cost more than code dependencies.

A decade and one foundation-model wave later, the ratio still holds. The model got easier to obtain, and everything around it, the integration into your order data, the eval suite, the review workflow, the monitoring, did not get easier at all. Build estimates that die in month 4 die in the surrounding system, almost never in the model.

The build case also carries a payroll assumption nobody prices: the team that builds it must exist in year 2 to maintain it, through the same model deprecations the buyer faces, plus the glue code that is now yours alone. Building converts a subscription line into a permanent headcount commitment, and that is the honest comparison, not license fee versus salaries-for-2-quarters.

Most retail AI is a workflow, which changes the build math

One input genuinely lowers the build estimate, and it comes from the model vendors themselves. Anthropic's building effective agents guidance draws a line between workflows, where the steps are known and orchestrated through predefined paths, and agents, where the model directs its own process, and recommends the simplest composable pattern that works over frameworks and autonomy.

Returns triage, attribute extraction, copy drafting and query rewriting are workflows. A knowable step count, a fixed graph, deterministic grading for most steps. Building a workflow is weeks of engineering on top of a bought model, not the 2-quarter platform the pitch describes, provided the team resists promoting it to an agent for the demo value.

This cuts against vendors too. A platform subscription priced for an autonomous agent is expensive cover for what is often a 5-step workflow against your own data. Asking a vendor which pattern their product actually runs is a clarifying conversation.

The decision table

Component	Default	Reasoning	When the default flips
Model	Buy	Commoditized, vendor-retired on a schedule, swappable behind an interface	Hard data-residency constraints or genuinely unusual domains push toward hosted open-weight models, which is still buying, differently
Orchestration	Build thin	Workflow logic encodes your policies; thin code on a bought model stays replaceable	Commodity workflows with no differentiation, like generic support deflection, are fine to buy whole
Integration	Build	Nobody else knows your systems' real shape, and every vendor quote assumes your data is cleaner than it is	If a vendor natively speaks your exact platform stack end to end, buying the connector beats rebuilding it
Evals + golden set	Own, always	Defines correct for your business; the portability layer that makes every other cell reversible	Never. Tooling around it can be bought; the examples and thresholds stay yours
Data contracts	Own, always	Schemas and feeds outlive every model and every vendor on this table	Never

The pattern in the right-hand columns is the actual thesis: buy where the market moves fastest, build where your context is the input, and own without exception the 2 components that make the other 3 reversible.

The seam of the decision table: rent what the market replaces, own what defines correct — and the eval suite makes the rented half reversible.

Reversibility is the criterion, not cost

Cost comparisons between a subscription and a build are mostly fiction on both sides, because the subscription hides the integration work and the build hides the maintenance tail. The comparison that stays honest is reversibility: how expensive is it to be wrong?

The honest comparison is never subscription versus salaries. It is the cost of being wrong under each option, and owned evals plus owned data contracts are the 2 assets that make being wrong cheap under both.

A bought feature with exported evals and owned data contracts is cheap to be wrong about. Swap the vendor, re-run the suite, migrate the labeled data. A bought feature without them is a rewrite disguised as a renewal negotiation.

The same test applies to building. A thin workflow on a bought model is cheap to abandon for a vendor product that catches up. A 2-quarter internal platform is a sunk-cost argument with a standup meeting. This is the same reasoning that governs replatform-versus-modernize decisions generally: the option that keeps the most future options open wins ties, and most matchups are closer to ties than either pitch admits.

Governance does not care who built it

One more asymmetry the buy side underweights: accountability does not transfer with the purchase order. The NIST AI Risk Management Framework structures the work as Govern, Map, Measure and Manage, and every one of those functions lands on the deploying organization regardless of whose model answers the API call.

Measure is the function that decides whether your buy-vs-build split works in practice. If the vendor's system misclassifies refunds, your risk committee will not accept the vendor's dashboard as the measurement, and they will be right, because the party measuring should not be the party being measured.

Which routes back to the same place every section of this post routes to. The eval suite is the governance artifact, the migration insurance, and the acceptance test, and it is the piece that decides whether either strategy works. Teams that skip it do not fail at buying or at building specifically. They fail at knowing, which is the stage where most AI pilots die.

How to run the decision in a week

The full analysis fits inside 5 working days, which is worth stating because both camps benefit from stretching it.

Day 1: decompose the feature into the 5 components and write the reversibility question for each.
Day 2: build the first 50 golden-set examples with the domain owner. This is useful under every outcome, including doing nothing.
Day 3: score the shortlisted vendor against the eval set, and ask the 2 export questions in writing.
Day 4: have the internal team estimate the thin-workflow version only, with the maintenance tail priced as headcount.
Day 5: fill the decision table, note which cells flipped from default, and record why.

The recorded table matters more than the verdict. Vendors get acquired, models get retired, and teams get reorganized, and the document that says which component was bought for which reason is what lets the successor CTO revisit one row instead of relitigating the whole question.

FAQ

Should we fine-tune a model to own more of the stack?

Fine-tuning buys you a per-model asset that dies with the base model's deprecation, so it deepens the maintenance commitment rather than reducing it. Exhaust prompting, retrieval and workflow design against your eval set first, and let measured gaps justify the step.

Does an abstraction layer over model providers solve lock-in?

It solves interface lock-in, which was the small part. A gateway like LiteLLM normalizes the API calls across providers, and that is genuinely useful plumbing. It does nothing about behavioral differences between models, which only your eval suite detects.

We have no ML engineers. Does that force buying everything?

It forces buying the model and probably the orchestration tooling, which the defaults already say. Evals and data contracts need domain owners and ordinary engineers, not ML specialists, and owning them keeps the eventual vendor honest.

How do we compare a vendor demo against an internal prototype fairly?

Same golden set, same slices, same graders, blind if you can manage it. Any comparison where each side brings its own examples measures example-picking skill.

What belongs in the contract if we buy?

Export rights over labeled data and corrections in a documented format, the right to run external evals, notice periods for model changes underneath the product, and per-slice quality floors with remedies. A vendor who resists the first 2 is telling you where their margin lives.

When is full build the right call?

When the feature sits on data no vendor can see, encodes policy that changes weekly, and stays a thin workflow. All 3 at once is rarer than internal pitches suggest, and the eval suite you built for the decision will tell you if you are the exception.

References

From the Destm engineering archive. For current work on this topic, start at Solutions or the blog.