AI·June 2026·15 min read

Building AI Applications with LangChain and MCP: The Future of Enterprise AI Architecture

MCP gives an agent a standard way to reach a tool. LangChain, and LangGraph specifically, decides which tool to reach for, in what order, and what to do with the result. Confusing the two is why most first attempts at agentic AI turn into a pile of custom glue code within two sprints.

Enterprise AI stopped being a chatbot problem two years ago. The real question now is architecture: how do you connect a model to a CRM, an ERP, and internal docs without writing a custom integration for every tool and every agent? LangChain and the Model Context Protocol (MCP) answer two different halves of that question, and knowing where one ends and the other begins is the first design decision that matters.

Get this split wrong and the symptoms show up fast. Teams either bolt tool access directly onto a LangGraph node — tight coupling, no reuse, a rewrite every time a vendor changes an API — or they build an MCP server and expect it to also handle planning, retries, and multi-step reasoning, which it was never designed to do. This post covers both halves properly, then goes further: agent runtime anatomy, MCP server design for real enterprise systems, evals, failure modes, build-vs-buy, and the team you actually need to run this in production.

What LangChain actually does

LangChain is an orchestration framework. It gives you the primitives for chaining prompts, calling tools, retrieving documents, and coordinating multi-step agent workflows in Python or TypeScript.

Its ecosystem has split into pieces that map to real jobs:

LangGraph — stateful, graph-based control flow for agents that need more than a single request-response loop. Nodes are functions, edges are transitions, and state persists across the whole run.
LangSmith — tracing and evaluation for what an agent actually did in production, not just what it was prompted to do.
LangChain core — the composable primitives (prompts, output parsers, retrievers, tool schemas) that both of the above build on.

LangChain does not know how to talk to a Salesforce instance or a Postgres database on its own. That's a separate problem, and it's the one MCP solves.

What MCP actually does

MCP is a protocol, not a framework. It standardizes how an AI application discovers tools, calls them, reads resources, and exchanges context, doing for model-to-tool communication roughly what HTTP does for web clients and servers.

The protocol defines three primitives an MCP server can expose, and mixing them up is a common early mistake:

Tools — actions with side effects: create a ticket, send an email, update a CRM record.
Resources — read-only context the model can pull in: a file, a database row, a document.
Prompts — reusable prompt templates the server exposes to any client that connects to it.

Before MCP, every agent needed a bespoke connector per tool: one for Slack, one for GitHub, one for an internal ticketing system. Each connector duplicated auth handling, error handling, and schema definitions.

MCP replaces those N-times-M custom integrations with N servers and M clients that speak the same wire format. The gain isn't intelligence — it's a shared interface so tool access doesn't get reinvented per agent.

Transport and where servers actually run

MCP servers run over one of two transports: stdio for local processes talking to a client on the same machine, or Streamable HTTP for remote servers a client connects to over a network. Most enterprise deployments end up on the HTTP transport, because the server sits behind an API gateway and multiple agents need to reach it.

That detail matters for capacity planning. A stdio server spins up and dies with the client process. An HTTP server is a long-running service with its own uptime, scaling, and on-call rotation — treat it like any other production API, not like a script.

Every MCP server you stand up is a production API with its own auth, its own SLAs, and its own on-call. Treat it like a script and it will fail like one.

Why they're not competitors

LangChain and MCP sit at different layers. LangChain decides what an agent should do next; MCP decides how that agent reaches a system to do it.

LangChain's tool-calling and agent abstractions can wrap MCP servers as callable tools. That gives you LangGraph's control flow and LangSmith's tracing on top of a standardized, swappable set of MCP-connected tools underneath. Neither replaces the other.

The orchestration layer plans and branches; MCP is the typed, audited seam every call to a real system passes through; LangSmith taps both layers for tracing and evals.

Anatomy of an agent runtime

Agent gets used loosely enough that it's worth naming the parts. A production agent runtime built on LangGraph has four components, and skipping any one of them is where prototypes stay prototypes.

The control loop

The loop reads the current state, decides the next action (call a tool, ask the user, or stop), executes it, and writes the result back into state. LangGraph implements this as a graph so the loop can branch, retry a specific node, or route to a human-review node without restarting the whole run.

A single linear chain, call the model, call a tool, call the model again, works for demos. It breaks the moment a task needs conditional branching: check inventory, only reorder if stock is below a threshold, only auto-approve the reorder if it's under a spend limit.

State and memory

State is what persists across steps in a single run: the conversation so far, intermediate results, which tools have already been called. Memory is what persists across runs: a customer's prior tickets, a user's stated preferences, a running summary of a long-lived project.

Conflating the two is a common source of bugs. State that should reset between runs, like a half-finished tool call, leaking into memory means an agent starts every new conversation confused about where it left off last time.

Human-in-the-loop gates

LangGraph's interrupt mechanism pauses a run at a named node and waits for external approval before continuing. That's the primitive that makes "let the agent draft the refund, but a human approves it before it processes" an architecture decision instead of a hope.

Skipping this for anything touching money, customer communication, or production infrastructure is the single most common mistake in early agent rollouts. Add the gate at design time. Retrofitting approval steps after an agent has already run unsupervised for months is a much harder conversation with legal.

Designing MCP servers for enterprise systems

Standing up an MCP server in front of a CRM is not the same exercise as writing a REST wrapper. The protocol gives you primitives; the design decisions are still yours.

Scope the surface area deliberately

Don't expose every CRM field and every CRUD operation as a tool because the API supports it. Expose the 5-10 operations an agent legitimately needs, like reading account history, creating a case, escalating a ticket, and nothing else.

A narrow, well-typed tool surface is easier for a model to use correctly and easier for a security team to review. Every additional tool is additional attack surface and additional ways for the model to pick the wrong one.

A tool you didn't expose is an action an agent can't accidentally take. Scoping the surface area is a security control, not just an API design choice.

Auth and scoping per caller

MCP's 2025-06-18 specification defines an OAuth 2.1-based authorization flow for HTTP-transport servers, and it's not optional for anything touching real data. The server should scope what a given client can see and do based on the identity of the calling agent or user, not a single shared service credential for every caller.

A support agent's MCP session should not have the same CRM write access as an internal ops agent. Model the permission boundary at the MCP server, not in a prompt instruction the model can be talked out of.

Schema stability and versioning

Treat a tool's input and output schema like a public API contract, because functionally it is one. Changing a field name or a required parameter breaks every agent calling that tool, often silently, because the model just starts passing the wrong shape and the failure looks like a reasoning error instead of a schema mismatch.

Version the server, not just the schema. Running v1 and v2 of an MCP server side by side during a migration window costs little and avoids a flag day across every agent that depends on it.

Approach	Coupling	Reuse across agents	Where auth lives
Custom per-agent integration	Tight, logic embedded in the agent code	None; rebuilt per agent	Scattered, often hardcoded
Shared internal SDK	Moderate, shared library, still in-process	Good within one language or runtime	Centralized in the SDK
MCP server	Loose, network boundary, typed schema	Full; any MCP client can connect	Centralized at the server, per the 2025-06-18 spec
Vendor-managed connector	Loose but vendor-controlled	Full, subject to vendor roadmap	Vendor's identity provider integration

Evals and observability — LangSmith and beyond

An agent that worked in the demo and fails silently in week three of production is the default outcome without an eval harness. LangSmith traces every step of a LangGraph run: which tools were called, what the model reasoned before calling them, and what came back.

Three categories of evaluation matter, and most teams only build the first one:

Offline evals — a fixed test set of inputs with known-good outputs, run against every model or prompt change before it ships.
Online evals — sampling live production traces and scoring them, either with a rubric or an LLM-as-judge, to catch drift the offline set doesn't cover.
Human review queues — routing a percentage of runs, or every run below a confidence threshold, to a person for manual grading.

Skipping offline evals means every prompt tweak is a coin flip. Skipping online evals means finding out about drift from a customer complaint instead of a dashboard.

An agent without an eval harness isn't in production. It's in an extended, unmonitored demo.

Failure modes worth designing against

These aren't hypothetical. They show up in the first few months of any real deployment.

Prompt injection through tool responses. A compromised or malicious data source returns content designed to redirect the agent's next action, a support ticket body containing instructions the model follows as if they came from the user.
Schema drift. A tool's underlying API changes shape and the MCP server isn't updated; the model starts passing arguments that used to work and silently fail now.
Runaway loops. A model retries a failing tool call indefinitely because the failure looks recoverable, burning tokens and, if the tool has side effects, retrying the side effect too.
Silent partial failures. A multi-step task completes four of five steps and the model reports success anyway because nothing in the loop checked that all five actually ran.
Cost blowups. A single user request fans out into a dozen tool calls and a long reasoning chain, and nobody set a budget ceiling per run.
Context poisoning from retrieval. A low-relevance chunk gets pulled into context anyway and the model treats it as authoritative, producing a confidently wrong answer sourced from the wrong document.

Failure mode	Where it's caught	Mitigation
Prompt injection via tool response	MCP server response filtering, LangSmith trace review	Sanitize and structure tool outputs; never let raw external content masquerade as a system instruction
Schema drift	Contract tests on the MCP server, error-rate alerting	Version the server; run contract tests against the real upstream API on every deploy
Runaway loops	Token and step budget per run	Hard cap on tool calls and reasoning steps per run, enforced in the LangGraph loop
Silent partial failure	Explicit completion checks, not model self-report	Verify state post-hoc against the source of truth, not the model's own claim of success
Cost blowup	Per-run spend tracking in LangSmith	Budget ceiling per run with a hard stop, alerted before it's hit
Context poisoning from retrieval	Relevance-score logging on every retrieval call	Enforce a minimum relevance threshold, cap chunks per query, log what was retrieved alongside the answer

Context and retrieval: the other half of enterprise AI

Tool calls answer what an agent should do. Retrieval answers what the agent needs to know before deciding. Most enterprise deployments need both, and treating retrieval as an afterthought is why answers end up citing a policy document that changed last week.

Chunking and freshness

A vector store is only as good as its ingestion pipeline. Chunk size, overlap, and re-indexing cadence matter more than which embedding model gets picked, because a stale index answers confidently from outdated source material regardless of how good the embeddings are.

MCP resources are a cleaner fit than a static vector store for anything that changes often. A resource server can return the current state of a record on every call, instead of relying on a nightly re-index to catch up with what changed.

Budgeting the context window

Every tool result, retrieved document, and turn of conversation history competes for the same context window. An agent that retrieves 20 documents just in case burns budget the actual reasoning step needed, and often degrades output quality instead of improving it.

Rank and filter before injecting into context, never after — a retrieval step returning the top 3 relevant chunks with a real relevance threshold beats one returning the top 20 and letting the model sort it out.
Cap total retrieved tokens per run and alert when a query consistently hits the cap, since that's usually a sign the underlying index needs better chunking, not a bigger context window.
Log the retrieved chunks next to the final answer in LangSmith, so a reviewer can see exactly what the model was working from, not just what it concluded.

Build vs buy

Not every enterprise needs to hand-roll an agent runtime. The decision comes down to how differentiated the workflow is and how much control the compliance function needs over the reasoning path.

Buy a vertical AI product or vendor agent platform when the workflow is common across the industry and the vendor's default behavior is close enough — expense report triage, standard support deflection.
Build with LangGraph plus MCP servers you control when the workflow touches proprietary systems, needs a specific approval chain, or is the differentiated part of the business.
Hybrid — buy the model and orchestration tooling, build the MCP servers in-house — is where most serious enterprise deployments land, because the systems being connected to are always internal.

The mistake to avoid is building a generic orchestration layer from scratch when LangGraph already solves the parts that aren't differentiated. Spend the engineering effort on the MCP servers and the domain logic, not on reinventing a state machine.

Total cost of ownership rarely shows up in the initial pitch. Model inference cost is the visible line item; the eval harness, the MCP server fleet, and the on-call rotation for both are the ones that determine whether the project is still funded in year two.

What team this actually takes

Hire an AI engineer undersells what a real deployment needs. Four distinct skill sets show up in a functioning team, sometimes as one person wearing multiple hats early on:

An orchestration engineer who owns the LangGraph runtime, the control flow, and the eval harness.
A systems integration engineer who owns the MCP servers and the contracts with each backend system.
A security and governance owner who defines the approval gates, the auth model, and the audit requirements before launch, not after an incident.
A domain reviewer, someone who actually knows the CRM workflow or the finance process well enough to grade whether the agent's output is actually correct, not just well-formatted.

Skipping the domain reviewer role is the most common gap. An engineer can tell you the agent ran without errors. Only someone who's done the underlying job can tell you if the answer was right.

A worked example, end to end

Someone asks an internal agent for a quarterly revenue forecast. LangGraph handles the plan: which sub-tasks run, in what order, with what state carried between steps.

MCP servers expose the CRM, the financial database, and an internal analytics API as callable tools with typed schemas. The agent calls each one the same way regardless of what sits behind it. The orchestration layer never needs to know the CRM's specific API shape, because that's the MCP server's job.

LangSmith traces the run so a finance reviewer can see exactly which numbers came from which system before the forecast goes into a deck. If the CRM query returns stale data, the trace shows that, instead of the discrepancy surfacing three weeks later in a board meeting.

Swap the CRM vendor next year, and the fix is one MCP server, not every agent that talks to it.

Common patterns worth building this way

Support agents that pull ticket history and account data before drafting a reply, with a human approval gate before the reply sends
Sales agents that read CRM pipeline data to generate a forecast, traced end to end for finance review
Internal developer agents that read a repository and open a pull request through the same tool interface, scoped to specific repos per team

None of this requires rewriting the underlying systems. It requires an MCP server in front of each one, an orchestration layer that knows how to call them, and an eval harness that catches drift before a customer does.

FAQ

Do I need both LangChain and MCP, or can I pick one?

They solve different problems. Skip MCP and you're back to bespoke connectors per tool. Skip LangGraph-style orchestration and MCP servers alone won't plan, branch, or retry. They just expose tools.

Can I use MCP without LangChain?

Yes. MCP is a protocol, not tied to any one orchestration framework. Any MCP-compatible client, including Claude's own agent tooling, can call an MCP server you build.

How many MCP servers should one team run?

Usually one per backend system, not one per agent. A CRM MCP server should serve every agent that needs CRM access, scoped by caller identity, rather than each agent shipping its own copy.

What's the biggest reason enterprise agent projects stall?

Missing evals and missing approval gates. Both get treated as post-launch polish. Both are the reason pilots get pulled after an incident instead of scaled.

Is LangGraph required, or can I write my own control loop?

You can write your own. Most teams that try end up rebuilding state persistence, interrupt handling, and retry logic that LangGraph already ships, at a cost in engineering time that rarely pays off.

How do I stop an MCP server from becoming a security hole?

Scope the tool surface narrowly, enforce the 2025-06-18 spec's OAuth-based authorization per caller, and treat every tool call as an auditable action, the same discipline as a database write.

Do I still need a vector store if I'm using MCP resources?

Often both. Use a vector store for unstructured, slow-changing content like documentation and past tickets; use MCP resources for anything with a live source of truth, like an order status or an account balance, where a nightly re-index would already be stale.

References

From the Destm engineering archive. For current work on this topic, start at Solutions or the blog.