Performance·February 2024·9 min read

Revolutionizing User Experience with AI in Web Development

An AI feature that works in a demo and one that works in production differ in exactly one place: what happens when the model call is slow, wrong, or unavailable. Search, personalization, and chat all need a latency budget and a fallback path before they ship. A good prompt alone doesn't cover that.

Three angles on AI in web development — this is the runtime one

"AI in web development" covers at least three different jobs, and conflating them is where most confusion starts. One is the design-process question: how tools like v0 and Figma Make generate a layout, covered in Automating Web Design: The Role of AI in Creative Processes. Another is the code-ownership question: who reviews AI-generated code before it ships, covered in Design Without Limits: How AI and No-Code Tools Are Reshaping UI Development.

This post is the third angle: runtime UX. Once a site ships an AI-powered search box, a personalized homepage module, or a chat assistant, the AI call becomes part of the page's live performance budget. That budget behaves nothing like a static component's.

AI-powered search: intent replaces exact match

Traditional site search matches query terms against an index and ranks by relevance signals computed offline. It's fast because there's no model call in the request path — the cost was paid at index time, not query time.

From keyword index to embeddings

Semantic search instead converts a query into a vector embedding and retrieves results by proximity in that vector space, which is what lets "warm running jacket" match a product titled "insulated shell" with no shared keywords. OpenAI's embeddings guide documents this pattern directly: convert text to a numeric vector, then compare vectors to find semantically related content (OpenAI Platform — Embeddings guide).

Algolia's NeuralSearch runs keyword and vector retrieval against the same query simultaneously rather than choosing one, which is the practical answer to a common failure mode: a pure vector search that returns confidently wrong results for an exact SKU or model number a keyword index would have matched instantly (Algolia — NeuralSearch).

Hybrid retrieval keeps keyword search as a floor

The practical lesson from shipping AI search: never let semantic retrieval fully replace keyword matching. Run both, merge the ranked results, and let keyword matches win ties on exact-match queries where a customer typed a SKU, a brand name, or a model number verbatim.

A search box that spins for two seconds while a vector index resolves has replaced an obvious autocomplete miss with an invisible one. Users don't distinguish "the model is slow" from "the site is broken" — they just leave.

Personalization surfaces: cold start is the default state

A personalized homepage module, a "recommended for you" rail, or a tailored hero banner only has signal once a visitor has done something. For every new session, cold start isn't an edge case — it's the first render for every single user, every single time.

What a personalization surface actually needs to know

Real-time recommendation services like Amazon Personalize build recommendations from a mix of historical interaction data and live event streams, explicitly designed to update as a session generates new signal rather than waiting for an offline batch job (AWS — What is Amazon Personalize). Typical inputs:

Browsing and click history within the current session, before any account exists.
Cart and wishlist contents, which carry stronger intent signal than page views.
Cohort-level defaults (referral source, device, geography) when no individual signal exists yet.
Authenticated purchase history, once available, which should outrank session-level guesses.

Personalization can't block the render

If a personalization service call sits in the critical path of an above-the-fold module, a slow response delays Largest Contentful Paint for every visitor, not just the ones the personalization actually helps. The fix is structural: render a sensible, non-personalized default immediately, then swap in the personalized version when it arrives.

Google Cloud's retail documentation frames this as parallel decisioning: retrieve a default result set fast, then layer personalization on top asynchronously (Google Cloud — Retail API and Vertex AI Search for commerce documentation).

A personalization surface with no default state isn't personalized — it's just slow for everyone until the model responds. Ship the generic version first; upgrade it in place.

Comparing runtime AI features by what breaks and how they recover

Feature	Latency budget	Typical failure mode	Fallback
Search	Perceived instant — results should start appearing within a few hundred milliseconds of the last keystroke	Vector index cold start or upstream timeout on the embedding call	Degrade to keyword-only ranked results, never a blank state
Personalization surface	Must not delay the page's Largest Contentful Paint	Recommendation service slower than the page's own render	Render a non-personalized default, swap in personalization asynchronously
Conversational assistant	Time to first token, not time to full completion	Model call hangs with no visible progress	Streamed partial output plus a timeout that offers a human handoff or FAQ link
Visual/image search	Tolerant of a slightly longer budget, since users expect image processing to take a moment	Slow image upload or embedding on constrained mobile networks	Text-search suggestion alongside a visibly progressing upload state

Latency budgets, measured against real metrics

Interaction to Next Paint (INP) is the Core Web Vitals metric for responsiveness: it measures the delay between a user's interaction and the next visual update, with a "good" threshold at or below 200 milliseconds (web.dev — Interaction to Next Paint). An AI feature triggered by a click or a keystroke is measured against this same threshold, whether or not the team building it thought of it as a "performance feature."

Core Web Vitals also tracks Largest Contentful Paint, which is why a personalization call in the critical render path is a Core Web Vitals problem, not just a UX nicety (web.dev — Web Vitals).

Set an explicit timeout on every AI service call — never let a client wait indefinitely for a model response.
Measure time to first visible feedback, not time to final result, since that's what a user actually perceives as speed.
Test the feature on a throttled connection and a mid-tier mobile device, not just the office network.
Confirm the loading state itself doesn't shift layout when the real content arrives, which is a Cumulative Layout Shift regression hiding inside an AI feature.

Streaming responses change the shape of the latency budget

A chat assistant that waits for a full model completion before showing anything will feel broken well before the 200-millisecond INP threshold, because the whole response can take several seconds to generate. Streaming reframes the budget: what matters is time to first token, not time to last token.

The WHATWG Streams standard defines the ReadableStream primitive that lets a browser start rendering a response before it finishes arriving (WHATWG — Streams Standard). Server-Sent Events, defined in the WHATWG HTML standard, is the transport most AI chat APIs use to push those incremental tokens from server to browser over a single long-lived connection (WHATWG HTML — Server-sent events).

A streamed response also needs its own fallback: if no token arrives within a short window, that's the signal to show a timeout state, not to keep an indefinite spinner running.

Fallbacks: what happens when the AI call is slow or wrong

The circuit breaker pattern, documented in Microsoft's architecture guidance, stops an application from repeatedly calling a service that's already failing, and lets it fail fast instead of piling up timeouts (Microsoft Learn — Circuit Breaker pattern). The same logic applies directly to an AI feature: if the model endpoint is timing out repeatedly, stop calling it and serve the deterministic fallback until it recovers.

Both branches always resolve to something rendered. The failure mode a runtime AI feature has to design against is an indefinite wait, not an occasional fallback.

Building the fallback path

Define the deterministic result before building the AI-powered one — a non-personalized module, a keyword-only search result set, a static FAQ link.
Set a timeout shorter than the point where a user perceives the page as stuck, not shorter than the model's average response time.
Track how often the fallback fires in production, since a rising fallback rate is an early signal of an upstream model or infrastructure problem.
Never let the fallback state look like an error. A generic search result or a default homepage module is a normal outcome, not a failure the user needs to see labeled as one.

Runtime accessibility follows the same logic. A live-captioned webinar or an AI-generated image description needs a fallback for when the model call fails, the same way a search box does — a missing caption stream shouldn't silently degrade to no captions at all without any indication to the user.

FAQ

What's a reasonable latency budget for an AI-powered search box?

Treat it like any other interaction measured by INP: aim for visible feedback within 200 milliseconds, even if that first feedback is a loading state rather than final results.

Should personalization ever block the initial page render?

No. Render a sensible default first and swap in the personalized version once it arrives, so a slow personalization call never delays Largest Contentful Paint for every visitor.

Is streaming worth the added complexity for a chat feature?

Usually yes, once a response takes more than a second or two to generate. Time to first token is what a user perceives as responsiveness, not time to the full completion.

How is this different from the AI design-tool and no-code posts?

Those cover building the UI with AI tools — layout generation and code review. This covers AI features running live in the shipped product, where latency and failure handling are the actual UX surface.

Does visual search need different fallback logic than text search?

The principle is the same, never leave a blank or indefinitely spinning state, but the budget is more forgiving, since users already expect image processing to take a visible moment.

What's the simplest sign an AI feature is missing a fallback?

If a slow or failed model call produces a blank section, an infinite spinner, or a visible error instead of a sensible default, the fallback path hasn't been built yet.

References

From the Destm engineering archive. For current work on this topic, start at Solutions or the blog.