# How to add an AI feature to your existing app in 2026

*By Roberto Lazar, founder of Dock30 · Published 2026-06-18 · Updated 2026-07-25 · 7 min read*

Where the model call should live, how to stream tokens from a Next.js route handler, and the routing, caching, and batch discounts that cut the bill.

To add an AI feature to an existing app, put the model call behind your own backend, **never in the client**. Your server owns the prompt, pulls in whatever context the model needs, calls the provider, checks the output, streams the result to the UI, and logs the whole exchange. The model is the easy part. The plumbing around it is the actual work, and in 2026 it is also where the money is won or lost.

The architecture in this post has not changed much since we started shipping it into client products at Dock30. Every step survived the last few model generations. The pricing math has changed a lot, though, so this update keeps the backbone and adds the current numbers for routing, caching, and batching. Those three decide whether your feature costs pocket change or a salary.

## First, check the feature needs a model at all

A sanity check before any architecture: plenty of "AI features" on 2026 roadmaps are a regex, a cron job, or a well-designed form wearing a trench coat. If the task has rules you can enumerate, write the rules. Models earn their keep on fuzzy inputs: free text, documents, images, anything where the logic cannot be written down. We have talked more than one client out of an LLM integration entirely, and that conversation is some of the best free consulting we do.

## Why the model call belongs on your backend

The most common mistake we see in code audits is a fetch to the model provider straight from the frontend. It works in the demo. In production it hands your API key to anyone who opens dev tools, and it leaves you with no control over anything that follows. The call belongs in a backend service you own:

- Your API keys never reach the browser.
- You control the prompt, the retrieved context, and the guardrails.
- You can cache, rate limit, and pick a cheaper model per task.
- You log every input and output, which becomes your eval data later.

Think of it as a thin AI service layer. The app, whether a web client or a [mobile app](/services/mobile-apps), talks to your backend, your backend talks to the provider, and nothing else in your stack has to know an LLM is involved.

## The request flow, end to end

One production request looks like this:

1. Validate and authorize the request on your server.
2. Retrieve context if the feature needs your data. Whether that means retrieval or something heavier is its own decision, covered in [RAG vs fine-tuning](/blog/rag-vs-fine-tuning).
3. Build the prompt from a versioned template plus that context.
4. Call the model with streaming enabled.
5. Validate the output before anything downstream trusts it.
6. Stream to the client and log the full interaction.

Every step is a seam where you can add a cache or a kill switch. That is the whole argument for keeping it on the server.

## Streaming from a Next.js route handler

A full model answer can take several seconds, which is an eternity by web standards. Streaming tokens as they generate turns that wait into something users read in real time. In Next.js 16 (App Router), a route handler does this with a plain ReadableStream returned as server-sent events, and it needs no extra infrastructure:

```ts
// app/api/assistant/route.ts
import Anthropic from "@anthropic-ai/sdk";
import { getSession } from "@/lib/auth";
import { SYSTEM_PROMPT } from "@/lib/prompts"; // versioned template, not an inline string

const anthropic = new Anthropic(); // API key stays server-side

export async function POST(request: Request) {
  const session = await getSession(request);
  if (!session) return new Response("Unauthorized", { status: 401 });

  const { message } = await request.json();
  if (typeof message !== "string" || message.length > 4000) {
    return Response.json({ error: "Invalid input" }, { status: 400 });
  }

  const stream = anthropic.messages.stream({
    model: "claude-haiku-4-5", // cheap tier; escalate hard requests to the frontier model
    max_tokens: 1024,
    system: [
      {
        type: "text",
        text: SYSTEM_PROMPT,
        cache_control: { type: "ephemeral" }, // provider caches this prefix on repeat calls
      },
    ],
    messages: [{ role: "user", content: message }],
  });

  const encoder = new TextEncoder();
  const body = new ReadableStream({
    async start(controller) {
      for await (const event of stream) {
        if (
          event.type === "content_block_delta" &&
          event.delta.type === "text_delta"
        ) {
          controller.enqueue(
            encoder.encode(`data: ${JSON.stringify(event.delta.text)}\n\n`)
          );
        }
      }
      controller.enqueue(encoder.encode("data: [DONE]\n\n"));
      controller.close();
    },
  });

  return new Response(body, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-store",
    },
  });
}
```

On the client, read it with fetch and a stream reader (EventSource only supports GET). Perceived latency drops from several seconds to under one, and total generation time has not changed at all. If the feature is a chat interface rather than a single endpoint, we covered the full UI side in [how to build an AI chatbot for your website](/blog/build-ai-chatbot-website).

## Validate output before you trust it

Guardrails sound like enterprise theater until the first time a model returns malformed JSON into a code path that assumed it couldn't. The defensive layers we ship on every feature:

- Cap input length and per-user request rate.
- If you expect structured output, parse it against a schema, retry once with the error appended, then fall back.
- Filter unsafe content in both directions.
- Return a graceful message when the model errors or times out, and give users a path to a human when the AI is unsure.

The schema check is a few lines with zod:

```ts
const Answer = z.object({
  summary: z.string().max(500),
  confidence: z.enum(["high", "medium", "low"]),
});

const parsed = Answer.safeParse(JSON.parse(raw));
if (!parsed.success) {
  // one retry with the validation error appended, then a static fallback
}
```

## Build an eval set before you ship

You cannot improve what you do not measure, and with LLMs you often cannot even tell it got worse. Before launch, collect 30 to 50 representative inputs with expected outputs and run them as a test suite whenever you change a prompt, swap a model, or bump a dependency. On projects we've shipped, this is the single habit that separates features that improve over time from features that quietly rot. It costs an afternoon. Skipping it costs a confused Slack thread every time someone "just tweaks the prompt".

## The 2026 cost levers, with numbers

Cost control used to be a vague "route to cheaper models" bullet in posts like this one. The spreads are now big enough to be a design input.

Model routing is real money. [Anthropic's pricing](https://platform.claude.com/docs/en/pricing) puts Claude Haiku 4.5 at $1 per million input tokens and $5 per million output, against Claude Opus 4.8 at $5 and $25: a **5x spread** on both sides of the call. OpenAI's GPT-5.6 tiers have the same shape, roughly $1/$6 at the cheap end and $5/$30 at the top, per [TLDL's July 2026 pricing roundup](https://www.tldl.io/resources/openai-api-pricing). Send simple requests (classification, short rewrites) to the cheap tier and escalate only the ones that need frontier reasoning.

Prompt caching stacks on top of routing. Cache reads bill at around 0.1x the base input price, about **90% off** repeated context like your system prompt and shared documents, while cache writes cost about 1.25x once, per the [Anthropic prompt caching docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching). OpenAI's cached input is likewise 10x cheaper than uncached. For chat features, where the same context is resent on every turn, this is the single biggest lever.

Anything that does not need an instant answer (nightly summaries, backfills, classification jobs) can go through [batch processing](https://platform.claude.com/docs/en/build-with-claude/batch-processing) for a flat **50% off** all token usage.

| Lever | Verified saving | Where it applies |
|---|---|---|
| Model routing | Up to 5x per request | Every call you can downgrade |
| Prompt caching | ~90% off cached input reads | System prompts, docs, multi-turn chat |
| Batch API | 50% off all tokens | Non-interactive jobs |
| Prompt trimming | Linear with tokens cut | Every call |
| Per-user limits | Caps worst-case spend | Anything public-facing |

Stack the first three and the math compounds: a nightly classification job on a small model with cached context costs a rounding error next to naive frontier calls. If you are budgeting a whole build rather than one feature, we broke down realistic totals in [what a custom AI agent costs in 2026](/blog/custom-ai-agent-cost-2026).

## A shipping checklist

- [ ] Model calls run on the backend, never the client
- [ ] Prompts are versioned templates, not inline strings
- [ ] Responses stream to the UI
- [ ] Output is parsed and validated before anything downstream trusts it
- [ ] Every interaction is logged
- [ ] An eval set runs on every prompt or model change
- [ ] Simple requests route to the cheap model tier
- [ ] The system prompt and shared context are cached
- [ ] Non-interactive jobs go through the batch API
- [ ] Per-user rate limits cap worst-case spend

If you can tick all ten, ship it. Most of the AI features we get called in to rescue fail on logging, evals, or caching. Almost never on model choice.

If you have backend capacity in house, this whole layer is a week or two of focused work, and you should build it yourselves. If you don't, this is what our [AI and automation team](/services/ai-automation) does daily: fixed scope, the exact price and delivery date in writing before we start ([projects from EUR 350](/pricing/project)), and 30 days of free support after launch. Either way, if you want a second pair of eyes on your architecture, [book a free 15-minute call](https://calendly.com/dock30/15min) and bring your diagram. I enjoy those calls.

## Frequently asked questions

**Should I call the LLM API from the frontend or the backend?**

Always the backend. Calling the model from the browser exposes your API key to anyone who opens dev tools, and it leaves you with no way to cache, rate limit, or validate output. A thin server layer between your app and the provider is the standard production pattern.

**How much does it cost to run an AI feature in 2026?**

For most single features the model bill is small compared to the engineering time. Claude Haiku 4.5 costs $1 per million input tokens and $5 per million output tokens, about 5x cheaper than the frontier tier on both sides. Add prompt caching (roughly 90 percent off repeated context) and batch processing (50 percent off non-interactive jobs) and a well-built feature usually costs tens of dollars a month, not thousands.

**How do I make an AI feature feel fast?**

Stream the response token by token with server-sent events instead of waiting for the full answer. Total generation time stays the same, but the user sees output within a second, which changes how the feature feels entirely. In Next.js this is a route handler returning a ReadableStream.

**Do I need RAG to add AI to my app?**

Only if the feature needs your private or frequently changing data. Generic tasks like summarizing, rewriting, or classifying work fine with a well-built prompt. Start without retrieval and add it when the model starts being wrong about your facts.

**What is prompt caching and how much does it save?**

Prompt caching lets the provider reuse the parts of your prompt that repeat across requests, like the system prompt or shared documents. Cached reads bill at about a tenth of the normal input price, roughly 90 percent off, while writing to the cache costs about 1.25x once. For chat-style features the savings are large because most of the context repeats every turn.

---

Written by Roberto Lazar, founder of Dock30. Book a call: https://dock30.com/contact