# RAG vs fine-tuning in 2026: how to customize an LLM

*By Roberto Lazar, founder of Dock30 · Published 2026-06-16 · Updated 2026-07-25 · 7 min read*

When prompting, caching, RAG, or fine-tuning is the right way to adapt an LLM in 2026, with current pricing and what changed for hosted fine-tuning.

To customize an LLM in 2026, work down a ladder: prompt engineering first, prompt caching to make large repeated prompts cheap, RAG when the model needs your private or frequently changing data, and fine-tuning only when you need behavior (a strict format, a specific tone, a narrow task) that prompting cannot make consistent. Fine-tuning also got much harder to buy this year. OpenAI is winding down its fine-tuning API and Anthropic offers no public fine-tuning of current Claude models, so tuning now mostly means training open-weight models yourself. Most production systems ship **prompting plus RAG** and never touch the last rung.

The core framework has held up since we first wrote this guide. What changed is the ground under one of the options, plus a whole rung that older articles skip.

## You mostly cannot fine-tune frontier models anymore

The original version of this article treated fine-tuning hosted frontier models as something you could just order. In mid-2026 you largely cannot. OpenAI is winding down its fine-tuning API and platform: it is closed to new users, and existing users can create jobs only "for the coming months" according to the [OpenAI model optimization docs](https://developers.openai.com/api/docs/guides/model-optimization). Only GPT-4.1 and GPT-4.1-mini remain tunable with supervised fine-tuning and DPO, plus o4-mini for reinforcement fine-tuning. The GPT-5.x family, the models you would actually want, cannot be fine-tuned at all. The [community discussion around the wind-down](https://community.openai.com/t/openai-is-winding-down-the-fine-tuning-api-and-platform-discussion-thread/1380522) makes clear this is a deliberate retreat, not a pause.

Anthropic never opened current Claude models for public fine-tuning either. So when someone says "we should fine-tune" in 2026, what they are really proposing is: pick an open-weight model (Llama, Qwen, Mistral), train a LoRA or QLoRA adapter, and serve it through a host like Together, Fireworks, or Bedrock. That is a real engineering commitment. You own the dataset, the evals, the serving setup, and the upgrade path.

One warning from experience: a fine-tuned model dies with its base model. When the provider deprecates the base, your carefully trained adapter goes with it, and you retrain from scratch on whatever replaced it. We have watched teams lose months of tuning work to a deprecation notice. All of this pushes the practical answer further toward retrieval.

## Which technique fixes which problem

| Technique | Fixes | Cost and effort | Reach for it when |
|---|---|---|---|
| Prompt engineering | Unclear instructions, missing context | Lowest, minutes to iterate | Always start here |
| Prompt caching | The same large context sent on every call | Near zero, a config change | High-volume calls sharing a big prefix |
| RAG (retrieval) | The model does not know your data | Medium, an index plus a pipeline | Private, large, or changing knowledge |
| Fine-tuning | The model will not behave consistently | Highest: data, training, evals, hosting | Strict format or tone at scale, open weights acceptable |

A quick test that settles most debates: if the model is wrong about facts, you need RAG. If it is right but will not hold the format or tone, you need fine-tuning (after harder prompting). If your bill is exploding because every call carries the same 30-page context, you need caching. If it just needs clearer direction, you need a better prompt.

## Prompt engineering: still the first move

Exhaust prompting before anything else. Clear instructions, a few well-chosen examples, structured output formats, and a system prompt that defines role and constraints get you further than most teams expect, especially with current models like Claude Sonnet 5 or GPT-5.6, which follow instructions far better than the models this advice was originally written for. Iterating a prompt costs minutes. Skipping straight to fine-tuning is still the most common over-engineering mistake we see when scoping [AI features for client products](/services/ai-automation).

## Prompt caching: the rung most old guides are missing

The old advice said: if you send the same huge prompt millions of times, fine-tune the behavior into the model and shrink the prompt. That advice is mostly obsolete. Prompt caching answers the same cost problem for a fraction of the effort. Anthropic prices cache reads at about 0.1x the base input rate, roughly **90 percent off** repeated context, per the [prompt caching docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching). OpenAI's cached input runs about 10x cheaper than uncached, per [TLDL's July 2026 pricing breakdown](https://www.tldl.io/resources/openai-api-pricing).

Concrete numbers, since the models have moved on from the ones 2024-era guides cite. Current [Anthropic pricing](https://platform.claude.com/docs/en/pricing) puts Claude Opus 4.8 at $5/$25 per million tokens in and out, Claude Sonnet 5 at $3/$15, and Claude Haiku 4.5 at $1/$5. OpenAI's GPT-5.6 tiers run roughly $1/$6 up to $5/$30 per [aipricing.guru](https://www.aipricing.guru/openai-pricing/). So a 20,000-token static prefix on Sonnet 5 costs about 6 cents per call uncached. Cached, roughly 0.6 cents. At 100,000 calls a month that is the difference between $6,000 and $600 for the prefix alone, with zero training data and zero maintenance. Do this before you even think about tuning.

## RAG: give the model your knowledge

Retrieval-augmented generation puts your data in front of the model at query time instead of baking it into weights. The pipeline is short:

1. Chunk and embed your documents into a vector store.
2. At query time, retrieve the most relevant chunks.
3. Pass them to the model as context alongside the question.

RAG wins whenever knowledge is private, large, or changes often: docs, policies, product catalogs, support history. You update an answer by updating a document, and you get citations back to the source, which matters more than people admit once a customer challenges an answer. It is the backbone of most useful business AI, including [chatbots grounded in your own data](/blog/build-ai-chatbot-website).

On the vector store: you probably do not need a dedicated one. pgvector runs free inside the Postgres you already operate and holds up well into the millions of vectors, per [Encore's pgvector vs Pinecone comparison](https://encore.dev/articles/pgvector-vs-pinecone). Pinecone earns its bill when you want zero-ops scale beyond that. Our default on client projects is pgvector, because one database is easier to run than two, and most products never reach the scale where that choice becomes wrong.

## Fine-tuning: change behavior, accept the ops bill

Fine-tuning adjusts weights on your examples. It does not reliably teach new facts; OpenAI's own [supervised fine-tuning guide](https://developers.openai.com/api/docs/guides/supervised-fine-tuning) frames it as a tool for behavior, format, and tone. Where it genuinely shines: a strict JSON shape a small model keeps drifting on, a brand voice you need at high volume, or a narrow classification task where a tuned small open model beats a prompted frontier model on cost and latency.

The newest hosted method worth knowing about is reinforcement fine-tuning on o4-mini, where you write programmable graders that score outputs and the training loop optimizes against them, per the [OpenAI RFT guide](https://developers.openai.com/api/docs/guides/reinforcement-fine-tuning). It works well for tasks you can score with code, and it is one of the few hosted tuning doors still open.

For everything else, budget honestly: a clean labeled dataset, an eval suite you trust, a training pipeline, serving infrastructure, and a plan for the day the base model is deprecated. If that list made you tired, good. That instinct is correct for about nine projects out of ten.

## The stack most teams actually ship

Real systems rarely pick one technique. The pattern we ship most often: prompting defines the task and guardrails, caching keeps the bill sane, RAG supplies current private knowledge with citations, and fine-tuning stays on the shelf unless volume and a measured quality gap justify it. Start at the top of the ladder, measure, and only descend when the data says you must. Each step down costs more and moves slower.

The short checklist:

- Answer is factually wrong or stale: RAG.
- Needs data that only lives in your systems: RAG.
- Right answer, inconsistent format or tone: harder prompting with examples first, then fine-tuning.
- Same huge prompt at high volume: prompt caching, not fine-tuning.
- Have not seriously iterated the prompt yet: stop and do that first.

If you are budgeting one of these builds, our breakdown of [what a custom AI agent costs in 2026](/blog/custom-ai-agent-cost-2026) covers the numbers side, and [adding AI to an existing app](/blog/add-ai-to-your-app) covers the integration side.

We have scoped this decision with a lot of founders since we started shipping AI features, and the honest answer is usually the cheap one: a better prompt, a cache header, or a small RAG pipeline over Postgres. If you want a second opinion on which rung your product actually needs, [book a free 15-minute call](https://calendly.com/dock30/15min) or reach us through the [contact page](/contact). We will tell you if the answer is "none of the above," because sometimes it is.

## Frequently asked questions

**Is RAG better than fine-tuning?**

They solve different problems. RAG gives the model knowledge it does not have, like your docs or product data, while fine-tuning changes how the model behaves in terms of format, tone, or a narrow task. For most business use cases that need private or current data, RAG is the better and cheaper starting point, and in 2026 it is also far easier to buy than fine-tuning.

**Can you still fine-tune GPT or Claude models in 2026?**

Mostly no. OpenAI is winding down its fine-tuning API: it is closed to new users, existing users can create jobs only for a limited period, and the GPT-5.x family cannot be fine-tuned at all. Anthropic offers no public fine-tuning of current Claude models. In practice, fine-tuning in 2026 means open-weight models like Llama, Qwen, or Mistral trained with LoRA or QLoRA on hosts such as Together, Fireworks, or Bedrock.

**Does fine-tuning teach an LLM new facts?**

Not reliably. Fine-tuning shapes behavior, format, and style, and OpenAI's own supervised fine-tuning guide frames it that way. For factual knowledge, especially data that changes, retrieval (RAG) is the right tool because you can update an answer by updating a document instead of retraining a model.

**What is prompt caching and how much does it save?**

Prompt caching lets the provider store the static part of your prompt so repeated calls reuse it at a steep discount. Anthropic prices cache reads at about 0.1x the base input rate, roughly 90 percent off repeated context, and OpenAI's cached input is about 10x cheaper than uncached. It replaces the old advice of fine-tuning a model just to shrink a large prompt.

**Can you use RAG and fine-tuning together?**

Yes, and some production systems do: a fine-tuned open-weight model handles consistent behavior or a narrow task, while RAG supplies current knowledge at query time. That said, most systems never need the fine-tuning half. Prompting plus RAG covers the large majority of real business use cases.

---

Written by Roberto Lazar, founder of Dock30. Book a call: https://dock30.com/contact