⚡How much does a RAG system cost? The short answer
RAG — retrieval-augmented generation — is the architecture where a language model answers from your documents instead of from memory: the system retrieves the relevant passages, then generates an answer grounded in them, usually with citations. The build cost is dominated not by the model but by the pipeline that gets your documents into a shape worth retrieving from.
These tiers assume a proper build: ingestion, parsing, chunking, hybrid retrieval, grounded generation with citations, and an evaluation set. A demo-quality pipeline assembled from tutorials is much cheaper and is what most failed RAG projects started as.
See our RAG development services
| Tier | Typical range | Timeline | What is included |
|---|---|---|---|
| Focused internal assistant | $15,000 – $40,000 | 4–8 weeks | One corpus, one interface, ingestion pipeline, hybrid search, citations, eval set |
| Production customer-facing RAG | $40,000 – $90,000 | 8–14 weeks | Multiple sources, reranking, guardrails, analytics, cost controls, monitoring |
| Enterprise knowledge platform | $90,000 – $200,000+ | 4–8 months | Connectors (SharePoint, Drive, Confluence), permission-aware retrieval, SSO, audit logging |
| Ongoing run cost | Generation-dominated — see below | Ongoing | LLM generation, vector storage, embeddings, re-indexing, eval maintenance |
🧩Where the money actually goes in a RAG build
Most people assume the model is the expensive part. In the build, it is the cheapest — the model is an API call. The cost lives in the pipeline around it, and each stage below is where accuracy is won or lost.
Ingestion and connectors
Pulling documents from SharePoint, Google Drive, Confluence, databases or email — with sync schedules, deletion handling and, critically, permission inheritance from the source system.
Parsing
PDFs with multi-column layouts, tables, scanned documents needing OCR. This unglamorous stage decides whether the downstream system has clean text or garbage, and it absorbs more engineering time than any other single stage on real corpora.
Chunking strategy
Splitting documents into retrievable units that respect structure — sections, tables, headings — rather than naive fixed-size cuts. Wrong chunking is the most common cause of "our RAG gives vague answers".
Retrieval quality
Hybrid search combining BM25 keyword matching with vector similarity, metadata filters, and a reranking model that re-orders the top candidates. Each layer is optional on a demo and necessary in production.
Grounded generation
Answers built only from retrieved passages, with citations the user can verify, and an explicit refusal path when retrieval returns nothing good enough. The refusal path is what makes it trustworthy.
Evaluation
A golden set of real questions with known-good answers, scored for retrieval recall and answer correctness. The line item demos skip — and the only way to know a pipeline change helped instead of hurt.
Permission-aware retrieval
Filtering at query time so users only receive answers from documents they are allowed to see. Non-negotiable for internal deployments, and genuinely hard to bolt on afterwards.
⚖️Build vs buy: the honest comparison
There are three real options, not two. You can build custom, buy a managed RAG or enterprise-search platform, or assemble from open frameworks (LangChain or LlamaIndex plus a vector database) — which is really "build, with a head start".
| Dimension | Custom build | Managed RAG platform | Framework assembly |
|---|---|---|---|
| Upfront cost | $15,000 – $200,000+ | Low — configuration, not engineering | $8,000 – $40,000 |
| Monthly cost | Run cost only (generation + infra) | Platform fee on top of run cost | Run cost only |
| Time to production | 4 weeks – 8 months | Days to weeks | 2–6 weeks |
| Fit to your data and ACLs | Exact | Bounded by the platform's connector and permission model | Good, if the team is disciplined |
| Lock-in | None beyond your own code | Meaningful — your index and config live in their system | Low |
| Best for | Permission-sensitive corpora, custom UX, cost control at scale | Standard corpora, speed over control | Proving value before committing to either |
📊What a RAG system costs to run
The retrieval infrastructure is the cheap part, which surprises people. Embedding models cost fractions of a cent per thousand tokens on published rate cards — indexing a large corpus is typically a one-time bill measured in tens of dollars, with small incremental costs as documents change. Vector storage at modest scale runs from near zero (pgvector inside your existing Postgres) to a few hundred dollars a month on a dedicated managed vector database.
Generation is the bill. Every answer re-reads the retrieved chunks as input tokens, so run cost is query volume multiplied by chunks per answer multiplied by the model rate. A system answering 50,000 questions a month with five retrieved chunks per answer is pushing hundreds of millions of input tokens through the generation model — a number worth computing against current pricing before you pick the model tier.
This is why reranking and context pruning are cost features, not just quality features. Sending the three best chunks instead of the ten cheapest-to-fetch ones cuts the generation bill roughly in half and improves answers at the same time.
In most RAG systems we audit, retrieval infrastructure is under 10 percent of the run bill. Generation is the bill — optimise what gets sent to the model before optimising anything else.
🎯When buy wins, and when build wins
LLM integration servicesRAG vs fine-tuning: which does your use case need?What the generation side costs per task
Buy when the corpus is standard and speed matters
If your documents live in mainstream tools, your permission model is simple, and you need answers this quarter, a managed platform gets you there without an engineering project.
Buy when nobody will own it
A custom RAG system needs an owner — someone who watches eval scores and fixes retrieval when sources change. If that person does not exist, the platform fee is buying you their job.
Build when permissions are real
If wrong answers leaking across permission boundaries is a legal or compliance event, custom permission-aware retrieval with audit logging is usually the defensible choice.
Build when the interface is the product
Customer-facing RAG where citation UX, latency and cost per answer are competitive features belongs in your own codebase, on your own roadmap.
Hybrid is normal
A managed platform for generic corporate knowledge and a custom pipeline for the one corpus that is actually your crown jewels is a common, sane architecture.