Home › Tools › RAG chatbot cost calculator

RAG Chatbot API Cost Calculator

RAG chatbots are expensive because every question carries thousands of retrieved tokens as input. At 5,000 conversations a month with 3,000 retrieved tokens each, the input bill alone ranges from $$1.90 on the cheapest model to $$190 on the most expensive. This page prices the full workload across all 28 models.

5,000Conversations / month

3,000Retrieved tokens / question

500Output tokens / answer

28Models compared

Top 5 cheapest models for this RAG workload

Monthly cost for 5,000 conversations, 3,000 retrieved tokens + 500 system + 200 history + 100 question per request, 500 output tokens per answer. Sorted by total.

RAG chatbot cost — 5,000 conversations/month, USD, rates verified October 2026
#	Model	Provider	Input	Output	Total
1	GPT-6 Luna	OpenAI	$1.90	$1.25	$3.15
2	GPT-5.6 Luna	OpenAI	$3.80	$3	$6.80
3	GPT-5.4 nano	OpenAI	$3.80	$3.13	$6.93
4	DeepSeek V4.1 Flash	DeepSeek	$5.70	$3	$8.70
5	Gemini 3.5 Flash-Lite	Google Gemini	$5.70	$6.25	$11.95

All models: RAG cost breakdown

Full comparison across all 28 models. Input includes retrieved context, system prompt, chat history, and user question.

RAG chatbot cost — all models, 5,000 conversations/month
#	Model	Provider	Input	Output	Total
1	GPT-6 Luna	OpenAI	$1.90	$1.25	$3.15
2	GPT-5.6 Luna	OpenAI	$3.80	$3	$6.80
3	GPT-5.4 nano	OpenAI	$3.80	$3.13	$6.93
4	DeepSeek V4.1 Flash	DeepSeek	$5.70	$3	$8.70
5	Gemini 3.5 Flash-Lite	Google Gemini	$5.70	$6.25	$11.95
6	Gemini 3.8 Flash	Google Gemini	$14.25	$9.38	$23.63
7	Gemini 3.7 Flash	Google Gemini	$14.25	$9.38	$23.63
8	GPT-5.4 mini	OpenAI	$14.25	$11.25	$25.50
9	Grok 4.3	xAI Grok	$23.75	$6.25	$30
10	Claude Haiku 4.5	Anthropic	$19	$12.50	$31.50
11	DeepSeek V4 Pro	DeepSeek	$25.08	$9.90	$34.98
12	Gemini 2.5 Pro	Google Gemini	$23.75	$25	$48.75
13	Gemini 3.5 Flash	Google Gemini	$28.50	$22.50	$51
14	Grok 4.6	xAI Grok	$38	$15	$53
15	GPT-4.1	OpenAI	$38	$20	$58
16	GPT-6.1 Sol	OpenAI	$38	$25	$63
17	GPT-6 Sol	OpenAI	$38	$25	$63
18	Claude Sonnet 5	Anthropic	$38	$25	$63
19	GPT-5.6 Terra	OpenAI	$38	$30	$68
20	Gemini 3.1 Pro (Preview)	Google Gemini	$38	$30	$68
21	GPT-5.2	OpenAI	$33.25	$35	$68.25
22	GPT-5.4	OpenAI	$47.50	$37.50	$85
23	Claude Sonnet 4.6	Anthropic	$57	$37.50	$94.50
24	GPT-5.6 Sol	OpenAI	$76	$50	$126
25	Claude Opus 5	Anthropic	$95	$62.50	$158
26	GPT-5.5	OpenAI	$95	$75	$170
27	GPT-6 Astra	OpenAI	$190	$125	$315
28	Claude Fable 5.1	Anthropic	$190	$125	$315

Input = 3,800 tokens x 5,000 requests. Output = 500 tokens x 5,000 requests. Prompt caching is not applied — add a 60-80% input discount if your knowledge base prefix is cacheable.

How to reduce RAG costs

Reduce chunk size. 500-token chunks instead of 3,000 save 83% on input.
Deduplicate retrieved chunks. Overlapping chunks waste tokens.
Cache the system prompt. System instructions repeat on every request — cache them.
Use a cheaper model for retrieval. The retrieval step does not need a flagship model.
Batch non-urgent queries. Batch API halves input and output rates.

Keep going

Calculate a real monthly bill

FAQ

What is RAG?

Retrieval-Augmented Generation: the model receives retrieved document chunks alongside the user question. The retrieved context is billed as input tokens on every request.

How much does a RAG chatbot cost per month?

A typical RAG chatbot with 5,000 conversations, 3,000 retrieved tokens per question, and 500 output tokens costs between $50 and $500 per month depending on the model and cache hit rate.

Does prompt caching help RAG?

Yes, if the retrieved context repeats across conversations. A shared knowledge base prefix can be cached, reducing input cost by 60-80%.

What is the biggest cost driver in RAG?

Retrieved context. At 3,000 tokens per question and 5,000 questions per month, that is 15 million input tokens — before any chat history or system prompt.

How do I reduce RAG costs?

Reduce chunk size, deduplicate retrieved chunks, cache the system prompt, use a cheaper model for retrieval, and batch non-urgent queries.

Keep going

Calculate a real monthly bill