Prompt caching enables Large Language Models to reuse pre-computed attention states across API requests, cutting input costs by 50% to 90% and reducing latency.
Prompt Caching (also referred to as Context Caching or Prefix Caching) is an inference optimization technique utilized by modern Large Language Model (LLM) API providers—such as Anthropic Claude, OpenAI GPT, and Google Gemini—that avoids recomputing transformer Key-Value (KV) attention states for repeated prompt prefixes. By persisting the mathematical representations of static system instructions, multi-turn conversational histories, code repositories, or large RAG (Retrieval-Augmented Generation) documents in GPU VRAM, providers reduce time-to-first-token (TTFT) and offer significant discounts (50% to 90% off standard input token rates) for cached tokens.
Model your cache read/write cost savings with our AI Cost Calculator, count your prompt token volume with the AI Token Counter, or assemble modular cached system messages using the AI Prompt Builder.
| Specification | Anthropic Claude | OpenAI (GPT-4o, o1, o3) | Google Gemini (2.5 Pro / Flash) |
|---|---|---|---|
| Caching Mechanism | Explicit cache_control: { type: 'ephemeral' } breakpoints |
Automatic prefix matching (exact prefix match) | Explicit context cache objects via SDK |
| Minimum Prompt Threshold | 1,024 tokens (Claude 3.5 Sonnet / Haiku / Opus) | 1,024 tokens | 32,768 tokens (1.5 Pro) / 1,024 tokens (Flash) |
| Cache Read Discount | 90% discount (e.g. $0.30/M vs $3.00/M standard) | 50% discount (e.g. $1.25/M vs $2.50/M standard) | 75% discount on input tokens |
| Cache Write Surcharge | +25% premium on initial cache write creation | No write surcharge (standard input rate) | Storage charge ($/hour) based on TTL |
| Cache Lifetime (TTL) | 5-minute rolling TTL (refreshed on every cache hit) | 5 to 10-minute dynamic heuristic eviction | Configurable TTL (default 1 hour, customizable) |
| Maximum Cache Breakpoints | Up to 4 explicit breakpoints per API request | Unlimited (automatic prefix tree) | Named cache resources |
In standard auto-regressive transformer inference, every input token must pass through multi-head self-attention layers to compute Key ($K$) and Value ($V$) tensors:
Request 1 (Cold Start):
[ System Instructions + 20,000 Token Knowledge Base ] ──► [ Full Attention Pass ] ──► Compute & Write KV Cache ──► Generate Response
│
▼
Request 2 (Warm Cache Hit): [ Stored in GPU VRAM ]
[ Cached Prefix (Billed at 10%) ] + [ New User Query ] ──► [ Lookup KV Cache ] ──► Skip Attention Pass ──► Generate Response Fast
While cache reads provide massive discounts, some providers (like Anthropic) charge a 25% write premium during initial cache creation. To achieve positive return on investment (ROI), cached prompts must receive enough repeated queries to offset the creation surcharge:
$$\text{Break-even Hits} = \frac{\text{Cache Write Surcharge}}{\text{Per-Call Read Discount}}$$
For Anthropic Claude 3.5 Sonnet ($3.00/M base, $3.75/M write, $0.30/M read):
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic();
// Static enterprise policy or code context (e.g. 15,000 tokens)
const enterpriseKnowledgeBase = `... large static documentation ...`;
export async function askEnterpriseAssistant(userQuestion: string) {
const response = await client.messages.create({
model: 'claude-3-7-sonnet-20250219',
max_tokens: 1000,
system: [
{
type: 'text',
text: 'You are a staff engineer analyzing architecture compliance.',
},
{
type: 'text',
text: enterpriseKnowledgeBase,
// Set explicit cache breakpoint at the end of static context
cache_control: { type: 'ephemeral' },
},
],
messages: [
{
role: 'user',
content: userQuestion,
},
],
});
// Track cache efficiency in telemetry
const usage = response.usage;
console.log(`Cache Read Tokens (90% off): ${usage.cache_read_input_tokens ?? 0}`);
console.log(`Cache Write Tokens: ${usage.cache_creation_input_tokens ?? 0}`);
console.log(`Uncached Input Tokens: ${usage.input_tokens}`);
return response.content[0];
}
No. Because transformer attention values for the cached prefix are mathematically identical to recomputing them from scratch, the model produces the exact same token probabilities and output quality.
Prompt Caching happens inside the model provider's GPU infrastructure at the tensor level, allowing dynamic, unique completions while reusing prompt prefill computations. Semantic Caching (e.g., using Redis or vector embeddings) stores and returns previously generated responses for similar user queries without invoking the LLM at all.
Use our AI Cost Calculator to enter your input, output, cache read, and cache write tokens to compare net monthly bills across OpenAI, Anthropic, and Google Gemini with live pricing rates.
Free, browser-based utilities to test, generate, and inspect Prompt Caching (LLM Context Caching & KV Cache) payloads directly.
Model and compare LLM API spend across 120+ providers for any workload with prompt caching and batch pricing.
Count tokens and estimate API costs for 100+ LLMs with exact BPE and provider tokenization.
Build structured LLM prompts with per-section token counting, variable injection, and provider-aware exports for OpenAI, Anthropic, and Google.