Prompt Caching
Reuse stable prompt prefixes to reduce model costs and response latency.
Prompt caching is supported on the OpenUI Embed route through the OpenAI explicit caching option.
Why prompt caching matters
Prompt caching reuses work when requests share the same prompt prefix. This provides three main benefits:
- Compute-efficient: Avoid recalculating a prompt prefix that the model has already processed.
- Cheaper input tokens: Pay the model's reduced cached-input rate for reused tokens.
- Faster: Reduce the time spent processing input before the response starts.
Enabling caching
Mark the end of the reusable prefix with prompt_cache_breakpoint on the corresponding content block. Content up to and including that block is cached.
client.chat.completions.create(
model="anthropic/claude-sonnet-5",
messages=[
{
"role": "system",
"content": [
{
"type": "text",
"text": SYSTEM_PROMPT,
"prompt_cache_breakpoint": {"mode": "explicit"},
}
],
},
{"role": "user", "content": "..."},
],
)Provider behavior
| Provider | Behavior |
|---|---|
| Anthropic (Claude) | Content up to and including a marked block is cached. Without a breakpoint, your content is not cached. |
| OpenAI (GPT) | A stable prompt prefix is cached automatically. Explicit caching requires GPT-5.6 or later (not supported). |
| Google (Gemini) and others | Prefixes are cached automatically where the provider supports it. The breakpoint is ignored. |
For Anthropic models, caching your content is opt-in. OpenAI and Gemini cache eligible prefixes automatically.
Limitations
- Setting a cache lifetime (
ttl) on an individual breakpoint is not supported. The cache lifetime is fixed.