This advisor estimates prompt-cache hit rate from call spacing and TTL, then estimates monthly savings for a stable prompt prefix.
Loading calculator...
It models average traffic, so bursty production traffic can differ.
How to use it
Enter stable prefix tokens, shared calls per hour, cache TTL, input price, and cached price percentage. Defaults are 3,000, 600, 5 minutes, $3, and 10%.
Use a byte-identical stable prefix at the front of the prompt.
Read the verdict, hit rate, average gap, saved per hit, and monthly saving.
Fields explained
Stable prefix tokens means reusable system prompt and docs.
Calls / hour sharing it defaults to 600.
Cache TTL (minutes) defaults to 5.
Cached price (% of input) defaults to 10%.
Reading the results
| Verdict | Rule |
|---|---|
| Cache it now | Hit rate at least 70% |
| Worth caching | Hit rate at least 30% |
| Marginal | Positive traffic below 30% |
600 calls per hour creates a 6-second average gap.
Warning: cache triggering requires the identical prefix first.
Calls at zero receive the Add traffic data verdict.
The formula
gap = 3600 / calls; hit = min(0.98, max(0, 1 - gap / TTLseconds)). Savings multiply monthly calls, hit rate, and full-price minus cached-price cost.
| Default | Value |
|---|---|
| Gap | 6 seconds |
| TTL | 300 seconds |
| Hit rate | 98% capped |
One full 3,000-token prefix costs $0.009; cached at 10% it costs $0.0009, saving $0.0081 per hit.
Info: real traffic is bursty, while this model uses average spacing.
Monthly calls equal calls per hour times 24 times 30.
Worked examples
Sparse calls. One call per hour and five-minute TTL gives zero estimated hits.
Default. The hit rate reaches its 98% cap.
Medium traffic. Twelve calls per hour and five-minute TTL estimate 80% hits.
Scale. 600 calls per hour yields 432,000 monthly calls before hit adjustment.
Common mistakes
Changing the prefix prevents cache matching.
Using a provider discount without checking its terms changes savings.
Assuming averages describe bursts can misstate hit rate.
Danger: a zero TTL produces zero hit rate.
Measure provider cache logs.
FAQ
Is 98% the maximum?
Yes.
The source caps the estimate at 0.98.
What is cached price?
It is a percentage of full input cost.
Default is 10%.
Does cache use output tokens?
No.
This is prefix input cost.
What if calls are zero?
Hit rate is zero.
The verdict asks for traffic data.
What should be tested?
Measure cache hits.
Compare them with the estimated rate.
Disclaimer
This is an educational estimate for planning. Results depend on model, provider, hardware, configuration and current pricing.
The live calculator is the source of truth. Verify with a small real test before scaling spend.
Been trying to integrate this into our Python FastAPI wrapper and honestly the documentation around the cache TTL calculation is pretty confusing. The formula gap = 3600 / calls makes sense mathematically, but I’m hitting issues when traffic fluctuates during peak hours. We’re running around 450 calls per hour on average but seeing spikes to 1200+, and the calculator assumes steady state which doesn’t match reality. Also the warning about byte-identical prefixes is kind of buried—spent two hours debugging why our cache hit rate was 0% before realizing we had a single space difference in the system prompt across different API calls. The SDK docs don’t really explain how to structure this properly for a production environment. Are there any code examples on GitHub showing this working end-to-end? The savings look attractive (we’d save like $80/month at our scale) but I need to understand the implementation better before pushing this to production.
Regarding the bursty traffic issue—you’re identifying something really important that the calculator explicitly flags. The model uses average spacing, so your 450 baseline with 1200+ spikes will indeed see lower actual hit rates than the estimate predicts. Here’s what we recommend: test with your real traffic patterns first. Pull your actual call logs for a 24-hour period, calculate the real inter-request gaps (not just average), and compare against the 300-second TTL you’re using. If you’re seeing gaps regularly exceed 5 minutes during off-peak, consider extending TTL or testing with a shorter stable prefix that gets higher traffic density. On the byte-identical prefix issue—this is a common stumbling block. The provider’s cache operates at the token level, so even whitespace differences in system prompts matter. We should improve the documentation here, but in practice: version control your exact prompt template, use environment variables only for the variable portions (user input), and store the stable prefix separately from your dynamic content. For GitHub examples, check out the LangChain and LlamaIndex repositories—both have prompt caching examples using Claude’s implementation that follow this pattern. The cost savings at your scale are real, but they materialize over weeks as you accumulate hits, so measure actual provider cache logs to validate before committing.