AWS Guide: Boosting GPT-5.5 Prompt Cache Hits on Bedrock Mantle

Select Language:

If you’re working with the Bedrock-Mantle Responses API and want to make sure your prompt caching is working correctly with the openai.gpt-5.5 model, here’s a simple guide to help you troubleshoot and understand what’s happening.

First, you might notice that even when you send multiple requests back-to-back with identical prompts — specifically with the same prompt_cache_key and a large prefix of about 9,000 tokens — the cache isn’t always being used. For example, if you run six requests in a row, only one of those may hit the cache, while the others go straight to processing. Sometimes, different runs show different patterns, and it can be confusing because there doesn’t seem to be a clear reason why the cache is or isn’t used on certain requests.

This behavior appears to contradict what the documentation states. According to Amazon’s Bedrock documentation, for models earlier than GPT-5.6 like openai.gpt-5.5, prompt caching is supposed to happen automatically for prompt prefixes of 1,024 tokens or longer. You shouldn’t need to add extra parameters for caching to work.

Here’s what you can do to verify caching is functioning:

Run your tests: Send multiple consecutive requests that use the same prompt_cache_key and similar prompt lengths.
Check the responses: Look at the raw usage data in the API responses. For the request that hits the cache, you should see details like "cached_tokens" being non-zero or other indicators of cache usage.
Compare behavior: If most requests show zero cached tokens but one shows a significant cache hit, it indicates that caching isn’t consistently functioning as expected.

If you want to observe this behavior more clearly, you can use a script similar to the one at this link [Gist URL], which automates sending requests and logs detailed usage data.

In summary:

Multiple identical requests may not always use the cache, especially with GPT-5.5.
The caching might be inconsistent or triggered under specific conditions.
Double-check your prompt length and make sure it meets requirements.
Keep an eye on the raw usage data, looking specifically for cache indicators.

If caching continues to be unreliable, consider reaching out to support or reviewing updates from the documentation, as there might be changes or issues specific to certain models or API versions.