Context caching

• Keep the shared part of your prompt at the very beginning and byte-identical between calls.
• Put anything that changes — user input, timestamps, random IDs — at the end.
• A single differing character at the start invalidates the whole prefix.

messages = [
    # 公共前缀放最前面，且逐字节不变 —— 这段会命中缓存
    {"role": "system", "content": LONG_SHARED_INSTRUCTIONS},
    {"role": "user",   "content": REFERENCE_DOCUMENT},

    # 变化的部分放最后
    {"role": "user",   "content": user_question},
]

The usage object reports how many input tokens hit the cache. The same split appears per request in your console usage log, so you can confirm the discount instead of guessing.

{
  "usage": {
    "prompt_tokens": 1000,
    "prompt_cache_hit_tokens": 700,
    "prompt_cache_miss_tokens": 300,
    "completion_tokens": 50
  }
}