Measure prompt tokens, completion tokens, retries, and model choice per request first. If the workload is conversational or agentic, group calls by session or workflow so you can see whether history or retrieval is inflating prompt size over time.

Typical fixes are to cap history growth, summarize after a few turns, trim oversized system or tool prompts, and route lower-complexity calls to cheaper models once you can compare cost by workflow.

Question context worth covering: Look for engineering questions about retrieval chunking, context inflation, and token observability.

Optional supporting resource: https://zenllm.io/rag-cost-optimization