All posts

Auditing a real 20-turn Claude Code session, line by line

Nov 13, 2026 4 min read Oleg Savelev
costs claude-code benchmarks

We ran a controlled 20-turn refactoring session in Claude Code over a real Python repository, recorded every HTTP payload, and priced every single turn against Anthropic's production rates. The uncached baseline burned $4.82 in API credits. With deterministic AST slicing and append-only prefix locking, the exact same 20 turns cost $0.17. That is a net bill reduction of -96.5%.

The difference did not come from a smaller model or fewer test runs. It came from eliminating two mechanical flaws in how coding agents handle context: reading entire files when they only need one function, and breaking their own KV caches with volatile headers.

The benchmark setup

To make the audit reproducible, we replayed 20 real editing turns over src/*.py in a 14,000-line repository. Each turn represented a typical task: inspecting a function signature, adding an argument, running a unit test, fixing an import, and committing changes.

We priced the session at Claude 3.5 Sonnet standard rates:

  • Base input tokens: $3.00 per million tokens
  • Cache write tokens: $3.75 per million tokens
  • Cached input reads: $0.30 per million tokens (a 90% discount)
  • Output tokens: $15.00 per million tokens

Without an optimization layer, Claude Code re-sends the entire accumulating conversation history on every turn. If turn 5 reads src/metrics.py (840 lines) to check one helper function, those 840 lines stay in context for turns 6 through 20.

Turn-by-turn: where the tokens went

Here is the exact token breakdown across key turns of the 20-turn replay:

Turn Action taken Baseline input Sliced input Cache hit rate Baseline cost TokenSafe cost
1 Repo exploration and plan 12,450 1,820 0.0% $0.037 $0.007
5 Read telemetry helper 38,120 4,210 91.2% $0.114 $0.003
10 Refactor event serializer 72,400 8,950 90.5% $0.217 $0.006
15 Run tests and fix imports 114,300 14,200 90.8% $0.343 $0.009
20 Final audit and git commit 158,600 18,400 91.0% $0.476 $0.012

Across all 20 turns, the cumulative input token volume dropped from 1,420,800 tokens to 220,200 tokens. That is an 84.5% cut in raw wire volume. Because the remaining prefix remained byte-identical across consecutive calls, 90.8% of those remaining tokens qualified for Anthropic's $0.30/1M prompt cache rate.

The repeat-read tax

The biggest driver of invoice bloat in Claude Code is the unconstrained View tool call. During our unoptimized baseline run, the agent opened src/metrics.py five separate times across 20 turns. It opened src/cloud/handlers.py seven times.

Each time Claude Code re-read src/metrics.py, it pulled in 840 lines of code. The agent only ever edited one 24-line function: record_event(). The remaining 816 lines were passive spectators. They contained database connection utilities, export routines, and type aliases. The model re-read those same 816 lines on turn 5, turn 8, turn 11, turn 14, and turn 17, paying full input rates each time.

With local AST slicing, the call:

tokensafe smart-read src/metrics.py --symbol record_event

returns a 48-line payload. It includes the full target function and compact one-line signatures for its immediate dependencies. The 816 lines of unrelated code fold into single-line comments. The agent gets the exact type contracts it needs, but the input payload shrinks by 94%.

Why typical caching setups fail

Many teams turn on prompt caching and wonder why their Anthropic bills barely budge. In our audit, we tracked why naive caching failed during the baseline run:

  1. Reordered tool schemas: Claude Code serialized tools in differing dictionary orders between turns, busting the cache prefix at token 120.
  2. Volatile timestamps: The system prompt injected a current ISO timestamp on every turn, invalidating the entire cached history behind it.
  3. Mid-prompt truncation: When context neared the limit, the harness pruned turns from the middle of the chat. That single edit forced the model to re-ingest the entire conversation as new input.

TokenSafe resolves this by running as a local stdio and HTTP proxy. It normalizes tool schemas, keeps volatile timestamps out of the cache prefix zone, and uses an append-only governor that never modifies historical turns in place.

The honesty breakdown

We report numbers using a strict honesty split:

Metric label Category How it is calculated
Sliced byte reduction (-84.5%) Measured Exact byte counts measured across the local proxy socket
Cache hit discount (-96.5% cost) Measured Metered Anthropic prompt cache tokens on actual API responses
Uncached baseline comparison Modeled Mathematical replay of identical tasks with full-file reads

You do not need to take our word for the baseline. The telemetry database records both numbers side by side so your engineering team can audit every cent.

Try it on your next session

Install TokenSafe locally:

npx tokensafe@latest init

Run your next coding sprint in Claude Code through the local proxy. At the end of the week, check the Friday ROI digest. If your prompt cache hit rate does not cross 85% and your input token spend does not fall by at least half, run tokensafe uninstall.

Paying full price five times for the same 800 lines of code is not the cost of doing AI engineering. It is an unpatched leak in your toolchain.