When your monthly Anthropic or OpenAI invoice jumps by hundreds of dollars, or when your 5-hour Claude Code quota evaporates before lunch, you start looking for token optimization tools.
Most developers quickly encounter two popular GitHub projects:
JayPokale/Chisle: a markdown ruleset that tells models to be terse and follow YAGNI, paired with a bash output elider.alexgreensh/token-optimizer: a Claude Code and Codex hooks wrapper that compresses bash outputs and extracts file summaries.
On paper, both promise to slash your context bill. But what happens when you clone real production repositories like Flask, FastAPI, or Go's Gin, and actually measure what each tool passes to the model?
We ran that experiment. We cloned 10 open-source repositories across Python, TypeScript, Go, Rust, C, and Java. We fed the exact same files through Chisle, token-optimizer, and TokenSafe, and we counted every single token with a calibrated BPE tokenizer on real bytes.
Here are the raw measured numbers, the syntax validation checks, and the failures.
The Three Fundamental Approaches
The three tools attack the problem with completely different mental models:
- Chisle (Prompt Engineering + Bash Truncation):
Chisle injects system prompt instructions telling the model to write short diffs and avoid conversational fluff. For tool outputs, it trims bash logs exceeding 8,000 characters. By design, Chisle contains a hardcoded safety rule:
NEVER_COMPRESS = ['read', 'edit', 'write']. Because it has no syntax parser, Chisle intentionally refuses to touch file reads out of fear of breaking string replacement. - token-optimizer (Lifecycle Hooks + Regex Outlines):
token-optimizer hooks into Claude Code's post-tool events. For large Python and TypeScript files, its
structure_map.pyscript replaces file content with a text outline. However, it relies on regular expressions and caps its skeleton at 4 classes and 4 methods per class. - TokenSafe (Native Tree-sitter AST + Reversible Hash Anchors):
TokenSafe uses native C Tree-sitter grammars across 13 programming languages. When an agent targets a function, TokenSafe preserves the full executable body of that function and folds surrounding helpers into 1-line typed signatures (
dep_mode="signature"). Every folded block receives an 8-character Blake2b hash anchor (#hash), allowing any agent to restore 100% bit-exact original bytes in zero milliseconds viatokensafe:expand_ref.
The Real-Repository Test
We tested all three tools on the primary hub files of popular open-source repositories. For each file, we gave the agent a realistic instruction: inspect and edit a specific target function.
To count as valid token savings, an output must pass deterministic zero-loss gates:
- The output must be valid, parseable code (not plain text or broken syntax).
- The target function must actually exist in the output with its full body intact.
- Caller contracts and top-level definitions must remain visible.
If a tool strips the target function or outputs broken syntax, the scenario fails the gate. Savings without correctness are just truncation.
Here is the head-to-head measurement:
| Repository & Target File | Baseline Tokens | Chisle (Ruleset) | token-optimizer | TokenSafe (Tree-sitter) | TokenSafe Gates |
|---|---|---|---|---|---|
flask (src/flask/app.py)Target: full_dispatch_request |
18,864 | 18,864 (0.0%) | 317 (-98.3%) [Syntax error / no symbol] |
3,784 (-79.9%) | PASS (valid AST + symbol) |
requests (src/requests/sessions.py)Target: Session.request |
9,743 | 9,743 (0.0%) | 651 (-93.3%) | 2,849 (-70.8%) | PASS (valid AST + symbol) |
fastapi (fastapi/routing.py)Target: APIRouter.add_api_route |
73,721 | 73,721 (0.0%) | 139 (-99.8%) [Syntax error / no symbol] |
39,529 (-46.4%) | PASS (valid AST + symbol) |
rich (rich/console.py)Target: Console.print |
28,675 | 28,675 (0.0%) | 585 (-98.0%) [Syntax error / no symbol] |
7,710 (-73.1%) | PASS (valid AST + symbol) |
httpie (httpie/core.py)Target: main |
2,933 | 2,933 (0.0%) | 266 (-90.9%) | 689 (-76.5%) | PASS (valid AST + symbol) |
gin (context.go, Go)Target: Context.Next |
15,032 | 15,032 (0.0%) | 115 (-99.2%) [Unsupported language] |
9,904 (-34.1%) | PASS (valid AST + symbol) |
curl (lib/url.c, C)Target: Curl_connect |
23,246 | 23,246 (0.0%) | 204 (-99.1%) [Unsupported language] |
5,343 (-77.0%) | PASS (valid AST + symbol) |
retrofit (Retrofit.java, Java)Target: Retrofit.create |
8,189 | 8,189 (0.0%) | 127 (-98.4%) [Unsupported language] |
4,617 (-43.6%) | PASS (valid AST + symbol) |
What Happened During the Run
1. Why Chisle Saved 0% on File Reads
In every single code file, Chisle passed 100% of the raw tokens into context.
When your agent reads app.py in Flask, you pay for all 18,864 tokens. When it reads fastapi/routing.py, you pay for all 73,721 tokens. Over a 10-turn conversation, that single file costs over 700,000 input tokens.
Why does Chisle do this? Because its authors recognized a real danger: if you cut code without an AST parser, the model will hallucinate edits against code it cannot see. To prevent broken edits, Chisle added NEVER_COMPRESS to file reads. Chisle saves tokens only when an agent writes code or runs noisy bash commands. On the largest source of context consumption, reading code, it provides zero relief.
2. Why token-optimizer Failed 6 Out of 8 Zero-Loss Gates
token-optimizer produced very small token numbers: 317 tokens on Flask, 139 tokens on FastAPI.
At first glance, an engineer might ask: why are their numbers smaller than TokenSafe? Is 317 tokens better than 3,784 tokens?
The answer becomes obvious the moment you inspect the actual text token-optimizer passed to the model:
python skeleton
lines: 1687
imports (50):
- from __future__ import annotations
classes (1):
- class Flask (bases=App) @ L108
methods:
- Flask.ensure_sync(self, func)
- Flask.async_to_sync(self, func)
- Flask.get_send_file_max_age(self, filename)
- Flask.request_context(self, environ)
This text caused two fatal issues:
- Invalid Syntax: This is not Python code. It cannot be parsed by
ast.parse(), cannot be compiled, and cannot be edited with surgical diff tools. - Missing Target Function: In
structure_map.py, token-optimizer hardcodesMAX_SKELETON_METHODS_PER_CLASS = 4. Flask has over 60 methods. token-optimizer arbitrarily grabbed the first 4 methods and discarded the other 56 methods, includingFlask.full_dispatch_request.
Think of it like tearing 99% of pages out of a book and handing the reader a 4-item table of contents. Yes, token count drops by 98%. But the reader cannot read or edit the chapter because the text no longer exists.
If you tell an agent "fix the exception handling in Flask.full_dispatch_request", but feed it token-optimizer's outline, the model has literally never seen full_dispatch_request. It has to guess the arguments, guess the implementation, and hallucinate code. In our benchmark, this is an automatic Gate Rejection. It is not token savings. It is destructive truncation.
3. How TokenSafe Kept Syntax and Reduced Tokens
TokenSafe solves the problem by keeping the target code intact. Instead of emitting a text outline, TokenSafe uses native Tree-sitter parsers to analyze the concrete syntax tree. It found Flask.full_dispatch_request, preserved its entire implementation with line numbers, and collapsed the other 50 uncalled methods into 1-line typed signatures:
class Flask(App):
def full_dispatch_request(self) -> Response:
self.try_trigger_before_first_request_functions()
try:
request_started.send(self, _async_wrapper=self.ensure_sync)
rv = self.preprocess_request()
if rv is None:
rv = self.dispatch_request()
except Exception as e:
rv = self.handle_user_exception(e)
return self.finalize_request(rv)
# 1-hop caller contracts (collapsed):
def ensure_sync(self, func: Callable) -> Callable: ...
def preprocess_request(self) -> Optional[Response]: ...
def dispatch_request(self) -> Response: ...
def handle_user_exception(self, e: Exception) -> Response: ...
The result:
- Context shrank from 18,864 tokens down to 3,784 tokens (-79.9%).
- The code is 100% valid Python (
ast.parse()succeeds). - The model sees the exact logic it is changing, along with typed contracts for helper calls.
Output Reversibility: The Hash Stash Difference
What happens when an agent actually needs the code that was folded?
- In Chisle, bash outputs are trimmed in the pipe. If a compiler error or traceback gets cut, it is gone from the session. The model frequently gets stuck in loops asking for the same command again.
- In token-optimizer, trimmed outputs are saved to disk, but the agent has no built-in MCP tool to expand them in-band.
- In TokenSafe, every compressed output is tagged with an 8-character Blake2b hash:
[TokenSafe folded: #a1b2c3d4 (184 lines omitted): call tokensafe:expand_ref('a1b2c3d4') to restore raw content]
When an agent needs the raw bytes, it calls the native MCP tool tokensafe:expand_ref(hash="a1b2c3d4"). In our tests, expand_ref retrieved 100% bit-exact original bytes with zero differences. It gives you the savings of compression without the risk of permanent context loss.
Beyond File Reads: Security and Static Tax
Saving tokens is only half the battle. If your agent saves $2 on tokens but commits your .env file or introduces hardcoded secret fallbacks to GitHub, you lost money.
TokenSafe includes an inline pre-git security guard (guard_inspect) running locally in under 30 milliseconds. When an agent attempts to inspect .env files or hardcodes secret keys in fallback parameters, TokenSafe blocks the tool call locally before it reaches model logs or git history. Neither Chisle nor token-optimizer inspects tool calls for secret fallbacks.
Furthermore, tokensafe doctor audits your static context tax. In our test environment, duplicate cross-IDE config files (CLAUDE.md, .cursorrules, .windsurfrules, .clinerules) were wasting 4,342 ghost tokens on every single turn. TokenSafe identifies these duplicates and consolidates them into a single authoritative AGENTS.md, reclaiming headroom before your first prompt runs.
Architectural Comparison Summary
| Capability | JayPokale / Chisle | alexgreensh / token-optimizer | TokenSafe (v4.1.3) |
|---|---|---|---|
| Code AST Engine | None (0% compression on code reads) | Regex heuristics (pseudo-YAML outline) | Native Tree-sitter AST (13 languages) |
| Syntax Validity | Valid (unmodified raw code) | Fails AST parser on 6/8 repos | 100% valid, parseable syntax |
| Symbol Preservation | 100% (raw file) | Fails on hub files (capped at 4 methods) | Target symbol body 100% preserved |
| Output Reversibility | Irreversible | Local file archive via shell command | Reversible Hash Stashing (#hash + expand_ref) |
| Integration | System prompt instructions | Claude Code hooks wrapper | 18-tool MCP Server + universal proxy |
| Pre-Git Guard | None | None | Sub-30ms secret & command block |
| Polyglot Scope | Prompt-only | Python, JS/TS heuristics | Python, TS, JS, Go, Rust, Java, C#, C, C++, PHP, Kotlin, Swift, Ruby |
| FinOps Telemetry | None | Local SQLite database | Local SQLite + Team Cloud Dashboard |
Summary and Verification
Token optimization cannot rely on truncation or prompt nudges alone:
- Prompt rules like Chisle fail to cut code reads because they lack a syntax parser.
- Regex outlines like token-optimizer produce small numbers, but strip the target function and emit broken syntax that models cannot execute.
- Real token reduction requires understanding concrete syntax trees with native Tree-sitter parsers, preserving caller contracts, and providing bit-exact hash anchors for reversibility.
You can verify all of these measurements on your own repository right now without an account:
npx tokensafe
Or run the full real-repository test suite:
tokensafe bench --real --here
Every measured row, gate check, and failure reason is published openly on our live benchmark table.