Tokens is a currency for generative AI. Every prompt you send, every document you attach, every response the model produces - all of it consumes tokens. Tokens costs money, time, and compute power. As you move from casual chat usage to autonomous agents and coding tools, a single workflow can burn your token budget in a blink of an eye.
The good news: it is easy to optimise. This post collects practical techniques, from zero-effort habits to architecture-level decisions.
Why token usage explodes
LLMs are stateless. They remember nothing between interactions. When you chat with a model, the entire conversation is passed back on every turn - your messages, its responses, tool outputs, everything.
This causes quadratic scaling: as a chat gets longer, each new turn re-processes everything before it. A 100-turn conversation doesn’t cost 100 units - it costs closer to the square of that. This is the single most important fact to internalise about token economics.
There is also a quality angle. Research on long-context behaviour shows what Anthropic calls context rot: as the token count in the context window grows, the model’s ability to accurately recall information from it degrades. Every token you add depletes a finite “attention budget”. Fewer, higher-signal tokens don’t just cost less - they produce better answers.
Level 1: Prompt hygiene
Cut the fluff
Politeness and filler burn tokens without changing the output. Compare:
Hello there! I would really appreciate it if you could act as a senior
backend developer. I am trying to figure out how to write a python script
that connects to the database and retrieves all of the information from
the user repository...versus:
As a senior backend developer: write a Python script that connects to
the DB and retrieves all users where retry_count >= 5 AND active = true.
Use env config, not hardcoded params. Output: JSON.Same intent, roughly half the tokens. A community experiment (Defluffer) showed that simple rule-based rewriting - removing filler phrases, collapsing verbose constructions, symbolising logic (“greater than or equal to” becomes >=) - cuts prompt size by around 45% with near-zero compute.
Be objective, not vague
Vague instructions force reasoning models to burn tokens debating what you meant. “Use as few words as possible” triggers internal deliberation; “answer in 10 words or less” doesn’t. Give measurable targets, templates to fill in, and expected output formats.
Start new chats often
Because the whole conversation is replayed every turn, a long-running chat is the most expensive place to work. Finish a task, start a fresh session. If you need continuity, carry over a short summary instead of the raw history.
Level 2: Context engineering
Anthropic’s engineering team frames the goal well: find the smallest possible set of high-signal tokens that maximise the likelihood of the desired outcome.
Keep system prompts minimal but sufficient
The failure modes are on both ends: brittle, hardcoded if-else logic in prompts, or vague guidance that assumes shared context. Aim for the middle - specific enough to guide behaviour, flexible enough to let the model work. Structure prompts into sections (background, instructions, tool guidance, output format).
Load context just-in-time
Instead of stuffing every potentially relevant document into the prompt, give agents lightweight identifiers - file paths, queries, links - and tools to load data on demand:
# Anti-pattern: preload everything
context = read_all_files("docs/") # 200k tokens, mostly irrelevant
# Better: give the agent references and retrieval tools
tools = [glob_search, grep_search, read_file]
# The agent reads only the 2-3 files it actually needsThis mirrors how humans work: we don’t memorise entire corpuses, we use file systems and bookmarks to retrieve information when needed.
Keep the tool set lean
Every connector, skill, or MCP server you have enabled is loaded into context on every turn - whether you use it or not. Worse, unused tool descriptions silently influence the task at hand. Before starting work, turn off everything the task doesn’t need.
Prefer lightweight document formats
Markdown, YAML, JSON, and plain text can be read natively by a model. Word documents, PDFs, and spreadsheets often require conversion or code execution, which burns extra tokens. For tabular data, YAML tends to work better than CSV - models can read it directly instead of writing Python to parse it.
Level 3: Workflow design
Plan big, act small
Model size correlates with capability - and with cost. The pattern that consistently saves the most:
- Plan with the biggest model available (requirements, specs, architecture diagrams)
- Execute with a smaller, cheaper model following the plan
- Review with the big model (QA, audit)
- Fix with the small model
If the plan is good, a small model can accomplish a lot. Using a frontier model to rename variables is like taking a fighter jet to the grocery store. Some tools bake this in - Claude Code’s “Opus Plan” mode plans with the large model and edits with the smaller one.
Spend tokens on planning, not rework
“Vibe coding” - talking in the general direction of the machine and course-correcting - is the most expensive way to work. Every correction round replays the growing conversation. Instead:
- Have the model produce a system diagram (Mermaid, markdown) before building. Mismatches between the diagram and your mental model surface problems cheaply.
- Run a pre-mortem: give the model your plan and ask “what could go wrong, what did I overlook?” Then fix the plan before execution starts.
Make AI take notes
Agentic sessions crash, compact, and lose coherence. If the agent writes terse progress notes to disk as it works, recovery costs a few hundred tokens instead of re-doing hours of work:
You will keep notes in ./notes/ using date-timestamped files.
Use maximum information density - terse, factual, no prose.
On restart, read the most recent note before continuing.Save and reuse everything
You already paid for every artifact the model produced. Reusable prompts, code fragments, skills, style samples - store them and feed them back instead of regenerating. Reinventing the wheel is the most common form of silent token waste.
Use Git for rollback
Version control gives you deterministic, zero-token rollback. If an agent produces 100 blog posts and numbers 73-78 are broken, you roll back to 72 and resume - instead of regenerating everything.
Level 4: Infrastructure
Prefer CLI tools over MCP servers
Command-line tools are deterministic and cost no tokens to execute. An MCP integration for a calendar service can take minutes and thousands of tokens to answer “what’s on my calendar today”; a CLI call returns structured data the model only has to interpret:
# Deterministic, fast, zero-token execution
gcalcli agenda --tsv | head -20
# The model interprets the output instead of
# reasoning through a probabilistic tool protocolThe same applies to parsing: tell the agent to use jq for JSON instead of letting it write (and rewrite) parsers.
Use prompt caching
Prompt caching stores the static prefix of your conversation (system prompt, tools, documents) so it isn’t re-processed at full price on every call. Most agentic harnesses support it; when choosing an inference provider or framework, verify how aggressively it caches - the difference in cost is large.
Route tasks to the right model
Model routers (OpenRouter, LiteLLM, or a homegrown dispatcher) classify each request and send it to the cheapest model that can handle it:
def route(task):
if task.type == "planning":
return "large-model" # expensive, rare
if task.type == "summarise":
return "small-fast-model" # cheap, frequent
return "mid-tier-model"Consider local models
For summarisation, extraction, formatting, and routine execution, local models running on your own hardware consume zero cloud budget - and keep data in-house. Plan with a frontier model in the cloud, execute locally.
What not to do: token maxing
Some organisations measure AI adoption by token consumption. This is exactly backwards - it incentivises waste, the same way measuring developers by lines of code incentivises bloat. Measure outcomes: are people getting to results faster, better, cheaper? The best AI users often consume fewer tokens, not more.
Summary
| Level | Technique | Effort | Impact |
|---|---|---|---|
| Prompt | Cut fluff, be specific, fresh chats | Minimal | High |
| Context | Just-in-time retrieval, lean tools, light formats | Medium | High |
| Workflow | Plan big / act small, notes, reuse, Git | Medium | Very high |
| Infra | CLIs over MCP, caching, routing, local models | High | Very high |
Token optimisation isn’t about being cheap - it’s about engineering discipline. The same practices that cut your bill (small contexts, clear plans, deterministic tools) are the ones that make AI systems reliable.
Sources & further reading
- Effective context engineering for AI agents - Anthropic Engineering
- 18 Ways To Save AI Token Budgets - Christopher Penn
- Defluffer - reduce token usage by 45% - GrahamTheDev
- Token optimization: The backbone of effective prompt engineering - IBM Developer