How to use AI more efficiently (and stop wasting tokens)
Using AI is easy; Understanding usage is not.
AI has become an integral part of many organisations’ internal tools and workflows. Most developers are well versed in using Agents, Subagents and tools for tasks like code generation, review, planning, and documentation. What they may not be as well versed in is how efficient their usage is, or could be.
Recently, some Codex users have reported their usage limits being consumed dramatically faster than expected. Combined with rapidly changing pricing models across AI providers, the time has never been more important to understand exactly how efficiently we are using AI.
Understanding what a model is consuming, what the usage costs, and then making a value judgement against simply writing the code yourself is not always as clear as we would like it to be. This is part of the ‘Black Box’ problem I outlined in my previous blog. If a business is to be serious about AI governance and transparency, it follows that to define why an answer was produced you must also be able to define what it cost to produce the answer.
One of the most important parts of understanding usage is understanding context, and why it matters. Dan Batty has written an excellent blog on the subject: Context engineering: the future of AI integration which provides information we do not need to reiterate here, but it poses the question; If context is valuable, how do we avoid wasting it?
AI efficiency isn’t about reducing context or consumption, it is about reducing waste and inefficiency. Models must be provided the right amount of useful information to complete their tasks effectively.
A small caveat before going further; different providers and models charge differently for input, output, reasoning and tools, and these prices change frequently, and some billing and costs can be frustratingly obscure. Any advice and examples here should be taken in a broad context of the time this blog was written, we do not know what the future will look like, but we can assume efficiency will always be a worthwhile goal in our AI usage.
What's a token?
Model billing is often defined by ‘token consumption’ but it is rare for them to actually go as far as to define what a token actually is.
Essentially, a token is a small unit of information processed by an AI model (helpful, I know). For text this might be a character, punctuation, part of a word or an entire word, depending on how the model tokenises the input. This is ambiguous mostly because of the ambiguity of how each individual model handles and bills tokens. There are 4 token categories worth being aware of:
Input tokens - This is essentially everything that comes from you, your prompts, conversation history, documents, and skills the model is consuming.
Output tokens - This is everything that comes from the model, its responses and outputs.
Cached tokens - Not every model does this, but some cache your previously processed context for reuse that is more efficient.
Reasoning/ thinking - Some models perform additional computations that may consume your usages and affect billing.
Another important caveat here; Usage and billing are not the same thing. This is where this might get a bit confusing again, we are discussing usage in terms of tokens, as this is the metric most models use for pricing, however, not all usage is measured in tokens, and not all payment plans or subscriptions log and track tokens. It is probably sufficient to keep talking in terms of tokens for consumption, but keep in mind that depending on each specific model or plan the only visible metric provided by the AI company might just be ‘usage’.
Fewer tokens ≠ lower cost
This is an intuitive assumption based on what we know so far. Less model throughput -> less consumption -> lower bills. Seems simple? Well, mostly it is, but not always.
In the appropriately titled Token reduction is not cost reduction Sarel Weinberger and
Amir Hozez demonstrated an experiment where they were able to demonstrate that substantially reducing tool-output tokens actually increased their billing costs;
“Aggressive compression reduced tokens by 38.4% but increased billed cost by 6.8% … We also find that compression can alter the agent trajectory by inducing additional retrieval, diagnosis, testing, or turns, offsetting local token savings.”
In short their conclusion is that token reduction and AI efficiency are not always the same thing. Context matters, the task matters, and removing useful information can force an agent to retrieve information again, take additional actions or require further turns.
Don’t focus on tokens spent
The title of this blog says ‘stop wasting tokens’ and you should, because it will save you money. But as we just saw, using less tokens doesn’t mean this happens automatically. You should aim to stop wasting them, which is an important distinction from stopping spending them.
Consider the case of a particularly difficult problem, a refactor, application rebase or review of a large number of documents. Lets call this 10,000 tokens. Now you could potentially generalise the problem, limit repository access or just give overviews of the application to reduce usage down to 3,000 tokens… You might get an output that looks correct, but what if that fails testing or PR 4 times? Now you have spent 12,000 tokens where 10,000 would have been quicker, more detailed and importantly more cost effective.
Your metrics should not be ‘How many tokens did this use?’ but rather ‘What did those tokens achieve?’. Consider the cost per successful task, the number of retries, time to completion, or the level of human intervention or rework required. With AI usage the critical factor is time and cost efficiency vs a human task. Don’t forget to also consider the time sunk into cost and token usage analysis, because this will also eat up your team's valuable project time.
When more context is too much context
Context is good, and well engineered context is even better. So surely a lot of well engineered context is the ideal?
Not necessarily.
Context pollution and context rot can both arise from too much or poorly maintained context. OpenAI has a good overview of the terms in their Subagents documentation and if you are looking for a more technical deep-dive Chroma have published an in-depth article on context rot. To summarise both here;
Context Pollution is too much noise in the context. This can take many forms; irrelevant docs in the repo, failed prompts in the chat windows, results of previous tasks, obsolete tools and completed reviews. All of this consumes tokens, reduces the accuracy of output and pollutes your context, even though much of it will have been essential or useful when it was added.
Context Rot (as covered in the Chroma report above) shows that model performance declines as context length increases. As the amount of irrelevant or distracting material grows, the agent can become less effective at identifying and using the information that actually matters.
When the two are combined together we can be a bit more precise about context; it is not necessarily the length that is the problem, it is the relevance.
For coding agents, large amounts of effort (and thus, tokens) can be spent looking through repositories, rereading chat logs, reviewing frameworks and analysing test outputs. To counteract this repository maps and persistent project instructions are essential, instead of forcing an agent to discover the same information repeatedly, provide them with clear directions on where to look for information and how the expected outputs should be created.
Writing for Medium, Joyal Saji reported a 95% drop in token usage by meticulously documenting their working repositories.
Handing off to Subagents
Subagents are not automatically cheaper. In fact, because every subagent performs its own model and tool work, a multi-agent workflow can consume more tokens than doing the same task with a single agent. The advantage comes from being more selective about where those tokens are spent. You might want to be writing code with GPT5.6 Sol with Extra High reasoning, but you probably don’t need to be using that model to search your repo for typos.
This is where Subagents can come in handy. With a little work detailing your agents, handoffs and tasks you can have your main agent (like GPT Sol) review a prompt and decide what level of reasoning is required for each of the sub-tasks that the work requires. This also protects your conversation context from being polluted with noisy exploratory work and allows cheaper agents to trawl through areas that are particularly messy.
For most models this can be done locally, for me using Codex I was able to ask it to set up a reusable local Codex multi-agent team for me. I instructed it to use the current Sol model only as the main lead/orchestrator for planning, architecture, and difficulty reasoning, and then hand off different parts of the task to different agents based on the task complexity, reducing models and reasoning down to cheaper levels. This does take a bit of instruction with some very detailed prompting (or just directly editing AGENTS.md and its override) but it is a very effective way to have much more targeted model usage.
Another caveat here though; setting up your own sub-agents for your own projects will take a bit of trial and error, and setting them imprecisely can increase your token usage rather than decrease it. The efficiency is worthwhile, and the OpenAI Subagents page is a
good starting point.
What you can do to effectively reduce token wastage
So now we have established what tokens (or broadly, usage) actually are and what is consuming more or less of them, what practical steps can be taken to ensure we are using them as effectively as possible?
Start a new conversation when switching problems
Continue in a chat window where the previous conversation is useful, start a new one as soon as it is not.
Select the right model
Before moving to subagents consider manually selecting a simpler model when conducting simpler tasks, aim for the minimum acceptable reasoning over time rather than always defaulting to the best or fastest.
Write better prompts
Give a clear definition, ensure the context is established and relevant and provide a clear output or stopping point to keep drift to a minimum.
Build repository maps and documentation
Persistent knowledge is important, and can be a huge factor in reducing the amount of discovery work that is required for each task.
Use Subagents for noisy or simplistic tasks
As noted above, simpler models are more than sufficient for discovery tasks, and can be much more economical.
Separate research, planning and implementation tasks
Trying to achieve too much in one prompt is expensive, and can lead to numerous retries. Separate tasks as much is reasonable and iterate over the outputs, updating skill files and instructions as you go.
Build efficiency through learning
In all stages, when you see a failure, or the model requires retries to reach a goal, pause after the execution and update the instructions so that future tasks have a clear solution to the problem.
Visibility and Governance
You can’t optimise what you can't see.
Individual optimisations are important, but when deploying AI at scale organisations need visibility into workload, token usage, retries, agent actions, and successful outcomes. This ties back again to my earlier blog about the Glass Box principles; you must understand the whole system, including the resources and costs that enable it in order to ensure it is efficient.
Before launching into a project slashing consumption, define how you will metric the results and have a clear definition of what efficiency actually means to your workflow or business. Reducing costs by 30% is meaningless if it increases delivery time by 50%.
Have the workflow capture the model used, actual token or credit usage reported by the provider, tool calls, retries, wall-clock duration and task outcome alongside the generated changelog.
So, how do we use AI more effectively?
Too little context makes the AI search, guess and retry.
Too much irrelevant context increases cost, context pollution, and context rot.
Reducing token consumption without clear goals or metrics can make your work less efficient, rather than more.
So don’t aim for a smallest cost, usage or token consumption, instead aim for the least possible waste required for the desired result. Efficient AI isn't about using fewer tokens. It's about making the tokens you use count.
Don’t forget to check the RADAPEX blog page and RADAPEX on LinkedIn to read more of our blogs and success stories.
