Token usage and context pressure
Every call in a DataMind Oda chat is measured: Oda keeps running counts of the tokens the model read and wrote, and of how full the model's reading space is. This page explains what those figures mean and why they are a reference rather than a bill. The chat around them is covered in Chat with Oda.
Where the figures appear
- Under the message box, the chat's figures show in a short form such as
66.8K tok · Cache hit 77%, beside the ring that shows how full the model's reading space is. Context usage covers the ring and its panel.
- A finished turn carries a usage pill such as
Usage 15.8K tok; it opens the Turn usage dialog. Actions on a message covers the pill and the dialog.
- Under the message box, the usage pill opens the Token usage dialog with the chat's totals, and the neighbouring pill carries the chat's timings. Chat navigation and stats covers the timing figures.
- These figures show while Performance & usage is Detailed (the default); Compact keeps only the available output speed and cache-hit. Chat display settings covers that row.
The four token figures
Each call records the same four figures:
| Figure | What it counts |
|---|
| Uncached input | The part of the request the model read fresh. |
| Cached input | The part the model served from its own cache. |
| Cache write | The part stored in the model's cache for later calls to reuse. |
| Output | What the model produced; the reasoning part is named inside it. |
- The input side is those three input figures together: read fresh, read from the cache and written to the cache. Cache hit is the cached share of that input.
- The figures arrive in a short form:
517, 12.2K, 517K, 1.2M.
- A fact that was not recorded leaves its row or segment out instead of showing a zero. The Cache write figure, for example, appears only once the cache has been written to.
How the counts build up
- A reply can take several calls, one per step, and a call that was retried is another call. The turn's figures add its calls up.
- While a call streams, a running figure fills from the first partial numbers. When the call settles, its final figure replaces the provisional one, so one call counts once.
- A retry that ran counts like any other call; a retry that was cancelled before it ran does not. Failures and recovery shows the retry row in a turn.
Where the numbers come from
- When the model service reports what a call used, Oda keeps that report and shows it.
- When the report is missing, Oda sizes the text the request carried itself, at about four characters to a token, and marks the figure with a
~ where it is shown, such as the token count on a Context compacted marker.
- An estimate can come out low for some material: long identifiers, dense data files and text in languages that write with wide characters can take more room than the four-character rule suggests.
- A figure appears only when Oda holds the whole record behind it. Where the record is incomplete or contradicts itself, Oda leaves the figure out rather than showing a partial count. A missing figure means it was not recorded, not that it was zero.
The context figure
- The ring shows how full the model's reading space is: the size of the newest request against the model's window.
- The figure follows the newest request rather than adding up: each request replaces it. It measures the prompt that was sent, not the reply being produced, so it holds still while an answer streams.
- A running figure also follows what has changed since the newest request, so setting an older part of the chat aside shows in it at once. It never reads below zero.
- When you switch models, the window follows the new model at once, while the used figure updates with the next reply. Until then, the two can come from different moments.
- What fills the figure, the system prompt, the tool definitions and the messages, is on Context usage.
A reference, not a bill
- These figures are Oda's own reading of the chat. The bill for the calls comes from the model service, not from this screen, and an estimated figure can differ from what the service counts.
- The context figure is a reference for reading the chat, not a billing record.
- The requests themselves are recorded in your company's log; Chat logging covers what travels with a request.
Questions, answered
What do the four token figures mean?
Uncached input is the part of the request the model read fresh, Cached input the part it read from its own cache, Cache write the part stored in that cache for later calls, and Output what the model produced, with the reasoning part named inside it. Cache hit is the cached share of the input.
Why does Cache hit sometimes disappear?
It needs input to measure. When nothing on the input side was read for what the figure covers, there is no share to show, so Cache hit is left out rather than shown as a zero.
Why does Cache hit show 99.96% instead of 100%?
A share reads 100% only when every input token came from the cache. A near miss keeps the extra digits it needs, such as 99.96%, so a partial hit is never rounded up into a full one.
Are these figures what the model service will bill?
No. They are Oda's kept record for the chat, a reference rather than a bill. The bill for the calls comes from the model service; where a figure is an estimate, the two can differ. Chat logging covers the record of requests your company keeps.
Why is a figure sometimes an estimate?
Not every call reports what it used. When the report is missing, Oda sizes the request's text itself, at about four characters to a token, and marks the figure with a ~. Dense material, such as data files or text that writes with wide characters, can come out low this way.
Why did the figures change while the reply was still running?
While a call streams, the figure fills from the first partial numbers. When the call settles, its final figure replaces the provisional one, so the count can move as a reply finishes; the call still counts once.
Do retried calls count too?
Yes. A retry is another call to the model, so it counts like any other. A retry that ran counts; one that was cancelled before it ran does not. Failures and recovery covers the retry row, and Model calls and failures covers which failures are retried.
Why didn't the chat's total shrink when an older part was compacted?
Compacting changes what the next call carries, not what already ran, so the chat's record of its calls keeps its figures. The context figure falls at once instead. What the marker records is on System prompt, context and compaction.
Why does the context figure hold still while the reply streams?
It measures the prompt that was sent, not the reply being produced, so it stays where it is while the answer streams. It moves again with a newer request.
Why does the ring disagree with the chat's token total?
They measure different things. The total adds up every call the chat has made; the ring shows how full the reading space is for the newest request. A long chat can carry a large total and still leave most of its window free, because the window holds one request at a time.