Loading the documentation…

Token usage and context pressure

Every call in a DataMind Oda chat is measured: Oda keeps running counts of the tokens the model read and wrote, and of how full the model's reading space is. This page explains what those figures mean, how the context ring and its panel show them, and why they are a reference rather than a bill. The chat around them is covered in Chat with Oda.

Where the figures appear

The four token figures

Each call records the same four figures:

FigureWhat it counts
Uncached inputThe part of the request the model read fresh.
Cached inputThe part the model served from its own cache.
Cache writeThe part stored in the model's cache for later calls to reuse.
OutputWhat the model produced; the reasoning part is named inside it.

How the counts build up

Where the numbers come from

The context figure

The context ring

The ring sits in the message box's control row, just before the model picker. The message box covers the row and its other controls.

The breakdown panel

When the context fills

When a long chat no longer fits the model's reading space, Oda shortens the older part of it, on its own or when you run /compact, and the ring follows the new figure at once. The marker the compaction leaves in the conversation, and its summary, are covered in System prompt, context and compaction.

A reference, not a bill

Questions, answered

What do the four token figures mean?

Uncached input is the part of the request the model read fresh, Cached input the part it read from its own cache, Cache write the part stored in that cache for later calls, and Output what the model produced, with the reasoning part named inside it. Cache hit is the cached share of the input.

Why does Cache hit sometimes disappear?

It needs input to measure. When nothing on the input side was read for what the figure covers, there is no share to show, so Cache hit is left out rather than shown as a zero.

Are these figures what the model service will bill?

No. They are Oda's kept record for the chat, a reference rather than a bill. The bill for the calls comes from the model service; where a figure is an estimate, the two can differ. Chat logging covers the record of requests your company keeps.

Why is a figure sometimes an estimate?

Not every call reports what it used. When the report is missing, Oda sizes the request's text itself, at about four characters to a token, and marks the figure with a ~. Dense material, such as data files or text that writes with wide characters, can come out low this way. In the breakdown panel, only the percentage is exact; the figures and the split are estimated, so they carry the ~ too.

Do retried calls count too?

Yes. A retry is another call to the model, so it counts like any other. A retry that ran counts; one that was cancelled before it ran does not. Failures and recovery covers the retry row, and Model calls and failures covers which failures are retried.

Why didn't the chat's total shrink when an older part was compacted?

Compacting changes what the next call carries, not what already ran, so the chat's record of its calls keeps its figures. The context figure falls at once instead. What the marker records is on System prompt, context and compaction.

Why does the context figure hold still while the reply streams?

It measures the prompt that was sent, not the reply being produced, so it stays where it is while the answer streams. It moves again with a newer request.

Why does the ring disagree with the chat's token total?

They measure different things. The total adds up every call the chat has made; the ring shows how full the reading space is for the newest request. A long chat can carry a large total and still leave most of its window free, because the window holds one request at a time.

What do the numbers in the breakdown panel mean?

The percentage is the share of the context in use, like 25% of context used; beside it, the chat's figure stands against the model's limit, like ~32K / 128K. The number after the slash is the model's context window. The three rows below split the figure between the system prompt, the tool definitions and the messages.

Why did the breakdown panel close by itself?

The panel closes when the model's capacity is no longer known, for example after a model switch that leaves it unknown. It stays closed even after the capacity returns, until you open it again from the ring.

Why does the ring sometimes show without a percentage?

When the control row runs out of space, the ring gives up its percentage first and keeps only the ring itself. A fresh chat shows no ring at all until both the chat's usage and the model's capacity are known.