The recorded-session snapshot suite checks Oda against committed evidence instead of a live model. Each scenario replays a recorded fixture: it starts the real agent as a subprocess, drives it with a scripted input, and compares the protocol transcript, the session log and the files the run wrote against what was committed. It is developer tooling for people who add or maintain these scenarios. To choose between the test tiers, see Testing tiers.
The suite runs from your checkout of the Oda source, in three modes:
pnpm run test:snapshot pnpm run test:snapshot:record pnpm run test:snapshot:refresh
| File | What it holds |
|---|---|
snapshot. | The scenario's manifest. |
input. | The deterministic script the run is driven with. |
session., session. | Committed session fixtures: the parent session and its children, one file per generation. |
stdout. | The normalised ACP transcript the run must match, one compact JSON frame per line. |
stdout. | The same transcript with native path separators, for scenarios that pin Windows output. |
system-prompt., system-prompt. | The full system prompt text behind the {{system}} tokens, per role. |
tool-schemas., tool-schemas. | The full tool schemas behind the {{tools}} tokens, per role. |
workspace/, workspace. | Files seeded into the run folder before it starts, and the expected end state when the scenario declares a final snapshot. |
replay. | Present exactly when a scenario needs scripted throws or hangs that the recorded stream cannot express; the shapes are on Session replay fixtures. |
Committed fixtures keep their generations side by side, under these shapes and rules:
version field (the filename encodes it), and the request header still carries the prompt and tool values as {{system}} and {{tools}}.version field and packs the streamed assistant text into text-chunks inside an assistant/ event, in place of the per-delta chunks of the older shape.step/ and step/ events, and the system prompt becomes its own system/ event with a surface operation: append, or replace bounded by startSeq and endSeq, plus the sourceEventSeqs the operation was derived from.session. with n counting from 1, roles are contiguous, and a version 0 generation is written without a version suffix. Every filename must agree with the header inside it.initial, series, change), and a pinned class counts the changes it expects rather than pinning the order in which they appear.A committed fixture replaces everything machine-specific with typed tokens.
{{session:N}}, {{message:N}}, {{approval:N}}, {{workflow:N}}, {{command:N}}, {{rpc:N}}, {{retry:N}}, {{id:N}}. The same id keeps the same token across a parent log and its children, so relationships survive.{{sessionId}}, {{cwd}}, {{system}}, {{tools}}, {{usedTokens}}, {{eventTime}}, {{eventOmittedBytes}}, {{spillLocator:<file>}}, {{sourceSessionFormatVersion}}. A generated working folder becomes {{cwd}}, a scratch file keeps only its name, and prompt text and tool schemas keep their presence without their content.snapshot. is a closed schema: unknown fields are refused, no JavaScript YAML tags are allowed, and names are lower-kebab-case. A minimal manifest is:
version: 1 profile: acp
version, scenario, profile, composition, recording, header, replay, platform, permission, environment, workspace, input, session, sessionFormat.profile is headless, sdk, acp or web; recording is live or authored; platform is posix or pwsh; permission is read-only, workspace-write or danger-full-access.header. must equal true when present, and the header counts are non-negative integers.sessionFormat is only valid when the scenario owns its session fixtures, and such a generation stays read-only while recording but still runs in replay and refresh.sha256: content id, a media type and base64 data.manifest., manifest., manifest., manifest., manifest., manifest., manifest., manifest..initialize, newSession, newSessionExpectError.prompt, promptContent, promptAndWaitForAgentMessage, promptExpectError, promptAndCancel.waitForTurnStart (with a minimum turn), waitForTurnEnd, waitForSubagentTurnEnd (a given child), waitForGoalPhase, waitForInboxMessage, waitForTitleAfterTurnEnd, waitForEventAfterTurnEnd, waitForFile. A wait polls for its evidence and fails when it does not appear; the default budget is ten seconds.cancel.allow_once, allow_always, reject_once, reject_always), because the option ids in a request are random. A request beyond the queue, or with no queue, is answered cancelled; a scripted kind the request does not offer fails the run: scripted permission answer <kind> not among the offered options [<offered>].no scenario pins the request-header content of class "<class>", and header class "<class>" pinned by both <a> and <b>: every header class needs exactly one pinning scenario.duplicate scenario name "<name>".identical prompt snapshots appear in <a> and <b>: share one source instead of committing the same text twice.plain..invalid session fixture name: <name>, duplicate session fixture generation: <name>, missing parent session fixture.session fixture roles must be contiguous: expected index <n>, found <name>.filename declares Session format v<n>, header declares v<m>.session fixture is empty, first record must be a Session header.<file> contains UNKNOWN_TOOL, <file> carries a non-canonical macOS cwd token, identity redaction fixed point.<file> has a request/, <file> carries an unscrubbed system prompt, <file> carries unscrubbed tool schemas.session "<id>" did not persist turn/, subagent child #<n> did not persist closed turn <m> within <ms>ms, workspace file "<path>" did not appear within <ms>ms.scenario failed: <detail>, and when teardown also fails, snapshot scenario and cleanup failed or snapshot cleanup failed.a changed workspace requires workspace..Record calls the live model: it re-records a live scenario and writes the new session generation and the sidecars. Refresh needs no key: it replays the committed scripts and rewrites the derived files. Authored scenarios are never re-recorded, so for them refresh is the only mode that writes.
No. Replay and refresh are keyless; only recording calls the live model.
Each role, the parent session and each child, is compared at its own highest generation. Older generations stay in the folder beside it and are not compared, which is also why record and refresh never rename or delete one. The format's own versioning rules are on Chat log format and compatibility.
The committed set is guarded, so an edit must pass the storage checks: the file must be a redaction fixed point and must not carry an unscrubbed system prompt or tool schemas, a request header with no preceding system message, a non-canonical workspace token, or a placeholder tool name. Regenerating is usually easier: refresh rewrites the derived files, and record re-records a live scenario.
{{system}}?In the sidecar file next to it: system-prompt., or a numbered sidecar for a child role, with the tool schemas in tool-schemas.. The fixture keeps the token to mark presence; the comparison restores the full text from the sidecar. A changed prompt is appended under a <!-- system/ marker.
The scenario table is checked before anything runs, and every header class must have exactly one scenario that pins its request-header content. With none, or with two, the suite refuses and names the class. A duplicate scenario name is refused too, and pin-only fields are refused on a scenario that does not pin.
Recording skips authored scenarios and retained historical generations; both still run in replay and refresh. A scenario that declares one platform runs only there, while the fixture guards run everywhere.
It is answered cancelled. Scripted answers are consumed in order and matched by option kind, so a request beyond the queue, or with no queue, is cancelled; a scripted kind the request does not offer fails the run.
The wait polled for that event in the stored log and its budget ran out, ten seconds by default. Check that the scripted input produces the event; the message names the event it waited for and the budget that expired.
Through workspace snapshots. A scenario can copy a committed workspace/ folder into the run folder before it starts, and declare a final snapshot: the run's end state is then compared entry by entry as text, binary, symlink or empty directory. A run that changes the workspace without a declared final snapshot is warned that a changed workspace requires workspace..