The mock model server

The mock model server is a standalone local process that stands in for the model with scripted responses, so a test or a demo can exercise both successes and failures without a key. It serves the messages endpoint that model adapters use, plays one scripted behaviour per accepted request, and never retries or interprets recovery policy of its own. To choose between the test tiers, see Testing tiers.

Run the server

Run the standalone process from your checkout of the Oda source:

pnpm run mock:llm \
  --port 8000 \
  --api-key mock-key \
  --sequence partial_disconnect,success \
  --partial-text "discard this half"

• No installable binary. The server runs from the project checkout; there is nothing to install.
• Defaults. It listens on the loopback address 127.0.0.1, port 8000, unless you change --host or --port. --sequence has no default and is required.
• Standard output is JSONL. Every line is one JSON object:
  • a ready record once the listener is up, carrying the /v1 base URL and the seed used for random selection, for example {"type":"ready","baseURL":"http://127.0.0.1:8000/v1","randomSeed":42};
  • with connection_refused first, an unavailable record before the listener delay, carrying the base URL and the delay;
  • a request record and a result record for every accepted request; both name the attempt, the scripted entry and the concrete behaviour it played (the two differ for random), and the result adds the outcome and how many stream chunks were sent.
• A bad option exits 1. The command writes the message and the usage text to standard error; --help prints the usage text.

Script the behaviours

• One entry per request. --sequence is a comma-separated first-in, first-out list; each accepted request consumes exactly one entry, in arrival order.
• Exhaustion is loud. When the list runs out, the next request gets a structured HTTP 500 with mock script exhausted. --repeat-last reuses the final entry instead.
• connection_refused is special. It is accepted only as the first entry and is not a request behaviour: it delays binding the port so that requests during --listen-delay-ms (default 750 ms) get a real TCP refusal. With it, a port of 0 is refused (connection_refused requires an explicit nonzero --port). The remaining entries begin once the listener is up.

The catalogue of 24 behaviours:

Random mode

• Seeded selection. A random entry draws its concrete behaviour at request time from seeded weights, so a run is reproducible from its printed seed. Omitting --seed generates one and prints it in the ready record.
• Weights. --random-weights takes behavior=weight entries with non-negative relative weights and requires at least one positive entry. Entries must name concrete behaviours (random is refused), and the same behaviour cannot be weighted twice.
• The default profile is stress, not reality. Without --random-weights, random uses a success-heavy default profile (success 48, slow_success 10, partial_disconnect 10, connection_reset 5, stream_disconnect 5, empty 5, rate_limit 5, server_error 4, max_tokens 2, stall 2, service_unavailable 2, partial_eof 1, malformed_json 1). It is test pressure, not an estimate of how often these failures happen in production.
• stall needs a client-side timeout. When the weights can produce stall, give the client under test a short stream-idle timeout so the scenario ends promptly.
• Reproducibility flags need random. --seed and --random-weights are refused when the sequence holds no random entry.

Timing, content and the key

Millisecond delays are whole numbers within Node's timer range; --retry-after-ms must also be positive.

Which requests consume the script

• One rule. Only a POST whose path ends in /v1/messages (any prefix before it is fine), carrying the configured key when one is set and a body that parses as JSON, consumes the script. An empty body is accepted.
• Everything else is an ordinary 4xx, and the sequence does not move:

A misconfigured client can burn retries this way without advancing the sequence.

The errors a script returns

• An error response still counts as a completed outcome for its request record.
• With --request-id set, these responses carry it as x-request-id; only rate_limit adds Retry-After.
• The last two are not catalogue entries: exhaustion answers once the sequence runs out, and the handler failure is the last-resort containment when a request was already admitted.

How a request ends

Each request record carries one outcome:

• completed: the behaviour ran to its end, error responses included.
• reset: a disconnect behaviour destroyed the connection.
• stalled: the stall behaviour is holding the connection open.
• client_closed: the client went away before the response finished.
• server_error: the handler itself failed.

Limits

• Arrival order is shared. Entries are served in arrival order, and concurrent callers share one cursor, so per-session determinism needs a separate server instance per session or test.
• The seed is your reproducibility handle. Record it from the ready record; the same seed and weights draw the same sequence of behaviours.
• A true connection refusal is a listener phase. The --listen-delay-ms window must overlap the client's attempt, and random cannot draw connection_refused, because a bound listener cannot refuse a connection.

Questions, answered

What should I do when the server answers mock script exhausted?

The sequence ran out. Lengthen --sequence, or pass --repeat-last so the final entry is reused for every further request.

Why did a request get a 4xx and not use a script entry?

Only valid requests consume the script: a POST whose path ends in /v1/messages, with the configured key when one is set and a JSON body. A wrong method, path, key or body gets its ordinary 4xx first, so a misconfigured client can retry without advancing the sequence.

How do I make a random run reproducible?

Pass a fixed --seed, or record the generated seed from the ready record, together with your weights. The same seed and weights draw the same sequence of behaviours.

How do I keep fault assignment deterministic across parallel tests?

Run a separate server instance per test or session. Entries are served in arrival order and concurrent callers share one cursor, so one instance cannot give each caller its own script.

What does connection_refused need to work?

It must be the first sequence entry, the port must not be 0, and its window is --listen-delay-ms (default 750 ms). During the window requests get a real TCP refusal; afterwards the remaining entries play in order.

Which behaviours end without the closing stream marker?

empty_body, stream_eof and partial_eof end cleanly without the closing marker, so clients that require it fail. Use empty when you want a well-formed completion with no content.

Which error responses add a Retry-After header?

Only rate_limit does, as a whole number of seconds rounded up from --retry-after-ms (default 1000 ms, so one second). Other errors never add the header.

What happens when a client disconnects mid-stream?

The request record settles as client_closed, and the server keeps serving other requests. Closing the server stops it accepting requests and ends stalled connections.

Must every request carry an API key?

Only when you start the server with --api-key. Then every request must carry that exact value in x-api-key, or it gets 401 invalid mock API key. Without --api-key, any token is accepted.

What timeout should a client use when the weights can produce stall?

Give the client under test a short stream-idle timeout. stall holds the connection open until the client or the server closes it, so the scenario ends only when one side gives up.