# FightBench dataset schema v1

One row per match, identical values in UTF-8 JSONL and Parquet (Snappy). Export only
records the requesting session may read. Request 1–20 IDs per download; repeat to
shard a corpus. Stable ordering follows requested IDs. No secrets or key-store
records are included. Full raw provider content can contain user-supplied text.

| Column | JSON / Parquet | Meaning |
|---|---|---|
| schemaVersion | integer / INT32 | 1 |
| matchId | string / UTF8 | source record ID |
| pool | string / UTF8 | official, live, private (preserve when aggregating) |
| mode | string / UTF8 | product mode |
| scenarioId | string / UTF8 | versioned scenario ID |
| stateHash | string / UTF8 | SHA-256 canonical final engine state |
| replay | string / UTF8 | JSON-encoded complete ReplayLog, including scenario, seed, config, actions and diagnostics |
| io | string / UTF8 | JSON array of per-turn controller output/status, call reference and timing |
| calls | string / UTF8 | JSON array of all provider call attempts, raw request/response, tokens, cost and latency |
| decisions | string / UTF8 | JSON array of observation, selected action, parse diagnostics and attempts |
| accounting | string / UTF8 | JSON object with cost/usage completeness; `null` if unrecorded |

Nested fields are JSON **strings in both formats** so heterogeneous provider
payloads and future fields round-trip losslessly. Decode explicitly with json.loads.
Unknown cost remains null; absent historical call lists become [] (not evidence of
zero tokens). Scripted controllers do not have fabricated LLM prompts/responses.
The authoritative nested replay schema is `/api/research/schema` → protocol.
Raw I/O structures follow the recorded runner result, not a normalized provider API.

```python
import json
import pyarrow.parquet as pq
row = pq.read_table('terminator-v1.parquet').to_pylist()[0]
replay = json.loads(row['replay'])
assert replay['engineVersion'] == '0.1.1'
```

Re-run the log using the matching engine and compare `stateHash`. The replay file
storage checksum is a separate checksum of serialized bytes. `/proof/determinism`
compares canonical engine state, validates scenario hash and rejects unsupported
engine versions, reordered turns and post-terminal actions.

Suite downloads contain `{id,createdAt,manifest}`; the outer metadata is not part
of the manifest hash. Verify `contentHash(manifest without hash)` against its hash.
New manifests also pin controller content and creation date inside the hash.
`pinned.entrants[].revision` is the content hash of that controller (model version,
prompt/code and settings). Legacy suites retain hashes but may lack controller
snapshots; the UI labels that limitation. Do not substitute today's model settings.

Changelog: 2026-09-23 v1 — initial JSONL/Parquet export and proof UI.