# HA voice-brain bench — receipts

Method `ha-voice-brain-v1` + `ha-voice-stt-tts-v1`, captured 2026-08-25/26 (US Central) on one
bench: Ryzen 7 7800X3D / RTX 5080 16GB / llama.cpp build 10507 / driver 610.88. License CC BY 4.0,
same as the summary CSV at `/data/ha-voice-brain-bench-2026-08.csv`. Article:
`/tutorials/home-assistant-local-llm-voice-2026/`.

## Files

- `ha_voice_corpus.json` — the frozen 24-command corpus, fixture smart home, and gold acceptance
  sets (v1.0.1; the `_changelog` key records the one gold-label fix).
- `ha_voice_bench.py` — the capture script: llama-server lifecycle, HA-2026.8.3-mirrored system
  prompt + Assist tool schemas, latency probes, scorer.
- `ha_voice_stt_tts_bench.py` — faster-whisper + Piper component bench.
- `ha_voice_rescore.py` — replays raw records against the current corpus, no inference needed.
- `raw_<model>_gpu-rtx5080.jsonl` — full per-request records (cold/warm probes + all 24 corpus
  responses) for the five published models.
- `stt_tts_raw.json` — per-command STT transcriptions/timings and Piper TTS timings.
- `rescore-summary.txt` — output of running `ha_voice_rescore.py` over these exact raw files.

## Scoring provenance (read this before recomputing)

The inline `"correct"` flags inside the raw JSONL files are the scores AT CAPTURE TIME. One gold
label was corrected between captures (corpus v1.0.0 → v1.0.1, see the corpus `_changelog`): the
14B's raw file therefore carries one `correct: false` that v1.0.1 scores as a pass. The
authoritative accuracy numbers are produced by replaying the raws against corpus v1.0.1:

    python ha_voice_rescore.py <this directory>

`rescore-summary.txt` is that command's output over these files and matches the summary CSV and
the article exactly. Raw records are receipts of what happened and are never edited after capture.