# Scored with auto-v3: 2026-09-23T11-33-54_claude-cli_claude-sonnet-5

Context mode: room; prompt agent-prompt-v1.1; json_mode false; max_tokens CLI default; temperature CLI default (not settable); timeout 180000 ms; reps 3.
Pass = heuristic auto-check against the frozen gold rubric (see eval/rubric.ts); schema-invalid answers fail because the server rejects them. Blind human grading: blind_sheet.md.

| Model | Runs | Pass | Pass by case | Schema-valid | Critical | False blocks | Conflict recall | Latency median / max | Tokens in / out (reasoning) | Total USD | USD per pass | Provider errors |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| claude-sonnet-5 (Claude Code CLI, subscription) | 18 | 16/18 | T1 1/3, T2 3/3, T3 3/3, T4 3/3, T5 3/3, T6 3/3 | 18/18 | 2 | 2 | 21/21 | 10.1 s / 24.1 s | 58352 / 17643 (6126) | $0.3330 | $0.02081 | 0 |

| Order | Case | Rep | Model | JSON | Schema | Pass | Critical | Latency ms | Tokens in/out | Notes |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | T1 | 1 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 13703 | 3244/1529 | - |
| 2 | T2 | 1 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 14373 | 3235/1148 | - |
| 3 | T3 | 1 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 10445 | 3221/788 | - |
| 4 | T4 | 1 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 7526 | 3235/697 | - |
| 5 | T5 | 1 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 14103 | 3231/1225 | - |
| 6 | T6 | 1 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 6103 | 3286/414 | - |
| 7 | T1 | 2 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | n | blocked an allowed change | 20920 | 3245/1989 | replacement text not introduced |
| 8 | T2 | 2 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 8748 | 3235/851 | - |
| 9 | T3 | 2 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 7562 | 3224/576 | - |
| 10 | T4 | 2 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 8895 | 3234/810 | - |
| 11 | T5 | 2 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 9723 | 3233/959 | - |
| 12 | T6 | 2 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 6395 | 3279/533 | - |
| 13 | T1 | 3 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | n | blocked an allowed change | 24145 | 3246/2141 | replacement text not introduced |
| 14 | T2 | 3 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 12248 | 3235/1154 | - |
| 15 | T3 | 3 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 6893 | 3223/595 | - |
| 16 | T4 | 3 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 11738 | 3233/998 | - |
| 17 | T5 | 3 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 10770 | 3232/791 | - |
| 18 | T6 | 3 | claude-sonnet-5 (Claude Code CLI, subscription) | y | y | y | - | 5714 | 3281/445 | - |