Archive first live benchmark as curated report

This commit is contained in:
2026-09-22 23:22:57 +08:00
parent fab07f80b0
commit 3cf3db7b12
2 changed files with 172 additions and 0 deletions
+18
View File
@@ -0,0 +1,18 @@
# Evaluation Records
This directory contains curated, immutable milestone reports rather than every benchmark run. Each committed report contains the human-readable evidence needed to understand the result:
- original answer-bearing skeletons;
- rendered problems;
- complete solver outputs and parsed answers;
- scores, run metadata, and qualitative anomalies.
Raw JSONL from ordinary runs belongs in the Gitea Actions artifact, not Git. Promote a run into this directory only when it establishes a baseline, changes the benchmark protocol, exposes an important failure mode, or is explicitly selected for long-term comparison. This keeps repository growth proportional to meaningful experimental milestones instead of CI volume.
## Runs
| Date | Run | Task | Cases | Accuracy | Commit |
| --- | --- | --- | ---: | ---: | --- |
| 2026-09-22 | [Gitea Actions #90](2026-09-22-run-90/README.md) | `short-smoke` | 5 | 100% | `fab07f8` |
Model identifiers were not emitted by run #90 and therefore are intentionally recorded as unknown rather than inferred. Future artifacts should capture the renderer and solver model IDs as non-secret metadata.