19 lines
1.1 KiB
Markdown
19 lines
1.1 KiB
Markdown
# Evaluation Records
|
|
|
|
This directory contains curated, immutable milestone reports rather than every benchmark run. Each committed report contains the human-readable evidence needed to understand the result:
|
|
|
|
- original answer-bearing skeletons;
|
|
- rendered problems;
|
|
- complete solver outputs and parsed answers;
|
|
- scores, run metadata, and qualitative anomalies.
|
|
|
|
Raw JSONL from ordinary runs belongs in the Gitea Actions artifact, not Git. Promote a run into this directory only when it establishes a baseline, changes the benchmark protocol, exposes an important failure mode, or is explicitly selected for long-term comparison. This keeps repository growth proportional to meaningful experimental milestones instead of CI volume.
|
|
|
|
## Runs
|
|
|
|
| Date | Run | Task | Cases | Accuracy | Commit |
|
|
| --- | --- | --- | ---: | ---: | --- |
|
|
| 2026-09-22 | [Gitea Actions #90](2026-09-22-run-90/README.md) | `short-smoke` | 5 | 100% | `fab07f8` |
|
|
|
|
Model identifiers were not emitted by run #90 and therefore are intentionally recorded as unknown rather than inferred. Future artifacts should capture the renderer and solver model IDs as non-secret metadata.
|