Score unparseable answers without aborting batches

This commit is contained in:
2026-09-23 00:52:36 +08:00
parent 06cdd9c556
commit 6f929437a2
3 changed files with 33 additions and 4 deletions
+1 -1
View File
@@ -59,7 +59,7 @@ Outputs under the task's `output_dir` are:
- `skeletons.jsonl`: full hidden programs, sampled behavior, exact probabilities, gold labels, and latent traces; written before any LLM call;
- `rendered.jsonl`: each natural-language problem, checkpointed immediately after rendering;
- `solver-outputs.jsonl`: each complete raw solver response, checkpointed before parsing;
- `results.jsonl`: rendered problems, complete solver responses, parsed labels, gold probabilities, and scores;
- `results.jsonl`: rendered problems, complete solver responses, parsed labels, parse errors, format compliance, gold probabilities, and scores;
- `summary.json`: aggregate accuracy and structural metadata.
## Gitea Actions one-shot test