Checkpoint rendered cases before solving

This commit is contained in:
2026-09-23 00:27:30 +08:00
parent afcd6c614e
commit b645190ea5
5 changed files with 31 additions and 4 deletions
+1
View File
@@ -56,6 +56,7 @@ python3 -m logicbench run tasks/short-smoke.json
Outputs under the task's `output_dir` are:
- `skeletons.jsonl`: full hidden programs, sampled behavior, exact probabilities, gold labels, and latent traces; written before any LLM call;
- `rendered.jsonl`: each natural-language problem, checkpointed immediately after rendering;
- `results.jsonl`: rendered problems, complete solver responses, parsed labels, gold probabilities, and scores;
- `summary.json`: aggregate accuracy and structural metadata.