Checkpoint rendered cases before solving
This commit is contained in:
@@ -56,6 +56,7 @@ python3 -m logicbench run tasks/short-smoke.json
|
||||
Outputs under the task's `output_dir` are:
|
||||
|
||||
- `skeletons.jsonl`: full hidden programs, sampled behavior, exact probabilities, gold labels, and latent traces; written before any LLM call;
|
||||
- `rendered.jsonl`: each natural-language problem, checkpointed immediately after rendering;
|
||||
- `results.jsonl`: rendered problems, complete solver responses, parsed labels, gold probabilities, and scores;
|
||||
- `summary.json`: aggregate accuracy and structural metadata.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user