Zayden-Jung/lean-bench
2
1
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(pilot): add full-cohort GPT-assisted diagnostic
ci / python (pull_request) Successful in 14s
Details
ci / lean (pull_request) Successful in 29s
Details
ci / image (pull_request) Has been skipped
Details

Browse Source
This commit is contained in:
Codex Bot
2026-10-28 19:28:07 +08:00
parent 5d531c9c5f
commit 2f0a217a94
6 changed files with 1942 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
experiments/P0-pilot/README.md
+13
View File
@@ -52,3 +52,16 @@ Labeling must be finished before anyone opens `results.jsonl`.
by the judge. No threshold, concept table or detector prompt was tuned for these labels.
- Raw passages, labeling HTML and model responses remain in the ignored `data/` directory.
Only verdicts, abstract annotation notes, hashes and text-free summaries belong in Git.
## Separate GPT-assisted diagnostic
The owner requested a complete 65-item reassessment by the current GPT assistant on
2026-10-28. This is separate from the incomplete Qwen run: no outputs are mixed and
neither the earlier labels nor Qwen files are overwritten. The assistant had prior
exposure to the reference labels and Lean; this is not blind or independent validation.
No GO/PARTIAL/NO-GO decision is issued.
See `docs/experiments/P0-gpt-diagnostic.md` for all counts, limitations, artifact locations
and the offline replay command. The tracked `results/gpt-diagnostic-*.json` files contain
only categorical/numeric outputs and audit metadata. The authored source structures,
reasons and full review page stay in the ignored `data/gpt-diagnostic/` directory.