Restructures the textbook pipeline around the distinction the body-derived
dataset had been blurring: a bench should be actual unsolved problems, and
a reusable premise library should only ever contain statements that have
really been proven, the way Mathlib itself works.
- library_candidates/goodman-1.6.jsonl (renamed from datasets/goodman-1.6.jsonl):
the book's own Theorem/Proposition/Lemma/Corollary claims from the main
text. Not bench targets -- premises waiting to be proven once via a real
lean-bench run, then assembled into the reusable library by
build_library.py (which already only keeps success:true entries with
their checked proof, unchanged by this commit).
- datasets/goodman-1.6-exercises.jsonl (new): the actual bench, sourced
from the book's Exercises 1.6 block via two new scripts,
textbook/extract_exercises.py (slices the Exercises block, lists its
numbered items, drops a known false-positive class where a wrapped
cross-reference like "...Proposition\n1.6.19." looks like a real
exercise heading) and textbook/formalize_exercises.py (LLM step, given
both the exercises and body text together since many exercises say
"Prove Proposition N.N.N" instead of restating the claim).
- Both formalize.py and formalize_exercises.py's validate() now enforce
that every checklist item is covered by at least one output entry
(included or explicitly excluded with a reason); formalize_exercises.py
did not have this yet and a first real run silently dropped 7 of 15
checklist items with no trace before this was added.
- All 12 model-produced exercise entries needed real fixes before commit,
the same review pass applied to the body pilot: 3 exercises literally
ask to prove an already-stated body claim (Exercise 1.6.1 -> Proposition
1.6.7, 1.6.2 -> Proposition 1.6.9, 1.6.7 -> Proposition 1.6.18, 1.6.9 ->
Corollary 1.6.20, 1.6.15 -> Lemma 1.6.26) and now reuse that claim's own
already-checked statement verbatim instead of an independently
re-derived (and in one case strictly weaker) restatement; two entries
(1.6.2, 1.6.11) had literal math prose like "I(m, n)" or
"gcd(n1,...,nk)" leaked into the statement field, which is not valid
Lean; several had untyped/ambiguous gcd or Prime usage fixed the same
way the earlier body fixes were (Nat.Prime, IsCoprime, explicit casts,
IsGreatest instead of max on a Set); and Exercise 1.6.12 part (a) turned
out to be a discussion question whose unconditional-identity reading is
actually false (6, 10, 15 are jointly coprime but not pairwise coprime),
so it was moved to the excluded file with a reason instead of committed
as a false theorem.
- Definitions can now be real, compiled Lean, not only JSON metadata:
Definition 1.6.22 (gcd of a list of integers, previously a null
placeholder since no single Mathlib constant matches it) now has a real
hand-written `def IsGCDOfList`, and textbook/build_definitions.py
compiles any manifest's non-null lean_code entries into
lib/Definitions.lean. goodman-1-6-24 (library) and the new
goodman-1-6-ex-11-b/c, goodman-1-6-ex-12 (bench) all now use
IsGCDOfList instead of repeating an inline existential.
- textbook/README.md rewritten around the two-track design and records
how this is meant to connect to lean-agent's eventual goal-directed
proof search against the growing library (not yet built).
- New tests for extract_exercises.py and build_definitions.py; full
local suite still green (18/19, the one failure is the pre-existing,
unrelated lean_agent import error in test_dataset.py).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vbLFFJiHSZqGZHEkjvG5x
- extract_section.py: slice a book section out of a pdftotext -layout dump and
list its labeled Theorem/Proposition/Lemma/Corollary/Definition items.
- formalize.py: LLM-formalize a section into Lean 4 candidate statements
(preferring Mathlib vocabulary) plus a definitions manifest.
- check_statements.py: elaboration-only sanity check against a real
Mathlib-backed Lean project (must run outside the Cowork sandbox, which has
no Lean toolchain and can't run macOS binaries anyway).
- build_library.py: assemble solved lean-bench results into a reusable .lean
file, reusing each winning proof verbatim.
- llm.py: standalone OpenAI-compatible chat client + .env loader, kept
independent of lean_agent so this pipeline works without it installed.
- Source PDFs and raw/layout text dumps are gitignored (copyrighted material);
only code and the final short Lean statement datasets are tracked.
claude-bot
codex-bot