Split pipeline into real bench (exercises) vs library candidates (body)

Restructures the textbook pipeline around the distinction the body-derived
dataset had been blurring: a bench should be actual unsolved problems, and
a reusable premise library should only ever contain statements that have
really been proven, the way Mathlib itself works.

- library_candidates/goodman-1.6.jsonl (renamed from datasets/goodman-1.6.jsonl):
  the book's own Theorem/Proposition/Lemma/Corollary claims from the main
  text. Not bench targets -- premises waiting to be proven once via a real
  lean-bench run, then assembled into the reusable library by
  build_library.py (which already only keeps success:true entries with
  their checked proof, unchanged by this commit).
- datasets/goodman-1.6-exercises.jsonl (new): the actual bench, sourced
  from the book's Exercises 1.6 block via two new scripts,
  textbook/extract_exercises.py (slices the Exercises block, lists its
  numbered items, drops a known false-positive class where a wrapped
  cross-reference like "...Proposition\n1.6.19." looks like a real
  exercise heading) and textbook/formalize_exercises.py (LLM step, given
  both the exercises and body text together since many exercises say
  "Prove Proposition N.N.N" instead of restating the claim).
- Both formalize.py and formalize_exercises.py's validate() now enforce
  that every checklist item is covered by at least one output entry
  (included or explicitly excluded with a reason); formalize_exercises.py
  did not have this yet and a first real run silently dropped 7 of 15
  checklist items with no trace before this was added.
- All 12 model-produced exercise entries needed real fixes before commit,
  the same review pass applied to the body pilot: 3 exercises literally
  ask to prove an already-stated body claim (Exercise 1.6.1 -> Proposition
  1.6.7, 1.6.2 -> Proposition 1.6.9, 1.6.7 -> Proposition 1.6.18, 1.6.9 ->
  Corollary 1.6.20, 1.6.15 -> Lemma 1.6.26) and now reuse that claim's own
  already-checked statement verbatim instead of an independently
  re-derived (and in one case strictly weaker) restatement; two entries
  (1.6.2, 1.6.11) had literal math prose like "I(m, n)" or
  "gcd(n1,...,nk)" leaked into the statement field, which is not valid
  Lean; several had untyped/ambiguous gcd or Prime usage fixed the same
  way the earlier body fixes were (Nat.Prime, IsCoprime, explicit casts,
  IsGreatest instead of max on a Set); and Exercise 1.6.12 part (a) turned
  out to be a discussion question whose unconditional-identity reading is
  actually false (6, 10, 15 are jointly coprime but not pairwise coprime),
  so it was moved to the excluded file with a reason instead of committed
  as a false theorem.
- Definitions can now be real, compiled Lean, not only JSON metadata:
  Definition 1.6.22 (gcd of a list of integers, previously a null
  placeholder since no single Mathlib constant matches it) now has a real
  hand-written `def IsGCDOfList`, and textbook/build_definitions.py
  compiles any manifest's non-null lean_code entries into
  lib/Definitions.lean. goodman-1-6-24 (library) and the new
  goodman-1-6-ex-11-b/c, goodman-1-6-ex-12 (bench) all now use
  IsGCDOfList instead of repeating an inline existential.
- textbook/README.md rewritten around the two-track design and records
  how this is meant to connect to lean-agent's eventual goal-directed
  proof search against the growing library (not yet built).
- New tests for extract_exercises.py and build_definitions.py; full
  local suite still green (18/19, the one failure is the pre-existing,
  unrelated lean_agent import error in test_dataset.py).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vbLFFJiHSZqGZHEkjvG5x
This commit is contained in:
2026-09-18 19:27:40 +00:00
parent 5aa048593c
commit 04d827b3b3
14 changed files with 702 additions and 81 deletions
+11
View File
@@ -0,0 +1,11 @@
-- Generated by textbook/build_definitions.py -- hand-written Lean
-- definitions for concepts with no Mathlib equivalent. Regenerate
-- from the definitions manifest(s) instead of hand-editing.
import Mathlib
-- Definition 1.6.22: greatest common divisor (gcd) of several integers
-- No single ready-made Mathlib constant for 'gcd of a `List ℤ`' matches the book's definition verbatim, so this concept gets a real, hand-written Lean `def` (see lean_code) instead of only a JSON pointer -- it is compiled into textbook/lib/Definitions.lean by build_definitions.py and inlined into any statement's `prelude` that needs it, so Lean can actually elaborate proofs that use it, not just read a description of it.
def IsGCDOfList (d : ℕ) (as : List ℤ) : Prop :=
(∀ a ∈ as, (d:ℤ) ∣ a) ∧ ∀ e : ℤ, (∀ a ∈ as, e ∣ a) → e ∣ (d:ℤ)