claude-bot
04d827b3b3
Restructures the textbook pipeline around the distinction the body-derived dataset had been blurring: a bench should be actual unsolved problems, and a reusable premise library should only ever contain statements that have really been proven, the way Mathlib itself works. - library_candidates/goodman-1.6.jsonl (renamed from datasets/goodman-1.6.jsonl): the book's own Theorem/Proposition/Lemma/Corollary claims from the main text. Not bench targets -- premises waiting to be proven once via a real lean-bench run, then assembled into the reusable library by build_library.py (which already only keeps success:true entries with their checked proof, unchanged by this commit). - datasets/goodman-1.6-exercises.jsonl (new): the actual bench, sourced from the book's Exercises 1.6 block via two new scripts, textbook/extract_exercises.py (slices the Exercises block, lists its numbered items, drops a known false-positive class where a wrapped cross-reference like "...Proposition\n1.6.19." looks like a real exercise heading) and textbook/formalize_exercises.py (LLM step, given both the exercises and body text together since many exercises say "Prove Proposition N.N.N" instead of restating the claim). - Both formalize.py and formalize_exercises.py's validate() now enforce that every checklist item is covered by at least one output entry (included or explicitly excluded with a reason); formalize_exercises.py did not have this yet and a first real run silently dropped 7 of 15 checklist items with no trace before this was added. - All 12 model-produced exercise entries needed real fixes before commit, the same review pass applied to the body pilot: 3 exercises literally ask to prove an already-stated body claim (Exercise 1.6.1 -> Proposition 1.6.7, 1.6.2 -> Proposition 1.6.9, 1.6.7 -> Proposition 1.6.18, 1.6.9 -> Corollary 1.6.20, 1.6.15 -> Lemma 1.6.26) and now reuse that claim's own already-checked statement verbatim instead of an independently re-derived (and in one case strictly weaker) restatement; two entries (1.6.2, 1.6.11) had literal math prose like "I(m, n)" or "gcd(n1,...,nk)" leaked into the statement field, which is not valid Lean; several had untyped/ambiguous gcd or Prime usage fixed the same way the earlier body fixes were (Nat.Prime, IsCoprime, explicit casts, IsGreatest instead of max on a Set); and Exercise 1.6.12 part (a) turned out to be a discussion question whose unconditional-identity reading is actually false (6, 10, 15 are jointly coprime but not pairwise coprime), so it was moved to the excluded file with a reason instead of committed as a false theorem. - Definitions can now be real, compiled Lean, not only JSON metadata: Definition 1.6.22 (gcd of a list of integers, previously a null placeholder since no single Mathlib constant matches it) now has a real hand-written `def IsGCDOfList`, and textbook/build_definitions.py compiles any manifest's non-null lean_code entries into lib/Definitions.lean. goodman-1-6-24 (library) and the new goodman-1-6-ex-11-b/c, goodman-1-6-ex-12 (bench) all now use IsGCDOfList instead of repeating an inline existential. - textbook/README.md rewritten around the two-track design and records how this is meant to connect to lean-agent's eventual goal-directed proof search against the growing library (not yet built). - New tests for extract_exercises.py and build_definitions.py; full local suite still green (18/19, the one failure is the pre-existing, unrelated lean_agent import error in test_dataset.py). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017vbLFFJiHSZqGZHEkjvG5x
47 lines
1.9 KiB
Python
47 lines
1.9 KiB
Python
import json
|
|
import tempfile
|
|
import unittest
|
|
from pathlib import Path
|
|
|
|
from textbook.build_definitions import build
|
|
|
|
|
|
def write_json(path, data):
|
|
path.write_text(json.dumps(data), encoding='utf-8')
|
|
|
|
|
|
class BuildDefinitionsTests(unittest.TestCase):
|
|
def test_only_entries_with_lean_code_are_emitted(self):
|
|
with tempfile.TemporaryDirectory() as directory:
|
|
path = Path(directory) / 'defs.json'
|
|
write_json(path, [
|
|
{"label": "Definition A", "concept": "has mathlib equivalent",
|
|
"mathlib_ref": "Nat.Prime", "lean_code": None, "notes": ""},
|
|
{"label": "Definition B", "concept": "needs hand-written def",
|
|
"mathlib_ref": None, "lean_code": "def Foo (n : Nat) : Prop := n = n",
|
|
"notes": "some note"},
|
|
])
|
|
text, emitted = build([path])
|
|
self.assertEqual(emitted, 1)
|
|
self.assertIn('def Foo (n : Nat) : Prop := n = n', text)
|
|
self.assertIn('Definition B', text)
|
|
self.assertNotIn('Definition A', text) # has a Mathlib ref, nothing to assemble
|
|
|
|
def test_multiple_manifests_are_concatenated(self):
|
|
with tempfile.TemporaryDirectory() as directory:
|
|
directory = Path(directory)
|
|
path1 = directory / 'defs1.json'
|
|
path2 = directory / 'defs2.json'
|
|
write_json(path1, [{"label": "D1", "concept": "c1", "mathlib_ref": None,
|
|
"lean_code": "def D1 := True", "notes": ""}])
|
|
write_json(path2, [{"label": "D2", "concept": "c2", "mathlib_ref": None,
|
|
"lean_code": "def D2 := True", "notes": ""}])
|
|
text, emitted = build([path1, path2])
|
|
self.assertEqual(emitted, 2)
|
|
self.assertIn('def D1 := True', text)
|
|
self.assertIn('def D2 := True', text)
|
|
|
|
|
|
if __name__ == '__main__':
|
|
unittest.main()
|