Generate the citable premise database, and goal-driven retrieval over it

The database was the missing half: extraction and assembly existed, but
nothing turned the book's main-content statements into something an agent
could actually cite mid-proof, and nothing searched it. Two new scripts.

textbook/build_database.py takes the main-content statements and emits
both halves of a usable premise library:
- an importable Lean file where every entry is a named declaration, so a
  proof can cite goodman_1_6_20 by name;
- a JSONL search index carrying each entry's name, statement, mentioned
  identifiers and status.

An entry is emitted as an `axiom` (asserted from the textbook) until a
real lean-bench run proves it; passing --results upgrades those entries
to real theorems carrying the exact verified proof. This mirrors how the
book itself works -- its exercises are posed assuming its body results,
the same way using Mathlib does not mean re-proving Mathlib. The cost is
explicit: anything proven through an axiom inherits it, so a garbled
extraction asserted as an axiom makes downstream proofs worthless.
Guards: --require-checked refuses entries that have not passed
check_statements.py (all 21 section 1.6 entries have, and are now marked
accordingly), and the verifier's existing #print axioms check surfaces
what a finished proof rests on.

textbook/search_database.py is the retrieval step the proof loop calls at
each line: given the current goal, it ranks premises by identifier and
symbol overlap, weighting rare identifiers far above common ones
(IsGCDOfList matching means much more than Int matching), preferring a
proven premise over an asserted one, and formats the top few for the
prompt. Verified against three real goals: the exactly-applicable premise
tops each ranking by a wide margin (11.4 vs 1.6 for the gcd goal, 5.1 vs
3.7 for the prime-divides-product goal). It is a ranking heuristic, not a
decision procedure -- only Lean can say whether a premise applies.
Replacing score_premise() is the whole job of swapping in embeddings.

Also: renamed library_candidates/ to database/ to match what it is;
generated lib/Goodman/Chapter01Section06.lean (21 premises, 0 proven, 21
asserted) and database/goodman-1.6.index.jsonl; noted a latent bug in the
older build_library.py, which attaches each entry's non-import prelude
per-theorem and so would emit a shared `def` twice in one file --
build_database.py hoists and dedupes, and has a test pinning that.
13 new tests, all green (33 total, the one failure remains the
pre-existing unrelated lean_agent import error in test_dataset.py).

Still not done, and worth being clear about: nothing has been proven yet.
The bench targets have not been elaboration-checked, no lean-bench run
has happened over either file, and the goal-directed loop that would
consume this retrieval does not exist -- lean-bench's solve() is still
single-shot. Separately, lean-bench cannot currently run at all in this
checkout, because lean_bench/cli.py imports lean_agent at module load and
lean-agent is not installed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vbLFFJiHSZqGZHEkjvG5x
This commit is contained in:
2026-09-18 19:36:18 +00:00
parent 04d827b3b3
commit 69c5b788b9
10 changed files with 557 additions and 51 deletions
+1
View File
@@ -10,3 +10,4 @@ results/
textbook/*.txt
textbook/*_raw.txt
.DS_Store
textbook/*.elab-report.jsonl