Record and gate proofs, so "proven" means Lean verified it

The index had no proof field and every entry was an axiom, so the
database asserted its premises rather than proving them. Adds the proof
half, with the verification gate that makes the status trustworthy.

- Database records now carry `proof` and `proof_checked`, and the search
  index exposes the proof, with status one of proved / proof_pending /
  asserted. build_database.py emits a real `theorem` only when a proof
  has been verified (by check_proofs.py, or by a lean-bench run via
  --results); a recorded but unverified proof stays an `axiom` and is
  reported as proof_pending, so the library never claims more than Lean
  has confirmed. A test pins that gating in both directions.

- textbook/check_proofs.py verifies proofs by compiling
  `theorem ... := by <proof>`. Stricter than check_statements.py in three
  ways: it rejects sorry/admit/native_decide outright, and it reads
  `#print axioms` to reject any proof depending on more than Lean's own
  three axioms (propext, Classical.choice, Quot.sound) -- which in
  particular catches a proof leaning on one of this database's own
  book-asserted axioms, since that is not an independent proof. With
  --update it writes proof_checked back, promoting entries on the next
  build.

- Ten candidate proofs recorded, all currently proof_checked=false
  pending a Lean run: goodman-1-6-1 (the eight algebraic identities are
  `ring`, plus a short argument for the nonzero conjunct), the four parts
  of Proposition 1.6.2 (Int.mul_eq_one_iff_eq_one_or_neg_one,
  associated_of_dvd_dvd with Int.associated_iff, dvd_trans, dvd_add with
  mul_left), 1-6-4 (Nat.exists_prime_and_dvd), 1-6-15
  (Int.gcd_eq_one_iff_coprime, where IsCoprime unfolds to exactly the
  target existential), 1-6-17 (IsCoprime.mul_dvd via
  Nat.isCoprime_iff_coprime), 1-6-19 (Int.Prime.dvd_mul') and 1-6-20
  (Prime.dvd_prod_iff). These are citations of existing Mathlib results,
  which is the realistic way to populate a textbook premise library --
  the remaining 11 (1-6-6, -7, -9, -10, -11, -13, -18, -21, -23, -24,
  -26) are the ones needing actual work and carry no proof yet.

Written without Lean available here, so expect a real failure rate on
first compile -- run check_proofs.py --update and send back the FAILED
ids with diagnostics, same loop as the statement fixes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vbLFFJiHSZqGZHEkjvG5x

textbook/README.md

@@ -169,7 +169,28 @@ task, matching how `check_statements.py` elaborates one task at a time), so
 `lib/Definitions.lean` is the single canonical copy and the inlined ones must stay
 in sync with it by construction (generated from the same manifest).
 
-### 7. Generate the citable premise database
+### 7. Verify recorded proofs
+
+A database entry may carry a `proof` (hand-written, or authored by an agent outside
+a bench run), but it stays an `axiom` until Lean compiles it -- an unverified proof
+never upgrades an entry, so the library cannot claim more than Lean confirmed.
+
+```sh
+python3 textbook/check_proofs.py database/goodman-1.6.jsonl \
+  --lean-project /path/to/your/mathlib-project \
+  --lean-command '["lake","env","lean"]' \
+  --report textbook/goodman-1.6.proof-report.jsonl \
+  --update
+```
+
+Stricter than check_statements.py: it compiles `theorem ... := by <proof>`, rejects
+`sorry`/`admit`/`native_decide`, and reads `#print axioms` to reject a proof that
+depends on anything beyond Lean's own three axioms -- notably one that leans on a
+book-asserted `axiom` from this very database, which would not be an independent
+proof. `--update` writes `proof_checked` back, which is what promotes an entry on
+the next build.
+
+### 8. Generate the citable premise database
 
 ```sh
 python3 textbook/build_database.py database/goodman-1.6.jsonl \

textbook/build_database.py

@@ -98,20 +98,33 @@ def build(entries, results=None, require_checked=False):
 
         name = decl_name(entry_id)
         result = results.get(entry_id)
-        proved = bool(result and result.get('success') and result.get('attempts'))
+        # A proof can come from a lean-bench run, or be recorded on the entry itself
+        # (hand-written, or authored by an agent outside a bench run). Either way it
+        # only counts once check_proofs.py has actually compiled it: an unverified
+        # proof stays an axiom, so the library never claims more than Lean confirmed.
+        run_proof = result['attempts'][-1]['proof'] if (
+            result and result.get('success') and result.get('attempts')) else None
+        own_proof = entry.get('proof') if entry.get('proof_checked') else None
+        proof = run_proof or own_proof
+        proved = bool(proof)
+        proof_pending = bool(not proved and entry.get('proof'))
         provenance = entry.get('source_label', entry_id)
 
         doc = f'/-- {provenance} (Goodman, Algebra: Abstract and Concrete).'
         if entry.get('natural_language'):
             doc += f' {entry["natural_language"].split("[fix note:")[0].strip()}'
-        doc += (' Proven by lean-bench.' if proved
-                else ' ASSERTED from the textbook, not proven here.') + ' -/'
+        if proved:
+            doc += ' Proof verified by Lean.'
+        elif proof_pending:
+            doc += ' ASSERTED from the textbook; a proof is recorded but not yet verified.'
+        else:
+            doc += ' ASSERTED from the textbook, not proven here.'
+        doc += ' -/'
         lines.append(doc)
 
         if proved:
-            proof = result['attempts'][-1]['proof']
             lines.append(f'theorem {name} : {entry["statement"]} := by')
-            lines.extend('  ' + line for line in proof.splitlines())
+            lines.extend('  ' + line for line in proof.strip().splitlines())
         else:
             lines.append(f'axiom {name} : {entry["statement"]}')
         lines.append('')
@@ -121,7 +134,8 @@ def build(entries, results=None, require_checked=False):
             "id": entry_id,
             "source_label": entry.get('source_label'),
             "kind": entry.get('kind'),
-            "status": "proved" if proved else "asserted",
+            "status": "proved" if proved else ("proof_pending" if proof_pending else "asserted"),
+            "proof": proof if proved else entry.get('proof'),
             "statement": entry['statement'],
             "natural_language": entry.get('natural_language', '').split('[fix note:')[0].strip(),
             "identifiers": statement_identifiers(entry['statement']),
@@ -154,7 +168,9 @@ def main():
             stream.write(json.dumps(record, ensure_ascii=False) + '\n')
 
     proved = sum(1 for r in index if r['status'] == 'proved')
-    print(f'{len(emitted)} premises -> {args.out_lean} ({proved} proven, {len(emitted) - proved} asserted as axioms)')
+    pending = sum(1 for r in index if r['status'] == 'proof_pending')
+    print(f'{len(emitted)} premises -> {args.out_lean} ({proved} proven, '
+          f'{len(emitted) - proved} asserted as axioms, of which {pending} have an unverified proof)')
     print(f'search index -> {args.out_index}')
     if skipped:
         print(f'{len(skipped)} skipped for not being elaboration-checked: {", ".join(skipped)}')

textbook/check_proofs.py

@@ -0,0 +1,127 @@
+"""Verify the database's proofs by compiling them, and record the verdict.
+
+check_statements.py asks "is this statement well-formed Lean"; this asks the much
+stronger question "does this proof actually prove it". A database entry is only
+allowed to claim proven status after passing here, so nothing enters the citable
+premise library on an author's confidence alone.
+
+What counts as passing:
+  - `theorem <name> : <statement> := by <proof>` compiles with no error, and
+  - the proof text contains no `sorry`/`admit`/`native_decide`, and
+  - `#print axioms <name>` reports no axiom dependency beyond Lean's own three
+    (propext, Classical.choice, Quot.sound) -- in particular it must not lean on
+    `sorryAx`, nor on any of the book-asserted `axiom` declarations, since a
+    premise proven *through* an asserted axiom is not independently proven.
+
+With --update, entries that pass get `proof_checked: true` written back into the
+database JSONL, which is what makes build_database.py emit them as real theorems
+instead of axioms. Needs Lean/Lake and a built Mathlib, same as check_statements.py.
+"""
+import argparse
+import json
+import re
+import subprocess
+import tempfile
+from dataclasses import dataclass
+from pathlib import Path
+
+BANNED = re.compile(r'\b(sorry|admit|native_decide)\b')
+# The three axioms Lean's own standard library is built on; anything else is ours.
+ALLOWED_AXIOMS = {'propext', 'Classical.choice', 'Quot.sound'}
+AXIOM_LINE = re.compile(r"^'[^']+' depends on axioms: \[(.*)\]$", re.M)
+
+
+@dataclass
+class ProofResult:
+    ok: bool
+    diagnostics: str
+
+
+def check_proof(name, statement, proof, prelude='', command=None, cwd=None, timeout=120):
+    command = command or ['lean']
+    if not proof or not proof.strip():
+        return ProofResult(False, 'no proof recorded')
+    if BANNED.search(proof):
+        return ProofResult(False, f'proof uses a banned escape hatch: {BANNED.search(proof).group(1)}')
+
+    body = '\n'.join('  ' + line for line in proof.strip().splitlines())
+    source = ((prelude + '\n' if prelude else '')
+              + f'theorem {name} : {statement} := by\n{body}\n'
+              + f'#print axioms {name}\n')
+
+    with tempfile.TemporaryDirectory(prefix='lean-proof-', dir=cwd) as directory:
+        path = Path(directory) / 'ProofCheck.lean'
+        path.write_text(source, encoding='utf-8')
+        try:
+            result = subprocess.run([*command, str(path)], cwd=cwd, capture_output=True,
+                                     text=True, timeout=timeout)
+        except subprocess.TimeoutExpired:
+            return ProofResult(False, 'Lean timeout')
+
+    diagnostics = result.stdout + result.stderr
+    if result.returncode != 0 or 'error:' in diagnostics:
+        return ProofResult(False, diagnostics[-8000:])
+
+    match = AXIOM_LINE.search(diagnostics)
+    if match:
+        used = {a.strip() for a in match.group(1).split(',') if a.strip()}
+        extra = used - ALLOWED_AXIOMS
+        if extra:
+            return ProofResult(False, f'proof depends on non-standard axioms: {sorted(extra)}\n'
+                                       + diagnostics[-2000:])
+    return ProofResult(True, diagnostics[-2000:])
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Compile and verify the proofs recorded in a database JSONL')
+    parser.add_argument('database', type=Path, help='e.g. database/goodman-1.6.jsonl')
+    parser.add_argument('--lean-command', default='["lean"]', help='JSON argv array; e.g. ["lake","env","lean"]')
+    parser.add_argument('--lean-project', help='Lean/Lake project working directory (needs Mathlib built)')
+    parser.add_argument('--timeout', type=float, default=120)
+    parser.add_argument('--report', type=Path, help='write JSONL of {id, ok, diagnostics} here')
+    parser.add_argument('--update', action='store_true',
+                        help='write proof_checked back into the database for entries that pass')
+    parser.add_argument('--only', nargs='*', help='check only these ids')
+    args = parser.parse_args()
+
+    command = json.loads(args.lean_command)
+    records = [json.loads(line) for line in args.database.read_text(encoding='utf-8').splitlines() if line.strip()]
+
+    reports, checked, failed, skipped = [], 0, 0, 0
+    for record in records:
+        if args.only and record['id'] not in args.only:
+            continue
+        if not record.get('proof'):
+            skipped += 1
+            continue
+        name = record['id'].replace('-', '_')
+        result = check_proof(name, record['statement'], record['proof'], record.get('prelude', ''),
+                             command, args.lean_project, args.timeout)
+        reports.append({"id": record['id'], "ok": result.ok, "diagnostics": result.diagnostics})
+        print(('PROVED ' if result.ok else 'FAILED ') + record['id'])
+        if result.ok:
+            checked += 1
+            if args.update:
+                record['proof_checked'] = True
+        else:
+            failed += 1
+            if args.update:
+                record['proof_checked'] = False
+
+    if args.update:
+        with args.database.open('w', encoding='utf-8') as stream:
+            for record in records:
+                stream.write(json.dumps(record, ensure_ascii=False) + '\n')
+        print(f'updated proof_checked in {args.database}')
+
+    if args.report:
+        with args.report.open('w', encoding='utf-8') as stream:
+            for report in reports:
+                stream.write(json.dumps(report, ensure_ascii=False) + '\n')
+
+    print(f'{checked} proved, {failed} failed, {skipped} entries have no proof recorded yet')
+    raise SystemExit(1 if failed else 0)
+
+
+if __name__ == '__main__':
+    main()