Files
lean-bench/tests/test_check_proofs.py
T
claude-bot fbcb0c3383 Record and gate proofs, so "proven" means Lean verified it
The index had no proof field and every entry was an axiom, so the
database asserted its premises rather than proving them. Adds the proof
half, with the verification gate that makes the status trustworthy.

- Database records now carry `proof` and `proof_checked`, and the search
  index exposes the proof, with status one of proved / proof_pending /
  asserted. build_database.py emits a real `theorem` only when a proof
  has been verified (by check_proofs.py, or by a lean-bench run via
  --results); a recorded but unverified proof stays an `axiom` and is
  reported as proof_pending, so the library never claims more than Lean
  has confirmed. A test pins that gating in both directions.

- textbook/check_proofs.py verifies proofs by compiling
  `theorem ... := by <proof>`. Stricter than check_statements.py in three
  ways: it rejects sorry/admit/native_decide outright, and it reads
  `#print axioms` to reject any proof depending on more than Lean's own
  three axioms (propext, Classical.choice, Quot.sound) -- which in
  particular catches a proof leaning on one of this database's own
  book-asserted axioms, since that is not an independent proof. With
  --update it writes proof_checked back, promoting entries on the next
  build.

- Ten candidate proofs recorded, all currently proof_checked=false
  pending a Lean run: goodman-1-6-1 (the eight algebraic identities are
  `ring`, plus a short argument for the nonzero conjunct), the four parts
  of Proposition 1.6.2 (Int.mul_eq_one_iff_eq_one_or_neg_one,
  associated_of_dvd_dvd with Int.associated_iff, dvd_trans, dvd_add with
  mul_left), 1-6-4 (Nat.exists_prime_and_dvd), 1-6-15
  (Int.gcd_eq_one_iff_coprime, where IsCoprime unfolds to exactly the
  target existential), 1-6-17 (IsCoprime.mul_dvd via
  Nat.isCoprime_iff_coprime), 1-6-19 (Int.Prime.dvd_mul') and 1-6-20
  (Prime.dvd_prod_iff). These are citations of existing Mathlib results,
  which is the realistic way to populate a textbook premise library --
  the remaining 11 (1-6-6, -7, -9, -10, -11, -13, -18, -21, -23, -24,
  -26) are the ones needing actual work and carry no proof yet.

Written without Lean available here, so expect a real failure rate on
first compile -- run check_proofs.py --update and send back the FAILED
ids with diagnostics, same loop as the statement fixes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017vbLFFJiHSZqGZHEkjvG5x
2026-09-18 19:43:54 +00:00

41 lines
1.8 KiB
Python

import unittest
from textbook.check_proofs import ALLOWED_AXIOMS, AXIOM_LINE, check_proof
class CheckProofsTests(unittest.TestCase):
"""Only the Lean-free logic is unit-tested here; actually compiling a proof needs
Lean and Mathlib, which is what running check_proofs.py on a real machine does."""
def test_missing_proof_fails_without_invoking_lean(self):
result = check_proof('t', 'True', '')
self.assertFalse(result.ok)
self.assertIn('no proof recorded', result.diagnostics)
def test_sorry_is_rejected_without_invoking_lean(self):
for escape in ('sorry', 'admit', 'native_decide'):
result = check_proof('t', 'True', f'intro h\n{escape}')
self.assertFalse(result.ok, escape)
self.assertIn(escape, result.diagnostics)
def test_proof_mentioning_sorry_only_as_substring_is_not_rejected_for_that(self):
# 'sorryAx' contains 'sorry' but \b...\b must not match a longer identifier,
# otherwise a legitimate name would be banned by accident.
from textbook.check_proofs import BANNED
self.assertIsNone(BANNED.search('exact notsorryreally'))
self.assertIsNotNone(BANNED.search('exact sorry'))
def test_axiom_line_parsing_separates_standard_from_ours(self):
line = "'goodman_1_6_1' depends on axioms: [propext, Classical.choice, Quot.sound]"
used = {a.strip() for a in AXIOM_LINE.search(line).group(1).split(',')}
self.assertEqual(used - ALLOWED_AXIOMS, set())
def test_axiom_line_parsing_flags_a_book_asserted_axiom(self):
line = "'x' depends on axioms: [propext, goodman_1_6_7, sorryAx]"
used = {a.strip() for a in AXIOM_LINE.search(line).group(1).split(',')}
self.assertEqual(used - ALLOWED_AXIOMS, {'goodman_1_6_7', 'sorryAx'})
if __name__ == '__main__':
unittest.main()