Validate repeated learned RMSNorm functions with CPU evidence

This commit is contained in:
2026-11-09 14:14:59 +08:00
parent 573cbedd98
commit 6d1acbc26a
108 changed files with 6526 additions and 33 deletions
+24 -1
View File
@@ -56,7 +56,8 @@ search extra candidates, mutate Git or delete weights. After reference checks, i
8-token real text capacity check, then the bounded functional pilot. Reference, preflight
and functional compute together are capped at two hours with a reporting reserve;
download wait is recorded separately. Projection memory is checked, RSS soft stop is
512 MiB, available-system-memory floor 256 MiB; sustained host swap IO also stops the run.
512 MiB, available-system-memory floor 256 MiB; sustained host swap IO combined with low available memory or >128 MiB process swap also stops the run.
A persistent ledger counts computation across restarts; early checks/preflight receive a conservative 180-second reserve.
`preflight.py` is a capacity smoke check, not a semantic oracle. `run.py` records exact
sample/token IDs, fixed configuration, source hashes, activations, per-text/per-head metrics,
@@ -66,3 +67,25 @@ saved data to produce PNG/PDF/SVG figures without reopening weights.
No generation throughput is measured in this short-text pilot. Any parameter-tying storage
benefit must be accounted for separately, including metadata. Neither same-input similarity
nor favorable tiny heldout NLL demonstrates broad mutual interchangeability or GPU speedup.
## Current primary result and completed extensions
The latest user goal is learned-function repetition, independent of costs. Existing
input RMSNorm modules at layer indices 12/13 have a proven worst two-direction real-arithmetic
error bound of 11.3402%, worst observed heldout relative error 4.5285%, minimum cosine 0.999487,
and small bidirectional heldout NLL changes. See [findings](../docs/CPU_FUNCTIONAL_FINDINGS.md).
- `neuron_reuse.py`: fixed four-layer scalar MLP function search, nontrivial activity
gates, calibration-only selection and independent heldout/substitution logic; no pair passed.
- `norm_repetition.py`: same-role learned RMSNorm bound across 552 pairs, one deterministic
selection independent of any input samples, real probes and two full-network substitutions.
- `gain_factorization.py`: architecture-derived identity on non-RoPE coordinates; a separate
reparameterization result, not a learned-function repetition example.
- `benchmark_factorization.py`: historical one-attempt CPU allocation optimization and
bitwise equivalence check. Performance is no longer a success condition.
- `storage.py`: explicit component layout accounting; the original candidate run selected
no pair. This does not affect the primary function-repetition conclusion.
Do not expand the final positive claim to full attention or MLP bodies. Retain the
fixed checkpoint until the user explicitly requests deletion. Full tests and audit are
run remotely; no local build or model execution is required to inspect the archived results.