Validate repeated learned RMSNorm functions with CPU evidence
This commit is contained in:
+24
-1
@@ -56,7 +56,8 @@ search extra candidates, mutate Git or delete weights. After reference checks, i
|
||||
8-token real text capacity check, then the bounded functional pilot. Reference, preflight
|
||||
and functional compute together are capped at two hours with a reporting reserve;
|
||||
download wait is recorded separately. Projection memory is checked, RSS soft stop is
|
||||
512 MiB, available-system-memory floor 256 MiB; sustained host swap IO also stops the run.
|
||||
512 MiB, available-system-memory floor 256 MiB; sustained host swap IO combined with low available memory or >128 MiB process swap also stops the run.
|
||||
A persistent ledger counts computation across restarts; early checks/preflight receive a conservative 180-second reserve.
|
||||
|
||||
`preflight.py` is a capacity smoke check, not a semantic oracle. `run.py` records exact
|
||||
sample/token IDs, fixed configuration, source hashes, activations, per-text/per-head metrics,
|
||||
@@ -66,3 +67,25 @@ saved data to produce PNG/PDF/SVG figures without reopening weights.
|
||||
No generation throughput is measured in this short-text pilot. Any parameter-tying storage
|
||||
benefit must be accounted for separately, including metadata. Neither same-input similarity
|
||||
nor favorable tiny heldout NLL demonstrates broad mutual interchangeability or GPU speedup.
|
||||
|
||||
## Current primary result and completed extensions
|
||||
|
||||
The latest user goal is learned-function repetition, independent of costs. Existing
|
||||
input RMSNorm modules at layer indices 12/13 have a proven worst two-direction real-arithmetic
|
||||
error bound of 11.3402%, worst observed heldout relative error 4.5285%, minimum cosine 0.999487,
|
||||
and small bidirectional heldout NLL changes. See [findings](../docs/CPU_FUNCTIONAL_FINDINGS.md).
|
||||
|
||||
- `neuron_reuse.py`: fixed four-layer scalar MLP function search, nontrivial activity
|
||||
gates, calibration-only selection and independent heldout/substitution logic; no pair passed.
|
||||
- `norm_repetition.py`: same-role learned RMSNorm bound across 552 pairs, one deterministic
|
||||
selection independent of any input samples, real probes and two full-network substitutions.
|
||||
- `gain_factorization.py`: architecture-derived identity on non-RoPE coordinates; a separate
|
||||
reparameterization result, not a learned-function repetition example.
|
||||
- `benchmark_factorization.py`: historical one-attempt CPU allocation optimization and
|
||||
bitwise equivalence check. Performance is no longer a success condition.
|
||||
- `storage.py`: explicit component layout accounting; the original candidate run selected
|
||||
no pair. This does not affect the primary function-repetition conclusion.
|
||||
|
||||
Do not expand the final positive claim to full attention or MLP bodies. Retain the
|
||||
fixed checkpoint until the user explicitly requests deletion. Full tests and audit are
|
||||
run remotely; no local build or model execution is required to inspect the archived results.
|
||||
|
||||
Reference in New Issue
Block a user