Validate repeated learned RMSNorm functions with CPU evidence

This commit is contained in:
2026-11-09 14:14:59 +08:00
parent 573cbedd98
commit 6d1acbc26a
108 changed files with 6526 additions and 33 deletions
+11 -6
View File
@@ -1,11 +1,15 @@
# Model parameter similarity experiments
Bounded, inference-free analysis of official unquantized model weights. The first experiment
is **Qwen3.5-2B**. Model weights are downloaded and analyzed on the authorized remote host,
then deleted after archived results pass validation for the completed static pilot.
Research on repeated learned functions in **Qwen3.5-2B**: a completed static weight baseline
and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm
functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks.
Cost is not a success condition. The completed static pilot was inference-free and its
weights were deleted after archival; the functional pilot executes streaming FP32 text forwards.
The current CPU functional checkpoint is retained until the user explicitly requests deletion.
No model weights are committed here.
- [CPU learned-function repetition findings](docs/CPU_FUNCTIONAL_FINDINGS.md)
- [Current learned-function repetition criterion](docs/LEARNED_FUNCTION_REPETITION.md)
- [CPU-first feasibility and validation](docs/CPU_FIRST_VALIDATION.md)
- [Training-free functional reuse plan (proposed; not executed)](docs/TRAINING_FREE_FUNCTIONAL_PLAN.md)
- [Primary-source literature and applicability](docs/FUNCTIONAL_REUSE_LITERATURE.md)
@@ -21,9 +25,10 @@ No model weights are committed here.
- `pilot/run.py`: one-model supervisor with bounded download wait and analysis timeouts.
- `tests/test_pilot.py`: synthetic numerical and 3D correctness tests.
The primary result is **verified net byte savings versus direct relative weight error**.
This does not demonstrate preserved task capability or faster inference. Different block
scales are alternative partitions and their savings must not be summed.
The static baseline reports **verified net byte savings versus direct relative weight error**.
The functional study separately reports same-input error, an all-input RMSNorm function bound
and tiny heldout NLL changes; these are not broad task-accuracy guarantees. Different static
block scales are alternative partitions and their savings must not be summed.
Dependencies are pinned in `requirements.txt`; install only prebuilt wheels on the remote host.
Full resolved versions and run provenance accompany each result. See the SOP for commands.