Validate repeated learned RMSNorm functions with CPU evidence
This commit is contained in:
@@ -1,11 +1,15 @@
|
||||
# Model parameter similarity experiments
|
||||
|
||||
Bounded, inference-free analysis of official unquantized model weights. The first experiment
|
||||
is **Qwen3.5-2B**. Model weights are downloaded and analyzed on the authorized remote host,
|
||||
then deleted after archived results pass validation for the completed static pilot.
|
||||
Research on repeated learned functions in **Qwen3.5-2B**: a completed static weight baseline
|
||||
and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm
|
||||
functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks.
|
||||
Cost is not a success condition. The completed static pilot was inference-free and its
|
||||
weights were deleted after archival; the functional pilot executes streaming FP32 text forwards.
|
||||
The current CPU functional checkpoint is retained until the user explicitly requests deletion.
|
||||
No model weights are committed here.
|
||||
|
||||
- [CPU learned-function repetition findings](docs/CPU_FUNCTIONAL_FINDINGS.md)
|
||||
- [Current learned-function repetition criterion](docs/LEARNED_FUNCTION_REPETITION.md)
|
||||
- [CPU-first feasibility and validation](docs/CPU_FIRST_VALIDATION.md)
|
||||
- [Training-free functional reuse plan (proposed; not executed)](docs/TRAINING_FREE_FUNCTIONAL_PLAN.md)
|
||||
- [Primary-source literature and applicability](docs/FUNCTIONAL_REUSE_LITERATURE.md)
|
||||
@@ -21,9 +25,10 @@ No model weights are committed here.
|
||||
- `pilot/run.py`: one-model supervisor with bounded download wait and analysis timeouts.
|
||||
- `tests/test_pilot.py`: synthetic numerical and 3D correctness tests.
|
||||
|
||||
The primary result is **verified net byte savings versus direct relative weight error**.
|
||||
This does not demonstrate preserved task capability or faster inference. Different block
|
||||
scales are alternative partitions and their savings must not be summed.
|
||||
The static baseline reports **verified net byte savings versus direct relative weight error**.
|
||||
The functional study separately reports same-input error, an all-input RMSNorm function bound
|
||||
and tiny heldout NLL changes; these are not broad task-accuracy guarantees. Different static
|
||||
block scales are alternative partitions and their savings must not be summed.
|
||||
|
||||
Dependencies are pinned in `requirements.txt`; install only prebuilt wheels on the remote host.
|
||||
Full resolved versions and run provenance accompany each result. See the SOP for commands.
|
||||
|
||||
Reference in New Issue
Block a user