2.8 KiB
Model parameter similarity experiments
Research on repeated learned functions in Qwen3.5-2B: a completed static weight baseline and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks. Cost is not a success condition. The completed static pilot was inference-free and its weights were deleted after archival; the functional pilot executes streaming FP32 text forwards. The current CPU functional checkpoint is retained until the user explicitly requests deletion. No model weights are committed here.
The latest cross-layer QKV study covers all 15 full-attention layer pairs. Its primary protocol had no passing candidate; an exploratory H23-conditioned self-token retrieval pattern is reported separately with its fresh-test failures.
- Cross-layer QKV findings and evidence limits
- Cross-layer QKV fixed protocol
- CPU learned-function repetition findings
- Current learned-function repetition criterion
- CPU-first feasibility and validation
- Training-free functional reuse plan (proposed; not executed)
- Primary-source literature and applicability
- Procedure and limits
- 2B pilot findings and limitations
- Raw result report, data and figures
pilot/download.py: revision-pinned official checkpoint download and verification.pilot/tensors.py: memory-mapped safetensors/BF16 reader and metrics.pilot/analyze.py: bounded tensor/block search, exact verification, representative groups.pilot/plot.py: scientific PNG/PDF/SVG figures and report from saved data.pilot/validate.py: result/group validation, checksums and guarded weight cleanup.pilot/audit.py: finite-metric checks, test evidence and archive provenance.pilot/run.py: one-model supervisor with bounded download wait and analysis timeouts.tests/test_pilot.py: synthetic numerical and 3D correctness tests.
The static baseline reports verified net byte savings versus direct relative weight error. The functional study separately reports same-input error, an all-input RMSNorm function bound and tiny heldout NLL changes; these are not broad task-accuracy guarantees. Different static block scales are alternative partitions and their savings must not be summed.
Dependencies are pinned in requirements.txt; install only prebuilt wheels on the remote host.
Full resolved versions and run provenance accompany each result. See the SOP for commands.