4.0 KiB
Model parameter similarity experiments
研究图册与固定在线副本：连续分布、联合消融及原始PDF/SVG。
Research on repeated learned functions in Qwen3.5-2B: a completed static weight baseline and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks. Cost is not a success condition. The completed static pilot was inference-free and its weights were deleted after archival; the functional pilot executes streaming FP32 text forwards. The current CPU functional checkpoint is retained until the user explicitly requests deletion. No model weights are committed here.
The latest cross-layer QKV study covers all 15 full-attention layer pairs. Its primary protocol had no passing candidate; an exploratory H23-conditioned self-token retrieval pattern is reported separately with its fresh-test failures.
- Cross-layer QKV findings and evidence limits
- Cross-layer QKV fixed protocol
- CPU learned-function repetition findings
- Current learned-function repetition criterion
- CPU-first feasibility and validation
- Training-free functional reuse plan (proposed; not executed)
- Primary-source literature and applicability
- Procedure and limits
- 2B pilot findings and limitations
- Raw result report, data and figures
pilot/download.py: revision-pinned official checkpoint download and verification.pilot/tensors.py: memory-mapped safetensors/BF16 reader and metrics.pilot/analyze.py: bounded tensor/block search, exact verification, representative groups.pilot/plot.py: scientific PNG/PDF/SVG figures and report from saved data.pilot/validate.py: result/group validation, checksums and guarded weight cleanup.pilot/audit.py: finite-metric checks, test evidence and archive provenance.pilot/run.py: one-model supervisor with bounded download wait and analysis timeouts.tests/test_pilot.py: synthetic numerical and 3D correctness tests.
The static baseline reports verified net byte savings versus direct relative weight error. The functional study separately reports same-input error, an all-input RMSNorm function bound and tiny heldout NLL changes; these are not broad task-accuracy guarantees. Different static block scales are alternative partitions and their savings must not be summed.
Dependencies are pinned in requirements.txt; install only prebuilt wheels on the remote host.
Full resolved versions and run provenance accompany each result. See the SOP for commands.
CPU block activation measurements
See measurement definitions and observed activation report for all 24 layers, contiguous output-channel blocks, exact/near-zero rates and output energy concentration. This is an observational SiLU-gated study, not a pruning or reuse quality claim.
Adjacent-layer head influence concentration
The CPU study measures all 23 adjacent transitions, observation-wise effective head counts and Q/K/V influence shares, with equal-size weak/strong/random joint removal controls. See definitions and reproduction, fixed protocol, and measured report. These are descriptive intervention effects, not additive contribution probabilities or validated final-LM pruning gains.
Existing GSM8K task-finetuning comparison
Paired study and reproduction · Fixed protocol · Results. This compares an existing pure-LoRA GRPO checkpoint to exactly the same original release; no training is performed here.