2.6 KiB
CPU learned-function repetition pilot
Primary question: do two distinct trained parts implement similar functions? Cost is not a success condition.
- Existing learned RMSNorm function positive: True
- Bidirectional network quality screen passed: True
- Learned scalar MLP neuron search positive: False
- Original 11 candidate run state: completed; selected: []
- All model weights are retained on the server until the user explicitly requests deletion.
Existing learned module evidence
Role: input_layernorm; layers 12 and 13; each maps 2048-dimensional input to output.
Uniform real-arithmetic relative-error bound, first-to-second: 0.101851851852; reverse: 0.113402061856.
Maximum heldout directional output relative L2: 0.04528510757179746; minimum cosine: 0.9994870299724158.
Same epsilon and nonzero gains: the common RMS denominator cancels. For any nonzero input x,
||f_a(x)-f_b(x)|| / ||f_a(x)|| <= max_j |a_j-b_j|/|a_j|; swap a/b for the reverse direction.
Selection used only the tight all-input bound across 552 same-role pairs, before inspecting inputs.
Real probes use four calibration and four heldout experiment-authored texts, 32 tokens each, and both source layers.
These are learned, nonconstant gain vectors. The claim concerns RMSNorm modules; it does not establish attention/MLP-body duplication.
- Layer 12 uses 13: mean heldout NLL delta 3.25651368766e-05; sequence P95 delta 0.000984639618287; screen pass=True.
- Layer 13 uses 12: mean heldout NLL delta -0.000363972997404; sequence P95 delta 0.000709721155035; screen pass=True.
Negative results and separate algebraic result
The original whole-attention, whole-MLP and within-layer Q/K norm candidates did not pass their gates. The scalar neuron search excludes low-RMS/near-constant profiles and selects at most one pair without heldout feedback; consult its search summary for coverage. Q/K non-RoPE gain consolidation is a separately verified real-arithmetic identity, not evidence that two existing learned modules duplicate functions. Earlier local CPU timing records are preserved for provenance; performance is outside the current success criterion.
Limits and provenance
Original BF16 weights were decoded into streaming FP32 CPU operators; no training, scale fitting, R fitting or GPU. The numerical reference includes 22 independent official Torch comparisons. Early adapter errors and two resource-guard stops are preserved as historical logs. The persistent computation ledger includes a conservative 180-second reserve for early checks; no budget reset after that reserve. The text sample is tiny and short. Network NLL checks are not broad task-accuracy validation.