Study cross-layer QKV functions and conditional retrieval overlap

Reviewed-on: #3

.gitattributes

@@ -3,3 +3,4 @@ results/** -text
 # Generated CSV uses RFC 4180 line endings; Matplotlib SVG paths contain spaces.
 results/**/*.csv -whitespace
 results/**/*.svg -whitespace
+results/**/*.log -whitespace

.gitignore

@@ -3,3 +3,8 @@ __pycache__/
 weights/
 *.partial
 *.log
+
+# Large reproducible functional activations remain on the experiment server.
+results/cpu-qwen35-2b/baseline-activations.npz
+results/cpu-qwen35-2b/neuron-reuse-activations.npz
+results/cpu-qwen35-2b/norm-repetition-input-profiles.npz

README.md

@@ -1,9 +1,21 @@
 # Model parameter similarity experiments
 
-Bounded, inference-free analysis of official unquantized model weights. The first experiment
-is **Qwen3.5-2B**. Model weights are downloaded and analyzed on the authorized remote host,
-then deleted after archived results pass validation. No model weights are committed here.
+Research on repeated learned functions in **Qwen3.5-2B**: a completed static weight baseline
+and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm
+functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks.
+Cost is not a success condition. The completed static pilot was inference-free and its
+weights were deleted after archival; the functional pilot executes streaming FP32 text forwards.
+The current CPU functional checkpoint is retained until the user explicitly requests deletion.
+No model weights are committed here.
 
+The latest cross-layer QKV study covers all 15 full-attention layer pairs. Its primary
+protocol had no passing candidate; an exploratory H23-conditioned self-token retrieval
+pattern is reported separately with its fresh-test failures.
+
+- [Cross-layer QKV findings and evidence limits](docs/CROSS_LAYER_QKV_FINDINGS.md)
+- [Cross-layer QKV fixed protocol](docs/CROSS_LAYER_QKV_PROTOCOL.md)
+- [CPU learned-function repetition findings](docs/CPU_FUNCTIONAL_FINDINGS.md)
+- [Current learned-function repetition criterion](docs/LEARNED_FUNCTION_REPETITION.md)
 - [CPU-first feasibility and validation](docs/CPU_FIRST_VALIDATION.md)
 - [Training-free functional reuse plan (proposed; not executed)](docs/TRAINING_FREE_FUNCTIONAL_PLAN.md)
 - [Primary-source literature and applicability](docs/FUNCTIONAL_REUSE_LITERATURE.md)
@@ -19,9 +31,10 @@ then deleted after archived results pass validation. No model weights are commit
 - `pilot/run.py`: one-model supervisor with bounded download wait and analysis timeouts.
 - `tests/test_pilot.py`: synthetic numerical and 3D correctness tests.
 
-The primary result is **verified net byte savings versus direct relative weight error**.
-This does not demonstrate preserved task capability or faster inference. Different block
-scales are alternative partitions and their savings must not be summed.
+The static baseline reports **verified net byte savings versus direct relative weight error**.
+The functional study separately reports same-input error, an all-input RMSNorm function bound
+and tiny heldout NLL changes; these are not broad task-accuracy guarantees. Different static
+block scales are alternative partitions and their savings must not be summed.
 
 Dependencies are pinned in `requirements.txt`; install only prebuilt wheels on the remote host.
 Full resolved versions and run provenance accompany each result. See the SOP for commands.

crossqkv/README.md

@@ -0,0 +1,42 @@
+# Cross-layer QKV CPU study
+
+This phase targets learned QKV functions across different layers, independent of cost.
+The six full-attention layers (indices 3/7/11/15/19/23) form 15 unordered pairs.
+No within-layer Q/K/V substitution, training, continuous R fitting, GPU, or new model download.
+
+Read [fixed primary protocol](../docs/CROSS_LAYER_QKV_PROTOCOL.md) and
+[findings with evidence limits](../docs/CROSS_LAYER_QKV_FINDINGS.md).
+The original main protocol had no passing candidate. A later fixed one-source audit
+showed near-self-token retrieval and small readout errors, but failed strict fresh
+mean-JS/row-JS-P95/row-TV-P95 criteria; it is not relabeled as a primary pass.
+
+- `core.py`: exact discrete head assignment, actual attention parts, common-space head
+  decomposition, fixed recipient V/gate/O transplants and temporal-cache controls.
+- `protocol.py`, `fresh_samples.json`: predeclared layer/unit/gate scope and untouched fresh texts.
+- `run.py`, `fresh.py`: calibration, frozen selection, old validation and gated fresh/network steps.
+- `conditional.py`: separately registered post-screen metric audit on one fixed H23 candidate;
+  no fallback selection or later threshold changes.
+- `audit_case.py`: recompute selected fresh probabilities, self mass, shared top5 mass and
+  actual-H3 versus same-H23 difference; no full-network intervention after failure.
+- `report.py`, `audit.py`: saved-data scientific plots, raw-count/matching/probability audits,
+  SHA256 and explicit small Git delivery with external large-cache metadata.
+
+Use the authorized remote CPU environment with the retained source checkpoint/cache:
+
+```sh
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_crossqkv -v
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 timeout 7200 venv/bin/python -B -m crossqkv.run --source runs/cpu-qwen35-2b --root runs/NEW_QKV_DIRECTORY
+```
+
+The main run requires a fresh directory and never overwrites prior protocol evidence.
+The recorded execution used `runs/cross-layer-qkv-v2` after fixing a controls-variable
+shadowing error in the first attempt; old logs and its profile cache are retained.
+The first attempt took about 10.3 seconds, then a 60-second reserve was deducted from
+the next 7140-second run limit. The main study finished in about 74.8 seconds.
+The fixed conditional audit used only remaining shared budget and took about 105.3 seconds.
+Final tests/plots/audits fit comfortably within the two-hour computation budget.
+
+All weights and caches stay on the server until explicit user deletion. Git contains
+all figure tables and the selected small probability matrices; large caches have exact
+paths/sizes/hashes in `external-caches.json`. Old validation texts were used in earlier
+experiments; the four fresh texts were fixed before the new cross-layer measurements.

crossqkv/__init__.py

@@ -0,0 +1 @@
+"""Cross-layer learned QKV function comparisons, CPU only."""

crossqkv/audit.py

@@ -0,0 +1,95 @@
+"""Audit and package only science outputs; retain all weights/profile caches."""
+import argparse
+import csv
+import hashlib
+import json
+from pathlib import Path
+import shutil
+import time
+import numpy as np
+from functional.run import write_json
+from .protocol import LAYERS,PAIRS
+from .core import assignment
+
+
+def digest(path):
+    h=hashlib.sha256()
+    with path.open('rb') as f:
+        while b:=f.read(4*1024*1024):h.update(b)
+    return h.hexdigest()
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);p.add_argument('--source',type=Path,required=True);p.add_argument('--output',type=Path,required=True);a=p.parse_args();root=a.root.resolve();source=a.source.resolve();out=a.output.resolve();assert root!=out
+    status=json.loads((root/'status.json').read_text());assert status['state']=='completed' and status['last_source_layer']==23 and status['last_pair']==[19,23]
+    expected={'full_attention':15,'qkv_head':960,'qk_map':30,'qk_head':960,'v_group':60};assert status['candidate_counts']==expected
+    summary=json.loads((root/'calibration-summary.json').read_text());assert len(summary)==2025
+    assert not status['selected'] and all(not x['strict_pass'] and not x['approximate_pass'] for x in summary)
+    selection=json.loads((root/'selection.json').read_text());assert selection['qk_head_prequalified']==selection['v_group_prequalified']==0
+    from collections import defaultdict
+    rawout=defaultdict(list)
+    for r in csv.DictReader((root/'calibration-output-metrics.csv').open()):rawout[r['candidate']].append(r)
+    for c in summary:
+        if c['output_verified']:
+            rr=rawout[c['id']];assert len(rr)==c['output_observations']
+            worst=max(max(float(r['error_a']),float(r['error_b'])) for r in rr)
+            assert abs(worst-c['worst_output_error'])<1e-12 and worst>.20
+    assert min(c['worst_mean_js'] for c in summary if c['kind']=='qk_head')>.05
+    assert min(c['worst_value_error'] for c in summary if c['kind']=='v_group')>.20
+
+    rows=list(csv.DictReader((root/'head-assignment-costs.csv').open()));assert len(rows)==960
+    for i,j in PAIRS:
+        r=[x for x in rows if int(x['i'])==i and int(x['j'])==j];cost=np.empty((8,8));chosen=[]
+        for x in r:
+            ha,hb=int(x['head_i']),int(x['head_j']);cost[ha,hb]=float(x['calibration_mean_js'])
+            if x['chosen']=='True':chosen.append((ha,hb))
+        assert len(chosen)==8;assert [b for _,b in sorted(chosen)]==assignment(cost)
+    tests=(root/'final-tests.txt').read_text();assert 'Ran 20 tests' in tests and '\nOK\n' in tests
+    cond=json.loads((root/'conditional-status.json').read_text());assert cond['state']=='completed' and cond['fresh_function_pass'] is False
+    case=json.loads((root/'conditional-case-audit.json').read_text());assert case['passed_recomputation'] and case['no_success_relabeling']
+    assert len(case['strict_failures'])==3 and case['network_intervention_not_run_due_to_fresh_gate_failure']
+    assert cond['shared_budget_seconds_including_60s_early_reserve']+600<7200
+    prob=np.load(root/'conditional-fresh-probabilities.npz');target=prob['target_layer23_head6'];donor=prob['donor_layer3_head7_on_H23']
+    from .core import distribution_metrics
+    from .conditional import tv_metrics
+    metric={**distribution_metrics(target,donor),**tv_metrics(target,donor)}
+    fresh=[r for r in csv.DictReader((root/'conditional-function-metrics.csv').open()) if r['split']=='fresh_test']
+    assert len(fresh)==4
+    for k,values in metric.items():
+        if k in fresh[0] and values[0] is not None:np.testing.assert_allclose(values,[float(r[k]) for r in fresh],atol=1e-12,rtol=1e-12)
+    for file in root.glob('*.csv'):
+        with file.open() as f:
+            for r in csv.DictReader(f):
+                for value in r.values():
+                    if value in ('',None):continue
+                    try:x=float(value)
+                    except ValueError:continue
+                    assert np.isfinite(x),(file,value)
+    weights=json.loads((source/'manifest.json').read_text());assert weights['revision']=='15852e8c16360a2fea060d615a32b45270f8a8fc'
+    for entry in weights['files']:
+        if entry['name'].endswith('.safetensors'):assert (source/'weights'/entry['name']).stat().st_size==entry['bytes'] and entry['sha256']==entry['upstream_sha256']
+    assert json.loads((source/'retention-policy.json').read_text())['automatic_cleanup'] is False
+    reference=json.loads((source/'reference-check.json').read_text());assert reference['implementation_sha256']==digest(Path('functional/model.py'))
+    write_json(root/'final-source-hashes.json',{str(file):digest(file) for file in [*sorted(Path('crossqkv').glob('*.py')),Path('crossqkv/fresh_samples.json'),Path('tests/test_crossqkv.py'),Path('functional/model.py')]})
+    caches=[]
+    for file in sorted((root/'cache').glob('*')):
+        if file.is_file():caches.append({'server_path':str(file),'bytes':file.stat().st_size,'sha256':digest(file),'retained':True})
+    old_cache=root.parent/'cross-layer-qkv'/'cache'/'calibration-source-3.npz'
+    if old_cache.exists():caches.append({'server_path':str(old_cache),'bytes':old_cache.stat().st_size,'sha256':digest(old_cache),'retained':True,'failed_attempt_cache':True})
+    write_json(root/'external-caches.json',{'host':'8.211.154.134','source_weights_directory':str(source/'weights'),'source_baseline_cache':str(source/'baseline-activations.npz'),
+               'files':caches,'automatic_deletion':False,'all_figure_source_tables_and_selected_probabilities_in_git':True})
+    (root/'DELIVERY.md').write_text('# Cross-layer QKV scientific delivery\n\nAll plot CSV/JSON, the selected fresh probability matrices, protocols, reference tests and failure logs are included.\nLarge intermediate profiles remain on 8.211.154.134 with locations/sizes/SHA256 in external-caches.json.\nThe fixed checkpoint and all caches are retained until explicit user deletion. No weights are committed.\nThe primary protocol has no passing candidate. The later fixed conditional audit is exploratory and its strict fresh checks failed; observed readout overlap is reported without relabeling it a complete pass.\n')
+    out.mkdir(exist_ok=True,parents=True)
+    files=[f for f in root.iterdir() if f.is_file() and f.name not in ('checksums.json','audit-validation.json','archive-files.json')]
+    files += list((root/'figures').glob('*'))
+    for file in files:
+        dest=out/file.relative_to(root);dest.parent.mkdir(exist_ok=True,parents=True);shutil.copy2(file,dest)
+    checks={str(f.relative_to(out)):digest(f) for f in sorted(out.rglob('*')) if f.is_file() and f.name not in ('checksums.json','audit-validation.json','archive-files.json')}
+    write_json(out/'checksums.json',checks)
+    write_json(out/'archive-files.json',[*checks,'checksums.json','audit-validation.json','archive-files.json'])
+    write_json(out/'audit-validation.json',{'passed':True,'validated_unix':time.time(),'checksummed_files':len(checks),'primary_counts_rechecked':expected,
+               'fresh_probabilities_independently_recomputed':True,'no_unrun_network_quality_claims':True,'all_weights_and_caches_retained':True})
+    print(json.dumps({'passed':True,'checksummed_files':len(checks),'external_cache_files':len(caches),'weights_retained':True}))
+
+
+if __name__=='__main__':main()

crossqkv/audit_case.py

@@ -0,0 +1,62 @@
+"""Recompute the frozen exploratory case for transparent plotted probabilities.
+
+No new selection, no threshold changes, no full-network intervention after failure.
+"""
+import argparse
+import json
+from pathlib import Path
+import numpy as np
+from functional.run import save_csv,write_json
+from .core import QKVModel,distribution_metrics
+from .conditional import tv_metrics
+
+
+def ranks_and_mass(a,b):
+    result=[]
+    for bi in range(a.shape[0]):
+        d={'target_self_mass':[],'donor_self_mass':[],'argmax_agreement':[],'shared_top5_mass_target':[],'shared_top5_mass_donor':[]}
+        for t in range(7,32):
+            x,y=a[bi,0,t,:t+1],b[bi,0,t,:t+1]
+            common=np.intersect1d(np.argsort(x)[-5:],np.argsort(y)[-5:])
+            d['target_self_mass'].append(float(x[t]));d['donor_self_mass'].append(float(y[t]))
+            d['argmax_agreement'].append(float(np.argmax(x)==np.argmax(y)))
+            d['shared_top5_mass_target'].append(float(x[common].sum()));d['shared_top5_mass_donor'].append(float(y[common].sum()))
+        result.append({k:float(np.mean(v)) for k,v in d.items()})
+    return result
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--source',type=Path,required=True);p.add_argument('--root',type=Path,required=True);args=p.parse_args();root=args.root;source=args.source
+    status=json.loads((root/'conditional-status.json').read_text());assert status['state']=='completed'
+    data=np.load(root/'cache'/'conditional-fresh-inputs.npz')
+    samples=json.loads((root/'conditional-fresh-samples.json').read_text())['samples']
+    lm=QKVModel(source/'weights',json.loads((source/'model_config.json').read_text()),seconds=300)
+    own=lm.parts(data['attn_23'],23);same=lm.parts(data['attn_23'],3);actual=lm.parts(data['attn_3'],3)
+    target=own['p'][:,6:7];donor=same['p'][:,7:8];cached=actual['p'][:,7:8]
+    np.savez(root/'conditional-fresh-probabilities.npz',target_layer23_head6=target,donor_layer3_head7_on_H23=donor,donor_layer3_head7_on_H3=cached)
+    rows=[]
+    from .run import outrows
+    for mode,profile,prob in [('same_H23',same,donor),('actual_H3_to_H23',actual,cached)]:
+        stats={**distribution_metrics(target,prob),**tv_metrics(target,prob)}
+        mass=ranks_and_mass(target,prob)
+        changed=lm.head_transplant_output(own,profile,23,6,7)
+        out=outrows(own['heads'][:,:,6],changed,samples,{},1e-6,1e-6)
+        for n,s in enumerate(samples):rows.append({'mode':mode,'sample':s['id'],**mass[n],**{k:(None if v[n] is None else float(v[n])) for k,v in stats.items()},'head_output_error':max(out[n]['error_a'],out[n]['error_b'])})
+    save_csv(root/'conditional-fresh-source-diagnostic.csv',rows)
+    with (root/'conditional-function-metrics.csv').open() as f:import csv;allrows=list(csv.DictReader(f))
+    failures=[]
+    for r in allrows:
+        if r['split']!='fresh_test':continue
+        for field,limit in [('js',.01),('js_p95',.05),('total_variation',.10),('total_variation_p95',.20)]:
+            if float(r[field])>limit:failures.append({'sample':r['sample'],'field':field,'value':float(r[field]),'limit':limit})
+    same_rows=[r for r in rows if r['mode']=='same_H23']
+    classification='self_token_dominated' if min(min(r['target_self_mass'],r['donor_self_mass']) for r in same_rows)>.85 else 'inspect detailed source diagnostic'
+    write_json(root/'conditional-case-audit.json',{'passed_recomputation':True,'strict_failures':failures,'pattern_classification':classification,
+                'minimum_same_input_target_self_mass':min(r['target_self_mass'] for r in same_rows),'minimum_same_input_donor_self_mass':min(r['donor_self_mass'] for r in same_rows),
+                'minimum_same_input_argmax_agreement':min(r['argmax_agreement'] for r in same_rows),
+                'minimum_shared_top5_probability_mass':min(min(r['shared_top5_mass_target'],r['shared_top5_mass_donor']) for r in same_rows),
+                'maximum_same_input_head_output_error':max(r['head_output_error'] for r in same_rows),
+                'no_success_relabeling':True,'network_intervention_not_run_due_to_fresh_gate_failure':True})
+
+
+if __name__=='__main__':main()

crossqkv/conditional.py

@@ -0,0 +1,113 @@
+"""One explicitly exploratory, fixed-candidate audit of unweighted top-k rejection.
+
+Primary protocol remains unchanged/negative. A new confirmation uses untouched
+fresh texts; no further candidate selection, threshold tuning, weights or R fitting.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import resource
+import time
+import traceback
+import numpy as np
+from tokenizers import Tokenizer
+from functional.run import write_json,save_csv
+from .core import InterventionLM,distribution_metrics,output_metrics
+from .run import head_floor,rows_for
+
+CANDIDATE={'id':'conditional_qk_head_3_23_7_6','kind':'qk_head','i':3,'j':23,'a':7,'b':6,'perm':None}
+PROTOCOL={'classification':'post-screen exploratory metric audit; not a primary-protocol positive',
+          'candidate':CANDIDATE,'conditioning_input_layer':23,'donor_projection_layer':3,'target_head':6,'donor_head':7,
+          'selection_basis':'one minimum worst-text JS among the already measured source-conditioned pairs; no raw readout/fresh data used',
+          'js_max':.01,'js_row_p95_max':.05,'mean_total_variation_max':.10,'row_total_variation_p95_max':.20,
+          'target_head_output_relative_l2_max':.20,'target_head_output_cosine_min':.98,
+          'primary_query_positions':list(range(7,32)),'top5_is_diagnostic_only':True,
+          'active_head_required':True,'no_candidate_fallback':True,'fresh_text_file':'crossqkv/fresh_samples.json',
+          'same_input_network_direction':'layer23 head6 uses QK probability of layer3 head7 recomputed on H23',
+          'actual_cache_separate':'layer3 probability on H3 reused at23; different-input diagnostic, not function-equivalence evidence',
+          'quality_gate':{'mean_nll_delta_max':.01,'per_text_p95_nll_delta_max':.05},'training':False,'R_fitting':False,'cost_required':False}
+
+
+def tv_metrics(a,b):
+    tv=.5*np.sum(np.abs(a.astype(np.float64)-b.astype(np.float64)),axis=-1)[:,:,7:]
+    return {'total_variation':tv.mean((1,2)),'total_variation_p95':np.quantile(tv.reshape(tv.shape[0],-1),.95,axis=1)}
+
+
+def evaluate(lm,h,samples,split,donor_h=None):
+    target=lm.parts(h,23);donor=lm.parts(h if donor_h is None else donor_h,3)
+    a,b=target['p'][:,6:7],donor['p'][:,7:8]
+    pm={**distribution_metrics(a,b),**tv_metrics(a,b)}
+    changed=lm.head_transplant_output(target,donor,23,6,7)
+    om=output_metrics(target['heads'][:,7:,6],changed[:,7:]);floor=head_floor(target)
+    uniform=np.zeros_like(a)
+    for t in range(a.shape[-1]):uniform[:,:,t,:t+1]=1/(t+1)
+    null=distribution_metrics(a,uniform)
+    rows=rows_for({**pm,**{f'output_{k}':v for k,v in om.items()},'target_uniform_js':null['js']},samples,{'split':split,'mode':'same_H23' if donor_h is None else 'actual_H3_to_H23'})
+    for n,r in enumerate(rows):r['active']=bool(om['rms_a'][n]>=floor[n] and om['rms_b'][n]>=floor[n])
+    passed=all(r['js']<=.01 and r['js_p95']<=.05 and r['total_variation']<=.10 and r['total_variation_p95']<=.20 and
+               max(r['output_error_a'],r['output_error_b'])<=.20 and r['output_cosine']>=.98 and r['active'] for r in rows)
+    return rows,passed
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--source',type=Path,required=True);p.add_argument('--root',type=Path,required=True);a=p.parse_args();source=a.source.resolve();root=a.root.resolve()
+    assert not (root/'conditional-protocol.json').exists(),'Fresh conditional audit protocol required'
+    write_json(root/'conditional-protocol.json',{**PROTOCOL,'registered_unix':time.time(),'fresh_text_sha256':hashlib.sha256(Path(__file__).with_name('fresh_samples.json').read_bytes()).hexdigest()})
+    prior=json.loads((root/'status.json').read_text());assert prior['state']=='completed' and not prior['selected']
+    reserve=prior['elapsed_seconds']+60
+    remaining=int(7200-reserve-180)
+    if remaining<120:raise TimeoutError('Insufficient shared two-hour budget')
+    started=time.monotonic();status={'state':'running','phase':'calibration_readout','primary_protocol_unchanged_and_negative':True}
+    rows=[]
+    try:
+        lm=InterventionLM(source/'weights',json.loads((source/'model_config.json').read_text()),seconds=remaining)
+        old=json.loads((source/'samples-tokenized.json').read_text())['samples'];data=np.load(source/'baseline-activations.npz')
+        for split in ('calibration','heldout'):
+            index=np.array([i for i,s in enumerate(old) if s['split']==split]);samples=[old[i] for i in index]
+            rr,passed=evaluate(lm,data['attn_23'][index],samples,split)
+            rows.extend(rr);save_csv(root/'conditional-function-metrics.csv',rows)
+            status[split+'_pass']=passed
+            write_json(root/'conditional-status.json',status)
+            if not passed:
+                status['stop_reason']=f'{split} probability-mass/readout/activity screen failed';break
+        else:
+            fresh=json.loads(Path(__file__).with_name('fresh_samples.json').read_text())['samples'];tokenizer=Tokenizer.from_file(str(source/'tokenizer.json'));ids=[]
+            for s in fresh:
+                tokens=tokenizer.encode(s['text'],add_special_tokens=False).ids;assert len(tokens)>=32
+                s.update(token_ids=tokens[:32],full_token_count=len(tokens),text_sha256=hashlib.sha256(s['text'].encode()).hexdigest());ids.append(tokens[:32])
+            ids=np.array(ids,dtype=np.int64);write_json(root/'conditional-fresh-samples.json',{'samples':fresh,'candidate_already_fixed':True})
+            status['phase']='fresh_baseline';write_json(root/'conditional-status.json',status)
+            h,inputs=lm.forward(ids,capture=[('attn',3),('attn',23)])
+            np.savez(root/'cache'/'conditional-fresh-inputs.npz',**inputs,final=h)
+            base=lm.nll(h,ids);np.save(root/'conditional-fresh-baseline-token-nll.npy',base)
+            rr,passed=evaluate(lm,inputs['attn_23'],fresh,'fresh_test');rows.extend(rr);save_csv(root/'conditional-function-metrics.csv',rows)
+            status['fresh_function_pass']=passed
+            if passed:
+                status['phase']='one_direction_same_input_network';write_json(root/'conditional-status.json',status)
+                lm.configure(CANDIDATE,reverse=True,cached=False);changed,_=lm.forward(ids);nll=lm.nll(changed,ids);np.save(root/'conditional-same-input-network-token-nll.npy',nll)
+                delta=nll.mean(1)-base.mean(1)
+                network={'mode':'layer23_head6_uses_layer3_head7_on_H23','sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_nll_delta':float(np.quantile(delta,.95)),
+                         'quality_pass':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05),'one_direction_only':True}
+                write_json(root/'conditional-network.json',network);status['same_input_network_quality_pass']=network['quality_pass']
+                # Check the actual-cache probability/readout separately, then run its network experiment.
+                cached_rows,cached_function_pass=evaluate(lm,inputs['attn_23'],fresh,'fresh_test_actual_cache',donor_h=inputs['attn_3'])
+                save_csv(root/'conditional-actual-cache-metrics.csv',cached_rows)
+                status['actual_cache_function_screen_pass']=cached_function_pass
+                lm.configure(CANDIDATE,reverse=True,cached=True);changed,_=lm.forward(ids);nll=lm.nll(changed,ids);np.save(root/'conditional-actual-cache-network-token-nll.npy',nll)
+                delta=nll.mean(1)-base.mean(1)
+                cache={'mode':'actual_layer3_H3_probability_reused_at23_H23','sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_nll_delta':float(np.quantile(delta,.95)),
+                       'quality_pass':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05),'different_hidden_inputs':True}
+                write_json(root/'conditional-actual-cache-network.json',cache);status['actual_cache_network_quality_pass']=cache['quality_pass']
+        status.update(state='completed',phase='done',conditional_function_positive=bool(status.get('fresh_function_pass')),
+                      position_pattern_classification='near_uniform' if rows and max(r['target_uniform_js'] for r in rows)<=.01 else 'inspect first-key/self/recent3 masses',
+                      claim_limit='Single-source H23-conditioned, single-head QK probability/readout similarity; not bidirectional, not V similarity, not full-QKV-head equivalence; exploratory addendum independently fresh-tested.')
+    except Exception as exc:
+        status.update(state='stopped',error_type=type(exc).__name__,error=str(exc));(root/'conditional-error.txt').write_text(traceback.format_exc());raise
+    finally:
+        status.update(elapsed_seconds=time.monotonic()-started,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
+                      shared_budget_seconds_including_60s_early_reserve=reserve+time.monotonic()-started)
+        write_json(root/'conditional-status.json',status)
+
+
+if __name__=='__main__':main()

crossqkv/core.py

@@ -0,0 +1,171 @@
+"""Attention functions and discrete diagnostics, no continuous alignment/fitting."""
+import numpy as np
+from functional.model import StreamingLM,PREFIX,attention,rope,sigmoid
+
+
+def assignment(cost):
+    """Exact minimum-cost bijection via O(n^2 2^n) dynamic programming."""
+    cost=np.asarray(cost,dtype=np.float64);n=cost.shape[0]
+    assert cost.shape==(n,n) and n<=12 and np.isfinite(cost).all()
+    states={0:(0.,())}
+    for i in range(n):
+        nxt={}
+        for mask,(value,perm) in states.items():
+            for j in range(n):
+                if mask&(1<<j):continue
+                key=mask|(1<<j);candidate=(value+float(cost[i,j]),perm+(j,))
+                if key not in nxt or candidate<nxt[key]:nxt[key]=candidate
+        states=nxt
+    return list(states[(1<<n)-1][1])
+
+
+def inverse(perm):
+    out=[0]*len(perm)
+    for i,j in enumerate(perm):out[j]=i
+    return out
+
+
+def output_metrics(a,b):
+    """Per-text directional error, cosine and output RMS. No fitted scaling."""
+    x=a.reshape(a.shape[0],-1).astype(np.float64)
+    y=b.reshape(b.shape[0],-1).astype(np.float64)
+    xx=np.sum(x*x,-1);yy=np.sum(y*y,-1);diff=np.sum((x-y)**2,-1)
+    return {'error_a':np.sqrt(diff/np.maximum(xx,1e-30)),
+            'error_b':np.sqrt(diff/np.maximum(yy,1e-30)),
+            'cosine':np.sum(x*y,-1)/np.maximum(np.sqrt(xx*yy),1e-30),
+            'rms_a':np.sqrt(xx/x.shape[1]),'rms_b':np.sqrt(yy/y.shape[1])}
+
+
+def distribution_metrics(a,b):
+    """Aligned heads; primary queries have at least 8 legal keys (t>=7)."""
+    a,b=a.astype(np.float64),b.astype(np.float64)
+    assert a.shape==b.shape and a.shape[-1]>=8
+    m=(a+b)*.5
+    def term(x):return np.where(x>0,x*np.log(np.maximum(x,1e-300)/np.maximum(m,1e-300)),0)
+    js=np.maximum(.5*np.sum(term(a)+term(b),axis=-1),0)
+    eligible=js[...,7:]
+    overlaps=[]
+    for t in range(7,a.shape[-2]):
+        aa=np.argsort(a[:,:,t,:t+1],axis=-1)[...,-5:]
+        bb=np.argsort(b[:,:,t,:t+1],axis=-1)[...,-5:]
+        overlaps.append(np.mean((aa[...,None]==bb[...,None,:]).any(-1),-1))
+    # First-key ablation is diagnostic, not an unreported filter of positional repetitions.
+    ar,br=a[...,1:],b[...,1:];ma,mb=ar.sum(-1),br.sum(-1)
+    valid=(ma>1e-8)&(mb>1e-8)
+    ar=ar/np.maximum(ma[...,None],1e-300);br=br/np.maximum(mb[...,None],1e-300)
+    mr=(ar+br)*.5
+    cond=.5*np.sum(np.where(ar>0,ar*np.log(np.maximum(ar,1e-300)/np.maximum(mr,1e-300)),0)+np.where(br>0,br*np.log(np.maximum(br,1e-300)/np.maximum(mr,1e-300)),0),-1)
+    sink=[]
+    for bi in range(a.shape[0]):
+        which=valid[bi,:,7:];values=cond[bi,:,7:]
+        sink.append(float(np.maximum(values[which],0).mean()) if which.any() else None)
+    diag=np.stack([a[:,:,t,t] for t in range(7,a.shape[-2])],-1).mean((1,2))
+    local=np.stack([a[:,:,t,max(0,t-2):t+1].sum(-1) for t in range(7,a.shape[-2])],-1).mean((1,2))
+    return {'js':eligible.mean((1,2)),'js_p95':np.quantile(eligible.reshape(a.shape[0],-1),.95,axis=1),
+            'top5':np.stack(overlaps,-1).mean((1,2)),'js_all_queries':js.mean((1,2)),
+            'first_key_mass_a':a[:,:,7:,0].mean((1,2)),'first_key_mass_b':b[:,:,7:,0].mean((1,2)),
+            'self_mass_a':diag,'recent3_mass_a':local,'js_without_first_key':sink}
+
+
+def cross_head_output_metrics(a,b):
+    """All head pairs, per text. a/b: [B,T,H,Dhidden]."""
+    aa=a.transpose(0,2,1,3).reshape(a.shape[0],a.shape[2],-1).astype(np.float64)
+    bb=b.transpose(0,2,1,3).reshape(b.shape[0],b.shape[2],-1).astype(np.float64)
+    xx=np.sum(aa*aa,-1);yy=np.sum(bb*bb,-1)
+    dot=np.einsum('bif,bjf->bij',aa,bb,optimize=True)
+    diff=np.maximum(xx[:,:,None]+yy[:,None,:]-2*dot,0)
+    return {'error_a':np.sqrt(diff/np.maximum(xx[:,:,None],1e-30)),
+            'error_b':np.sqrt(diff/np.maximum(yy[:,None,:],1e-30)),
+            'cosine':dot/np.maximum(np.sqrt(xx[:,:,None]*yy[:,None,:]),1e-30),
+            'rms_a':np.broadcast_to(np.sqrt(xx/aa.shape[-1])[:,:,None],dot.shape),
+            'rms_b':np.broadcast_to(np.sqrt(yy/bb.shape[-1])[:,None,:],dot.shape)}
+
+
+def value_gram_cka(a,b):
+    """Centered token Gram agreement, diagnostic only; not interchangeability."""
+    a=a.astype(np.float64)-a.mean(axis=1,keepdims=True,dtype=np.float64)
+    b=b.astype(np.float64)-b.mean(axis=1,keepdims=True,dtype=np.float64)
+    ga=np.einsum('btd,bsd->bts',a,a);gb=np.einsum('btd,bsd->bts',b,b)
+    return np.sum(ga*gb,(1,2))/np.maximum(np.sqrt(np.sum(ga*ga,(1,2))*np.sum(gb*gb,(1,2))),1e-30)
+
+
+class QKVModel(StreamingLM):
+    def project(self,context,gate,layer):
+        return self.linear((context*sigmoid(gate)).reshape(*context.shape[:2],-1),f'layers.{layer}.self_attn.o_proj')
+
+    def project_heads(self,context,gate,layer):
+        """Each head's contribution in the common residual output coordinates."""
+        c=context*sigmoid(gate);b,t,h,d=c.shape
+        weight=self.tensors[PREFIX+f'layers.{layer}.self_attn.o_proj.weight']
+        outdim,indim=weight.shape;assert indim==h*d
+        out=np.empty((b,t,h,outdim),dtype=np.float32)
+        for start in range(0,outdim,256):
+            self.guard();end=min(start+256,outdim)
+            w=weight.read(slice(start*indim,end*indim)).reshape(end-start,indim)
+            for head in range(h):out[:,:,head,start:end]=c[:,:,head,:]@w[:,head*d:(head+1)*d].T
+            del w;weight.evict()
+        return out
+
+    def parts(self,h,layer,heads=True):
+        p=f'layers.{layer}.self_attn.';c=self.config;d=c['head_dim'];nh=c['num_attention_heads'];nk=c['num_key_value_heads']
+        mixed=self.linear(h,p+'q_proj').reshape(*h.shape[:2],nh,2*d)
+        q,gate=mixed[...,:d],mixed[...,d:].copy()
+        k=self.linear(h,p+'k_proj').reshape(*h.shape[:2],nk,d)
+        v=self.linear(h,p+'v_proj').reshape(*h.shape[:2],nk,d)
+        q=self.normalization(q,p+'q_norm');k=self.normalization(k,p+'k_norm')
+        r=c['rope_parameters'];q,k=(rope(x,r['rope_theta'],r['partial_rotary_factor']) for x in (q,k))
+        context,probs=attention(q,k,v)
+        out={'q':q,'k':k,'p':probs,'v':v,'gate':gate,'context':context,'output':self.project(context,gate,layer)}
+        if heads:out['heads']=self.project_heads(context,gate,layer)
+        return out
+
+    def transplant(self,own,donor,layer,kind,target_head=None,donor_head=None,perm=None):
+        repeats=self.config['num_attention_heads']//self.config['num_key_value_heads']
+        if kind=='full_attention':return donor['output']
+        if kind=='qkv_head':return own['output']-own['heads'][:,:,target_head]+donor['heads'][:,:,donor_head]
+        probs=own['p'].copy();v=own['v'].copy()
+        if kind=='qk_map':probs=donor['p'][:,perm]
+        elif kind=='qk_head':probs[:,target_head]=donor['p'][:,donor_head]
+        elif kind=='v_group':v[:,:,target_head]=donor['v'][:,:,donor_head]
+        else:raise ValueError(kind)
+        context=np.einsum('bhts,bshd->bthd',probs,np.repeat(v,repeats,axis=2))
+        return self.project(context,own['gate'],layer)
+
+    def head_transplant_output(self,own,donor,layer,target_head,donor_head):
+        """QK-only transplanted target head contribution; all other heads omitted."""
+        repeats=self.config['num_attention_heads']//self.config['num_key_value_heads']
+        c=own['context'].copy()
+        c[:,:,target_head]=np.einsum('bts,bsd->btd',donor['p'][:,donor_head],own['v'][:,:,target_head//repeats])
+        # Zero remaining head contexts before projection to isolate this learned subfunction.
+        for head in range(c.shape[2]):
+            if head!=target_head:c[:,:,head]=0
+        return self.project(c,own['gate'],layer)
+
+    def value_group_output(self,own,donor,layer,target_group,donor_group):
+        repeats=self.config['num_attention_heads']//self.config['num_key_value_heads'];c=np.zeros_like(own['context'])
+        for head in range(target_group*repeats,(target_group+1)*repeats):
+            c[:,:,head]=np.einsum('bts,bsd->btd',own['p'][:,head],donor['v'][:,:,donor_group])
+        return self.project(c,own['gate'],layer)
+
+
+class InterventionLM(QKVModel):
+    def configure(self,candidate,reverse=False,cached=False):
+        self.candidate=candidate;self.reverse=reverse;self.use_cache=cached;self.donor_cache=None
+        self.target_layer=candidate['j'] if reverse else candidate['i']
+        self.donor_layer=candidate['i'] if reverse else candidate['j']
+        self.target_head=candidate.get('b') if reverse else candidate.get('a')
+        self.donor_head=candidate.get('a') if reverse else candidate.get('b')
+        self.perm=inverse(candidate['perm']) if reverse and candidate.get('perm') else candidate.get('perm')
+        if cached:assert self.donor_layer<self.target_layer,'Cannot reuse future layer state'
+
+    def full_attention(self,h,layer,shared_p=None):
+        if not hasattr(self,'candidate'):return super().full_attention(h,layer,shared_p)
+        if self.use_cache and layer==self.donor_layer:
+            self.donor_cache=self.parts(h,layer,heads=self.candidate['kind']=='qkv_head')
+            return self.donor_cache['output'],self.donor_cache['p']
+        if layer!=self.target_layer:return super().full_attention(h,layer,shared_p)
+        own=self.parts(h,layer,heads=self.candidate['kind']=='qkv_head')
+        donor=self.donor_cache if self.use_cache else self.parts(h,self.donor_layer,heads=self.candidate['kind']=='qkv_head')
+        assert donor is not None
+        out=self.transplant(own,donor,layer,self.candidate['kind'],self.target_head,self.donor_head,self.perm)
+        return out,own['p']

crossqkv/fresh.py

@@ -0,0 +1,56 @@
+"""Fresh heldout confirmation and same-input versus actual-state interventions."""
+import hashlib
+import json
+from pathlib import Path
+import time
+import numpy as np
+from functional.run import write_json,save_csv
+from .protocol import LAYERS
+
+
+def run_fresh(source,root,c,tier,lm,status):
+    from tokenizers import Tokenizer
+    from .run import validate_selected,candidate_pass,summarize
+    samples=json.loads(Path(__file__).with_name('fresh_samples.json').read_text())['samples']
+    tokenizer=Tokenizer.from_file(str(source/'tokenizer.json'));ids=[]
+    for s in samples:
+        encoded=tokenizer.encode(s['text'],add_special_tokens=False).ids
+        assert len(encoded)>=32,s['id']
+        s.update(token_ids=encoded[:32],full_token_count=len(encoded),text_sha256=hashlib.sha256(s['text'].encode()).hexdigest())
+        ids.append(encoded[:32])
+    ids=np.array(ids,dtype=np.int64)
+    write_json(root/'fresh-samples-tokenized.json',{'samples':samples,'selection_already_frozen':True})
+    status['phase']='fresh_test_baseline';write_json(root/'status.json',status)
+    h,inputs=lm.forward(ids,capture=[('attn',i) for i in LAYERS])
+    np.savez(root/'cache'/'fresh-baseline-activations.npz',final=h,**inputs)
+    nll=lm.nll(h,ids);np.save(root/'fresh-baseline-token-nll.npy',nll)
+    checked=validate_selected(lm,c,inputs,samples,np.arange(len(samples)))
+    passed=candidate_pass(checked,tier)
+    write_json(root/'fresh-function-summary.json',summarize(checked))
+    save_csv(root/'fresh-output-metrics.csv',checked['output_rows']);save_csv(root/'fresh-probability-metrics.csv',checked['probability_rows']);save_csv(root/'fresh-value-metrics.csv',checked['value_rows'])
+    status.update(fresh_function_pass=passed,fresh_confirmation_tier=tier)
+    if not passed:return
+    status['phase']='same_input_network_substitution';write_json(root/'status.json',status)
+    interventions=[]
+    for reverse in (False,True):
+        lm.configure(c,reverse=reverse,cached=False)
+        t=time.monotonic();changed,_=lm.forward(ids);loss=lm.nll(changed,ids)
+        label=f'{lm.target_layer}_uses_{lm.donor_layer}'
+        np.save(root/f'network-{label}-token-nll.npy',loss)
+        delta=loss.mean(1)-nll.mean(1)
+        rec={'mode':'same_input_donor_function','target_layer':lm.target_layer,'donor_layer':lm.donor_layer,
+             'target_head_or_group':lm.target_head,'donor_head_or_group':lm.donor_head,
+             'sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_nll_delta':float(np.quantile(delta,.95)),
+             'quality_pass':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05),'elapsed_seconds':time.monotonic()-t}
+        interventions.append(rec);write_json(root/'network-interventions.json',interventions)
+    # Temporal reuse is a separate intervention and only goes from earlier to later.
+    status['phase']='earlier_to_later_actual_cache';write_json(root/'status.json',status)
+    lm.configure(c,reverse=True,cached=True)
+    changed,_=lm.forward(ids);loss=lm.nll(changed,ids);np.save(root/'network-earlier-actual-cache-token-nll.npy',loss)
+    delta=loss.mean(1)-nll.mean(1)
+    cache={'mode':'earlier_actual_state_reused_later','donor_layer':lm.donor_layer,'target_layer':lm.target_layer,
+           'sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_nll_delta':float(np.quantile(delta,.95)),
+           'quality_pass':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05),
+           'not_same_input_comparison':True,'no_reverse_future_state_claim':True}
+    write_json(root/'actual-cache-intervention.json',cache)
+    status.update(bidirectional_same_input_network_pass=all(r['quality_pass'] for r in interventions),actual_cache_quality_pass=cache['quality_pass'])

crossqkv/fresh_samples.json

@@ -0,0 +1,9 @@
+{
+  "origin":"Four experiment-authored fresh plain texts, fixed for this cross-layer QKV goal before measurements; no chat template, first 32 tokenizer tokens.",
+  "samples":[
+    {"id":"fresh-zh-1","language":"zh","text":"植物学家在不同海拔采集了叶片样本，并为每一份标本记录温度、光照和土壤湿度。回到实验室后，她按地点整理标签，先核对缺失记录，再比较不同环境中的叶片形状。"},
+    {"id":"fresh-zh-2","language":"zh","text":"如果一个盒子里有十二枚蓝色棋子和八枚白色棋子，随机取出一枚后不放回，第二次取到蓝色的概率需要根据第一次的结果分别计算。请先列出各个事件，再检查它们的概率之和。"},
+    {"id":"fresh-en-1","language":"en","text":"A technician recorded the vibration of a small motor before and after replacing its bearings. She used the same mounting position and sampling interval for both measurements. The final notebook included the raw traces, instrument settings, and a brief explanation of unusual observations."},
+    {"id":"fresh-en-2","language":"en","text":"The function walks through a list of integers and stores each running total in a new list. An empty input should return an empty result. Before changing the implementation, compare a short example by hand and make sure negative numbers preserve the intended behavior."}
+  ]
+}

crossqkv/protocol.py

@@ -0,0 +1,30 @@
+"""Fixed protocol, constants frozen before any new cross-layer measurements."""
+from itertools import combinations
+
+LAYERS=[3,7,11,15,19,23]
+PAIRS=list(combinations(LAYERS,2))
+TIERS={
+ 'strict':{'output_error':.05,'output_cosine':.99,'js':.01,'js_p95':.05,'top5':.8,'value_error':.05,'value_cosine':.99},
+ 'approximate':{'output_error':.20,'output_cosine':.98,'js':.05,'js_p95':.15,'top5':.6,'value_error':.20,'value_cosine':.98}}
+PROTOCOL={'layers':LAYERS,'layer_pairs':PAIRS,'heads':8,'kv_heads':2,'head_dim':256,
+ 'calibration_samples':['cal-zh-1','cal-zh-2','cal-en-1','cal-en-2'],
+ 'validation_samples':['test-zh-1','test-zh-2','test-en-1','test-en-2'],
+ 'fresh_test_file':'crossqkv/fresh_samples.json','fresh_test_count':4,'tokens_per_text':32,
+ 'query_positions_for_primary_metrics':list(range(7,32)),
+ 'candidate_types':['full_attention','qkv_head','qk_map','qk_head','v_group'],
+ 'discrete_head_matching':'Exact 8-head minimum mean-JS bijection, two source-layer calibration input distributions; frozen before validation; P injection only',
+ 'continuous_alignment':False,'training':False,'scale_fitting':False,'gpu':False,'cost_required':False,
+ 'tiers':TIERS,'max_qk_head_output_candidates':32,'max_v_group_output_candidates':16,
+ 'all_qk_map_output_candidates':30,'max_final_candidate':1,
+ 'selection':'strict before approximate; type priority full_attention/qkv_head/qk_map/qk_head/v_group; then worst output error, JS, ID; no validation-driven fallback',
+ 'head_output_activity_floor':'max(1e-6, 0.1 * within-layer median head RMS) for every calibration/validation/test text',
+ 'fresh_test_run_only_if_frozen_candidate_passes_validation':True,
+ 'network_interventions':'two same-input donor-function substitution directions; plus one earlier-to-later actual cached-state reuse test reported separately',
+ 'network_quality':{'mean_token_nll_delta_max':.01,'per_text_p95_nll_delta_max':.05},
+ 'retention':'all weights and caches retained until explicit user deletion request',
+ 'seconds':7200,'controls':['identity','uniform causal P','seeded within-causal-row key shuffle','first-key ablation','self/recent3 mass','value Gram to input','value Gram of two seeded Gaussian projections'],
+ 'positional_repeat_labels':{'sink_dominated':.90,'self_dominated':.90,'recent3_dominated':.90},
+ 'limitations':['Old validation texts were used in prior experiments; four fresh texts are reserved for this new goal',
+ 'Direct V-coordinate disagreement does not disprove V/O functional equivalence; centered value Gram CKA is diagnostic only',
+ 'Arbitrary P-head assignment does not authorize arbitrary GQA KV-weight permutation',
+ 'Short text-only pilot does not establish broad task equivalence or linear-attention reuse']}

crossqkv/report.py

@@ -0,0 +1,126 @@
+"""Scientific plots from saved tables/probabilities only; no model access."""
+import argparse
+import csv
+import json
+from pathlib import Path
+import numpy as np
+import matplotlib
+matplotlib.use('Agg')
+import matplotlib.pyplot as plt
+from matplotlib.patches import Rectangle
+from .protocol import LAYERS,PAIRS
+
+
+def read_csv(path):
+    with path.open() as f:return list(csv.DictReader(f))
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);args=p.parse_args();root=args.root;out=root/'figures';out.mkdir(exist_ok=True)
+    plt.rcParams.update({'font.size':9,'pdf.fonttype':42,'svg.fonttype':'none','savefig.bbox':'tight'})
+    def save(fig,name):
+        for ext in ('png','pdf','svg'):fig.savefig(out/f'{name}.{ext}',dpi=180)
+        plt.close(fig)
+    status=json.loads((root/'status.json').read_text());summary=json.loads((root/'calibration-summary.json').read_text())
+    matrices=[]
+    for mode in ('qk_head','identity','matched'):
+        matrix=np.full((6,6),np.nan)
+        for i,j in PAIRS:
+            rows=[r for r in summary if r['i']==i and r['j']==j and ((mode=='qk_head' and r['kind']=='qk_head') or (mode!='qk_head' and r['id']==f'qk_map_{i}_{j}_{mode}'))]
+            value=min(r['worst_mean_js'] for r in rows)
+            a,b=LAYERS.index(i),LAYERS.index(j);matrix[a,b]=matrix[b,a]=value
+        matrices.append(matrix)
+    fig,axes=plt.subplots(1,3,figsize=(13,4.7));vmax=max(np.nanmax(m) for m in matrices)
+    for ax,m,title in zip(axes,matrices,['Best single head pair','All heads: original order','All heads: calibration bijection']):
+        im=ax.imshow(m,cmap='YlOrRd',vmin=0,vmax=vmax)
+        for a in range(6):
+            ax.add_patch(Rectangle((a-.5,a-.5),1,1,facecolor='#eeeeee',edgecolor='#aaaaaa',hatch='///',lw=.3))
+            for b in range(6):
+                if a!=b:ax.text(b,a,f'{m[a,b]:.3f}',ha='center',va='center',fontsize=7,color='black' if m[a,b]<vmax*.65 else 'white')
+        ax.set_xticks(range(6),LAYERS);ax.set_yticks(range(6),LAYERS);ax.set_xlabel('Layer index');ax.set_ylabel('Layer index');ax.set_title(title)
+    fig.suptitle('QK retrieval divergence: worst-text mean JS over both source input sets\nCalibration only; single-head panel minimizes over 64 cross-layer head pairs; hatched diagonal unused')
+    fig.subplots_adjust(top=.77,bottom=.18,right=.9,wspace=.35);cax=fig.add_axes([.92,.2,.012,.5]);fig.colorbar(im,cax=cax,label='JS (natural log); lower is closer');save(fig,'cross_layer_qk_divergence')
+    order=['full_attention','qkv_head','qk_map','qk_head','v_group'];labels=['Whole attention','QKV+gate/O head','QK map transplant','QK head transplant','V group transplant']
+    fig,ax=plt.subplots(figsize=(10,4.9));rng=np.random.default_rng(0)
+    for idx,(kind,label) in enumerate(zip(order,labels)):
+        rows=[r for r in summary if r['kind']==kind and r['output_verified']]
+        if rows:
+            x=idx+rng.uniform(-.15,.15,len(rows));y=[r['worst_output_error'] for r in rows]
+            ax.scatter(x,y,s=12,alpha=.35,color='#606060')
+            ax.text(idx,max(y)*1.1,f'n={len(rows)}',ha='center',fontsize=8)
+        else:ax.text(idx,.3,'not output-probed\ninitial gate failed',ha='center',va='center',fontsize=8)
+    ax.axhline(.05,color='#b2182b',ls='--',label='Strict output error = 0.05')
+    ax.axhline(.2,color='#d95f02',ls=':',label='Approximate output error = 0.20')
+    ax.set_yscale('log');ax.set_xlim(-.5,4.5);ax.set_xticks(range(5),labels,rotation=20,ha='right');ax.set_ylabel('Worst directional relative L2, queries 7–31')
+    ax.set_title('Primary protocol: calibrated cross-layer output comparisons\nEach point is one unit; no candidate passed all predeclared gates')
+    ax.legend(loc='upper left',bbox_to_anchor=(0,-.28),fontsize=8);fig.tight_layout();save(fig,'cross_layer_output_errors')
+    values=read_csv(root/'calibration-value-metrics.csv');unit={}
+    for r in values:unit.setdefault(r['candidate'],[]).append(r)
+    xx=[np.mean([float(r['centered_value_gram_cka']) for r in rows]) for rows in unit.values()]
+    yy=[max(max(float(r['error_a']),float(r['error_b'])) for r in rows) for rows in unit.values()]
+    controls=read_csv(root/'controls.csv');random_cka=[float(r['gram_cka']) for r in controls if r['control']=='value_gram_random_projection_pair']
+    fig,ax=plt.subplots(figsize=(7,4.5));ax.scatter(xx,yy,s=24,color='#377eb8',label='60 cross-layer KV-head pairs')
+    ax.axvspan(min(random_cka),max(random_cka),color='gray',alpha=.15,label='Random-projection Gram CKA range')
+    ax.axvline(np.median(random_cka),color='gray',ls=':')
+    ax.axhline(.2,color='#b2182b',ls='--',label='Approximate direct V error = 0.20')
+    ax.set_xlabel('Mean centered token-Gram CKA');ax.set_ylabel('Worst direct V relative L2');ax.set_title('V geometry is a diagnostic, not substitution evidence')
+    ax.legend(loc='upper left',bbox_to_anchor=(0,-.19),fontsize=8);fig.tight_layout();save(fig,'value_geometry_and_direct_error')
+    conditional_path=root/'conditional-status.json'
+    cond=json.loads(conditional_path.read_text()) if conditional_path.exists() else {}
+    if (root/'conditional-function-metrics.csv').exists():
+        rows=read_csv(root/'conditional-function-metrics.csv');labels=[r['sample'] for r in rows];x=np.arange(len(rows))
+        fig,axes=plt.subplots(1,2,figsize=(13,5))
+        axes[0].plot(x,[float(r['top5']) for r in rows],'o-',label='Unweighted top-5 overlap')
+        axes[0].plot(x,[1-float(r['total_variation']) for r in rows],'o-',label='Probability-mass overlap (1−TV)')
+        axes[0].set_ylim(0,1.03);axes[0].set_ylabel('Overlap');axes[0].legend(loc='lower left',fontsize=8)
+        axes[1].plot(x,[max(float(r['output_error_a']),float(r['output_error_b'])) for r in rows],'o-',color='#238b45')
+        axes[1].set_ylim(0,max(.05,max(max(float(r['output_error_a']),float(r['output_error_b'])) for r in rows)*1.15))
+        axes[1].set_ylabel('Head readout relative L2');axes[1].set_title('Fixed layer23 V/gate/O',fontsize=9)
+        for ax in axes:
+            for pos in (3.5,7.5):ax.axvline(pos,color='gray',lw=.6)
+            ax.set_xticks(x,labels,rotation=50,ha='right');ax.set_xlabel('Calibration / old validation / fresh tests, 4 each')
+        fig.suptitle('Exploratory fixed case: layer3 head7 vs layer23 head6, both applied to H23\nFresh JS / tail-mass limits not fully passed; network substitution not run')
+        fig.tight_layout();save(fig,'conditional_probability_mass_and_readout')
+    if (root/'conditional-fresh-probabilities.npz').exists():
+        p=np.load(root/'conditional-fresh-probabilities.npz');a=p['target_layer23_head6'][0,0];b=p['donor_layer3_head7_on_H23'][0,0];d=p['donor_layer3_head7_on_H3'][0,0]
+        fig,axes=plt.subplots(2,2,figsize=(9,8));images=[a,b,np.abs(a-b),d]
+        titles=['Layer23 head6 on H23','Layer3 head7 on the same H23','Absolute same-input probability difference','Layer3 head7 on its actual H3 (different input)']
+        for idx,(ax,z,title) in enumerate(zip(axes.flat,images,titles)):
+            im=ax.imshow(z,cmap='magma',vmin=0,vmax=(max(float(z.max()),1e-8) if idx==2 else 1.));fig.colorbar(im,ax=ax,fraction=.045,pad=.03)
+            ax.set_xticks([0,7,15,23,31]);ax.set_yticks([0,7,15,23,31]);ax.set_xlabel('Key position');ax.set_ylabel('Query position');ax.set_title(title,fontsize=9)
+        fig.suptitle('One fresh Chinese text: cross-layer conditional self-token retrieval\nRows 0–6 shown for context; primary metrics use queries 7–31')
+        fig.tight_layout();save(fig,'conditional_cross_layer_probability_maps')
+    text=['# Cross-layer QKV CPU study','',
+          'Primary protocol and exploratory addendum are reported separately; no post-hoc success relabeling.',
+          f'- Primary state: {status["state"]}; selected candidate: {status.get("selected",[])}.',
+          f'- Candidate counts: {status.get("candidate_counts")}.',
+          f'- Passing counts (approximate tier, all gates): {status.get("passing_counts")}.',
+          f'- Primary computation seconds: {status.get("elapsed_seconds")}; peak RSS KiB: {status.get("peak_rss_kib")}.','',
+          '## What was measured','',
+          'Six full-attention layers, indices 3/7/11/15/19/23; all 15 cross-layer pairs. No within-layer Q/K/V interchange.',
+          'Same normalized hidden inputs from both source layers; frozen Q/K norm, partial RoPE, GQA, V, output gate and O.',
+          'QK head probabilities, whole-map injection in original/frozen matched head order, direct V functions, complete head contributions in residual coordinates, and complete branches were separated.',
+          'Primary metrics use queries 7–31. Discrete matching uses calibration only; no continuous alignment, scale fitting or training.',
+          'No primary candidate passed, so primary validation/fresh/network intervention stages were not entered.','']
+    for kind in order:
+        group=[r for r in summary if r['kind']==kind];outrows=[r for r in group if r['worst_output_error'] is not None];p=[r for r in group if r['worst_mean_js'] is not None];v=[r for r in group if r['worst_value_error'] is not None]
+        text += [f'- {kind}: n={len(group)}, best worst output error={min((r["worst_output_error"] for r in outrows),default=None)}, minimum worst-text mean JS={min((r["worst_mean_js"] for r in p),default=None)}, best worst direct V error={min((r["worst_value_error"] for r in v),default=None)}.']
+    if cond:
+        text+=['','## Exploratory metric audit: frozen one-source candidate','',
+               'After inspecting primary calibration results, one candidate was frozen: layer3 head7 vs layer23 head6 on H23.',
+               'This addendum audited the instability of unweighted top-k tail ranking; original primary results were not changed.',
+               'Four old validation and four previously unused fresh texts were evaluated with frozen extra gates and no candidate fallback.',
+               f'- Addendum fresh function pass: {cond.get("fresh_function_pass")}.',
+               f'- Overall addendum state: {cond.get("state")}; primary unchanged/negative: {cond.get("primary_protocol_unchanged_and_negative")}.',
+               'Fresh probability-mass and fixed V/gate/O readout measurements show conditional behavioral overlap; strict fresh mean-JS, row-JS-P95 and row-TV-P95 limits still failed.',
+               'No full-network substitution was run after that failure. V similarity, bidirectionality and complete QKV-head equivalence are not established.',
+               'The actual-H3 cache probability is a separate different-input diagnostic, not the same-input function comparison.']
+    text+=['','## Limits and retention','',
+           'This is a short text-only pilot, not proof of absence of repetition elsewhere. Exact arithmetic, matrix coordinates, retrieval behavior and final residual output are distinct questions.',
+           'Centered value Gram CKA can be high even for independent Gaussian projections of common inputs; it is not substitution evidence.',
+           'All weights and large profile caches remain on the authorized server. Git delivery includes all plot tables, probabilities, protocols and cache location/hash metadata.',
+           'No GPU, new checkpoint download, model training, continuous R fitting or automatic weight/cache deletion.']
+    (root/'REPORT.md').write_text('\n'.join(text)+'\n')
+
+
+if __name__=='__main__':main()

crossqkv/run.py

@@ -0,0 +1,279 @@
+"""Bounded 15-pair CPU study, calibration selection before any heldout feedback."""
+import argparse
+from collections import defaultdict
+import hashlib
+import json
+from pathlib import Path
+import resource
+import time
+import traceback
+import numpy as np
+from functional.run import save_csv,write_json
+from .core import (QKVModel,InterventionLM,assignment,inverse,output_metrics,
+                   cross_head_output_metrics,distribution_metrics,value_gram_cka)
+from .protocol import LAYERS,PAIRS,PROTOCOL,TIERS
+
+PRIMARY=slice(7,None)
+TYPE_ORDER={'full_attention':0,'qkv_head':1,'qk_map':2,'qk_head':3,'v_group':4}
+
+
+def candidate_id(kind,i,j,a=None,b=None,suffix=''):
+    return f'{kind}_{i}_{j}'+(f'_{a}_{b}' if a is not None else '')+suffix
+
+
+def rows_for(metrics,samples,meta):
+    rows=[]
+    for n,s in enumerate(samples):
+        r={**meta,'sample':s['id']}
+        for key,value in metrics.items():
+            v=value[n]
+            r[key]=None if v is None else float(v)
+        rows.append(r)
+    return rows
+
+
+def outrows(a,b,samples,meta,floor_a=1e-6,floor_b=1e-6):
+    m=output_metrics(a[:,PRIMARY],b[:,PRIMARY]);rows=rows_for(m,samples,meta)
+    for n,r in enumerate(rows):r['active']=bool(m['rms_a'][n]>=np.broadcast_to(floor_a,m['rms_a'].shape)[n] and m['rms_b'][n]>=np.broadcast_to(floor_b,m['rms_b'].shape)[n])
+    return rows
+
+
+def head_floor(profile):
+    rms=np.sqrt(np.mean(profile['heads'][:,7:].astype(np.float64)**2,axis=(1,3)))
+    return np.maximum(1e-6,.1*np.median(rms,axis=1))
+
+
+def candidate_pass(c,tier):
+    t=TIERS[tier];out=c.get('output_rows',[])
+    if not out or not all(r['active'] and max(r['error_a'],r['error_b'])<=t['output_error'] and r['cosine']>=t['output_cosine'] for r in out):return False
+    if c['kind'] in ('qk_head','qk_map'):
+        p=c.get('probability_rows',[])
+        if not p or not all(r['js']<=t['js'] and r['js_p95']<=t['js_p95'] and r['top5']>=t['top5'] for r in p):return False
+    if c['kind']=='v_group':
+        v=c.get('value_rows',[])
+        if not v or not all(max(r['error_a'],r['error_b'])<=t['value_error'] and r['cosine']>=t['value_cosine'] and r['active'] for r in v):return False
+    return True
+
+
+def preliminary_probability(c):
+    t=TIERS['approximate'];r=c['probability_rows']
+    return bool(r) and all(x['js']<=t['js'] and x['js_p95']<=t['js_p95'] and x['top5']>=t['top5'] for x in r)
+
+
+def preliminary_value(c):
+    t=TIERS['approximate'];r=c['value_rows']
+    return bool(r) and all(x['active'] and max(x['error_a'],x['error_b'])<=t['value_error'] and x['cosine']>=t['value_cosine'] for x in r)
+
+
+def load_profile(root,source,layer):
+    with np.load(root/'cache'/f'calibration-source-{source}.npz') as data:
+        prefix=f'layer{layer}_'
+        return {key[len(prefix):]:data[key] for key in data.files if key.startswith(prefix)}
+
+
+def map_profile_values(p):return {k:v for k,v in p.items()}
+
+
+def new_bank():return {}
+
+
+def get_candidate(bank,kind,i,j,a=None,b=None,suffix='',perm=None):
+    name=candidate_id(kind,i,j,a,b,suffix)
+    if name not in bank:
+        bank[name]={'id':name,'kind':kind,'i':i,'j':j,'a':a,'b':b,'perm':perm,
+                    'output_rows':[],'probability_rows':[],'value_rows':[]}
+    return bank[name]
+
+
+def add_output(c,a,b,samples,source,target=None,floor_a=1e-6,floor_b=1e-6):
+    c['output_rows'].extend(outrows(a,b,samples,{'candidate':c['id'],'source_input_layer':source,'target':target},floor_a,floor_b))
+
+
+def observe_pair(lm,bank,pi,pj,i,j,source,samples,diagnostics):
+    fi,fj=head_floor(pi),head_floor(pj)
+    whole=get_candidate(bank,'full_attention',i,j)
+    add_output(whole,pi['output'],pj['output'],samples,source)
+    hm=cross_head_output_metrics(pi['heads'][:,7:],pj['heads'][:,7:])
+    qm=cross_head_output_metrics(pi['q'][:,7:],pj['q'][:,7:])
+    repeats=lm.config['num_attention_heads']//lm.config['num_key_value_heads']
+    ki,kj=np.repeat(pi['k'],repeats,axis=2),np.repeat(pj['k'],repeats,axis=2)
+    km=cross_head_output_metrics(ki[:,7:],kj[:,7:])
+    for a in range(pi['p'].shape[1]):
+        for b in range(pj['p'].shape[1]):
+            qk=get_candidate(bank,'qk_head',i,j,a,b);qkv=get_candidate(bank,'qkv_head',i,j,a,b)
+            pm=distribution_metrics(pi['p'][:,a:a+1],pj['p'][:,b:b+1])
+            rr=rows_for(pm,samples,{'candidate':qk['id'],'source_input_layer':source})
+            qk['probability_rows'].extend(rr);qkv['probability_rows'].extend(rr)
+            for n,s in enumerate(samples):
+                rec={'candidate':qkv['id'],'source_input_layer':source,'sample':s['id'],**{key:float(v[n,a,b]) for key,v in hm.items()}}
+                rec['active']=bool(rec['rms_a']>=fi[n] and rec['rms_b']>=fj[n]);qkv['output_rows'].append(rec)
+                diagnostics.append({'i':i,'j':j,'head_i':a,'head_j':b,'source_input_layer':source,'sample':s['id'],
+                                    'q_error':float(max(qm['error_a'][n,a,b],qm['error_b'][n,a,b])),
+                                    'q_cosine':float(qm['cosine'][n,a,b]),'k_error':float(max(km['error_a'][n,a,b],km['error_b'][n,a,b])),
+                                    'k_cosine':float(km['cosine'][n,a,b]),'qkv_head_output_error':max(rec['error_a'],rec['error_b'])})
+    vm=cross_head_output_metrics(pi['v'][:,7:],pj['v'][:,7:])
+    vfi=np.maximum(1e-6,.1*np.median(np.sqrt(np.mean(pi['v'][:,7:].astype(np.float64)**2,axis=(1,3))),axis=1))
+    vfj=np.maximum(1e-6,.1*np.median(np.sqrt(np.mean(pj['v'][:,7:].astype(np.float64)**2,axis=(1,3))),axis=1))
+    for a in range(pi['v'].shape[2]):
+        for b in range(pj['v'].shape[2]):
+            c=get_candidate(bank,'v_group',i,j,a,b)
+            gram=value_gram_cka(pi['v'][:,7:,a],pj['v'][:,7:,b])
+            for n,s in enumerate(samples):
+                rec={'candidate':c['id'],'source_input_layer':source,'sample':s['id'],**{key:float(v[n,a,b]) for key,v in vm.items()},'centered_value_gram_cka':float(gram[n])}
+                rec['active']=bool(rec['rms_a']>=vfi[n] and rec['rms_b']>=vfj[n]);c['value_rows'].append(rec)
+
+
+def probe_candidate(lm,c,pi,pj,source,samples):
+    i,j=c['i'],c['j'];kind=c['kind'];fi,fj=head_floor(pi),head_floor(pj)
+    if kind=='qk_map':
+        perm=c['perm'];p=distribution_metrics(pi['p'],pj['p'][:,perm])
+        c['probability_rows'].extend(rows_for(p,samples,{'candidate':c['id'],'source_input_layer':source}))
+        changed_i=lm.transplant(pi,pj,i,kind,perm=perm)
+        changed_j=lm.transplant(pj,pi,j,kind,perm=inverse(perm))
+        add_output(c,pi['output'],changed_i,samples,source,i)
+        add_output(c,pj['output'],changed_j,samples,source,j)
+    elif kind=='qk_head':
+        a,b=c['a'],c['b']
+        changed_i=lm.head_transplant_output(pi,pj,i,a,b)
+        changed_j=lm.head_transplant_output(pj,pi,j,b,a)
+        add_output(c,pi['heads'][:,:,a],changed_i,samples,source,i,fi,fi)
+        add_output(c,pj['heads'][:,:,b],changed_j,samples,source,j,fj,fj)
+    elif kind=='v_group':
+        a,b=c['a'],c['b'];rep=lm.config['num_attention_heads']//lm.config['num_key_value_heads']
+        own_i=pi['heads'][:,:,a*rep:(a+1)*rep].sum(2);own_j=pj['heads'][:,:,b*rep:(b+1)*rep].sum(2)
+        changed_i=lm.value_group_output(pi,pj,i,a,b);changed_j=lm.value_group_output(pj,pi,j,b,a)
+        add_output(c,own_i,changed_i,samples,source,i,fi,fi);add_output(c,own_j,changed_j,samples,source,j,fj,fj)
+    else:raise ValueError(kind)
+
+
+def summarize(c):
+    out=c['output_rows'];p=c['probability_rows'];v=c['value_rows']
+    m={'id':c['id'],'kind':c['kind'],'i':c['i'],'j':c['j'],'a':c['a'],'b':c['b'],'perm':c['perm'],
+       'output_verified':bool(out),'output_observations':len(out),'probability_observations':len(p),'value_observations':len(v),
+       'strict_pass':candidate_pass(c,'strict'),'approximate_pass':candidate_pass(c,'approximate')}
+    m['worst_output_error']=max((max(r['error_a'],r['error_b']) for r in out),default=None)
+    m['minimum_output_cosine']=min((r['cosine'] for r in out),default=None)
+    m['worst_mean_js']=max((r['js'] for r in p),default=None)
+    m['minimum_top5']=min((r['top5'] for r in p),default=None)
+    m['worst_value_error']=max((max(r['error_a'],r['error_b']) for r in v),default=None)
+    return m
+
+
+def controls_for_profiles(profiles,source,samples,rows,h):
+    rng=np.random.default_rng(20261109+source)
+    ra=(h@rng.standard_normal((h.shape[-1],256),dtype=np.float32))/np.float32(np.sqrt(h.shape[-1]))
+    rb=(h@rng.standard_normal((h.shape[-1],256),dtype=np.float32))/np.float32(np.sqrt(h.shape[-1]))
+    random_cka=value_gram_cka(ra[:,7:],rb[:,7:])
+    for n,s in enumerate(samples):rows.append({'control':'value_gram_random_projection_pair','source_input_layer':source,'sample':s['id'],'gram_cka':float(random_cka[n])})
+    for layer,p in profiles.items():
+        for kv in range(p['v'].shape[2]):
+            for label,other in [('value_gram_to_input',h),('value_gram_to_random_projection',ra)]:
+                vcka=value_gram_cka(p['v'][:,7:,kv],other[:,7:])
+                for n,s in enumerate(samples):rows.append({'control':label,'source_input_layer':source,'projection_layer':layer,'kv_head':kv,'sample':s['id'],'gram_cka':float(vcka[n])})
+        probs=p['p'];uniform=np.zeros_like(probs);shuffled=probs.copy()
+        for t in range(probs.shape[-1]):
+            uniform[:,:,t,:t+1]=1/(t+1)
+            for bi in range(probs.shape[0]):
+                for head in range(probs.shape[1]):shuffled[bi,head,t,:t+1]=probs[bi,head,t,rng.permutation(t+1)]
+        for name,other in [('identity',probs),('uniform_causal',uniform),('shuffled_legal_keys',shuffled)]:
+            for head in range(probs.shape[1]):
+                m=distribution_metrics(probs[:,head:head+1],other[:,head:head+1])
+                rows.extend(rows_for(m,samples,{'control':name,'source_input_layer':source,'projection_layer':layer,'head':head}))
+
+
+def validate_selected(lm,selected,inputs,samples,indices):
+    i,j=selected['i'],selected['j'];c={k:v for k,v in selected.items() if k not in ('output_rows','probability_rows','value_rows')}
+    c.update(output_rows=[],probability_rows=[],value_rows=[])
+    for source in (i,j):
+        h=inputs[f'attn_{source}'][indices];pi,pj=lm.parts(h,i),lm.parts(h,j)
+        temporary={};diag=[];observe_pair(lm,temporary,pi,pj,i,j,source,samples,diag)
+        if c['kind'] in ('full_attention','qkv_head','qk_head','v_group'):
+            original=temporary[candidate_id(c['kind'],i,j,c['a'],c['b'])]
+            c['probability_rows'].extend(original['probability_rows']);c['value_rows'].extend(original['value_rows'])
+            if c['kind'] in ('full_attention','qkv_head'):c['output_rows'].extend(original['output_rows'])
+        if c['kind'] in ('qk_map','qk_head','v_group'):probe_candidate(lm,c,pi,pj,source,samples)
+    return c
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--source',required=True,type=Path);p.add_argument('--root',required=True,type=Path);p.add_argument('--seconds',type=int,default=7200)
+    args=p.parse_args();source=args.source.resolve();root=args.root.resolve();root.mkdir(parents=True,exist_ok=True);(root/'cache').mkdir(exist_ok=True)
+    assert not (root/'protocol.json').exists(),'Fresh experiment directory required; do not overwrite evidence'
+    protocol={**PROTOCOL,'registered_unix':time.time(),'source_run':str(source),'seconds':args.seconds,
+              'fresh_sample_sha256':hashlib.sha256(Path(__file__).with_name('fresh_samples.json').read_bytes()).hexdigest()}
+    write_json(root/'protocol.json',protocol)
+    write_json(root/'retention-policy.json',{'automatic_cleanup':False,'weights_and_caches':'retain_until_explicit_user_deletion'})
+    start=time.monotonic();status={'state':'running','phase':'calibration_profiles','started_unix':time.time()}
+    bank={};diagnostics=[];controls=[]
+    try:
+        manifest=json.loads((source/'manifest.json').read_text());assert manifest['revision']=='15852e8c16360a2fea060d615a32b45270f8a8fc'
+        reference=json.loads((source/'reference-check.json').read_text());assert reference['passed']
+        assert reference['implementation_sha256']==hashlib.sha256(Path('functional/model.py').read_bytes()).hexdigest()
+        config=json.loads((source/'model_config.json').read_text());lm=InterventionLM(source/'weights',config,seconds=args.seconds-180)
+        assert [i for i,t in enumerate(lm.config['layer_types']) if t=='full_attention']==LAYERS
+        samples=json.loads((source/'samples-tokenized.json').read_text())['samples'];cal=[s for s in samples if s['split']=='calibration'];valid=[s for s in samples if s['split']=='heldout']
+        ci=np.array([i for i,s in enumerate(samples) if s['split']=='calibration']);vi=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+        write_json(root/'source-samples.json',{'samples':samples,'note':'old validation samples used in earlier studies; fresh test separately fixed'})
+        inputs=np.load(source/'baseline-activations.npz')
+        write_json(root/'source-provenance.json',{'revision':manifest['revision'],'weights_manifest':str(source/'manifest.json'),
+                'baseline_activation_sha256':hashlib.sha256((source/'baseline-activations.npz').read_bytes()).hexdigest(),
+                'source_hashes':{str(f):hashlib.sha256(f.read_bytes()).hexdigest() for f in sorted(Path('crossqkv').glob('*')) if f.is_file()}})
+        for src in LAYERS:
+            lm.guard();h=inputs[f'attn_{src}'][ci];profiles={layer:lm.parts(h,layer) for layer in LAYERS}
+            np.savez(root/'cache'/f'calibration-source-{src}.npz',**{f'layer{layer}_{key}':value for layer,profile in profiles.items() for key,value in profile.items()})
+            controls_for_profiles(profiles,src,cal,controls,h)
+            for i,j in PAIRS:
+                if src in (i,j):observe_pair(lm,bank,profiles[i],profiles[j],i,j,src,cal,diagnostics)
+            del profiles
+            status.update(last_source_layer=src,elapsed_seconds=time.monotonic()-start);write_json(root/'status.json',status)
+            save_csv(root/'raw-qk-diagnostics.csv',diagnostics);save_csv(root/'controls.csv',controls)
+            print(json.dumps(status),flush=True)
+        status['phase']='discrete_head_matching_and_output_probes';write_json(root/'status.json',status)
+        mapping_rows=[]
+        qk_pre=[c for c in bank.values() if c['kind']=='qk_head' and preliminary_probability(c)]
+        qk_probe=sorted(qk_pre,key=lambda c:(max(r['js'] for r in c['probability_rows']),c['id']))[:32]
+        v_pre=[c for c in bank.values() if c['kind']=='v_group' and preliminary_value(c)]
+        v_probe=sorted(v_pre,key=lambda c:(max(max(r['error_a'],r['error_b']) for r in c['value_rows']),c['id']))[:16]
+        for i,j in PAIRS:
+            cost=np.empty((8,8),dtype=np.float64)
+            for a in range(8):
+                for b in range(8):cost[a,b]=np.mean([r['js'] for r in bank[candidate_id('qk_head',i,j,a,b)]['probability_rows']])
+            perm=assignment(cost)
+            for a in range(8):
+                for b in range(8):mapping_rows.append({'i':i,'j':j,'head_i':a,'head_j':b,'calibration_mean_js':float(cost[a,b]),'chosen':perm[a]==b})
+            maps=[get_candidate(bank,'qk_map',i,j,suffix='_identity',perm=list(range(8))),get_candidate(bank,'qk_map',i,j,suffix='_matched',perm=perm)]
+            for src in (i,j):
+                pi,pj=load_profile(root,src,i),load_profile(root,src,j)
+                for c in [*maps,*[c for c in qk_probe+v_probe if c['i']==i and c['j']==j]]:
+                    lm.guard();probe_candidate(lm,c,pi,pj,src,cal)
+            status.update(last_pair=[i,j],elapsed_seconds=time.monotonic()-start);write_json(root/'status.json',status)
+        save_csv(root/'head-assignment-costs.csv',mapping_rows)
+        summaries=[summarize(c) for c in bank.values()];write_json(root/'calibration-summary.json',summaries)
+        save_csv(root/'calibration-output-metrics.csv',[r for c in bank.values() for r in c['output_rows']])
+        save_csv(root/'calibration-probability-metrics.csv',[r for c in bank.values() if c['kind']!='qkv_head' for r in c['probability_rows']])
+        save_csv(root/'calibration-value-metrics.csv',[r for c in bank.values() for r in c['value_rows']])
+        eligible=[c for c in summaries if c['strict_pass'] or c['approximate_pass']]
+        eligible.sort(key=lambda c:(not c['strict_pass'],TYPE_ORDER[c['kind']],c['worst_output_error'],c['worst_mean_js'] or 0,c['id']))
+        selected=eligible[:1]
+        write_json(root/'selection.json',{'selected':selected,'qk_head_prequalified':len(qk_pre),'qk_head_output_verified':len(qk_probe),
+                    'v_group_prequalified':len(v_pre),'v_group_output_verified':len(v_probe),'selection_uses_validation':False})
+        status['selected']=selected
+        if selected:
+            c=selected[0];status['phase']='old_validation';write_json(root/'status.json',status)
+            checked=validate_selected(lm,c,inputs,valid,vi);tier='strict' if c['strict_pass'] else 'approximate'
+            write_json(root/'validation-summary.json',summarize(checked));save_csv(root/'validation-output-metrics.csv',checked['output_rows']);save_csv(root/'validation-probability-metrics.csv',checked['probability_rows']);save_csv(root/'validation-value-metrics.csv',checked['value_rows'])
+            passed=candidate_pass(checked,tier);status['validation_pass']=passed
+            if passed:
+                from .fresh import run_fresh
+                run_fresh(source,root,c,tier,lm,status)
+        status.update(state='completed',phase='done',elapsed_seconds=time.monotonic()-start,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
+                      candidate_counts={kind:sum(c['kind']==kind for c in summaries) for kind in TYPE_ORDER},
+                      passing_counts={kind:sum(c['kind']==kind and c['approximate_pass'] for c in summaries) for kind in TYPE_ORDER},
+                      original_11_and_norm_positive_not_used_as_qkv_evidence=True)
+    except Exception as exc:
+        status.update(state='stopped',error_type=type(exc).__name__,error=str(exc),elapsed_seconds=time.monotonic()-start,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        (root/'error.txt').write_text(traceback.format_exc());raise
+    finally:write_json(root/'status.json',status)
+
+
+if __name__=='__main__':main()

docs/CPU_FIRST_VALIDATION.md

@@ -28,7 +28,7 @@ NumPy 2.5.4，OPENBLAS_NUM_THREADS=1、OMP_NUM_THREADS=1，远程 timeout 60 秒
    使用受信任的算子实现及小数组正确性检查；模型仓库代码不直接执行。
    先确认首个短样本能以受控内存运行。不用不正确的“简化 Qwen”产生看似有价值的结果。
 2. **真实分布上的候选**：batch=1、32 tokens，校准和留出各 4 段真实文本。
-   按顺序加载、计算、释放每层，保留很小的入口激活；探测选定的 3 个 full-attention 层对和 2 个 MLP 对。
+   按顺序加载、计算、释放每层，保留很小的入口激活；探测预先固定的 3 个 full-attention 层对、2 个 MLP 对，以及 6 个 full-attention 层内的 Q/K 归一化模块对，共 11 对。
    每对在两个来源层的隐藏输入上分别测试，记录 JS、top-k、完整分支输出误差。
 3. **最小干预证据**：最多对 1 个候选做单处替换；冻结参数，在留出文本上看输出变化，
    条件允许时流式计算 token NLL。词表输出头分块归约 logsumexp，避免一次物化大 logits。
@@ -48,8 +48,28 @@ NumPy 2.5.4，OPENBLAS_NUM_THREADS=1、OMP_NUM_THREADS=1，远程 timeout 60 秒
 - 先实现最小正确路径；线性状态递推、长上下文和生成速度不塞入首轮。
 - 总计算 120 分钟，阶段预算见功能方案；无候选、超时或数值/实现核对失败均停止。
 - 容量探测已完成，真实模型阶段尚未执行。后续需要恢复本实验自有的固定 revision 权重，
-  分析后按既有规则校验归档并删除；不增加模型规模或自动租用 GPU。
+  权重持续保留，只有用户明确要求删除时才清理；不增加模型规模或自动租用 GPU。
 
 CPU 上“找到输出接近的候选”能支持继续投入 GPU；
 “在指定小样本中单处复用保持质量且节省明确资源”才是初步正例。
 两者都不能替代大规模能力验证。CPU 时延不外推为 GPU 加速率。
+
+## 当前权重保留规则
+
+用户于 2026-11-09 明确要求：本轮模型权重保留，只有用户明确要求删除时才删除。
+试验完成、超时、失败、归档或结果校验均不得自动触发清理。该要求覆盖先前逐模型分析后删除的约定。
+上一轮已删除的历史记录保留；本轮恢复下载的权重受新规则保护。
+
+## 本轮执行补充
+
+实际实验分支为 `codex/cpu-reuse-experiment`。在读取真实权重/功能指标前，新增固定的
+6 个层内 Q/K RMSNorm 增益共享候选，以检查较小粒度的功能复用；不是看过留出结果后扩充搜索。
+最多选择一个候选，必须分别验证两个替换方向；完整模块失败不会被小模块通过所掩盖。
+归一化共享最多节省少量字节，不预设能显著减少计算。
+采集真实隐藏输入必须正确执行前置线性注意力；它不进入首轮复用候选，但其递推计算纳入官方参考对照。
+
+## 用户最新目标修订
+
+不再以成本/加速作为成功条件；主问题是现有学习部分的功能重复。
+新增一次固定四层的 MLP 神经元函数搜索、独立留出和双向替换，
+见 [功能重复判据](LEARNED_FUNCTION_REPETITION.md)。解析增益合并单独归类，不冒充学习部分重复。

docs/CPU_FUNCTIONAL_FINDINGS.md

@@ -0,0 +1,86 @@
+# CPU 先导结论：存在已训练归一化模块的相似功能重复
+
+本轮主目标按用户最新要求为功能重复的存在性，不以成本/加速为成功条件。
+全部计算在 2 核、约 1.7 GiB RAM 的 CPU 服务器完成，无训练、缩放拟合或旋转搜索。
+层编号均为从 0 开始的索引。
+
+## 一个具体正例
+
+固定官方 Qwen3.5-2B checkpoint 的两个独立已训练模块：
+
+- `model.language_model.layers.12.input_layernorm`
+- `model.language_model.layers.13.input_layernorm`
+
+二者均为 2048→2048 的 RMSNorm，eps=1e-6，使用训练后的有效增益 gamma=1+weight。
+增益标准差分别约 0.1901、0.1685，最小值约 0.5547、0.5703；不是零函数或恒定增益。
+
+| 证据 | 结果 |
+|---|---|
+| 全输入域数学界（相对于层12输出） | 输出相对 L2 误差≤10.1852% |
+| 反方向全输入域数学界（相对于层13输出） | 输出相对 L2 误差≤11.3402% |
+| 4 条留出文本、两个来源层输入的最坏双向误差 | 4.5285% |
+| 相同留出比较中的最低余弦 | 0.999487 |
+| 层12使用层13，留出平均 token NLL 增量 | +0.0000325651 nat/token |
+| 层13使用层12，留出平均 token NLL 增量 | −0.0003639730 nat/token |
+
+两方向逐文本 NLL 增量 P95 分别约 0.00098464、0.00070972，均通过原定小样本质量门槛。
+所有数值来自保存的逐文本、逐 token 原始数据。留出为 4 条不同的实验原创短文本，
+每条截取 32 tokens；校准另有 4 条。它们不是广泛任务能力评测。
+
+## 为什么不只是“权重看起来接近”
+
+对相同 eps 和维数，f_a(x)=diag(a)x/sqrt(mean(x²)+eps)。任意非零实数输入满足：
+
+||f_a(x)-f_b(x)|| / ||f_a(x)|| ≤ max_j |a_j-b_j|/|a_j|。
+
+共同归一化分母消去；坐标基向量可达到最大值，因而该界是精确的最坏情况。
+反向分母换成 |b_j|。本轮先穷举 24 层同角色 input/post-attention norm 的 552 对，
+仅按最小双向最坏界选定一对，未使用任何输入样本作选择。≤20% 界有 18 对，≤10% 无配对；
+本轮只验证最优的一对，不根据留出结果换选。
+
+这个确定性界直接给出两个已训练函数整体接近的证据。真实输入上的误差更小，
+与只关心目标输入分布的目标一致。浮点实测与实数数学界分别报告。
+重新采集输入时，未修改的完整前向逐 bit 复现了最初保存的基线输出。
+输入探测分别使用两个来源层，对相同输入同时应用两套增益，避免比较不同输入的输出。
+从已加增益的激活还原共同归一化输入时有额外 FP32 舍入，未将其称为 bitwise 等价。
+
+结论限定为两个已有 RMSNorm 模块实现相似映射；不把共同包含 RMS 操作本身当作证据。
+二者学习增益不同且非恒定，输出非零。完整 attention/MLP 主体的重复仍未得到正例。
+
+## 已完成的阴性搜索
+
+1. 原定 3 对完整 attention、2 对完整 MLP、6 对层内 Q/K norm 均未通过预设门槛。
+   attention 检索分布 JS 范围约 0.145～0.318，也未通过 0.01 的门槛；
+   未进行头排列、连续坐标变换或训练对齐，不能外推为不存在其他可复用表示。
+2. 在层 5、6、20、21 各 6144 个 MLP scalar 子函数中，排除低活跃和近常量单元后，
+   共精确筛查 7,450,139 个唯一同层配对。要求双向相对误差≤20%、余弦≥0.98、
+   Pearson≥0.95，无通过者。没有针对留出集不断更换候选或放宽门槛。
+3. 这些结果只覆盖声明的模块、输入和阈值，不否定其他粒度/角色的功能重复。
+
+## 单独归类的解析恒等式
+
+Q/K 未参与 RoPE 的 192 个维度，其增益只以乘积影响点积，可合并到任一侧。
+96 组真实输入检查最大分数差 3.8147e-6、最大概率差 8.0466e-7；固定第3层两个方向
+留出 NLL 变化接近 FP32 舍入。详见 [推导](QK_GAIN_FACTORIZATION.md)。
+这是重参数化等价，不作为“两个已有学习部分功能重复”的正例。先前性能测试原始数据
+保留用于溯源，成本不属于当前结论的成功条件。
+
+## 正确性、资源与保留规则
+
+- 固定模型 revision：15852e8c16360a2fea060d615a32b45270f8a8fc，原始权重 SHA256 校验通过。
+- 流式读取原始 BF16，逐块转为 FP32，不同时载入全模型。
+- 与固定 Transformers 官方定义比较的 22 项检查通过，最大差约 5.4e-7。
+- 保留早期参考适配器错误、临时分区空间不足及两次过于保守的换页停止记录。
+  只归还自身临时堆页，未改变系统 swap 设置；后续有界前向完成。
+- 计算预算跨重启累计，初始保守计入 180 秒先前检查预留；具体时长及峰值见归档日志。
+- 当前权重按用户最新指令持续保留，只在用户明确要求删除时清理。
+
+原始表、样本/token IDs、激活、NLL、证明界与图表见 `results/cpu-qwen35-2b/`。
+后续 GPU 阶段可增加长度、语言和任务来验证正例的稳健性；本轮不自动启动新模型或扩展搜索。
+
+## Git 交付与保留缓存
+
+全部图表源 CSV/JSON、逐 token NLL、证明界和日志已交付。三份较大的中间激活缓存
+保留在原服务器及本地完整归档，路径、大小、SHA256 见结果目录的 external-activation-cache.json。
+其完整原实验校验表为 checksums-full-server.json；Git交付包校验表为 checksums-functional.json。
+没有删除模型或缓存；这些缓存可由固定代码和样本重新生成，也可从服务器按哈希取回。

docs/CROSS_LAYER_QKV_FINDINGS.md

@@ -0,0 +1,90 @@
+# 跨层 QKV：主检验阴性，发现条件性的自 token 检索重合
+
+全部在 CPU 上运行，冻结 Qwen3.5-2B 权重，无训练、连续 R 拟合或缩放拟合。
+层索引从0开始。成本不作为判据；归一化相似不作为本轮 QKV 正例。
+
+## 结论分级
+
+**主协议没有候选通过全部门槛。** 覆盖六个 full-attention 层的全部15个跨层配对，
+同时测试两个来源层的真实输入，不能将该结论扩大为模型不存在任何功能重复。
+
+**探索性补充有一条具体行为观察，但严格新测试仍未全部通过：**
+把层23的同一隐藏输入 H23 分别交给层3头7、层23头6的 QK 检索函数，
+它们主要都读取当前 token；在4条新文本上，query位置7～31的 argmax 完全相同。
+这不是同层 Q/K/V 互换，不是两个完整 attention 头等价，也没有证明 V 相似。
+
+| 新文本证据（逐文本平均或最坏值） | 结果 |
+|---|---:|
+| 层23头6分配给当前token的平均概率，四文本最小值 | 95.71% |
+| 层3头7在同一H23上分配给当前token的平均概率，最小值 | 92.07% |
+| 两者最高概率key一致率 | 100% |
+| 两套top5交集覆盖各自概率质量的最小值 | 97.97% |
+| 分布总变差的最大文本平均值 | 6.96% |
+| 保留层23 V/gate/O后的最坏双向头读出相对误差 | 3.82% |
+| 最坏文本平均JS | 0.018921 |
+
+新英文文本 fresh-en-2 超过补充协议的三条严格线：平均 JS 0.018921>0.01、
+行 JS P95 0.072943>0.05、行总变差 P95 0.238731>0.20。
+因此补充试验的 `fresh_function_pass=false`，**没有重标为完全通过，也未运行替换后的全LM NLL**。
+已计算新文本原模型基线，不能把该基线文件当作替换实验结果。
+可支持的观察是特定输入条件下的主要检索行为和实际读出接近；严格可互换尚未确认。
+
+## 主协议实际覆盖
+
+| 对象 | 初筛/直接比较数量 | 后续输出核验 |
+|---|---:|---|
+| 完整 attention 分支 | 15 | 全部测量 |
+| QKV+gate/O 的完整头函数 | 960 | 全部测量，输出在共同residual坐标中 |
+| 全8头P图映射（原序/冻结匹配） | 30 | 全部保留接收层V/gate/O做替换读出 |
+| 单头联合QK检索 | 960 | 无候选通过初筛，主协议未进入单头读出 |
+| 跨层KV组的直接V函数 | 60 | 无候选通过原始V初筛，未进入V组替换 |
+
+主协议最接近的若干项，指标来自不同候选，不能拼接成同一个正例：
+
+- QK两来源检索：层7头0↔层11头6，最坏文本平均JS约0.08609，最低top5约70.4%。
+- 完整头输出：同一对头的最坏方向相对误差约134.4%。检索接近不意味着V/O组成的输出接近。
+- 直接V函数：层7 KV头0↔层11 KV头1，最坏方向误差约150.3%。
+- 完整分支：层7↔层11的最坏方向误差约185.3%。
+- 保留目标V/O的完整P图映射：最小最坏输出误差约70.3%（层15↔19、冻结头匹配）。
+
+V token-Gram CKA另与输入Gram和两个独立高斯投影比较。共同输入几何可以造成高Gram相似，
+不能据此宣布V可替换；完整头输出检验已经包含各自的O/gate，避免仅因内部坐标差异就漏掉输出等价。
+
+## 为什么增加单独的条件性审计
+
+主协议要求两个来源层的输入分布均通过，并把无权重top5重合率作为硬门槛。
+复核全部已保存的单来源指标后，固定了唯一一对：层3头7与层23头6，条件为H23。
+它在校准集的JS很小，却因概率很低的尾部排名不同而top5重合低。
+
+无权重top-k在近似并列处不连续：可以构造总变差趋近0而top5集合仍不同的例子，
+本轮已加入独立回归测试。补充协议在观察该候选保留接收端V/O的读出、旧验证或新测试之前固定。
+它改为单一H23输入条件，将top5作为诊断，并固定概率质量/实际读出/活跃度约束；
+不换候选、不重新拟合参数。
+它使用4条原验证文本，再使用本轮事先冻结的4条新文本，保留全部未通过项。
+这是探索性补充，不改写主协议，不是事后声称原协议成功。
+
+原主检验的图与条件性图使用不同输入集合：主检验取两个来源分布的最坏情况；
+条件性图只研究H23。条件性相似不能推广到H3，也不能把早层在H3上已算好的P直接当作P_3(H23)。
+实际H3缓存与H23接收端的差异另有概率/读出诊断，未做失败后追加的全模型替换。
+
+## 图表阅读
+
+- `cross_layer_qk_divergence`：15对层的JS，左为各配对最佳单头，中为原头序，右为冻结双射；
+  颜色越浅分布越近，斜线对角不作经验数据。全部仅为校准结果。
+- `cross_layer_output_errors`：各类输出误差。QK单头和V组未通过初筛时，不画成零误差，而标未输出核验。
+- `value_geometry_and_direct_error`：V Gram几何与直接V误差，灰区为随机投影对照，仅诊断。
+- `conditional_probability_mass_and_readout`：同一条件候选在校准、旧验证和新测试上的top5、质量重合与实际读出差异。
+- `conditional_cross_layer_probability_maps`：一条新文本的具体P图；层3在H23上的结果与层3在实际H3上的结果分开。
+
+## 复现、预算与保留
+
+20项完整远程测试通过。主计算约74.8秒，补充条件试验约105.3秒；峰值进程RSS分别约168/174 MiB。
+另有出图、审计和测试，仍在两小时预算内。早期对照生成器的变量遮蔽错误及10.3秒失败记录保留，
+修复后在新目录按相同配对和门槛重跑；没有删除原缓存。
+
+模型revision 15852e8c16360a2fea060d615a32b45270f8a8fc，与此前官方参考校验的流式CPU实现一致。
+全部权重和大型profile缓存继续保留；Git只上传代码、全部图表源表、选定的新文本概率矩阵、
+日志、协议和缓存位置/SHA。没有新模型下载或自动清理。
+
+后续若继续，应重新预注册“单向输入条件”和概率质量/读出判据，使用新文本及更长上下文；
+当前结果不能证明跨上下文稳定，也不应把归一化正例或代数重参数化当作QKV主体验证。

docs/CROSS_LAYER_QKV_PROTOCOL.md

@@ -0,0 +1,59 @@
+# 跨层 QKV 功能重复：CPU 预注册方案
+
+目标：两个不同层中的学习部分是否实现相似检索、内容变换或完整头函数。不以成本为条件。
+冻结 Qwen3.5-2B 权重；不训练、不拟合连续 R 或缩放，不使用 GPU。权重与缓存持续保留。
+
+## 配对与数据
+
+仅研究完整注意力层索引 3/7/11/15/19/23（从0开始）的全部15个跨层配对。
+18个线性注意力层不套用 softmax 概率指标，不在本轮自动扩大搜索。
+复用已采集的同一隐藏输入，四条原校准文本选对，四条旧留出文本作验证；
+后者曾用于其他试验，不称完全未见的最终测试。另固定4条新文本作新测试，32 tokens，中文英文各半。
+仅对冻结候选通过验证时采集新测试输入，不根据测试结果重选。
+
+## 拆开回答的问题
+
+- QK：比较实际 Q/K norm、部分 RoPE 和 GQA 后产生的逐头检索概率，保留接收层 V/gate/O 时替换检索分布。
+- V：比较跨层两个 KV 头的 V(H)；保留接收层 P/gate/O，再替换一个 KV 组，覆盖该组4个Q头。
+  token Gram/CKA仅用于诊断坐标表示问题，不能代替实际输出替换；另与输入Gram、固定seed的两个高斯投影Gram比较，以控制共同输入结构造成的高相似。
+- 完整头：比较 P V、gate、O 共同形成的每个头在 residual 坐标中的输出函数。
+  它不要求内部 V 坐标一致，能够避免仅比较内部元素漏掉表示差异。
+- 完整分支：比较全部8个头组成的完整 attention 输出。
+
+因此原始 Q/K/V 输出相似、QK 检索相似、V 可替换、完整头可替换分别报告，归一化单独相似不作主正例。
+同一层 Q、K、V 互相比较不进入主实验。
+
+## 头匹配与位置对照
+
+15对层各在校准集选择一次最小平均JS的8头双射（精确离散DP，不求连续R）。
+该排列只用于给接收Q头注入 donor P；接收层V/gate/O保持顺序，不宣称任意GQA权重排列合法。
+全部960个跨层Q头配对、60个跨层KV头配对和960个完整头函数配对均作初筛。
+完整P映射考察原顺序及冻结排列，共30项；完整分支共15项。
+
+主指标只用 query 位置7～31，避免第一行概率必为1、早期合法key过少的平凡相似。
+记录均匀因果P、固定seed的合法key打乱、去首key后重归一化、首key/self/最近3key质量。
+首key或局部位置主导的真实重复单独标为位置型，不冒充内容语义重复。
+
+## 固定门槛与预算
+
+两档门槛在读取新跨层指标前固定，不依据结果放宽：
+
+| 档位 | 输出双向相对L2 | 输出余弦 | 每文本平均JS | JS行P95 | top5重合 |
+|---|---:|---:|---:|---:|---:|
+| strict | ≤0.05 | ≥0.99 | ≤0.01 | ≤0.05 | ≥0.8 |
+| approximate | ≤0.20 | ≥0.98 | ≤0.05 | ≤0.15 | ≥0.6 |
+
+V另需同档原始V直接误差/余弦；QK候选另需同档检索门槛。
+每个来源层、每条文本、两个替换方向分别检查，输出不含共同 residual 输入。
+头输出需达到该层头RMS中位数的10%，且绝对RMS≥1e-6，避免近零头制造假正例。
+
+QK单头最多精确核验32个初筛通过者，V组最多16个；完整P的30个输出均核验。
+只按校准选定1个最终候选：先strict，再approximate；同档优先完整分支、完整头、完整P、QK单头、V组。
+之后按最坏输出误差/JS/ID排序。不按验证或新测试结果换选。
+
+同输入函数验证通过后，做两个方向的真实网络干预，计算新测试token NLL（均值增量≤0.01、
+逐文本增量P95≤0.05）。另外只做一次时间顺序合法的“早层实际缓存→晚层”复用，单独报告；
+同输入函数可互换不代表来自不同实际输入的已计算激活可以直接复制。
+
+CPU总计算限两小时，保留阶段覆盖率和停止原因；没有候选即停止，不自动扩模型、层数或阈值。
+较大的中间profile缓存在服务器，Git交付全部图表源表、协议、NLL和缓存位置/SHA。

docs/LEARNED_FUNCTION_REPETITION.md

@@ -0,0 +1,58 @@
+# 当前目标：已有学习部分是否重复实现相似函数
+
+用户于 2026-11-09 明确：不以成本或加速作为成功条件，只关心相似功能重复是否存在。
+权重继续保留，只有用户明确要求删除才清理；不训练、不拟合旋转或缩放。
+
+## 什么可以作为正例
+
+两个不同的已训练部分，在同一真实输入上产生接近、且并非恒定/接近零的输出；
+独立留出输入仍满足预先固定的门槛。实际双向替换只作补充验证。
+不能仅因替换一小部分后整模型 NLL 几乎不变，就认定它们实现相同函数。
+
+Q/K 增益的解析合并证明同一计算的重参数化自由度，不等于两个现成学习部分功能重复。
+前一轮 11 对完整/归一化模块未通过门槛，也不能否定更小粒度的重复。
+不将两个模块都包含 SiLU、矩阵乘法等共同运算本身作为重复功能证据。
+
+## 新增一次有界搜索：MLP 学习神经元
+
+在已保存的第 5、6、20、21 层真实 MLP 入口输入上，研究
+
+f_(l,j)(h) = SiLU(w_gate_(l,j)^T h) × (w_up_(l,j)^T h)。
+
+每层 6,144 个神经元具有不同学习参数。比较同一层内两个 scalar 子函数，
+不是比较来自不同输入的两个观测值，也不是把整层或整 tensor 称作重复。
+下游 down-projection 的列保持各自原样；替换发生在两个投影和 SiLU 之后的激活值处。
+
+1. 4 条校准文本、4 条留出文本，每条 32 tokens，中文/英文各半，沿用先前冻结样本。
+2. 校准阶段只保留 RMS 至少为该层神经元中位数、std/RMS≥0.25、能量有效 token 数≥16
+   的非平凡单元。能量有效样本数定义为 (sum f²)² / sum f⁴，不是独立统计样本量。
+3. 对这些单元做分块穷举；实际配对数必须记录。不用随机 hidden 向量作主要证据。
+4. 预先固定门槛：双向直接相对 L2≤20%、余弦≥0.98、Pearson≥0.95；同时输出 5/10/20%
+   三档校准配对数量。没有拟合线性缩放、偏置或 R。
+5. 全部层中仅按校准的最小最坏方向误差选择 1 对，冻结选择；无通过者停止。
+6. 留出集要求同样的汇总函数门槛及活跃度条件，逐文本双向误差≤35%。不根据留出失败换候选。
+7. 只有函数验证通过，才做两个方向的独立网络替换；留出平均 NLL 增量≤0.01 nat/token，
+   逐文本增量 P95≤0.05。它不是主要的功能重复判据，防止因单元贡献很小而得到假正例。
+
+本阶段使用先前两小时累计预算的剩余额度，复用现有隐藏输入；不增加模型、样本规模或成本测试。
+神经元层面的经验正例不证明完整层可以互换、不证明全输入空间等价，也不能推断独立任务能力。
+留出只有 4 条文本，结论必须限定到此短文本先导分布；后续扩展文本才能检验稳健性。
+
+## 同角色学习 RMSNorm：确定性函数误差界
+
+静态试验已经发现部分语言层 post-attention norm 的参数接近。新增一次独立检验：
+比较 24 层各自的 input/post-attention RMSNorm（同角色内共 552 对），
+仅按训练后增益的全输入最坏误差界选 1 对，完全不依赖校准或留出输入。
+增益需非零且不是常量向量（std≥0.001），不把没有学习参数的共同归一化运算当作正例。
+
+相同 eps、相同形状下，f_a(x)=diag(a)x/sqrt(mean(x²)+eps)。对任意非零 x，
+
+||f_a(x)-f_b(x)|| / ||f_a(x)|| ≤ max_j |a_j-b_j|/|a_j|。
+
+反方向分母换为 |b_j|。该上界由相应坐标基向量达到，是实数算术下的精确最坏界。
+因此可先证明两个现成学习模块的函数整体接近，而不只比较原始参数 w；实际增益为 1+w。
+选择门槛：最坏双向界≤20%；同时记录≤5%、≤10%、≤20% 配对数量。只选最优界的一对。
+随后重新采集其实际输入，要求留出逐文本直接双向输出误差≤5%，再测两个方向网络替换。
+函数接近与替换后网络 NLL 分开报告；后者沿用已有 0.01/0.05 门槛，不决定函数存在性本身。
+选定后不根据留出结果换配对；不搜索视觉模块、不增加模型、不以成本为条件。
+归一化层正例不能推广为 attention 或 MLP 主体存在同样重复。

docs/QK_GAIN_FACTORIZATION.md

@@ -0,0 +1,46 @@
+# Q/K 未旋转维度的确定性增益合并（已数值验证，归为重参数化）
+
+这是从已核对架构直接推导的恒等式，不是看留出指标后扩大近邻搜索。
+原定 11 对候选保留原协议；本项是独立的解析对照，不训练、不求 R，也不选择层。
+预先固定完整网络干预层为第 3 层，分别验证两个移动方向；不根据数据换层。
+
+## 恒等式与前提
+
+以完成 RMS 单位化、尚未乘可学习增益的 q/k 为 u/v，Q/K 增益向量为 a/b。
+固定 checkpoint 的 norm 参数实际使用 a=1+w_Q，b=1+w_K。
+模型 head_dim=256，partial_rotary_factor=0.25：前 64 维旋转，后 192 维不旋转。
+逐头共享的增益向量与 GQA 结构已由官方代码确认。
+
+对于后 192 维，任意 query/key 位置 p/s：
+
+sum_j (a_j u_pj)(b_j v_sj) = sum_j u_pj ((a_j b_j) v_sj)
+                         = sum_j ((a_j b_j) u_pj) v_sj。
+
+前 64 维保留原增益和位置旋转。因而完整 QK 点积在实数算术中相等，softmax、V、
+输出门控和 O 不变时，完整 attention 分支也是同一个函数。这个结论不依赖某一输入分布。
+浮点乘法次序改变，不能承诺逐 bit 相等；需报告实际误差并验证留出 NLL。
+不能把这一规则用于旋转维度，或未经核对的逐头独立增益架构。
+
+## 验证与成本边界
+
+1. 在已有真实 q/k 输入上，检查 6 个 full-attention 层的原始/两个合并方向分数、概率及输出。
+2. 固定第 3 层进行两个方向的独立网络替换，不叠加，使用原留出文本和质量阈值。
+3. 记录预先缓存增益后、相同 CPU 环境下原版与合并版局部归一化耗时；不外推 GPU 性能。
+4. 双精度数学推导和 FP32 实测分开。没有消除 Q/K 投影或 RMS 方差计算。
+
+将 Q 的未旋转增益移入 K 时，每 token、每层少做 8×192=1,536 次逐元素增益乘法；
+反方向少做 2×192=384 次。它们只是归一化里的小部分操作，不是整个模型的百分比加速。
+NumPy 切片/拼接可能抵消这点收益，必须报告测得的局部时延。
+
+参数量从两个 256 维增益变为两个 64 维旋转增益及一个 192 维联合增益，即 512→320 个数。
+但原增益以 BF16 存储、联合乘积以 FP32 保留时，载荷仍是 1,024 字节，没有 checkpoint
+净压缩；元数据还可能增加。FP32 运行时缓存增益的理论载荷 2,048→1,280 字节，
+不等于已测得整个模型 RSS 降低。另行 BF16 量化乘积会增加误差，不属于本次恒等式验证。
+
+这是同一注意力计算内部的重参数化与冗余消除，不是两个现成 attention 层能互换的证明。
+若得到正例，应将它作为小粒度确定性优化机会；较大模块互换仍由原候选试验单独回答。
+
+## 完成状态
+
+96 组真实输入分数/概率检查及固定第3层双向网络检查均通过。
+该结果独立归类；主目标的已训练函数重复正例见 [CPU 功能结论](CPU_FUNCTIONAL_FINDINGS.md)。

docs/SOP.md

@@ -121,3 +121,20 @@ validation. See the [bounded plan](TRAINING_FREE_FUNCTIONAL_PLAN.md) and
 [primary-source literature](FUNCTIONAL_REUSE_LITERATURE.md). No forward pass, new download,
 or new model is started by adopting this document. The original static pilot remains
 inference-free; the proposed follow-up requires a separately confirmed execution environment.
+
+## Current functional-run retention override
+
+On 2026-11-09 the user explicitly changed retention: keep the CPU functional pilot
+checkpoint until the user explicitly requests deletion. Completion, timeout, failure
+and archive validation do not authorize cleanup. The earlier pilot cleanup remains
+a historical action; do not apply its automatic cleanup step to the current run.
+
+## CPU functional follow-up outcome
+
+The executed CPU study found a learned RMSNorm function-repetition example at text
+layer indices 12/13. Use same-input function error and a uniform analytic bound as
+primary evidence, with separate heldout network substitution checks. Costs are not
+a success condition under the latest user instruction. See the
+[executed findings](CPU_FUNCTIONAL_FINDINGS.md) and
+[function-repetition criterion](LEARNED_FUNCTION_REPETITION.md).
+Keep whole-module, neuron, normalization and algebraic-reparameterization results distinct.

functional/README.md

@@ -0,0 +1,91 @@
+# Bounded CPU functional pilot
+
+This is the execution implementation of the CPU-first plan, not the completed static
+parameter pilot. Frozen Qwen3.5-2B weights are decoded to FP32 a projection block at a
+time. CPU only, no autograd/optimization, no rotation fitting, no model-hub Python code.
+The user explicitly requires retaining this checkpoint until they request deletion.
+
+## Fixed scope
+
+- Batch of eight distinct experiment-authored plain texts, first 32 tokens each:
+  four calibration and four heldout, two Chinese/two English in each split.
+- Full attention pairs: (3,7), (11,15), (19,23).
+- MLP pairs: (5,6), (20,21).
+- Smaller-granularity Q/K RMSNorm pairs inside layers 3,7,11,15,19,23.
+  Added before observing real weights/functions for this pilot; these are small storage
+  opportunities, not a claim of substantial compute savings.
+- At most one calibration-selected candidate, two separate substitution directions.
+  Selection prefers component size, then worst symmetric output error. No heldout-driven retry.
+- Same-input relative L2 <=0.05 in both directions and token P95 <=0.1 in both directions;
+  attention additionally requires mean JS <=0.01 and top-5 overlap >=0.8.
+- Heldout mean NLL increase <=0.01 nat/token and per-sequence P95 increase <=0.05.
+  These are pilot decision criteria, not mathematical capability guarantees.
+
+## Source and checks
+
+`reference.json` pins the official Transformers Qwen3.5 source commit, URL and hash.
+`reference_check.py` extracts only named mathematical definitions from that exact file,
+removes optional fused-kernel decorators, and compares the NumPy implementation against
+Torch operators, full toy decoder layers and a two-layer text forward. Chunked delta and
+recurrent delta implementations are both checked, including sequence padding boundaries.
+The official source is Apache-2.0, copyright Qwen Team and HuggingFace; the reference file
+is fetched for testing and is not bundled here. FP32 equivalence is checked, not equality
+to every deployed BF16 kernel.
+
+`tests/test_functional.py` independently checks streamed projection/embedding/NLL,
+causality, causal convolution, normalization conventions and basic metric identities.
+These tests use tiny synthetic arrays. Real model evidence is produced only by `run.py`.
+
+## Remote execution
+
+Use the previously authorized CPU host and existing NumPy/Matplotlib environment. Install
+only binary wheels: tokenizers==0.23.3 from HTTPS PyPI; torch==2.15.0+cpu from the official
+CPU wheel index for independent reference checks. Use an experiment-owned on-disk TMPDIR:
+the shared host's /tmp tmpfs was too small during the first wheel download. Do not change
+system swap or shared temporary files.
+
+```sh
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_functional -v
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m functional.reference_check --source runs/cpu-qwen35-2b/reference.py --output runs/cpu-qwen35-2b/reference-check.json
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m functional.supervise --root runs/cpu-qwen35-2b
+```
+
+The supervisor waits at most one hour for binary dependencies and four hours for the
+already started pinned, verified checkpoint download. It does not install, retry downloads,
+search extra candidates, mutate Git or delete weights. After reference checks, it runs one
+8-token real text capacity check, then the bounded functional pilot. Reference, preflight
+and functional compute together are capped at two hours with a reporting reserve;
+download wait is recorded separately. Projection memory is checked, RSS soft stop is
+512 MiB, available-system-memory floor 256 MiB; sustained host swap IO combined with low available memory or >128 MiB process swap also stops the run.
+A persistent ledger counts computation across restarts; early checks/preflight receive a conservative 180-second reserve.
+
+`preflight.py` is a capacity smoke check, not a semantic oracle. `run.py` records exact
+sample/token IDs, fixed configuration, source hashes, activations, per-text/per-head metrics,
+selection, per-token NLL, intervention results and truthful stop coverage. `report.py` reads
+saved data to produce PNG/PDF/SVG figures without reopening weights.
+
+No generation throughput is measured in this short-text pilot. Any parameter-tying storage
+benefit must be accounted for separately, including metadata. Neither same-input similarity
+nor favorable tiny heldout NLL demonstrates broad mutual interchangeability or GPU speedup.
+
+## Current primary result and completed extensions
+
+The latest user goal is learned-function repetition, independent of costs. Existing
+input RMSNorm modules at layer indices 12/13 have a proven worst two-direction real-arithmetic
+error bound of 11.3402%, worst observed heldout relative error 4.5285%, minimum cosine 0.999487,
+and small bidirectional heldout NLL changes. See [findings](../docs/CPU_FUNCTIONAL_FINDINGS.md).
+
+- `neuron_reuse.py`: fixed four-layer scalar MLP function search, nontrivial activity
+  gates, calibration-only selection and independent heldout/substitution logic; no pair passed.
+- `norm_repetition.py`: same-role learned RMSNorm bound across 552 pairs, one deterministic
+  selection independent of any input samples, real probes and two full-network substitutions.
+- `gain_factorization.py`: architecture-derived identity on non-RoPE coordinates; a separate
+  reparameterization result, not a learned-function repetition example.
+- `benchmark_factorization.py`: historical one-attempt CPU allocation optimization and
+  bitwise equivalence check. Performance is no longer a success condition.
+- `storage.py`: explicit component layout accounting; the original candidate run selected
+  no pair. This does not affect the primary function-repetition conclusion.
+
+Do not expand the final positive claim to full attention or MLP bodies. Retain the
+fixed checkpoint until the user explicitly requests deletion. Full tests and audit are
+run remotely; no local build or model execution is required to inspect the archived results.

functional/__init__.py

@@ -0,0 +1 @@
+"""CPU-only, training-free functional reuse pilot."""

functional/audit.py

@@ -0,0 +1,86 @@
+"""Audit saved functional evidence and archive only outputs, never checkpoint weights."""
+import argparse
+import csv
+import hashlib
+import json
+from pathlib import Path
+import platform
+import subprocess
+import time
+import numpy as np
+from .model import StreamingLM
+from .norm_repetition import uniform_bounds
+from .run import write_json
+
+EXCLUDE={'tokenizer.json','reference.py','checksums-functional.json','audit-validation.json','archive-files.json'}
+
+
+def archive_files(root):
+    files=[p for p in root.iterdir() if p.is_file() and p.name not in EXCLUDE and not p.name.endswith(('.partial','.safetensors'))]
+    files += [p for p in (root/'figures').glob('*') if p.is_file()]
+    return sorted(files)
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    manifest=json.loads((root/'manifest.json').read_text())
+    assert manifest['revision']=='15852e8c16360a2fea060d615a32b45270f8a8fc'
+    for f in manifest['files']:
+        if f['name'].endswith('.safetensors'):
+            path=root/'weights'/f['name']
+            assert path.is_file() and path.stat().st_size==f['bytes']
+            assert f['sha256']==f['upstream_sha256']
+    retention=json.loads((root/'retention-policy.json').read_text());assert retention['automatic_cleanup'] is False
+    reference=json.loads((root/'reference-check.json').read_text())
+    assert reference['passed'] and len(reference['tests'])==22
+    assert reference['implementation_sha256']==hashlib.sha256(Path('functional/model.py').read_bytes()).hexdigest()
+    tests=(root/'final-tests.txt').read_text();assert 'Ran 12 tests' in tests and '\nOK\n' in tests
+    result=json.loads((root/'norm-repetition-summary.json').read_text());assert result['state']=='completed'
+    assert result['learned_function_repetition_positive'] and result['bidirectional_network_quality_pass']
+    cfg=json.loads((root/'model_config.json').read_text());lm=StreamingLM(root/'weights',cfg,seconds=60)
+    c=result['selected'];ga=1+lm.small(f'layers.{c["a"]}.{c["role"]}.weight');gb=1+lm.small(f'layers.{c["b"]}.{c["role"]}.weight')
+    bounds=uniform_bounds(ga,gb)
+    np.testing.assert_allclose(bounds,[c['a_to_b_uniform_bound'],c['b_to_a_uniform_bound']],atol=1e-14,rtol=1e-14)
+    with (root/'norm-repetition-uniform-bounds.csv').open() as f:assert len(list(csv.DictReader(f)))==552
+    baseline=np.load(root/'baseline-heldout-token-nll.npy');assert baseline.shape==(4,31)
+    for rec in result['interventions']:
+        nll=np.load(root/f'norm-repetition-{rec["target"]}-uses-{rec["donor"]}-token-nll.npy')
+        delta=nll.mean(1)-baseline.mean(1)
+        np.testing.assert_allclose(delta,rec['sequence_nll_deltas'],atol=1e-13,rtol=1e-13)
+        assert abs(delta.mean()-rec['mean_nll_delta'])<1e-13
+    with (root/'norm-repetition-function-metrics.csv').open() as f:rows=list(csv.DictReader(f))
+    held=[r for r in rows if r['split']=='heldout'];assert len(held)==8
+    worst=max(max(float(r['relative_l2']),float(r['reverse_relative_l2'])) for r in held)
+    assert abs(worst-result['worst_heldout_relative_l2'])<1e-14
+    neuron=json.loads((root/'neuron-reuse-search-summary.json').read_text());assert sum(x['eligible_unique_pairs_checked'] for x in neuron)==7450139
+    ledger=json.loads((root/'compute-budget-ledger.json').read_text());assert ledger['spent_seconds']+600<7200
+    environment={'python':platform.python_version(),'platform':platform.platform(),'numpy':np.__version__,
+                 'packages':subprocess.run([str(Path(__import__('sys').executable)),'-m','pip','freeze'],capture_output=True,text=True,check=True).stdout.splitlines(),
+                 'weights_retained':True,'no_automatic_cleanup':True,'ledger_seconds_including_initial_reserve':ledger['spent_seconds'],
+                 'final_tests_report_audit_additional_budget_upper_reserve_seconds':600,
+                 'historical_error_logs':'Early reference adapter errors and preflight pressure stops; latest summaries/tests are authoritative.'}
+    write_json(root/'functional-environment.json',environment)
+    source_paths=[*Path('functional').glob('*.py'),Path('functional/reference.json'),Path('pilot/tensors.py'),Path('tests/test_functional.py')]
+    write_json(root/'final-source-hashes.json',{str(p):hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(source_paths)})
+    files=archive_files(root)
+    # Every numeric CSV value must be finite. Empty diagnostic fields stay unknown.
+    for path in files:
+        if path.suffix=='.csv':
+            with path.open() as f:
+                for row in csv.DictReader(f):
+                    for value in row.values():
+                        if value in (None,''):continue
+                        try:number=float(value)
+                        except ValueError:continue
+                        assert np.isfinite(number),(path,value)
+    checksums={str(p.relative_to(root)):hashlib.sha256(p.read_bytes()).hexdigest() for p in files}
+    write_json(root/'checksums-functional.json',checksums)
+    selected=[*checksums,'checksums-functional.json','audit-validation.json','archive-files.json']
+    write_json(root/'archive-files.json',selected)
+    write_json(root/'audit-validation.json',{'passed':True,'validated_unix':time.time(),'checksummed_files':len(checksums),
+               'proof_recomputed_from_retained_weights':True,'heldout_nll_recomputed_from_saved_arrays':True,
+               'weights_present_and_retained':True,'archive_excludes_weights_and_tokenizer':True})
+    print(json.dumps({'passed':True,'checksummed_files':len(checksums),'weights_retained':True}))
+
+
+if __name__=='__main__':main()

functional/benchmark_factorization.py

@@ -0,0 +1,83 @@
+"""One allocation-reduction attempt, with a similarly optimized baseline.
+
+Does not change weights or run the network again. Checks bitwise equality to the
+network-tested factorization on every captured real Q/K input before timing.
+"""
+import argparse
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM
+from .gain_factorization import consolidate,normalized,LAYERS
+from .run import save_csv,write_json
+
+
+def baseline_inplace(q,k,a,b):
+    q=normalized(q);q*=a
+    k=normalized(k);k*=b
+    return q,k
+
+
+def factored_inplace(q,k,a,b,r,combined,move='q_to_k'):
+    q,k=normalized(q),normalized(k)
+    if move=='q_to_k':
+        q[...,:r]*=a[:r];k*=combined
+    else:
+        q*=combined;k[...,:r]*=b[:r]
+    return q,k
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);a=p.parse_args();root=a.root
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text());remaining=int(7200-ledger['spent_seconds']-20)
+    if remaining<60:raise TimeoutError('Cumulative budget insufficient')
+    start=time.monotonic();result={'state':'running'}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());lm=StreamingLM(root/'weights',cfg,seconds=min(remaining,300))
+        r=64;data=np.load(root/'baseline-activations.npz');checks=[]
+        for layer in LAYERS:
+            lm.guard();pfx=f'layers.{layer}.self_attn.'
+            ag,bg=1+lm.small(pfx+'q_norm.weight'),1+lm.small(pfx+'k_norm.weight')
+            q,k=data[f'qraw_{layer}'],data[f'kraw_{layer}']
+            for move in ('q_to_k','k_to_q'):
+                combined=np.concatenate([(bg if move=='q_to_k' else ag)[:r],ag[r:]*bg[r:]])
+                expected=consolidate(q,k,ag,bg,r,move)
+                actual=factored_inplace(q,k,ag,bg,r,combined,move)
+                for x,y in zip(expected,actual):np.testing.assert_array_equal(x,y)
+                checks.append({'layer':layer,'direction':move,'query_elements':int(q.size),'key_elements':int(k.size),'bitwise_equal':True})
+        q,k=data['qraw_3'][:1],data['kraw_3'][:1];pfx='layers.3.self_attn.'
+        ag,bg=1+lm.small(pfx+'q_norm.weight'),1+lm.small(pfx+'k_norm.weight')
+        combined=np.concatenate([bg[:r],ag[r:]*bg[r:]])
+        funcs={'baseline_legacy':lambda:(normalized(q)*ag,normalized(k)*bg),
+               'baseline_inplace':lambda:baseline_inplace(q,k,ag,bg),
+               'factored_inplace':lambda:factored_inplace(q,k,ag,bg,r,combined)}
+        for _ in range(3):
+            for fn in funcs.values():fn()
+        timing=[];names=list(funcs)
+        for repeat in range(30):
+            order=names[repeat%3:]+names[:repeat%3]
+            for name in order:
+                lm.guard();t=time.perf_counter()
+                for _ in range(50):funcs[name]()
+                timing.append({'repeat':repeat,'implementation':name,'seconds_per_call':(time.perf_counter()-t)/50,'calls_per_block':50})
+        save_csv(root/'gain-factorization-optimized-timing.csv',timing)
+        values={name:np.array([row['seconds_per_call'] for row in timing if row['implementation']==name]) for name in names}
+        ratio=values['baseline_inplace']/values['factored_inplace']
+        medians={name:float(np.median(v)) for name,v in values.items()}
+        result.update(state='completed',bitwise_checks=checks,medians_seconds=medians,
+                      paired_baseline_over_factored_median=float(np.median(ratio)),
+                      paired_ratio_p10=float(np.quantile(ratio,.1)),paired_ratio_p90=float(np.quantile(ratio,.9)),
+                      fraction_blocks_factored_faster=float(np.mean(ratio>1)),
+                      descriptive_local_gain=bool(np.median(ratio)>1.05 and np.mean(ratio>1)>=.8 and np.quantile(ratio,.1)>1),
+                      claim_limit='Hot-array CPU normalization microbenchmark against equally allocation-optimized baseline; no full-model speedup or independent statistical significance claim.')
+    except Exception as exc:
+        result.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-start;result.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.benchmark_factorization','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'gain-factorization-optimized-summary.json',result)
+
+
+if __name__=='__main__':main()

functional/gain_factorization.py

@@ -0,0 +1,135 @@
+"""Architecture-derived Q/K gain consolidation on non-RoPE dimensions only.
+
+Fixed layer3 interventions; no fitting, layer selection or learned alignment.
+Run only after baseline/candidate pilot finishes, within the same cumulative budget.
+"""
+import argparse
+import csv
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM,norm,rope
+from .run import probability_metrics,write_json,save_csv
+
+LAYERS=[3,7,11,15,19,23]
+
+
+def normalized(x):
+    return x/np.sqrt(np.mean(x*x,axis=-1,keepdims=True)+np.float32(1e-6))
+
+
+def consolidate(q,k,a,b,r,move,joint=None):
+    q,k=normalized(q),normalized(k)
+    c=a[r:]*b[r:] if joint is None else joint
+    if move=='q_to_k':
+        q=np.concatenate([q[...,:r]*a[:r],q[...,r:]],axis=-1)
+        k=np.concatenate([k[...,:r]*b[:r],k[...,r:]*c],axis=-1)
+    else:
+        q=np.concatenate([q[...,:r]*a[:r],q[...,r:]*c],axis=-1)
+        k=np.concatenate([k[...,:r]*b[:r],k[...,r:]],axis=-1)
+    return q,k
+
+
+class FactoredLM(StreamingLM):
+    def configure(self,layer,move):
+        self.factor_layer=layer;self.move=move
+        p=f'layers.{layer}.self_attn.'
+        self.a=1+self.small(p+'q_norm.weight');self.b=1+self.small(p+'k_norm.weight')
+        self.r=int(self.config['head_dim']*self.config['rope_parameters']['partial_rotary_factor'])
+        self.joint=self.a[self.r:]*self.b[self.r:]
+
+    def normalization(self,x,name):
+        p=f'layers.{self.factor_layer}.self_attn.'
+        if name not in (p+'q_norm',p+'k_norm'):return super().normalization(x,name)
+        # This path does not multiply the unit-gain tail, unlike a full-vector mask.
+        unit=x/np.sqrt(np.mean(x*x,axis=-1,keepdims=True)+self.eps)
+        isq=name==p+'q_norm';head=self.a if isq else self.b
+        skip=(isq and self.move=='q_to_k') or (not isq and self.move=='k_to_q')
+        tail=unit[...,self.r:] if skip else unit[...,self.r:]*self.joint
+        return np.concatenate([unit[...,:self.r]*head[:self.r],tail],axis=-1)
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);a=p.parse_args();root=a.root
+    # Protocol is fixed before inspecting functional measurements.
+    protocol={'fixed_intervention_layer':3,'directions':['q_to_k','k_to_q'],'diagnostic_layers':LAYERS,'no_selection':True,'no_training':True,'precision':'FP32 product; no BF16 requantization','quality_gate':{'mean_nll_delta':.01,'sequence_p95_nll_delta':.05}}
+    protocol_path=root/'gain-factorization-protocol.json'
+    if protocol_path.exists():
+        old=json.loads(protocol_path.read_text());assert all(old[k]==v for k,v in protocol.items()), 'Protocol differs from preregistration'
+    else:write_json(protocol_path,{**protocol,'registered_unix':time.time()})
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text())
+    remaining=int(7200-ledger['spent_seconds']-30)
+    if remaining<60:raise TimeoutError('No remaining cumulative budget for deterministic identity verification')
+    start=time.monotonic()
+    report={'state':'running','records':[],'interventions':[]}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());c=cfg['text_config'];r=int(c['head_dim']*c['rope_parameters']['partial_rotary_factor'])
+        assert c['rms_norm_eps']==1e-6
+        assert c['head_dim']==256 and r==64 and c['num_attention_heads']==8 and c['num_key_value_heads']==2
+        lm=FactoredLM(root/'weights',cfg,seconds=remaining)
+        data=np.load(root/'baseline-activations.npz');samples=json.loads((root/'samples-tokenized.json').read_text())['samples']
+        rp=c['rope_parameters'];repeats=c['num_attention_heads']//c['num_key_value_heads']
+        records=[]
+        for layer in LAYERS:
+            lm.guard()
+            pfx=f'layers.{layer}.self_attn.'
+            aw=lm.small(pfx+'q_norm.weight');bw=lm.small(pfx+'k_norm.weight');ag,bg=1+aw,1+bw
+            q,k=data[f'qraw_{layer}'],data[f'kraw_{layer}']
+            qr,kr=rope(norm(q,aw),rp['rope_theta'],rp['partial_rotary_factor']),rope(norm(k,bw),rp['rope_theta'],rp['partial_rotary_factor'])
+            ref=np.einsum('bthd,bshd->bhts',qr,np.repeat(kr,repeats,axis=2))/np.float32(np.sqrt(c['head_dim']))
+            mask=np.triu(np.ones(ref.shape[-2:],dtype=bool),1)
+            def probs(s):
+                s=np.where(mask,-np.inf,s);e=np.exp(s-s.max(-1,keepdims=True));return e/e.sum(-1,keepdims=True)
+            pref=probs(ref)
+            for move in ('q_to_k','k_to_q'):
+                qq,kk=consolidate(q,k,ag,bg,r,move)
+                qq,kk=rope(qq,rp['rope_theta'],rp['partial_rotary_factor']),rope(kk,rp['rope_theta'],rp['partial_rotary_factor'])
+                changed=np.einsum('bthd,bshd->bhts',qq,np.repeat(kk,repeats,axis=2))/np.float32(np.sqrt(c['head_dim']))
+                pp=probs(changed)
+                for bi,s in enumerate(samples):
+                    diff=changed[bi].astype(np.float64)-ref[bi]
+                    records.append({'layer':layer,'direction':move,'sample_id':s['id'],'split':s['split'],
+                                    'score_max_abs_error':float(np.abs(diff).max()),'score_relative_l2':float(np.linalg.norm(diff)/max(np.linalg.norm(ref[bi].astype(np.float64)),1e-12)),
+                                    'probability_max_abs_error':float(np.abs(pp[bi]-pref[bi]).max()),**probability_metrics(pref[bi:bi+1],pp[bi:bi+1])})
+            save_csv(root/'gain-factorization-score-metrics.csv',records)
+        held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout']);ids=np.array([s['token_ids'] for s in samples],dtype=np.int64)
+        base=np.load(root/'baseline-heldout-token-nll.npy').mean(1)
+        for move in ('q_to_k','k_to_q'):
+            lm.configure(3,move)
+            h,_=lm.forward(ids[held])
+            nll=lm.nll(h,ids[held]);np.save(root/f'gain-factorization-{move}-token-nll.npy',nll)
+            delta=nll.mean(1)-base
+            report['interventions'].append({'layer':3,'direction':move,'sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_delta':float(np.quantile(delta,.95)),'passed':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)})
+            write_json(root/'gain-factorization-summary.json',report)
+        # Local normalization timing uses precomputed gains for both implementations.
+        q,k=data['qraw_3'][:1],data['kraw_3'][:1];pfx='layers.3.self_attn.'
+        ag,bg=1+lm.small(pfx+'q_norm.weight'),1+lm.small(pfx+'k_norm.weight')
+        def baseline():return normalized(q)*ag,normalized(k)*bg
+        joint=ag[r:]*bg[r:]
+        def factored():return consolidate(q,k,ag,bg,r,'q_to_k',joint=joint)
+        for _ in range(3):baseline();factored()
+        timing=[]
+        for repeat in range(10):
+            for label,fn in (('baseline',baseline),('factored',factored)) if repeat%2==0 else (('factored',factored),('baseline',baseline)):
+                lm.guard();t=time.perf_counter()
+                for _ in range(20):fn()
+                timing.append({'repeat':repeat,'implementation':label,'seconds_per_call':(time.perf_counter()-t)/20})
+        save_csv(root/'gain-factorization-kernel-timing.csv',timing)
+        report.update(state='completed',maximum_score_abs_error=max(x['score_max_abs_error'] for x in records),maximum_probability_abs_error=max(x['probability_max_abs_error'] for x in records),
+                      bidirectional_heldout_pass=all(x['passed'] for x in report['interventions']),
+                      elementwise_gain_multiplies_saved_per_token_layer={'q_to_k':1536,'k_to_q':384},
+                      original_bf16_payload_bytes=1024,factored_lossless_mixed_payload_bytes=1024,
+                      fp32_cached_gain_payload_before=2048,fp32_cached_gain_payload_after=1280,
+                      claim_limit='Exact real-arithmetic identity only on non-RoPE coordinates; empirical FP32/NLL checks on tiny text pilot. Not evidence that two existing full layers interchange, not model-wide speedup.')
+    except Exception as exc:
+        report.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-start
+        report.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.gain_factorization','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'gain-factorization-summary.json',report)
+
+
+if __name__=='__main__':main()

functional/model.py

@@ -0,0 +1,247 @@
+"""Streaming FP32 text-only Qwen3.5 reference, original BF16 weights.
+
+Mathematical semantics checked against Transformers Qwen3.5, commit
+fd3cf0140de03f7619ac95b5cfed05b711ff9e22 (Apache-2.0, Qwen Team/HuggingFace).
+No cached decode, padding, images, generation, fused kernels or autograd.
+Full-sequence gated delta recurrence replaces the algebraically equivalent
+chunk algorithm; both are independently compared in remote tests.
+"""
+import ctypes
+import json
+import time
+from pathlib import Path
+import numpy as np
+from pilot.tensors import load_tensors, decode
+
+PREFIX = 'model.language_model.'
+
+
+def sigmoid(x):
+    return np.exp(-np.logaddexp(np.float32(0), -x))
+
+
+def silu(x):
+    return x * sigmoid(x)
+
+
+def norm(x, weight, eps=1e-6, centered=True):
+    return (x / np.sqrt(np.mean(x*x, axis=-1, keepdims=True) + eps)
+            * (1 + weight if centered else weight)).astype(np.float32)
+
+
+def rope(x, theta=10000000., fraction=.25):
+    # Text-only: all three MRoPE position axes coincide.
+    dim = int(x.shape[-1] * fraction)
+    inv = np.float32(1) / np.float32(theta) ** (np.arange(0, dim, 2, dtype=np.float32) / dim)
+    freq = np.arange(x.shape[1], dtype=np.float32)[:, None] * inv[None, :]
+    angle = np.concatenate([freq, freq], axis=-1)[None, :, None, :]
+    a = x[..., :dim]
+    rotated = np.concatenate([-a[..., dim//2:], a[..., :dim//2]], axis=-1)
+    return np.concatenate([a*np.cos(angle) + rotated*np.sin(angle), x[..., dim:]], axis=-1)
+
+
+def delta_rule(q, k, v, g, beta):
+    q = q / np.sqrt(np.sum(q*q, axis=-1, keepdims=True) + 1e-6)
+    k = k / np.sqrt(np.sum(k*k, axis=-1, keepdims=True) + 1e-6)
+    q = q / np.float32(np.sqrt(q.shape[-1]))
+    state = np.zeros((q.shape[0], q.shape[2], q.shape[3], v.shape[3]), dtype=np.float32)
+    out = np.empty_like(v)
+    for t in range(q.shape[1]):
+        state *= np.exp(g[:, t, :, None, None])
+        predicted = np.sum(state * k[:, t, :, :, None], axis=-2)
+        delta = (v[:, t] - predicted) * beta[:, t, :, None]
+        state += k[:, t, :, :, None] * delta[:, :, None, :]
+        out[:, t] = np.sum(state * q[:, t, :, :, None], axis=-2)
+    return out, state
+
+
+def causal_conv(x, kernel):
+    out = np.zeros_like(x)
+    width = kernel.shape[-1]
+    for j in range(width):
+        lag = width - 1 - j
+        if lag < x.shape[1]:
+            out[:, lag:] += x[:, :x.shape[1]-lag] * kernel[:, j]
+    return silu(out)
+
+
+def attention(q, k, v):
+    repeats = q.shape[2] // k.shape[2]
+    k, v = np.repeat(k, repeats, axis=2), np.repeat(v, repeats, axis=2)
+    scores = np.einsum('bthd,bshd->bhts', q, k) / np.float32(np.sqrt(q.shape[-1]))
+    mask = np.triu(np.ones(scores.shape[-2:], dtype=bool), 1)
+    scores = np.where(mask, -np.inf, scores)
+    exp = np.exp(scores - scores.max(axis=-1, keepdims=True))
+    p = exp / exp.sum(axis=-1, keepdims=True)
+    return np.einsum('bhts,bshd->bthd', p, v), p
+
+
+class StreamingLM:
+    def __init__(self, weights, config, seconds=7200, enforce_memory=True):
+        self.tensors = load_tensors(Path(weights))
+        self.config = config.get('text_config', config)
+        self.deadline = time.monotonic() + seconds
+        self.enforce_memory = enforce_memory
+        self.eps = self.config['rms_norm_eps']
+        self.norm_replacement = None
+        self.capture_qk = False
+        self.qk_inputs = {}
+        self.swap_observation = None
+        self.swap_pressure_windows = 0
+        self.malloc_trim = getattr(ctypes.CDLL(None), 'malloc_trim', None)
+
+    def guard(self):
+        if time.monotonic() > self.deadline:
+            raise TimeoutError('CPU computation deadline reached')
+        if not self.enforce_memory:
+            return
+        status = dict(line.split(':', 1) for line in Path('/proc/self/status').read_text().splitlines() if ':' in line)
+        rss_kib = int(status['VmRSS'].split()[0])
+        available = int(next(line.split()[1] for line in Path('/proc/meminfo').read_text().splitlines() if line.startswith('MemAvailable:')))
+        now = time.monotonic()
+        if self.swap_observation is None or now-self.swap_observation[0] >= 5:
+            counters = dict(line.split() for line in Path('/proc/vmstat').read_text().splitlines())
+            count = int(counters['pswpin'])+int(counters['pswpout'])
+            if self.swap_observation is not None:
+                elapsed = now-self.swap_observation[0]
+                rate = (count-self.swap_observation[1])*4096/elapsed
+                self.swap_pressure_windows = self.swap_pressure_windows+1 if rate > 4*1024**2 else 0
+                own_swap_kib = int(status.get('VmSwap', '0 kB').split()[0])
+                if self.swap_pressure_windows >= 3 and (available < 384*1024 or own_swap_kib > 128*1024):
+                    raise MemoryError(f'Sustained host swap pressure plus constrained memory: available={available} KiB, process_swap={own_swap_kib} KiB')
+            self.swap_observation = (now,count)
+        if rss_kib > 512*1024 or available < 256*1024:
+            raise MemoryError(f'Soft memory stop: RSS {rss_kib} KiB, available {available} KiB')
+
+    def small(self, name):
+        t = self.tensors[PREFIX + name]
+        if t.numel > 1048576:
+            raise ValueError('Use blockwise linear for large tensors')
+        x = t.read().reshape(t.shape)
+        t.evict()
+        return x
+
+    def linear(self, x, name, rows=256):
+        t = self.tensors[PREFIX + name + '.weight']
+        outdim, indim = t.shape
+        flat = x.reshape(-1, indim)
+        result = np.empty((flat.shape[0], outdim), dtype=np.float32)
+        for start in range(0, outdim, rows):
+            self.guard()
+            end = min(start + rows, outdim)
+            w = t.read(slice(start*indim, end*indim)).reshape(end-start, indim)
+            result[:, start:end] = flat @ w.T
+            del w
+            t.evict()
+        if self.malloc_trim is not None:
+            self.malloc_trim(0)  # Release only this process's freed temporary heap pages.
+        return result.reshape(*x.shape[:-1], outdim)
+
+    def embedding(self, ids):
+        t = self.tensors[PREFIX + 'embed_tokens.weight']
+        raw = t.raw.reshape(t.shape)
+        result = decode(raw[ids], t.dtype)
+        t.evict()
+        return result
+
+    def normalization(self, x, name):
+        if getattr(self, 'norm_replacement', None) and name == self.norm_replacement[0]:
+            name = self.norm_replacement[1]
+        return norm(x, self.small(name + '.weight'), self.eps)
+
+    def mlp(self, h, layer):
+        p = f'layers.{layer}.mlp.'
+        gate = silu(self.linear(h, p+'gate_proj'))
+        up = self.linear(h, p+'up_proj')
+        return self.linear(gate*up, p+'down_proj')
+
+    def full_attention(self, h, layer, shared_p=None):
+        p = f'layers.{layer}.self_attn.'
+        c = self.config
+        heads, kvheads, dim = c['num_attention_heads'], c['num_key_value_heads'], c['head_dim']
+        # Q and output gate are interleaved per head, not globally split.
+        mixed = self.linear(h, p+'q_proj').reshape(*h.shape[:-1], heads, 2*dim)
+        q, gate = mixed[..., :dim], mixed[..., dim:]
+        k = self.linear(h, p+'k_proj').reshape(*h.shape[:-1], kvheads, dim)
+        v = self.linear(h, p+'v_proj').reshape(*h.shape[:-1], kvheads, dim)
+        if getattr(self, 'capture_qk', False):
+            self.qk_inputs[layer] = (q.copy(), k.copy())
+        q = self.normalization(q, p+'q_norm')
+        k = self.normalization(k, p+'k_norm')
+        rp = c['rope_parameters']
+        q, k = (rope(x, rp['rope_theta'], rp['partial_rotary_factor']) for x in (q,k))
+        out, probs = attention(q,k,v)
+        if shared_p is not None:
+            out = np.einsum('bhts,bshd->bthd', shared_p, np.repeat(v, heads//kvheads, axis=2))
+        out = (out*sigmoid(gate)).reshape(*h.shape[:-1], -1)
+        return self.linear(out, p+'o_proj'), probs
+
+    def linear_attention(self, h, layer):
+        p = f'layers.{layer}.linear_attn.'
+        c = self.config
+        kh, vh = c['linear_num_key_heads'], c['linear_num_value_heads']
+        kd, vd = c['linear_key_head_dim'], c['linear_value_head_dim']
+        qkv = self.linear(h, p+'in_proj_qkv')
+        qkv = causal_conv(qkv, self.small(p+'conv1d.weight')[:,0,:])
+        q, k, v = np.split(qkv, [kh*kd, 2*kh*kd], axis=-1)
+        q = q.reshape(*h.shape[:-1], kh, kd)
+        k = k.reshape(*h.shape[:-1], kh, kd)
+        v = v.reshape(*h.shape[:-1], vh, vd)
+        if vh != kh:
+            q, k = np.repeat(q, vh//kh, axis=2), np.repeat(k, vh//kh, axis=2)
+        z = self.linear(h, p+'in_proj_z').reshape(*h.shape[:-1], vh, vd)
+        beta = sigmoid(self.linear(h, p+'in_proj_b'))
+        a = self.linear(h, p+'in_proj_a')
+        g = -np.exp(self.small(p+'A_log')) * np.logaddexp(np.float32(0), a+self.small(p+'dt_bias'))
+        out, _ = delta_rule(q,k,v,g,beta)
+        out = norm(out, self.small(p+'norm.weight'), self.eps, centered=False)*silu(z)
+        return self.linear(out.reshape(*h.shape[:-1], -1), p+'out_proj')
+
+    def forward(self, ids, capture=None, replacement=None, progress=None):
+        h = self.embedding(ids)
+        saved = {}
+        capture = set(capture or [])
+        for layer, kind in enumerate(self.config['layer_types']):
+            self.guard()
+            raw = h
+            n = self.normalization(h, f'layers.{layer}.input_layernorm')
+            if ('attn', layer) in capture:
+                saved[f'attn_{layer}'] = n.copy()
+            donor = replacement[2] if replacement and replacement[:2] == ('attn',layer) else layer
+            if kind == 'full_attention':
+                out, _ = self.full_attention(n,donor)
+            else:
+                out = self.linear_attention(n,donor)
+            h = raw + out
+            n = self.normalization(h, f'layers.{layer}.post_attention_layernorm')
+            if ('mlp',layer) in capture:
+                saved[f'mlp_{layer}'] = n.copy()
+            donor = replacement[2] if replacement and replacement[:2] == ('mlp',layer) else layer
+            h = h + self.mlp(n,donor)
+            if not np.isfinite(h).all():
+                raise FloatingPointError(f'Nonfinite hidden states at layer {layer}')
+            if progress:
+                progress(layer)
+        return self.normalization(h,'norm'), saved
+
+    def nll(self, h, ids, rows=512):
+        # Tied output embedding; online logsumexp avoids full vocabulary logits.
+        x = h[:,:-1].reshape(-1,h.shape[-1])
+        targets = ids[:,1:].reshape(-1)
+        t = self.tensors[PREFIX+'embed_tokens.weight']
+        mx = np.full(x.shape[0],-np.inf,dtype=np.float64)
+        total = np.zeros(x.shape[0],dtype=np.float64)
+        correct = np.empty(x.shape[0],dtype=np.float64)
+        for start in range(0,t.shape[0],rows):
+            self.guard()
+            end=min(start+rows,t.shape[0])
+            w=t.read(slice(start*t.shape[1],end*t.shape[1])).reshape(end-start,t.shape[1])
+            scores=x@w.T
+            m=np.maximum(mx,scores.max(axis=-1))
+            total=total*np.exp(mx-m)+np.exp(scores-m[:,None]).sum(axis=-1)
+            mx=m
+            which=(targets>=start)&(targets<end)
+            correct[which]=scores[np.flatnonzero(which),targets[which]-start]
+            del w,scores
+            t.evict()
+        return (mx+np.log(total)-correct).reshape(ids.shape[0],-1)

functional/neuron_reuse.py

@@ -0,0 +1,134 @@
+"""Find repeated learned scalar MLP functions on real hidden inputs, no training.
+
+Four fixed layers; calibration-only selection; one frozen pair; independent heldout
+functional test and two actual network substitutions. No scale fitting or R.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM,silu
+from .run import write_json,save_csv
+
+LAYERS=[5,6,20,21]
+
+
+def pair_metrics(x,y):
+    x,y=x.astype(np.float64),y.astype(np.float64)
+    xx,yy=float(x@x),float(y@y);delta=float((x-y)@(x-y))
+    xc,yc=x-x.mean(),y-y.mean()
+    return {'relative_l2_a_to_b':float(np.sqrt(delta/max(xx,1e-30))),
+            'relative_l2_b_to_a':float(np.sqrt(delta/max(yy,1e-30))),
+            'cosine':float(x@y/max(np.sqrt(xx*yy),1e-30)),
+            'pearson':float(xc@yc/max(np.linalg.norm(xc)*np.linalg.norm(yc),1e-30)),
+            'rms_a':float(np.sqrt(xx/x.size)),'rms_b':float(np.sqrt(yy/y.size)),
+            'std_over_rms_a':float(np.std(x)/max(np.sqrt(xx/x.size),1e-30)),
+            'std_over_rms_b':float(np.std(y)/max(np.sqrt(yy/y.size),1e-30)),
+            'effective_energy_tokens_a':float(xx*xx/max(float((x**4).sum()),1e-30)),
+            'effective_energy_tokens_b':float(yy*yy/max(float((y**4).sum()),1e-30))}
+
+
+class NeuronReuseLM(StreamingLM):
+    def configure(self,layer,target,donor):self.reused_layer,self.target,self.donor=layer,target,donor
+    def mlp(self,h,layer):
+        p=f'layers.{layer}.mlp.'
+        z=silu(self.linear(h,p+'gate_proj'))*self.linear(h,p+'up_proj')
+        if layer==self.reused_layer:z[...,self.target]=z[...,self.donor]
+        return self.linear(z,p+'down_proj')
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    protocol={'layers':LAYERS,'unit':'post_silu_gate_times_up_scalar_function; keep each output column',
+              'thresholds':[.05,.10,.20],'maximum_directional_relative_l2':.20,'cosine_min':.98,'pearson_min':.95,
+              'rms_floor':'at least within-layer median calibration neuron RMS','std_over_rms_min':.25,
+              'effective_energy_tokens_min':16,'max_selected_pairs':1,'selection':'smallest worst directional calibration error; then layer/id',
+              'heldout_per_text_error_max':.35,'heldout_quality_mean_nll_delta_max':.01,'heldout_quality_sequence_p95_delta_max':.05,
+              'training':False,'scale_fitting':False,'cost_required':False}
+    protocol_path=root/'neuron-reuse-protocol.json'
+    if protocol_path.exists():
+        old=json.loads(protocol_path.read_text());assert all(old[k]==v for k,v in protocol.items())
+    else:write_json(protocol_path,{**protocol,'registered_unix':time.time()})
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text());remaining=int(7200-ledger['spent_seconds']-30)
+    if remaining<120:raise TimeoutError('Remaining cumulative budget insufficient')
+    started=time.monotonic();result={'state':'running','selected':None,'function_pass':False,'interventions':[]}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());lm=NeuronReuseLM(root/'weights',cfg,seconds=remaining)
+        data=np.load(root/'baseline-activations.npz');samples=json.loads((root/'samples-tokenized.json').read_text())['samples']
+        cal=np.array([i for i,s in enumerate(samples) if s['split']=='calibration']);held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+        activations={};summaries=[];best=None
+        for layer in LAYERS:
+            h=data[f'mlp_{layer}'];pfx=f'layers.{layer}.mlp.'
+            z=silu(lm.linear(h,pfx+'gate_proj'))*lm.linear(h,pfx+'up_proj');activations[layer]=z
+            # Profiles use all calibration tokens; never use heldout in selection.
+            x=np.ascontiguousarray(z[cal].reshape(-1,z.shape[-1]).T,dtype=np.float64)
+            n=x.shape[1];normsq=(x*x).sum(1);means=x.mean(1);variance=np.maximum(normsq-n*means**2,0)
+            rms=np.sqrt(normsq/n);floor=float(np.median(rms));effective=normsq**2/np.maximum((x**4).sum(1),1e-30)
+            active=(rms>=floor)&(np.sqrt(variance/np.maximum(normsq,1e-30))>=.25)&(effective>=16)
+            counts={str(t):0 for t in protocol['thresholds']};tested=0;layerbest=None
+            for start in range(0,x.shape[0],256):
+                lm.guard();end=min(start+256,x.shape[0]);dot=x[start:end]@x.T
+                delta=np.maximum(normsq[start:end,None]+normsq[None,:]-2*dot,0)
+                error=np.sqrt(delta/np.maximum(np.minimum(normsq[start:end,None],normsq[None,:]),1e-30))
+                cosine=dot/np.maximum(np.sqrt(normsq[start:end,None]*normsq[None,:]),1e-30)
+                pearson=(dot-n*means[start:end,None]*means[None,:])/np.maximum(np.sqrt(variance[start:end,None]*variance[None,:]),1e-30)
+                unique=np.arange(start,end)[:,None]<np.arange(x.shape[0])[None,:]
+                allowed=unique&active[start:end,None]&active[None,:]&(cosine>=.98)&(pearson>=.95)
+                tested+=int((unique&active[start:end,None]&active[None,:]).sum())
+                for threshold in protocol['thresholds']:counts[str(threshold)]+=int((allowed&(error<=threshold)).sum())
+                mask=allowed&(error<=.20)
+                if mask.any():
+                    score=np.where(mask,error,np.inf);row,col=np.unravel_index(np.argmin(score),score.shape)
+                    candidate={'layer':layer,'a':int(start+row),'b':int(col),'calibration_worst_relative_l2':float(score[row,col])}
+                    key=lambda c:(c['calibration_worst_relative_l2'],c['layer'],c['a'],c['b'])
+                    if layerbest is None or key(candidate)<key(layerbest):layerbest=candidate
+                    if best is None or key(candidate)<key(best):best=candidate
+            summaries.append({'layer':layer,'neurons':int(x.shape[0]),'eligible_neurons':int(active.sum()),'eligible_unique_pairs_checked':tested,'rms_floor':floor,'calibration_pair_counts':counts,'best':layerbest})
+            write_json(root/'neuron-reuse-search-summary.json',summaries)
+            print(json.dumps(summaries[-1]),flush=True)
+        np.savez(root/'neuron-reuse-activations.npz',**{f'layer{k}':v for k,v in activations.items()})
+        write_json(root/'neuron-reuse-selection.json',{'selected':best,'selection_uses_heldout':False})
+        result['selected']=best
+        if best is not None:
+            layer,i,j=best['layer'],best['a'],best['b'];z=activations[layer]
+            metrics=[]
+            for split,index in [('calibration',cal),('heldout',held)]:
+                metrics.append({'split':split,'scope':'pooled',**pair_metrics(z[index,:,i].reshape(-1),z[index,:,j].reshape(-1))})
+                for idx in index:metrics.append({'split':split,'scope':samples[idx]['id'],**pair_metrics(z[idx,:,i],z[idx,:,j])})
+            save_csv(root/'neuron-reuse-function-metrics.csv',metrics)
+            pooled=next(r for r in metrics if r['split']=='heldout' and r['scope']=='pooled')
+            floor=next(r['rms_floor'] for r in summaries if r['layer']==layer)
+            function_pass=(max(pooled['relative_l2_a_to_b'],pooled['relative_l2_b_to_a'])<=.20 and pooled['cosine']>=.98 and pooled['pearson']>=.95 and
+                           min(pooled['std_over_rms_a'],pooled['std_over_rms_b'])>=.25 and min(pooled['effective_energy_tokens_a'],pooled['effective_energy_tokens_b'])>=16 and
+                           min(pooled['rms_a'],pooled['rms_b'])>=floor and
+                           all(max(r['relative_l2_a_to_b'],r['relative_l2_b_to_a'])<=.35 for r in metrics if r['split']=='heldout' and r['scope']!='pooled'))
+            result.update(function_pass=bool(function_pass),heldout_function_metrics=pooled)
+            # Show parameter similarity separately from actual functional similarity.
+            wd=[]
+            for role in ('gate_proj','up_proj'):
+                t=lm.tensors[f'model.language_model.layers.{layer}.mlp.{role}.weight'];d=t.shape[1]
+                wa,wb=t.read(slice(i*d,(i+1)*d)),t.read(slice(j*d,(j+1)*d))
+                wd.append({'role':role,'weight_cosine':float(wa.astype(np.float64)@wb/max(np.linalg.norm(wa.astype(np.float64))*np.linalg.norm(wb.astype(np.float64)),1e-30))});t.evict()
+            result['weight_diagnostics']=wd
+            if function_pass:
+                ids=np.array([s['token_ids'] for s in samples],dtype=np.int64);base=np.load(root/'baseline-heldout-token-nll.npy').mean(1)
+                for target,donor in ((i,j),(j,i)):
+                    lm.configure(layer,target,donor);h,_=lm.forward(ids[held]);nll=lm.nll(h,ids[held])
+                    np.save(root/f'neuron-reuse-{target}-uses-{donor}-token-nll.npy',nll)
+                    delta=nll.mean(1)-base
+                    result['interventions'].append({'target':target,'donor':donor,'sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_delta':float(np.quantile(delta,.95)),'passed':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)})
+                    write_json(root/'neuron-reuse-summary.json',result)
+            # No fallback after looking at heldout.
+        result.update(state='completed',empirical_repeated_function_positive=bool(result['function_pass'] and len(result['interventions'])==2 and all(r['passed'] for r in result['interventions'])),
+                      claim_limit='Distinct learned scalar MLP subfunctions on the specified short-text distribution; not whole-layer duplication, global equivalence or broad task-capability proof. No performance requirement.')
+    except Exception as exc:result.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-started;result.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.neuron_reuse','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'neuron-reuse-summary.json',result)
+
+
+if __name__=='__main__':main()

functional/norm_repetition.py

@@ -0,0 +1,108 @@
+"""Existing learned RMSNorm modules: uniform function bound + independent probes.
+
+Deterministic exhaustive selection by a mathematical all-input error bound, not
+calibration/heldout fitting. Same role, shape and eps; one pair only.
+"""
+import argparse
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM,norm
+from .run import write_json,save_csv,error_rows
+
+ROLES=['input_layernorm','post_attention_layernorm']
+
+
+def uniform_bounds(a,b):
+    if np.any(a==0) or np.any(b==0):return None
+    diff=np.abs(a.astype(np.float64)-b.astype(np.float64))
+    return float(np.max(diff/np.abs(a))),float(np.max(diff/np.abs(b)))
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);args=p.parse_args();root=args.root
+    protocol={'roles':ROLES,'layers':list(range(24)),'same_role_only':True,'same_eps':1e-6,
+              'uniform_worst_error_limit':.20,'empirical_function_worst_relative_error_limit':.05,
+              'selection':'smallest maximum two-direction uniform bound, then role/layer; no input data used',
+              'minimum_gain_standard_deviation':.001,'max_selected_pairs':1,'heldout_nll_mean_delta_limit':.01,
+              'heldout_nll_sequence_p95_delta_limit':.05,'training':False,'cost_required':False}
+    path=root/'norm-repetition-protocol.json'
+    if path.exists():
+        existing=json.loads(path.read_text());assert all(existing[k]==v for k,v in protocol.items())
+    else:write_json(path,{**protocol,'registered_unix':time.time()})
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text());remaining=int(7200-ledger['spent_seconds']-30)
+    if remaining<120:raise TimeoutError('Insufficient cumulative computation budget')
+    started=time.monotonic();result={'state':'running','selected':None,'interventions':[]}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());lm=StreamingLM(root/'weights',cfg,seconds=remaining)
+        assert lm.eps==1e-6
+        candidates=[];gains={}
+        for role in ROLES:
+            for layer in range(24):gains[(role,layer)]=1+lm.small(f'layers.{layer}.{role}.weight')
+            for i in range(24):
+                for j in range(i+1,24):
+                    a,b=gains[(role,i)],gains[(role,j)]
+                    bounds=uniform_bounds(a,b)
+                    if bounds is None:continue
+                    eligible=min(float(np.std(a)),float(np.std(b)))>=.001
+                    candidates.append({'role':role,'a':i,'b':j,'a_to_b_uniform_bound':bounds[0],'b_to_a_uniform_bound':bounds[1],
+                                       'worst_uniform_bound':max(bounds),'eligible_nonconstant_gains':eligible,
+                                       'gain_std_a':float(np.std(a)),'gain_std_b':float(np.std(b)),
+                                       'gain_min_a':float(np.min(a)),'gain_min_b':float(np.min(b))})
+        save_csv(root/'norm-repetition-uniform-bounds.csv',candidates)
+        feasible=[c for c in candidates if c['eligible_nonconstant_gains'] and c['worst_uniform_bound']<=.20]
+        counts={str(t):sum(c['eligible_nonconstant_gains'] and c['worst_uniform_bound']<=t for c in candidates) for t in (.05,.10,.20)}
+        selected=sorted(feasible,key=lambda c:(c['worst_uniform_bound'],c['role'],c['a'],c['b']))[:1]
+        write_json(root/'norm-repetition-selection.json',{'selected':selected,'uniform_bound_counts':counts,'selection_reads_no_input_data':True})
+        result['uniform_bound_counts']=counts
+        if selected:
+            c=selected[0];result['selected']=c
+            # Selection is frozen before reading tokenized inputs or their activations.
+            samples=json.loads((root/'samples-tokenized.json').read_text())['samples'];ids=np.array([s['token_ids'] for s in samples],dtype=np.int64)
+            held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+            kind='attn' if c['role']=='input_layernorm' else 'mlp'
+            captures=[(kind,c['a']),(kind,c['b'])]
+            h,saved=lm.forward(ids,capture=captures)
+            # The untouched recomputation must reproduce the saved real baseline.
+            original=np.load(root/'baseline-activations.npz')['final']
+            np.testing.assert_array_equal(h,original)
+            np.savez(root/'norm-repetition-input-profiles.npz',**saved)
+            ga,gb=gains[(c['role'],c['a'])],gains[(c['role'],c['b'])]
+            rows=[]
+            for source in (c['a'],c['b']):
+                n=saved[f'{kind}_{source}'];g=gains[(c['role'],source)]
+                # n = u * gamma_source. Reconstruct common normalized input u;
+                # source gains are nonzero. Additional rounding is ~FP32 only.
+                unit=n/g
+                fa,fb=unit*ga,unit*gb
+                rec=error_rows(fa,fb,samples,{'source_layer':source,'role':c['role'],'a':c['a'],'b':c['b']})
+                rows.extend(rec)
+            save_csv(root/'norm-repetition-function-metrics.csv',rows)
+            heldrows=[r for r in rows if r['split']=='heldout']
+            function_pass=all(max(r['relative_l2'],r['reverse_relative_l2'])<=.05 for r in heldrows)
+            result.update(uniform_function_similarity_proven=True,heldout_function_pass=bool(function_pass),
+                          worst_heldout_relative_l2=max(max(r['relative_l2'],r['reverse_relative_l2']) for r in heldrows),
+                          worst_heldout_cosine=min(r['cosine'] for r in heldrows),untouched_full_forward_bitwise_matches_baseline=True)
+            if function_pass:
+                baseline=np.load(root/'baseline-heldout-token-nll.npy').mean(1)
+                for target,donor in ((c['a'],c['b']),(c['b'],c['a'])):
+                    lm.norm_replacement=(f'layers.{target}.{c["role"]}',f'layers.{donor}.{c["role"]}')
+                    changed,_=lm.forward(ids[held]);nll=lm.nll(changed,ids[held]);np.save(root/f'norm-repetition-{target}-uses-{donor}-token-nll.npy',nll)
+                    delta=nll.mean(1)-baseline
+                    result['interventions'].append({'target':target,'donor':donor,'sequence_nll_deltas':delta.tolist(),
+                         'mean_nll_delta':float(delta.mean()),'sequence_p95_nll_delta':float(np.quantile(delta,.95)),
+                         'passed_quality':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)})
+                    write_json(root/'norm-repetition-summary.json',result)
+            result['learned_function_repetition_positive']=bool(function_pass)
+            result['bidirectional_network_quality_pass']=bool(len(result['interventions'])==2 and all(x['passed_quality'] for x in result['interventions']))
+        result.update(state='completed',claim_limit='Existing learned RMSNorm module functions with identical eps and shape; uniform real-arithmetic relative bound and tiny real-text probes. Does not establish attention/MLP-body duplication or broad task capability.')
+    except Exception as exc:result.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-started;result.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.norm_repetition','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'norm-repetition-summary.json',result)
+
+
+if __name__=='__main__':main()

functional/package_delivery.py

@@ -0,0 +1,46 @@
+"""Package code-independent scientific outputs, retain large activation caches externally.
+
+The experiment directory remains immutable and complete. The delivery directory
+contains all figure source tables/NLL arrays plus hashes/locations for three large
+reproducible activation caches. Never deletes model weights or cache files.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import shutil
+from .run import write_json
+
+CACHES={'baseline-activations.npz','neuron-reuse-activations.npz','norm-repetition-input-profiles.npz'}
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);p.add_argument('--output',required=True,type=Path);a=p.parse_args()
+    root,out=a.root.resolve(),a.output.resolve();assert root!=out
+    out.mkdir(parents=True,exist_ok=True)
+    full=json.loads((root/'checksums-functional.json').read_text())
+    files=json.loads((root/'archive-files.json').read_text())
+    for name in files:
+        src=root/name
+        if name in CACHES:continue
+        dest=out/name;dest.parent.mkdir(parents=True,exist_ok=True);shutil.copy2(src,dest)
+    external=[]
+    for name in sorted(CACHES):
+        src=root/name;digest=hashlib.sha256(src.read_bytes()).hexdigest();assert digest==full[name]
+        external.append({'name':name,'server_path':str(src),'bytes':src.stat().st_size,'sha256':digest,'retained':True,'reproducible_by':'functional.run / functional.neuron_reuse / functional.norm_repetition'})
+    write_json(out/'external-activation-cache.json',{'host':'8.211.154.134','files':external,'not_model_weights':True,
+               'reason':'Large intermediate caches retained on the authorized server; all figure source tables and token NLL arrays are delivered in Git.'})
+    shutil.copy2(root/'checksums-functional.json',out/'checksums-full-server.json')
+    (out/'DELIVERY.md').write_text('# Scientific delivery and retained caches\n\nAll figure CSV/JSON source data, per-token NLL, protocols, reports, metadata and historical logs are included.\nThree large reproducible intermediate activation caches remain on 8.211.154.134 and in the original full local archive;\n`external-activation-cache.json` records exact locations, sizes and SHA256. No weight or cache was deleted.\nThe original full experiment manifest is `checksums-full-server.json`; the delivery-only manifest is\n`checksums-functional.json`. No model checkpoint or tokenizer payload is in this Git delivery.\n')
+    validation=json.loads((out/'audit-validation.json').read_text());validation.update(delivery_excludes_external_activation_caches=True,full_experiment_checksums='checksums-full-server.json',figure_source_tables_and_token_nll_complete=True)
+    write_json(out/'audit-validation.json',validation)
+    checks={}
+    for file in sorted(out.rglob('*')):
+        if file.is_file() and file.name not in ('checksums-functional.json','audit-validation.json','archive-files.json'):
+            checks[str(file.relative_to(out))]=hashlib.sha256(file.read_bytes()).hexdigest()
+    write_json(out/'checksums-functional.json',checks)
+    write_json(out/'archive-files.json',[*checks,'checksums-functional.json','audit-validation.json','archive-files.json'])
+    print(json.dumps({'delivery_files':len(checks)+3,'checksummed_files':len(checks),'external_caches':len(external),'weights_and_caches_retained':True}))
+
+
+if __name__=='__main__':main()

functional/preflight.py

@@ -0,0 +1,39 @@
+"""One 8-token real text, bounded memory/semantic capacity smoke check."""
+import argparse
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM
+from .run import tokenize,write_json
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);a=p.parse_args()
+    samples,ids=tokenize(a.root)
+    start=time.time()
+    lm=StreamingLM(a.root/'weights',json.loads((a.root/'model_config.json').read_text()),seconds=1200)
+    snapshots=[]
+    result={'passed':False,'sample_id':samples[0]['id'],'tokens':8}
+    def progress(layer):
+        status=dict(line.split(':',1) for line in Path('/proc/self/status').read_text().splitlines() if ':' in line)
+        snapshots.append({'layer':layer,'seconds':time.time()-start,'rss_kib':int(status['VmRSS'].split()[0]),
+                          'process_swap_kib':int(status.get('VmSwap','0 kB').split()[0]),
+                          'available_kib':int(next(line.split()[1] for line in Path('/proc/meminfo').read_text().splitlines() if line.startswith('MemAvailable:')))})
+        print(json.dumps(snapshots[-1]),flush=True)
+    try:
+        h,_=lm.forward(ids[:1,:8],progress=progress)
+        nll=lm.nll(h,ids[:1,:8])
+        if not np.isfinite(nll).all():raise FloatingPointError('Nonfinite NLL')
+        result.update(passed=True,mean_nll=float(nll.mean()),token_nll=nll.tolist())
+    except Exception as exc:
+        result.update(error_type=type(exc).__name__,error=str(exc))
+        raise
+    finally:
+        result.update(elapsed_seconds=time.time()-start,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
+                      resource_snapshots=snapshots,limitation='Real-weight capacity check, not an independent correctness oracle or quality benchmark')
+        write_json(a.root/'preflight.json',result)
+
+
+if __name__=='__main__':main()

functional/reference.json

@@ -0,0 +1,5 @@
+{
+  "commit": "fd3cf0140de03f7619ac95b5cfed05b711ff9e22",
+  "url": "https://raw.githubusercontent.com/huggingface/transformers/fd3cf0140de03f7619ac95b5cfed05b711ff9e22/src/transformers/models/qwen3_5/modeling_qwen3_5.py",
+  "sha256": "4463b8a108f678c166755101d95f188179f54143305953d78147de0389b80ebc"
+}

functional/reference_check.py

@@ -0,0 +1,142 @@
+"""Remote-only numerical comparison with pinned official Torch definitions.
+
+Reads only a known official Transformers source file; extracts named mathematical
+operators/classes via AST, removing optional kernel decorators. Never imports the
+complete model or executes checkpoint repository code. No compilation/build.
+"""
+import argparse
+import ast
+import hashlib
+import json
+from pathlib import Path
+from types import SimpleNamespace
+import numpy as np
+from .model import StreamingLM, delta_rule, rope, norm
+
+EXPECTED_COMMIT = 'fd3cf0140de03f7619ac95b5cfed05b711ff9e22'
+NAMES = {'apply_mask_to_padding_states', 'l2norm', 'torch_chunk_gated_delta_rule', 'torch_recurrent_gated_delta_rule',
+         'causal_conv1d_fn', 'Qwen3_5RMSNormGated', 'Qwen3_5GatedDeltaNet',
+         'rotate_half', 'apply_rotary_pos_emb', 'repeat_kv', 'eager_attention_forward',
+         'Qwen3_5Attention', 'Qwen3_5MLP', 'Qwen3_5RMSNorm', 'Qwen3_5DecoderLayer',
+         'Qwen3_5TextRotaryEmbedding'}
+
+
+def official_namespace(source):
+    import torch
+    import torch.nn.functional as F
+    tree = ast.parse(source)
+    nodes = []
+    for node in tree.body:
+        if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in NAMES:
+            for child in ast.walk(node):
+                if isinstance(child, (ast.FunctionDef, ast.ClassDef)):
+                    child.decorator_list = [d for d in child.decorator_list if isinstance(d, ast.Name) and d.id in ('staticmethod','classmethod')]
+            nodes.append(node)
+    assert {x.name for x in nodes} == NAMES
+    tree = ast.Module(body=[ast.ImportFrom(module='__future__',names=[ast.alias(name='annotations')],level=0),*nodes],type_ignores=[])
+    ns = {'torch':torch, 'nn':torch.nn, 'F':F, 'ACT2FN':{'silu':F.silu},
+          'GradientCheckpointingLayer':torch.nn.Module,
+          'is_torchdynamo_exporting':lambda:False,
+          'ALL_ATTENTION_FUNCTIONS':SimpleNamespace(get_interface=lambda name, fallback:fallback)}
+    exec(compile(ast.fix_missing_locations(tree),'pinned_transformers_reference','exec'),ns)
+    return ns
+
+
+class ArrayLM(StreamingLM):
+    def __init__(self, arrays, config):
+        self.arrays, self.config, self.eps = arrays,config,config['rms_norm_eps']
+    def guard(self):
+        pass
+    def small(self,name):
+        return self.arrays[name]
+    def linear(self,x,name,rows=256):
+        return x@self.arrays[name+'.weight'].T
+    def embedding(self,ids):
+        return self.arrays['embed_tokens.weight'][ids]
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--source',type=Path,required=True);p.add_argument('--output',type=Path,required=True)
+    a=p.parse_args()
+    import torch
+    torch.set_num_threads(1)
+    torch.manual_seed(42)
+    provenance=json.loads(Path(__file__).with_name('reference.json').read_text())
+    assert hashlib.sha256(a.source.read_bytes()).hexdigest()==provenance['sha256'], 'Reference source hash mismatch'
+    ns=official_namespace(a.source.read_text())
+    cfg={'hidden_size':16,'num_attention_heads':2,'num_key_value_heads':1,'head_dim':8,
+         'intermediate_size':24,'hidden_act':'silu','attention_bias':False,'attention_dropout':0.,
+         'linear_num_value_heads':2,'linear_num_key_heads':2,'linear_key_head_dim':4,
+         'linear_value_head_dim':4,'linear_conv_kernel_dim':4,'rms_norm_eps':1e-6,
+         'layer_types':['linear_attention','full_attention'],'max_position_embeddings':4096,
+         'rope_parameters':{'rope_type':'default','rope_theta':10000000.,'partial_rotary_factor':.5,'mrope_section':[1,0,0]},
+         '_attn_implementation':'eager'}
+    config=SimpleNamespace(**cfg)
+    layers=[ns['Qwen3_5DecoderLayer'](config,i).eval() for i in range(2)]
+    for layer in layers:
+        for name,t in layer.named_parameters():
+            with torch.no_grad():
+                t.uniform_(-.2,.2)
+                if name=='linear_attn.norm.weight':t.add_(1.)
+    arrays={f'layers.{i}.'+k:v.detach().numpy().copy() for i,l in enumerate(layers) for k,v in l.state_dict().items()}
+    lm=ArrayLM(arrays,cfg)
+    rng=np.random.default_rng(123)
+    x=rng.standard_normal((2,7,16),dtype=np.float32)
+    tx=torch.from_numpy(x)
+    positions=torch.arange(7)[None,:]
+    rotary=ns['Qwen3_5TextRotaryEmbedding'](config)
+    cos,sin=rotary(tx,positions)
+    mask=torch.triu(torch.full((7,7),-float('inf')),diagonal=1)[None,None]
+    evidence=[]
+    def check(name,actual,expected,atol=2e-5,rtol=2e-5):
+        actual=np.asarray(actual);expected=expected.detach().numpy() if hasattr(expected,'detach') else np.asarray(expected)
+        np.testing.assert_allclose(actual,expected,atol=atol,rtol=rtol)
+        evidence.append({'test':name,'shape':list(actual.shape),'max_abs_error':float(np.max(np.abs(actual-expected))),'atol':atol,'rtol':rtol,'passed':True})
+    with torch.no_grad():
+        for i,layer in enumerate(layers):
+            n=lm.normalization(x,f'layers.{i}.input_layernorm')
+            tn=layer.input_layernorm(torch.from_numpy(x))
+            check(f'layer{i}_rmsnorm',n,tn)
+            if i==0:
+                out=lm.linear_attention(n,i)
+                tout=layer.linear_attn(tn)
+            else:
+                out,pr=lm.full_attention(n,i)
+                tout,tpr=layer.self_attn(tn,(cos,sin),mask)
+                check('attention_probabilities',pr,tpr)
+            check(f'layer{i}_mixer',out,tout)
+            h=x+out
+            n=lm.normalization(h,f'layers.{i}.post_attention_layernorm')
+            check(f'layer{i}_mlp',lm.mlp(n,i),layer.mlp(layer.post_attention_layernorm(torch.from_numpy(h))))
+            actual=h+lm.mlp(n,i)
+            expected=layer(torch.from_numpy(x),(cos,sin),attention_mask=mask if i else None)
+            check(f'layer{i}_complete',actual,expected)
+        # Independent recurrence/chunk equality, including nonmultiple sequence length.
+        for length in (1,7,33):
+            q=rng.standard_normal((2,length,2,4),dtype=np.float32)
+            k=rng.standard_normal(q.shape,dtype=np.float32)
+            v=rng.standard_normal((2,length,2,5),dtype=np.float32)
+            g=-np.abs(rng.standard_normal((2,length,2),dtype=np.float32))
+            beta=rng.uniform(size=g.shape).astype(np.float32)
+            out,state=delta_rule(q,k,v,g,beta)
+            args=[torch.from_numpy(z) for z in (q,k,v,g,beta)]
+            for kind in ('torch_recurrent_gated_delta_rule','torch_chunk_gated_delta_rule'):
+                ref,refstate=ns[kind](*args,output_final_state=True,use_qk_l2norm_in_kernel=True)
+                check(f'{kind}_length{length}_output',out,ref)
+                check(f'{kind}_length{length}_state',state,refstate)
+        # Full two-layer residual forward, final norm and embedding lookup.
+        arrays['embed_tokens.weight']=rng.standard_normal((31,16),dtype=np.float32)
+        arrays['norm.weight']=rng.standard_normal((16,),dtype=np.float32)*.1
+        ids=rng.integers(0,31,size=(2,7))
+        h=torch.from_numpy(arrays['embed_tokens.weight'][ids])
+        for i,l in enumerate(layers):h=l(h,(cos,sin),attention_mask=mask if i else None)
+        expected=h*torch.rsqrt(h.square().mean(-1,keepdim=True)+1e-6)*(1+torch.from_numpy(arrays['norm.weight']))
+        actual,_=lm.forward(ids)
+        check('complete_two_layer_text_forward',actual,expected)
+    result={'passed':True,'reference_commit':EXPECTED_COMMIT,'source_sha256':hashlib.sha256(a.source.read_bytes()).hexdigest(),
+            'torch':torch.__version__,'numpy':np.__version__,'implementation_sha256':hashlib.sha256(Path(__file__).with_name('model.py').read_bytes()).hexdigest(),'tests':evidence,'limitations':'FP32 CPU semantics on tiny synthetic configurations; not full-checkpoint BF16 kernel equivalence'}
+    a.output.write_text(json.dumps(result,indent=2))
+    print(json.dumps({'passed':True,'checks':len(evidence),'maximum_absolute_discrepancy':max(x['max_abs_error'] for x in evidence)}))
+
+
+if __name__=='__main__':main()

functional/report.py

@@ -0,0 +1,104 @@
+"""Render scientific pilot figures using saved data; never reopen model weights."""
+import argparse
+import csv
+import json
+from pathlib import Path
+import numpy as np
+import matplotlib
+matplotlib.use('Agg')
+import matplotlib.pyplot as plt
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    out=root/'figures';out.mkdir(exist_ok=True)
+    plt.rcParams.update({'font.size':10,'savefig.bbox':'tight','pdf.fonttype':42,'svg.fonttype':'none'})
+    def save(fig,name):
+        for ext in ('png','pdf','svg'):fig.savefig(out/f'{name}.{ext}',dpi=180)
+        plt.close(fig)
+    status=json.loads((root/'functional-status.json').read_text())
+    if (root/'candidate-summary.json').exists():
+        c=json.loads((root/'candidate-summary.json').read_text())
+        fig,ax=plt.subplots(figsize=(10,4.5))
+        ax.bar(np.arange(len(c)),[max(v['maximum_symmetric_relative_error'],1e-8) for v in c],color=['#238b45' if v['passed'] else '#9e9e9e' for v in c])
+        ax.axhline(.05,color='#b2182b',linestyle='--',label='Relative L2 screen = 0.05; other gates also apply')
+        ax.set_yscale('log');ax.set_xticks(np.arange(len(c)),[v['id'] for v in c],rotation=45,ha='right')
+        ax.set_ylabel('Worst calibration directional relative L2')
+        ax.set_title('Fixed candidates; same-input function comparison\n4 calibration texts, 32 tokens; no training')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.38),fontsize=8);fig.tight_layout();save(fig,'candidate_function_errors')
+    if (root/'interventions.json').exists():
+        interventions=json.loads((root/'interventions.json').read_text())
+        fig,ax=plt.subplots(figsize=(8,4.5))
+        for idx,r in enumerate(interventions):
+            values=r['sequence_nll_deltas'];x=np.arange(len(values))
+            ax.plot(x,values,'o-',label=r['direction'])
+        ax.axhline(0,color='gray',linewidth=.7);ax.axhline(.01,color='#b2182b',linestyle='--',label='Mean-delta budget (not per-sequence bound)')
+        ax.set_xticks(np.arange(4),['heldout zh1','heldout zh2','heldout en1','heldout en2'])
+        ax.set_ylabel('Token NLL change (nat/token)');ax.set_title('Single-site interventions; frozen candidate\n4 heldout texts, 32 tokens; limited pilot evidence')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.18),fontsize=8);fig.tight_layout();save(fig,'heldout_intervention_nll')
+    norm_path=root/'norm-repetition-summary.json'
+    repetition=json.loads(norm_path.read_text()) if norm_path.exists() else {}
+    if (root/'norm-repetition-function-metrics.csv').exists():
+        with (root/'norm-repetition-function-metrics.csv').open() as f:rows=list(csv.DictReader(f))
+        sample_ids=list(dict.fromkeys(row['sample_id'] for row in rows))
+        fig,ax=plt.subplots(figsize=(9,4.8))
+        for source in sorted({row['source_layer'] for row in rows},key=int):
+            rr=[r for r in rows if r['source_layer']==source]
+            ax.plot(range(len(rr)),[max(float(r['relative_l2']),float(r['reverse_relative_l2'])) for r in rr],'o-',label=f'Input captured at layer {source}')
+        ax.axhline(.05,color='#b2182b',ls='--',label='Heldout empirical screen = 0.05')
+        chosen=repetition.get('selected')
+        if chosen:ax.axhline(chosen['worst_uniform_bound'],color='#6a51a3',ls=':',label='Proven uniform all-input bound')
+        ax.axvline(3.5,color='gray',linewidth=.8)
+        ax.set_xticks(range(len(sample_ids)),sample_ids,rotation=30,ha='right');ax.set_ylabel('Worst of two directional relative L2 errors')
+        ax.set_title('Learned input RMSNorm at layer indices 12/13: same-input comparison\nCalibration (left) / independent heldout texts (right), 32 tokens each')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.25),fontsize=8);fig.tight_layout();save(fig,'learned_norm_function_errors')
+    if repetition.get('interventions'):
+        fig,ax=plt.subplots(figsize=(8,4.5))
+        for v in repetition['interventions']:
+            ax.plot(range(4),v['sequence_nll_deltas'],'o-',label=f'Layer {v["target"]} uses {v["donor"]}; mean={v["mean_nll_delta"]:.5g}')
+        ax.axhline(0,color='gray',linewidth=.8);ax.set_xticks(range(4),['zh1','zh2','en1','en2'])
+        ax.set_xlabel('Independent heldout text (32 tokens)');ax.set_ylabel('Mean token NLL change (nat/token)')
+        ax.set_title('Bidirectional single-site substitution of existing learned RMSNorm modules')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.18),fontsize=8);fig.tight_layout();save(fig,'learned_norm_network_substitution')
+    if (root/'gain-factorization-score-metrics.csv').exists():
+        with (root/'gain-factorization-score-metrics.csv').open() as f:rr=list(csv.DictReader(f))
+        layers=sorted({int(v['layer']) for v in rr});fig,axes=plt.subplots(1,2,figsize=(10,4))
+        for ax,key,label in zip(axes,['score_max_abs_error','probability_max_abs_error'],['Maximum absolute score error','Maximum absolute probability error']):
+            for direction in ('q_to_k','k_to_q'):
+                ax.plot(layers,[max(float(v[key]) for v in rr if int(v['layer'])==layer and v['direction']==direction) for layer in layers],'o-',label=direction)
+            ax.set_xlabel('Full-attention layer index');ax.set_xticks(layers);ax.set_ylabel(label);ax.ticklabel_format(axis='y',style='sci',scilimits=(0,0));ax.legend(loc='upper left',fontsize=8)
+        fig.suptitle('Supplement: algebraic reparameterization, separate from learned-function repetition');fig.tight_layout();save(fig,'algebraic_gain_identity')
+    neuron_path=root/'neuron-reuse-summary.json'
+    neuron=json.loads(neuron_path.read_text()) if neuron_path.exists() else {}
+    text=['# CPU learned-function repetition pilot','',
+          'Primary question: do two distinct trained parts implement similar functions? Cost is not a success condition.',
+          f'- Existing learned RMSNorm function positive: {repetition.get("learned_function_repetition_positive",False)}',
+          f'- Bidirectional network quality screen passed: {repetition.get("bidirectional_network_quality_pass",False)}',
+          f'- Learned scalar MLP neuron search positive: {neuron.get("empirical_repeated_function_positive",False)}',
+          f'- Original 11 candidate run state: {status["state"]}; selected: {status.get("selected_candidate",[])}',
+          '- All model weights are retained on the server until the user explicitly requests deletion.','']
+    if repetition.get('selected'):
+        c=repetition['selected'];text += ['## Existing learned module evidence','',
+          f'Role: {c["role"]}; layers {c["a"]} and {c["b"]}; each maps 2048-dimensional input to output.',
+          f'Uniform real-arithmetic relative-error bound, first-to-second: {c["a_to_b_uniform_bound"]:.12g}; reverse: {c["b_to_a_uniform_bound"]:.12g}.',
+          f'Maximum heldout directional output relative L2: {repetition.get("worst_heldout_relative_l2")}; minimum cosine: {repetition.get("worst_heldout_cosine")}.',
+          'Same epsilon and nonzero gains: the common RMS denominator cancels. For any nonzero input x,',
+          '`||f_a(x)-f_b(x)|| / ||f_a(x)|| <= max_j |a_j-b_j|/|a_j|`; swap a/b for the reverse direction.',
+          'Selection used only the tight all-input bound across 552 same-role pairs, before inspecting inputs.',
+          'Real probes use four calibration and four heldout experiment-authored texts, 32 tokens each, and both source layers.',
+          'These are learned, nonconstant gain vectors. The claim concerns RMSNorm modules; it does not establish attention/MLP-body duplication.','']
+        for v in repetition.get('interventions',[]):text += [f'- Layer {v["target"]} uses {v["donor"]}: mean heldout NLL delta {v["mean_nll_delta"]:.12g}; sequence P95 delta {v["sequence_p95_nll_delta"]:.12g}; screen pass={v["passed_quality"]}.']
+    text += ['', '## Negative results and separate algebraic result','',
+             'The original whole-attention, whole-MLP and within-layer Q/K norm candidates did not pass their gates.',
+             'The scalar neuron search excludes low-RMS/near-constant profiles and selects at most one pair without heldout feedback; consult its search summary for coverage.',
+             'Q/K non-RoPE gain consolidation is a separately verified real-arithmetic identity, not evidence that two existing learned modules duplicate functions.',
+             'Earlier local CPU timing records are preserved for provenance; performance is outside the current success criterion.','',
+             '## Limits and provenance','',
+             'Original BF16 weights were decoded into streaming FP32 CPU operators; no training, scale fitting, R fitting or GPU.',
+             'The numerical reference includes 22 independent official Torch comparisons. Early adapter errors and two resource-guard stops are preserved as historical logs.',
+             'The persistent computation ledger includes a conservative 180-second reserve for early checks; no budget reset after that reserve.',
+             'The text sample is tiny and short. Network NLL checks are not broad task-accuracy validation.','']
+    (root/'FUNCTIONAL_REPORT.md').write_text('\n'.join(text))
+
+
+if __name__=='__main__':main()

functional/run.py

@@ -0,0 +1,227 @@
+"""Bounded CPU pilot. One fixed model; calibration selects, heldout only validates."""
+import argparse
+import csv
+import hashlib
+import json
+import os
+from pathlib import Path
+import resource
+import time
+import traceback
+import urllib.request
+import numpy as np
+from .model import StreamingLM, norm
+
+ATTN_PAIRS=[(3,7),(11,15),(19,23)]
+MLP_PAIRS=[(5,6),(20,21)]
+NORM_LAYERS=[3,7,11,15,19,23]
+REVISION='15852e8c16360a2fea060d615a32b45270f8a8fc'
+
+
+def write_json(path,obj):
+    tmp=path.with_suffix('.tmp')
+    tmp.write_text(json.dumps(obj,indent=2,allow_nan=False)+'\n')
+    tmp.replace(path)
+
+
+def error_rows(a,b,samples,metadata):
+    records=[]
+    for x,y,sample in zip(a,b,samples):
+        xf,yf=x.astype(np.float64).reshape(-1),y.astype(np.float64).reshape(-1)
+        xn,yn=np.linalg.norm(xf),np.linalg.norm(yf)
+        delta=np.linalg.norm(xf-yf)
+        tokenerr=np.linalg.norm((x-y).reshape(x.shape[0],-1).astype(np.float64),axis=-1)/np.maximum(np.linalg.norm(x.reshape(x.shape[0],-1).astype(np.float64),axis=-1),1e-12)
+        records.append({**metadata,'sample_id':sample['id'],'split':sample['split'],
+                        'relative_l2':float(delta/max(xn,1e-12)),
+                        'reverse_relative_l2':float(delta/max(yn,1e-12)),
+                        'token_relative_p95':float(np.quantile(tokenerr,.95)),
+                        'reverse_token_relative_p95':float(np.quantile(np.linalg.norm((x-y).reshape(x.shape[0],-1).astype(np.float64),axis=-1)/np.maximum(np.linalg.norm(y.reshape(x.shape[0],-1).astype(np.float64),axis=-1),1e-12),.95)),
+                        'cosine':float(xf@yf/max(xn*yn,1e-30)),
+                        'low_norm_tokens':int((np.linalg.norm(x.reshape(x.shape[0],-1),axis=-1)<1e-8).sum())})
+    return records
+
+
+def probability_metrics(a,b):
+    m=(a+b)*.5
+    def term(p):
+        return np.where(p>0,p*np.log(np.maximum(p,1e-30)/np.maximum(m,1e-30)),0)
+    js=(term(a)+term(b)).sum(axis=-1)*.5
+    overlap=[]
+    for row in range(a.shape[-2]):
+        k=min(5,row+1)
+        ai=np.argsort(a[:,:,row,:row+1],axis=-1)[...,-k:]
+        bi=np.argsort(b[:,:,row,:row+1],axis=-1)[...,-k:]
+        overlap.append(np.mean((ai[...,None]==bi[...,None,:]).any(axis=-1),axis=-1))
+    # Attention-sink control: remove key zero, renormalize valid remaining mass.
+    aa,bb=a[...,1:],b[...,1:]
+    mass_a,mass_b=aa.sum(-1),bb.sum(-1)
+    valid=(mass_a>1e-12)&(mass_b>1e-12)
+    aa=aa/np.maximum(mass_a[...,None],1e-30)
+    bb=bb/np.maximum(mass_b[...,None],1e-30)
+    mm=(aa+bb)*.5
+    conditional_js=.5*(np.where(aa>0,aa*np.log(np.maximum(aa,1e-30)/np.maximum(mm,1e-30)),0)+np.where(bb>0,bb*np.log(np.maximum(bb,1e-30)/np.maximum(mm,1e-30)),0)).sum(-1)
+    return {'first_key_mass_a':float(a[...,0].mean()),'first_key_mass_b':float(b[...,0].mean()),
+            'js_without_first_key':float(conditional_js[valid].mean()) if valid.any() else None,
+            'without_first_key_valid_rows':int(valid.sum()),'js_mean':float(js.mean()),'js_p95':float(np.quantile(js,.95)),
+            'js_without_first_query':float(js[...,1:].mean()),
+            'top5_overlap':float(np.stack(overlap,axis=-1).mean())}
+
+
+def save_csv(path,rows):
+    if not rows:return
+    keys=list(dict.fromkeys(k for r in rows for k in r))
+    with path.open('w',newline='') as f:
+        w=csv.DictWriter(f,fieldnames=keys);w.writeheader();w.writerows(rows)
+
+
+def tokenize(root):
+    from tokenizers import Tokenizer
+    path=root/'tokenizer.json'
+    if not path.exists():
+        url=f'https://huggingface.co/Qwen/Qwen3.5-2B/resolve/{REVISION}/tokenizer.json'
+        with urllib.request.urlopen(url,timeout=180) as r:path.write_bytes(r.read())
+    tokenizer=Tokenizer.from_file(str(path))
+    samples=json.loads(Path(__file__).with_name('samples.json').read_text())['samples']
+    ids=[]
+    for s in samples:
+        tokens=tokenizer.encode(s['text'],add_special_tokens=False).ids
+        if len(tokens)<32:raise ValueError(f'Text too short: {s["id"]}')
+        s['text_sha256']=hashlib.sha256(s['text'].encode()).hexdigest()
+        s['full_token_count']=len(tokens);s['token_ids']=tokens[:32]
+        ids.append(tokens[:32])
+    write_json(root/'samples-tokenized.json',{'revision':REVISION,'tokenizer_sha256':hashlib.sha256(path.read_bytes()).hexdigest(),'samples':samples})
+    return samples,np.array(ids,dtype=np.int64)
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);p.add_argument('--seconds',type=int,default=7200)
+    args=p.parse_args();root=args.root;root.mkdir(exist_ok=True,parents=True)
+    write_json(root/'retention-policy.json',{'weights':'retain_until_explicit_user_delete_request','automatic_cleanup':False,'instruction_date':'2026-11-09'})
+    samples,ids=tokenize(root)
+    cal=np.array([i for i,s in enumerate(samples) if s['split']=='calibration'])
+    held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+    # Freeze all candidates/thresholds before opening weights or measuring functions.
+    experiment={'revision':REVISION,'attention_pairs':ATTN_PAIRS,'mlp_pairs':MLP_PAIRS,'norm_layers':NORM_LAYERS,
+                'seconds':args.seconds,'output_relative_l2_limit':.05,'token_p95_limit':.1,
+                'attention_js_limit':.01,'attention_top5_min':.8,'heldout_nll_mean_delta_limit':.01,
+                'heldout_nll_sequence_p95_delta_limit':.05,'max_intervention_candidates':1,
+                'directions_per_selected_candidate':2,'training':False,'search_R':False,'automatic_weight_cleanup':False,
+                'dtype':'original weights decoded to FP32; not a BF16 backend benchmark',
+                'blas_threads':os.environ.get('OPENBLAS_NUM_THREADS')}
+    write_json(root/'experiment.json',experiment)
+    started=time.time();status={'state':'running','started_unix':started,'phase':'baseline'}
+    write_json(root/'functional-status.json',status)
+    rows=[];headrows=[];interventions=[]
+    try:
+        config=json.loads((root/'model_config.json').read_text())
+        lm=StreamingLM(root/'weights',config,seconds=args.seconds)
+        overall_deadline=lm.deadline
+        lm.deadline=min(overall_deadline,time.monotonic()+2400)
+        lm.capture_qk=True
+        capture=[('attn',i) for pair in ATTN_PAIRS for i in pair]+[('mlp',i) for pair in MLP_PAIRS for i in pair]
+        def progress(layer):
+            status.update(last_completed_layer=layer,elapsed_seconds=time.time()-started)
+            write_json(root/'functional-status.json',status)
+            print(json.dumps({'phase':status['phase'],'layer':layer,'elapsed':time.time()-started}),flush=True)
+        h,saved=lm.forward(ids,capture=capture,progress=progress)
+        np.savez(root/'baseline-activations.npz',**saved,**{f'qraw_{k}':v[0] for k,v in lm.qk_inputs.items()},**{f'kraw_{k}':v[1] for k,v in lm.qk_inputs.items()},final=h)
+        # Baseline heldout NLL uses no selection information.
+        nll=lm.nll(h[held],ids[held]);np.save(root/'baseline-heldout-token-nll.npy',nll)
+        baseline_nll=nll.mean(axis=1)
+        lm.capture_qk=False
+        lm.deadline=min(overall_deadline,time.monotonic()+1800)
+        status['phase']='calibration';write_json(root/'functional-status.json',status)
+        candidates=[]
+        for kind,pairs in [('attn',ATTN_PAIRS),('mlp',MLP_PAIRS)]:
+            for i,j in pairs:
+                pairid=f'{kind}_{i}_{j}';pairrows=[]
+                for source in (i,j):
+                    x=saved[f'{kind}_{source}'][cal]
+                    if kind=='attn':
+                        a,pa=lm.full_attention(x,i);b,pb=lm.full_attention(x,j)
+                        pm={}
+                        for bi,si in enumerate(cal):
+                            for hi in range(pa.shape[1]):
+                                hpm=probability_metrics(pa[bi:bi+1,hi:hi+1],pb[bi:bi+1,hi:hi+1])
+                                headrows.append({'candidate':pairid,'source':source,'sample_id':samples[si]['id'],'head':hi,**hpm})
+                        save_csv(root/'attention-head-metrics.csv',headrows)
+                    else:
+                        a,b=lm.mlp(x,i),lm.mlp(x,j);pm={}
+                    rec=error_rows(a,b,[samples[t] for t in cal],{'candidate':pairid,'kind':kind,'source':source,**pm})
+                    if kind=='attn':
+                        for bi,r in enumerate(rec):r.update(probability_metrics(pa[bi:bi+1],pb[bi:bi+1]))
+                    rows.extend(rec);pairrows.extend(rec)
+                    save_csv(root/'calibration-metrics.csv',rows)
+                passing=all(r['relative_l2']<=.05 and r['reverse_relative_l2']<=.05 and r['token_relative_p95']<=.1 and r['reverse_token_relative_p95']<=.1 for r in pairrows)
+                if kind=='attn':passing=passing and all(r['js_mean']<=.01 and r['top5_overlap']>=.8 for r in pairrows)
+                prefix=f'model.language_model.layers.{i}.'+('self_attn.' if kind=='attn' else 'mlp.')
+                size=sum(t.bytes for name,t in lm.tensors.items() if name.startswith(prefix))
+                candidates.append({'id':pairid,'kind':kind,'i':i,'j':j,'passed':passing,'maximum_symmetric_relative_error':max(max(r['relative_l2'],r['reverse_relative_l2']) for r in pairrows),'donor_component_bytes':size})
+        for layer in NORM_LAYERS:
+            pairid=f'qk_norm_{layer}';pfx=f'layers.{layer}.self_attn.'
+            qw,kw=lm.small(pfx+'q_norm.weight'),lm.small(pfx+'k_norm.weight')
+            pairrows=[]
+            for source,x in zip(('q','k'),lm.qk_inputs[layer]):
+                a,b=norm(x[cal],qw,lm.eps),norm(x[cal],kw,lm.eps)
+                rec=error_rows(a,b,[samples[t] for t in cal],{'candidate':pairid,'kind':'qk_norm','source':source})
+                pairrows.extend(rec);rows.extend(rec)
+            passing=all(r['relative_l2']<=.05 and r['reverse_relative_l2']<=.05 and r['token_relative_p95']<=.1 and r['reverse_token_relative_p95']<=.1 for r in pairrows)
+            candidates.append({'id':pairid,'kind':'qk_norm','i':layer,'j':layer,'passed':passing,'maximum_symmetric_relative_error':max(max(r['relative_l2'],r['reverse_relative_l2']) for r in pairrows),'donor_component_bytes':lm.tensors['model.language_model.'+pfx+'q_norm.weight'].bytes})
+        save_csv(root/'calibration-metrics.csv',rows)
+        write_json(root/'candidate-summary.json',candidates)
+        passed=[c for c in candidates if c['passed']]
+        # Prefer larger potential savings, then smaller worst calibration error; never consult heldout.
+        selected=sorted(passed,key=lambda c:(-c['donor_component_bytes'],c['maximum_symmetric_relative_error'],c['id']))[:1]
+        write_json(root/'selection.json',{'selected':selected,'rule':'calibration pass; descending component bytes, ascending maximum symmetric error; maximum one candidate'})
+        if selected:
+            c=selected[0]
+            heldrows=[]
+            if c['kind']=='qk_norm':
+                pfx=f'layers.{c["i"]}.self_attn.'
+                qw,kw=lm.small(pfx+'q_norm.weight'),lm.small(pfx+'k_norm.weight')
+                for source,x in zip(('q','k'),lm.qk_inputs[c['i']]):
+                    heldrows.extend(error_rows(norm(x[held],qw,lm.eps),norm(x[held],kw,lm.eps),[samples[t] for t in held],{'candidate':c['id'],'kind':'qk_norm','source':source}))
+            else:
+                for source in (c['i'],c['j']):
+                    x=saved[f'{c["kind"]}_{source}'][held]
+                    if c['kind']=='attn':
+                        a,_=lm.full_attention(x,c['i']);b,_=lm.full_attention(x,c['j'])
+                    else:a,b=lm.mlp(x,c['i']),lm.mlp(x,c['j'])
+                    heldrows.extend(error_rows(a,b,[samples[t] for t in held],{'candidate':c['id'],'kind':c['kind'],'source':source}))
+            save_csv(root/'heldout-function-metrics.csv',heldrows)
+            held_function_pass=all(r['relative_l2']<=.05 and r['reverse_relative_l2']<=.05 and r['token_relative_p95']<=.1 and r['reverse_token_relative_p95']<=.1 for r in heldrows)
+            status['heldout_function_pass']=held_function_pass
+            lm.deadline=min(overall_deadline,time.monotonic()+1800)
+            status['phase']='intervention';write_json(root/'functional-status.json',status)
+            for reverse in (False,True):
+                replacement=None;lm.norm_replacement=None
+                if c['kind']=='qk_norm':
+                    pfx=f'layers.{c["i"]}.self_attn.'
+                    target,donor=('q_norm','k_norm') if not reverse else ('k_norm','q_norm')
+                    lm.norm_replacement=(pfx+target,pfx+donor)
+                    label=target+'_uses_'+donor
+                else:
+                    target,donor=(c['i'],c['j']) if not reverse else (c['j'],c['i'])
+                    replacement=(c['kind'],target,donor);label=f'{target}_uses_{donor}'
+                t=time.monotonic();modified,_=lm.forward(ids[held],replacement=replacement,progress=progress)
+                changed=lm.nll(modified,ids[held]);np.save(root/f'intervention-{label}-token-nll.npy',changed)
+                delta=changed.mean(axis=1)-baseline_nll
+                rec={'candidate':c['id'],'direction':label,'elapsed_seconds':time.monotonic()-t,
+                     'nll_mean_delta':float(delta.mean()),'nll_sequence_p95_delta':float(np.quantile(delta,.95)),
+                     'sequence_nll_deltas':delta.tolist(),'passed_quality':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)}
+                interventions.append(rec);write_json(root/'interventions.json',interventions)
+            # No fallback candidate if heldout fails.
+        status.update(state='completed',phase='done',selected_candidate=selected,
+                      positive_bidirectional_quality=bool(interventions and status.get('heldout_function_pass') and all(x['passed_quality'] for x in interventions)),
+                      elapsed_seconds=time.time()-started,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
+                      claim_limit='Small text-only heldout pilot; no global interchangeability proof, no measured GPU speedup; component storage savings require explicit layout accounting.')
+    except Exception as exc:
+        status.update(state='stopped',error_type=type(exc).__name__,error=str(exc),elapsed_seconds=time.time()-started,
+                      peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        (root/'functional-error.txt').write_text(traceback.format_exc())
+        raise
+    finally:
+        write_json(root/'functional-status.json',status)
+
+
+if __name__=='__main__':main()

functional/samples.json

@@ -0,0 +1,13 @@
+{
+  "origin": "Experiment-authored plain text, no external corpus; fixed before functional metrics. Chinese/English balanced in each split. No chat template. First 32 tokenizer tokens retained.",
+  "samples": [
+    {"id":"cal-zh-1","split":"calibration","language":"zh","text":"清晨，研究员打开实验记录本，检查昨天保存的数据。他先确认文件完整，再比较两组观测结果。为了避免偶然误差，他为每一步留下时间和条件，并将尚未验证的推测单独记录。"},
+    {"id":"cal-zh-2","split":"calibration","language":"zh","text":"一辆列车从山间驶向海边，经过村庄、桥梁和开阔的田野。旅客望着窗外变化的景色，计划抵达后先步行到旅馆，再去市场购买晚餐需要的食材。天气预报说傍晚可能下雨。"},
+    {"id":"cal-en-1","split":"calibration","language":"en","text":"A small library opens every morning beside the river. The librarian checks returned books, repairs damaged covers, and places new arrivals on a wooden table. Visitors often stop to read before continuing their journey across the old bridge."},
+    {"id":"cal-en-2","split":"calibration","language":"en","text":"To compare two measurements, first record their units and the conditions under which they were collected. Compute the difference, inspect unusual observations, and repeat the procedure with independent examples. A short report should separate observations from explanations and unresolved questions."},
+    {"id":"test-zh-1","split":"heldout","language":"zh","text":"午后，工程师检查温室中的传感器。温度记录出现一个异常值，她没有立刻删除，而是查看设备状态并补做一次测量。确认原因以后，她调整了采样计划，在报告中说明这次变化。"},
+    {"id":"test-zh-2","split":"heldout","language":"zh","text":"周末，家人沿着湖边的小路散步。他们带了水和简单的午餐，打算在树林旁休息。途中遇到一处正在维修的桥，于是参考路标选择另一条道路，终于在日落前回到车站。"},
+    {"id":"test-en-1","split":"heldout","language":"en","text":"The gardener planted several rows of beans behind the house. Each evening she checked the soil and wrote down how much water the plants received. After a week of warm weather, the first green shoots appeared near the fence beside the apple tree."},
+    {"id":"test-en-2","split":"heldout","language":"en","text":"When a program reads a table, it should check column names and handle missing entries explicitly. The developer tested a small example, compared the output with a hand calculation, and saved the input alongside the result so another person could reproduce the check."}
+  ]
+}

functional/storage.py

@@ -0,0 +1,62 @@
+"""Measure an explicit two-component alias storage layout, not inference speed.
+
+Payloads remain on the server under weights/ and must not be committed or cleaned
+without the user's explicit deletion request. Manifest/byte/hash evidence may be archived.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import numpy as np
+from pilot.tensors import load_tensors,decode
+from .run import write_json
+
+
+def payload(t,path):
+    digest=hashlib.sha256()
+    with path.open('wb') as f:
+        for start in range(0,t.raw.size,262144):
+            b=t.raw[start:start+262144].tobytes();f.write(b);digest.update(b)
+            t.evict()
+    return {'file':path.name,'dtype':t.dtype,'shape':list(t.shape),'bytes':path.stat().st_size,'sha256':digest.hexdigest()}
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    selected=json.loads((root/'selection.json').read_text())['selected']
+    if not selected:
+        write_json(root/'storage-evidence.json',{'measured':False,'reason':'No calibration candidate selected'});return
+    c=selected[0]
+    if c['kind']!='qk_norm':
+        write_json(root/'storage-evidence.json',{'measured':False,'reason':'Only two-gain alias layout implemented in this bounded pilot; larger-module savings remain theoretical'});return
+    tensors=load_tensors(root/'weights')
+    prefix=f'model.language_model.layers.{c["i"]}.self_attn.'
+    names=[prefix+'q_norm.weight',prefix+'k_norm.weight']
+    # Original pair plus two alternative layouts; each is counted independently.
+    path=root/'component-layout'/'weights';path.mkdir(parents=True,exist_ok=True)
+    original={'format':'two_component_raw_v1','tensors':{}}
+    for short,name in zip(('q','k'),names):original['tensors'][name]=payload(tensors[name],path/f'{short}.bin')
+    original_bytes=json.dumps(original,separators=(',',':'),sort_keys=True).encode()
+    (path/'original-layout.json').write_bytes(original_bytes)
+    evidence=[]
+    for target,donor in ((names[0],names[1]),(names[1],names[0])):
+        entry=original['tensors'][donor]
+        layout={'format':'two_component_raw_v1','tensors':{donor:entry},'aliases':{target:donor}}
+        encoded=json.dumps(layout,separators=(',',':'),sort_keys=True).encode()
+        file=path/('share-'+('k' if donor==names[1] else 'q')+'.json');file.write_bytes(encoded)
+        t=tensors[donor]
+        raw=np.fromfile(path/entry['file'],dtype=t.raw.dtype)
+        reconstructed=decode(raw,t.dtype).reshape(t.shape)
+        np.testing.assert_array_equal(reconstructed,t.read().reshape(t.shape));t.evict()
+        before=sum(e['bytes'] for e in original['tensors'].values())+len(original_bytes)
+        after=entry['bytes']+len(encoded)
+        evidence.append({'target':target,'donor':donor,'original_payload_bytes':sum(e['bytes'] for e in original['tensors'].values()),
+                         'shared_payload_bytes':entry['bytes'],'original_descriptor_bytes':len(original_bytes),
+                         'shared_descriptor_bytes':len(encoded),'original_total_bytes':before,'shared_total_bytes':after,
+                         'net_saved_bytes':before-after,'shared_descriptor_sha256':hashlib.sha256(encoded).hexdigest(),
+                         'decoded_donor_exactly_matches_intervention_gain':True})
+    write_json(root/'storage-evidence.json',{'measured':True,'candidate':c['id'],'layouts':evidence,
+               'claim_limit':'Measured standalone component layout including descriptors, not an integrated compressed checkpoint, latency gain or full-model RSS reduction. Two sharing directions are alternatives and must not be added.'})
+
+
+if __name__=='__main__':main()

functional/supervise.py

@@ -0,0 +1,95 @@
+"""One bounded CPU run; waits for verified download and binary dependencies.
+
+No installation, retry search, cleanup, training, GPU selection or Git mutation.
+"""
+import argparse
+import hashlib
+import importlib.util
+import json
+from pathlib import Path
+import subprocess
+import sys
+import time
+import traceback
+from .run import write_json
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);p.add_argument('--download-wait-seconds',type=int,default=14400)
+    a=p.parse_args();root=a.root.resolve();start=time.monotonic()
+    state={'state':'waiting','phase':'dependencies','weight_retention':'until_explicit_user_delete_request'}
+    def status(**kwargs):
+        state.update(kwargs,supervisor_elapsed_seconds=time.monotonic()-start)
+        write_json(root/'supervisor-status.json',state)
+    def execute(module,args,timeout,log):
+        ledger_path=root/'compute-budget-ledger.json'
+        ledger=json.loads(ledger_path.read_text()) if ledger_path.exists() else {'spent_seconds':0.,'calls':[]}
+        timeout=min(timeout,int(7200-ledger['spent_seconds']))
+        if timeout<1:raise TimeoutError('Cumulative two-hour compute budget exhausted')
+        t=time.monotonic()
+        try:
+            with (root/log).open('w') as out:
+                r=subprocess.run([sys.executable,'-B','-m',module,*args],stdout=out,stderr=subprocess.STDOUT,timeout=timeout)
+        finally:
+            elapsed=time.monotonic()-t
+            ledger['spent_seconds']+=elapsed
+            ledger['calls'].append({'module':module,'seconds':elapsed})
+            write_json(ledger_path,ledger)
+        if r.returncode:raise RuntimeError(f'{module} failed: exit {r.returncode}, see {log}')
+    status()
+    try:
+        while True:
+            ready=False
+            if all(importlib.util.find_spec(name) for name in ('torch','tokenizers')):
+                try:
+                    r=subprocess.run([sys.executable,'-B','-c','import torch, tokenizers; torch.empty(1)'],capture_output=True,timeout=30)
+                    ready=r.returncode==0
+                except subprocess.TimeoutExpired:pass
+            if ready:break
+            if time.monotonic()-start>3600:raise TimeoutError('Binary reference dependencies not ready within one hour')
+            status()
+            time.sleep(15)
+        compute_start=time.monotonic()
+        status(state='running',phase='reference_check')
+        provenance={str(p):hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(Path('functional').glob('*')) if p.is_file()}
+        write_json(root/'functional-code-hashes.json',provenance)
+        execute('functional.reference_check',['--source',str(root/'reference.py'),'--output',str(root/'reference-check.json')],300,'reference-check.log')
+        reference_seconds=time.monotonic()-compute_start
+        status(state='waiting',phase='verified_weights',reference_seconds=reference_seconds)
+        while True:
+            path=root/'manifest.json'
+            try:
+                manifest=json.loads(path.read_text())
+                weights=[f for f in manifest.get('files',[]) if f['name'].endswith('.safetensors')]
+                ready=(manifest.get('completed_unix') is not None and weights and
+                       all((root/'weights'/f['name']).is_file() and (root/'weights'/f['name']).stat().st_size==f['bytes'] and f['sha256']==f['upstream_sha256'] for f in weights))
+            except (OSError,ValueError,KeyError):ready=False
+            if ready:break
+            if time.monotonic()-start>a.download_wait_seconds:raise TimeoutError('Verified weight download not ready before wait deadline')
+            status()
+            time.sleep(15)
+        status(state='running',phase='real_weight_preflight')
+        compute_start=time.monotonic()
+        execute('functional.preflight',['--root',str(root)],1200,'preflight.log')
+        preflight_seconds=time.monotonic()-compute_start
+        remaining=int(7200-json.loads((root/'compute-budget-ledger.json').read_text())['spent_seconds']-60)
+        if remaining<60:raise TimeoutError('No remaining bounded functional computation budget')
+        status(phase='functional_analysis',preflight_seconds=preflight_seconds,remaining_seconds=remaining)
+        execute('functional.run',['--root',str(root),'--seconds',str(remaining)],remaining+15,'functional-run.log')
+        status(phase='storage_accounting')
+        execute('functional.storage',['--root',str(root)],120,'storage-accounting.log')
+        status(phase='report')
+        execute('functional.report',['--root',str(root)],120,'functional-report.log')
+        status(state='completed',phase='done')
+    except Exception as exc:
+        status(state='stopped',error_type=type(exc).__name__,error=str(exc))
+        (root/'supervisor-error.txt').write_text(traceback.format_exc())
+        # Retain all weights/results. No automatic retries or candidate expansion.
+        if not (root/'functional-status.json').exists():
+            write_json(root/'functional-status.json',{'state':'stopped','phase':state['phase'],'error_type':type(exc).__name__,'error':str(exc)})
+        try:execute('functional.report',['--root',str(root)],120,'functional-report.log')
+        except Exception:pass
+        raise
+
+
+if __name__=='__main__':main()

requirements-functional.txt

@@ -0,0 +1,5 @@
+# Existing requirements.txt supplies NumPy and plotting dependencies.
+# Install binary wheels only. Torch is a reference-check dependency, not the streaming runtime.
+tokenizers==0.23.3
+# Install separately with --index-url https://download.pytorch.org/whl/cpu:
+# torch==2.15.0+cpu

results/cpu-qwen35-2b/DELIVERY.md

@@ -0,0 +1,7 @@
+# Scientific delivery and retained caches
+
+All figure CSV/JSON source data, per-token NLL, protocols, reports, metadata and historical logs are included.
+Three large reproducible intermediate activation caches remain on 8.211.154.134 and in the original full local archive;
+`external-activation-cache.json` records exact locations, sizes and SHA256. No weight or cache was deleted.
+The original full experiment manifest is `checksums-full-server.json`; the delivery-only manifest is
+`checksums-functional.json`. No model checkpoint or tokenizer payload is in this Git delivery.

results/cpu-qwen35-2b/FUNCTIONAL_REPORT.md

@@ -0,0 +1,36 @@
+# CPU learned-function repetition pilot
+
+Primary question: do two distinct trained parts implement similar functions? Cost is not a success condition.
+- Existing learned RMSNorm function positive: True
+- Bidirectional network quality screen passed: True
+- Learned scalar MLP neuron search positive: False
+- Original 11 candidate run state: completed; selected: []
+- All model weights are retained on the server until the user explicitly requests deletion.
+
+## Existing learned module evidence
+
+Role: input_layernorm; layers 12 and 13; each maps 2048-dimensional input to output.
+Uniform real-arithmetic relative-error bound, first-to-second: 0.101851851852; reverse: 0.113402061856.
+Maximum heldout directional output relative L2: 0.04528510757179746; minimum cosine: 0.9994870299724158.
+Same epsilon and nonzero gains: the common RMS denominator cancels. For any nonzero input x,
+`||f_a(x)-f_b(x)|| / ||f_a(x)|| <= max_j |a_j-b_j|/|a_j|`; swap a/b for the reverse direction.
+Selection used only the tight all-input bound across 552 same-role pairs, before inspecting inputs.
+Real probes use four calibration and four heldout experiment-authored texts, 32 tokens each, and both source layers.
+These are learned, nonconstant gain vectors. The claim concerns RMSNorm modules; it does not establish attention/MLP-body duplication.
+
+- Layer 12 uses 13: mean heldout NLL delta 3.25651368766e-05; sequence P95 delta 0.000984639618287; screen pass=True.
+- Layer 13 uses 12: mean heldout NLL delta -0.000363972997404; sequence P95 delta 0.000709721155035; screen pass=True.
+
+## Negative results and separate algebraic result
+
+The original whole-attention, whole-MLP and within-layer Q/K norm candidates did not pass their gates.
+The scalar neuron search excludes low-RMS/near-constant profiles and selects at most one pair without heldout feedback; consult its search summary for coverage.
+Q/K non-RoPE gain consolidation is a separately verified real-arithmetic identity, not evidence that two existing learned modules duplicate functions.
+Earlier local CPU timing records are preserved for provenance; performance is outside the current success criterion.
+
+## Limits and provenance
+
+Original BF16 weights were decoded into streaming FP32 CPU operators; no training, scale fitting, R fitting or GPU.
+The numerical reference includes 22 independent official Torch comparisons. Early adapter errors and two resource-guard stops are preserved as historical logs.
+The persistent computation ledger includes a conservative 180-second reserve for early checks; no budget reset after that reserve.
+The text sample is tiny and short. Network NLL checks are not broad task-accuracy validation.

results/cpu-qwen35-2b/archive-files.json

@@ -0,0 +1,88 @@
+[
+  "DELIVERY.md",
+  "FUNCTIONAL_REPORT.md",
+  "attention-head-metrics.csv",
+  "baseline-heldout-token-nll.npy",
+  "calibration-metrics.csv",
+  "candidate-summary.json",
+  "checksums-full-server.json",
+  "compute-budget-ledger.json",
+  "download.log",
+  "experiment.json",
+  "external-activation-cache.json",
+  "figures/algebraic_gain_identity.pdf",
+  "figures/algebraic_gain_identity.png",
+  "figures/algebraic_gain_identity.svg",
+  "figures/candidate_function_errors.pdf",
+  "figures/candidate_function_errors.png",
+  "figures/candidate_function_errors.svg",
+  "figures/learned_norm_function_errors.pdf",
+  "figures/learned_norm_function_errors.png",
+  "figures/learned_norm_function_errors.svg",
+  "figures/learned_norm_network_substitution.pdf",
+  "figures/learned_norm_network_substitution.png",
+  "figures/learned_norm_network_substitution.svg",
+  "final-report.log",
+  "final-source-hashes.json",
+  "final-tests.txt",
+  "functional-code-hashes.json",
+  "functional-environment.json",
+  "functional-report.log",
+  "functional-run.log",
+  "functional-status.json",
+  "gain-factorization-k_to_q-token-nll.npy",
+  "gain-factorization-kernel-timing.csv",
+  "gain-factorization-optimized-summary.json",
+  "gain-factorization-optimized-timing.csv",
+  "gain-factorization-optimized.log",
+  "gain-factorization-protocol.json",
+  "gain-factorization-q_to_k-token-nll.npy",
+  "gain-factorization-score-metrics.csv",
+  "gain-factorization-summary.json",
+  "gain-factorization.log",
+  "install-torch-disk-tmp.log",
+  "install-torch.log",
+  "manifest.json",
+  "model_config.json",
+  "neuron-reuse-protocol.json",
+  "neuron-reuse-search-summary.json",
+  "neuron-reuse-selection.json",
+  "neuron-reuse-summary.json",
+  "neuron-reuse.log",
+  "norm-repetition-12-uses-13-token-nll.npy",
+  "norm-repetition-13-uses-12-token-nll.npy",
+  "norm-repetition-function-metrics.csv",
+  "norm-repetition-protocol.json",
+  "norm-repetition-selection.json",
+  "norm-repetition-summary.json",
+  "norm-repetition-uniform-bounds.csv",
+  "norm-repetition.log",
+  "preflight-first-attempt.log",
+  "preflight-second-attempt.json",
+  "preflight-second-attempt.log",
+  "preflight.json",
+  "preflight.log",
+  "reference-check-first-attempt.log",
+  "reference-check-second-attempt.log",
+  "reference-check.json",
+  "reference-check.log",
+  "retention-policy.json",
+  "samples-tokenized.json",
+  "selection.json",
+  "source_metadata.json",
+  "storage-accounting.log",
+  "storage-evidence.json",
+  "supervisor-error.txt",
+  "supervisor-first-attempt-error.txt",
+  "supervisor-first-attempt.json",
+  "supervisor-second-attempt.json",
+  "supervisor-status.json",
+  "supervisor-third-attempt.json",
+  "supervisor.log",
+  "tokenizer-download.log",
+  "tokenizers-wheel.log",
+  "unit-tests.txt",
+  "checksums-functional.json",
+  "audit-validation.json",
+  "archive-files.json"
+]

results/cpu-qwen35-2b/attention-head-metrics.csv

@@ -0,0 +1,193 @@
+candidate,source,sample_id,head,first_key_mass_a,first_key_mass_b,js_without_first_key,without_first_key_valid_rows,js_mean,js_p95,js_without_first_query,top5_overlap
+attn_3_7,3,cal-zh-1,0,0.178242027759552,0.44051653146743774,0.2887135446071625,31,0.2728434205055237,0.42193013429641724,0.2816448211669922,0.5875
+attn_3_7,3,cal-zh-1,1,0.14070242643356323,0.6567835807800293,0.154605433344841,31,0.273810476064682,0.45863786339759827,0.2826431095600128,0.5375
+attn_3_7,3,cal-zh-1,2,0.11500269174575806,0.4787790775299072,0.3218191862106323,31,0.35086748003959656,0.46183231472969055,0.3621857762336731,0.53125
+attn_3_7,3,cal-zh-1,3,0.17224811017513275,0.6523524522781372,0.22310931980609894,31,0.28077495098114014,0.43149834871292114,0.289832204580307,0.5
+attn_3_7,3,cal-zh-1,4,0.11534158885478973,0.6260660886764526,0.19559672474861145,31,0.2952086329460144,0.470927894115448,0.30473145842552185,0.53125
+attn_3_7,3,cal-zh-1,5,0.0896417498588562,0.34022626280784607,0.22164911031723022,31,0.2483729124069214,0.3841992914676666,0.256384938955307,0.54375
+attn_3_7,3,cal-zh-1,6,0.1247040405869484,0.7327346801757812,0.26632776856422424,31,0.37061089277267456,0.5524277091026306,0.38256609439849854,0.525
+attn_3_7,3,cal-zh-1,7,0.05083201080560684,0.44692301750183105,0.04695317894220352,31,0.20246444642543793,0.4956357181072235,0.20899559557437897,0.65625
+attn_3_7,3,cal-zh-2,0,0.14007353782653809,0.6036521196365356,0.2504415214061737,31,0.30694448947906494,0.4879894554615021,0.31684595346450806,0.59375
+attn_3_7,3,cal-zh-2,1,0.14963164925575256,0.7174584865570068,0.16050270199775696,31,0.2995006740093231,0.44737952947616577,0.3091619610786438,0.58125
+attn_3_7,3,cal-zh-2,2,0.1916198432445526,0.6128076314926147,0.24314163625240326,31,0.28848910331726074,0.42006126046180725,0.29779520630836487,0.63125
+attn_3_7,3,cal-zh-2,3,0.11550820618867874,0.6839271783828735,0.22851790487766266,31,0.3617430627346039,0.4926488697528839,0.373412162065506,0.43125
+attn_3_7,3,cal-zh-2,4,0.1432671993970871,0.714910089969635,0.16698074340820312,31,0.29421237111091614,0.44453173875808716,0.30370306968688965,0.55
+attn_3_7,3,cal-zh-2,5,0.15627557039260864,0.4676961600780487,0.22456194460391998,31,0.24842627346515656,0.35863032937049866,0.25644001364707947,0.53125
+attn_3_7,3,cal-zh-2,6,0.10968590527772903,0.8703562021255493,0.2677042484283447,31,0.44428783655166626,0.6032188534736633,0.45861977338790894,0.4625
+attn_3_7,3,cal-zh-2,7,0.07148394733667374,0.5658342838287354,0.050552818924188614,31,0.24003271758556366,0.5258023738861084,0.24777570366859436,0.6875
+attn_3_7,3,cal-en-1,0,0.11021165549755096,0.7443452477455139,0.2832939028739929,31,0.4321710467338562,0.5805379152297974,0.4461120665073395,0.5
+attn_3_7,3,cal-en-1,1,0.14157001674175262,0.6908999681472778,0.15555045008659363,31,0.3069734573364258,0.46220970153808594,0.3168758153915405,0.5562499999999999
+attn_3_7,3,cal-en-1,2,0.22632445394992828,0.6502443552017212,0.3039384186267853,31,0.33335912227630615,0.4760979115962982,0.34411266446113586,0.575
+attn_3_7,3,cal-en-1,3,0.1636783331632614,0.749056875705719,0.18588410317897797,31,0.31960076093673706,0.4827522933483124,0.3299104571342468,0.5062500000000001
+attn_3_7,3,cal-en-1,4,0.18975511193275452,0.5745540857315063,0.20487342774868011,31,0.2522231340408325,0.39599278569221497,0.26035937666893005,0.60625
+attn_3_7,3,cal-en-1,5,0.1436786651611328,0.45787984132766724,0.1883384883403778,31,0.2209610939025879,0.3852173388004303,0.2280888706445694,0.6187499999999999
+attn_3_7,3,cal-en-1,6,0.08422182500362396,0.8810427784919739,0.24220214784145355,31,0.4833376407623291,0.6262145638465881,0.49892911314964294,0.45625
+attn_3_7,3,cal-en-1,7,0.09002429246902466,0.49231916666030884,0.053437694907188416,31,0.19592277705669403,0.4695172905921936,0.20224282145500183,0.6624999999999999
+attn_3_7,3,cal-en-2,0,0.09620732069015503,0.7015095949172974,0.2738996148109436,31,0.3950675129890442,0.5772150158882141,0.40781161189079285,0.5874999999999999
+attn_3_7,3,cal-en-2,1,0.1168593019247055,0.7301980257034302,0.160212442278862,31,0.3232686221599579,0.4884706735610962,0.3336966037750244,0.55625
+attn_3_7,3,cal-en-2,2,0.21824826300144196,0.655203104019165,0.3244048058986664,31,0.3314599394798279,0.489634245634079,0.3421522378921509,0.49375
+attn_3_7,3,cal-en-2,3,0.15237370133399963,0.7099946737289429,0.253733366727829,31,0.34891438484191895,0.4875939190387726,0.3601696491241455,0.4
+attn_3_7,3,cal-en-2,4,0.20830751955509186,0.6707655191421509,0.16886863112449646,31,0.23844638466835022,0.41475459933280945,0.24613820016384125,0.58125
+attn_3_7,3,cal-en-2,5,0.20089492201805115,0.4360402226448059,0.20298565924167633,31,0.19701015949249268,0.3117577135562897,0.20336531102657318,0.5875
+attn_3_7,3,cal-en-2,6,0.08693217486143112,0.8322523832321167,0.28768372535705566,31,0.45854610204696655,0.604308545589447,0.47333794832229614,0.4625
+attn_3_7,3,cal-en-2,7,0.12079077959060669,0.5237122178077698,0.08786331117153168,31,0.1976630836725235,0.4211302399635315,0.2040393203496933,0.71875
+attn_3_7,7,cal-zh-1,0,0.08316324651241302,0.1214274913072586,0.22053609788417816,31,0.2072795033454895,0.32793501019477844,0.21396592259407043,0.5625
+attn_3_7,7,cal-zh-1,1,0.11647684127092361,0.187703475356102,0.09203154593706131,31,0.09238047897815704,0.16611215472221375,0.09536049515008926,0.66875
+attn_3_7,7,cal-zh-1,2,0.06550130248069763,0.07431766390800476,0.28803399205207825,31,0.2721514105796814,0.46315261721611023,0.28093045949935913,0.47500000000000003
+attn_3_7,7,cal-zh-1,3,0.10703553259372711,0.24631237983703613,0.1426287740468979,31,0.150299072265625,0.26700741052627563,0.1551474630832672,0.56875
+attn_3_7,7,cal-zh-1,4,0.04052526503801346,0.28515881299972534,0.11374201625585556,31,0.1819339096546173,0.2913167476654053,0.1878027319908142,0.5875
+attn_3_7,7,cal-zh-1,5,0.03951618820428848,0.12756675481796265,0.12323599308729172,31,0.13915526866912842,0.22263947129249573,0.14364413917064667,0.6062500000000001
+attn_3_7,7,cal-zh-1,6,0.09479819983243942,0.28205406665802,0.14449526369571686,31,0.16065017879009247,0.30267569422721863,0.16583245992660522,0.6375000000000001
+attn_3_7,7,cal-zh-1,7,0.031424712389707565,0.10597768425941467,0.029422910884022713,31,0.05383709818124771,0.10532202571630478,0.055573780089616776,0.60625
+attn_3_7,7,cal-zh-2,0,0.0902467742562294,0.11627861112356186,0.21221575140953064,31,0.19495421648025513,0.317206472158432,0.201243057847023,0.6187499999999999
+attn_3_7,7,cal-zh-2,1,0.11616441607475281,0.1248721182346344,0.0952492356300354,31,0.08693543076515198,0.16702036559581757,0.08973980695009232,0.5874999999999999
+attn_3_7,7,cal-zh-2,2,0.11578349769115448,0.08946777880191803,0.2738611102104187,31,0.2561575174331665,0.45070335268974304,0.26442068815231323,0.5
+attn_3_7,7,cal-zh-2,3,0.11586448550224304,0.16896863281726837,0.10842260718345642,31,0.10158580541610718,0.21354182064533234,0.1048627570271492,0.60625
+attn_3_7,7,cal-zh-2,4,0.044329218566417694,0.22534678876399994,0.15060655772686005,31,0.19093912839889526,0.3356643319129944,0.19709844887256622,0.61875
+attn_3_7,7,cal-zh-2,5,0.05062376335263252,0.12924723327159882,0.1047384962439537,31,0.11341066658496857,0.2176104635000229,0.1170690730214119,0.6312500000000001
+attn_3_7,7,cal-zh-2,6,0.08418147265911102,0.28032630681991577,0.1446688026189804,31,0.16730237007141113,0.24755442142486572,0.1726992279291153,0.5125
+attn_3_7,7,cal-zh-2,7,0.0315190926194191,0.11244469881057739,0.02086171880364418,31,0.04779163748025894,0.10416773706674576,0.04933330416679382,0.65
+attn_3_7,7,cal-en-1,0,0.08185911923646927,0.10868857800960541,0.24351894855499268,31,0.22597578167915344,0.35052549839019775,0.23326534032821655,0.66875
+attn_3_7,7,cal-en-1,1,0.12180840969085693,0.12996530532836914,0.1148008182644844,31,0.10937604308128357,0.1990041583776474,0.11290428787469864,0.6625
+attn_3_7,7,cal-en-1,2,0.12156520038843155,0.07304315268993378,0.32663044333457947,31,0.31218329071998596,0.4966602325439453,0.3222537338733673,0.4375
+attn_3_7,7,cal-en-1,3,0.09531049430370331,0.21274563670158386,0.13099102675914764,31,0.14007462561130524,0.2635233998298645,0.14459317922592163,0.5874999999999999
+attn_3_7,7,cal-en-1,4,0.051348961889743805,0.27256035804748535,0.17027397453784943,31,0.22669382393360138,0.33905521035194397,0.23400652408599854,0.59375
+attn_3_7,7,cal-en-1,5,0.05210518091917038,0.14343655109405518,0.11446496844291687,31,0.12723194062709808,0.23429512977600098,0.13133619725704193,0.64375
+attn_3_7,7,cal-en-1,6,0.09278014302253723,0.3040335178375244,0.20964911580085754,31,0.21833369135856628,0.34637004137039185,0.22537672519683838,0.53125
+attn_3_7,7,cal-en-1,7,0.031563661992549896,0.11319160461425781,0.02739104814827442,31,0.053969405591487885,0.1002146303653717,0.055710360407829285,0.5625000000000001
+attn_3_7,7,cal-en-2,0,0.07943743467330933,0.1426258534193039,0.21740850806236267,31,0.21001917123794556,0.35335251688957214,0.2167939990758896,0.58125
+attn_3_7,7,cal-en-2,1,0.13507506251335144,0.15443003177642822,0.10350806266069412,31,0.09826720505952835,0.18229186534881592,0.10143710672855377,0.575
+attn_3_7,7,cal-en-2,2,0.07290980964899063,0.08657881617546082,0.26718389987945557,31,0.259122759103775,0.4706645905971527,0.26748156547546387,0.45625
+attn_3_7,7,cal-en-2,3,0.06336543709039688,0.2253912389278412,0.11350395530462265,31,0.1445191204547882,0.30314192175865173,0.1491810381412506,0.5875
+attn_3_7,7,cal-en-2,4,0.04387465864419937,0.2312193661928177,0.14195290207862854,31,0.1880926638841629,0.3466520309448242,0.19416013360023499,0.5125
+attn_3_7,7,cal-en-2,5,0.05407806113362312,0.1308087259531021,0.15231285989284515,31,0.16046105325222015,0.3131849467754364,0.16563721001148224,0.5874999999999999
+attn_3_7,7,cal-en-2,6,0.10013270378112793,0.30412447452545166,0.17993809282779694,31,0.19388216733932495,0.34072187542915344,0.2001364380121231,0.5437500000000001
+attn_3_7,7,cal-en-2,7,0.03161787986755371,0.14472994208335876,0.04679951071739197,31,0.0830649882555008,0.13866016268730164,0.08574451506137848,0.65625
+attn_11_15,11,cal-zh-1,0,0.2780076861381531,0.40727436542510986,0.08935041725635529,31,0.08769044280052185,0.1605687141418457,0.09051916748285294,0.6937500000000001
+attn_11_15,11,cal-zh-1,1,0.599311351776123,0.1917644888162613,0.1712544858455658,31,0.22863143682479858,0.39283493161201477,0.23600666224956512,0.5750000000000001
+attn_11_15,11,cal-zh-1,2,0.15545204281806946,0.5101379156112671,0.08800674974918365,31,0.15655069053173065,0.3642641007900238,0.16160069406032562,0.73125
+attn_11_15,11,cal-zh-1,3,0.11032958328723907,0.4199199676513672,0.08535198122262955,31,0.14290741086006165,0.27922654151916504,0.14751732349395752,0.7
+attn_11_15,11,cal-zh-1,4,0.18725907802581787,0.3965998888015747,0.12378748506307602,31,0.1280820369720459,0.24588972330093384,0.13221368193626404,0.6937500000000001
+attn_11_15,11,cal-zh-1,5,0.3307006359100342,0.43053850531578064,0.12539583444595337,31,0.1220257431268692,0.27262142300605774,0.1259620636701584,0.6187499999999999
+attn_11_15,11,cal-zh-1,6,0.17290356755256653,0.4159870147705078,0.10268668830394745,31,0.12616759538650513,0.27286267280578613,0.13023751974105835,0.625
+attn_11_15,11,cal-zh-1,7,0.12546084821224213,0.4914926290512085,0.11725502461194992,31,0.17813995480537415,0.28658801317214966,0.18388642370700836,0.6625000000000001
+attn_11_15,11,cal-zh-2,0,0.2187591940164566,0.42793723940849304,0.09548557549715042,31,0.12088388949632645,0.2345586121082306,0.12478335946798325,0.6875
+attn_11_15,11,cal-zh-2,1,0.5546900033950806,0.1175224781036377,0.16409580409526825,31,0.25557559728622437,0.37492382526397705,0.26381993293762207,0.6125
+attn_11_15,11,cal-zh-2,2,0.11856245994567871,0.44309669733047485,0.08432265371084213,31,0.1634930670261383,0.37660637497901917,0.16876700520515442,0.65
+attn_11_15,11,cal-zh-2,3,0.09019850194454193,0.3149072527885437,0.09793759882450104,31,0.1334252804517746,0.24160735309123993,0.13772933185100555,0.71875
+attn_11_15,11,cal-zh-2,4,0.09969134628772736,0.20464956760406494,0.10986322164535522,31,0.11441972851753235,0.2605915665626526,0.11811067909002304,0.7
+attn_11_15,11,cal-zh-2,5,0.16705867648124695,0.412482887506485,0.20107370615005493,31,0.21316100656986237,0.44814038276672363,0.22003716230392456,0.6000000000000001
+attn_11_15,11,cal-zh-2,6,0.126181960105896,0.38109123706817627,0.10754827409982681,31,0.14080098271369934,0.2554462254047394,0.14534294605255127,0.6687500000000001
+attn_11_15,11,cal-zh-2,7,0.11085975170135498,0.5181387662887573,0.10885247588157654,31,0.2060571014881134,0.35480910539627075,0.2127041071653366,0.66875
+attn_11_15,11,cal-en-1,0,0.2882137894630432,0.45660895109176636,0.07201986759901047,31,0.08352888375520706,0.1487027406692505,0.08622336387634277,0.73125
+attn_11_15,11,cal-en-1,1,0.597124457359314,0.13483837246894836,0.22424918413162231,31,0.29736971855163574,0.529407799243927,0.3069622814655304,0.51875
+attn_11_15,11,cal-en-1,2,0.17265087366104126,0.4521681070327759,0.08010748028755188,31,0.13888895511627197,0.29661378264427185,0.14336924254894257,0.7375
+attn_11_15,11,cal-en-1,3,0.10834433138370514,0.319663405418396,0.08455760031938553,31,0.11557236313819885,0.24554768204689026,0.11930052191019058,0.84375
+attn_11_15,11,cal-en-1,4,0.157099187374115,0.3456929326057434,0.10397545993328094,31,0.11958309262990952,0.20836150646209717,0.12344060838222504,0.73125
+attn_11_15,11,cal-en-1,5,0.30003029108047485,0.3683816194534302,0.11273742467164993,31,0.11081759631633759,0.26781079173088074,0.11439235508441925,0.6875
+attn_11_15,11,cal-en-1,6,0.16862526535987854,0.5013703107833862,0.12122618407011032,31,0.1663668006658554,0.29310786724090576,0.17173348367214203,0.65
+attn_11_15,11,cal-en-1,7,0.14973583817481995,0.5537562966346741,0.1512605994939804,31,0.22106251120567322,0.3912945091724396,0.22819358110427856,0.64375
+attn_11_15,11,cal-en-2,0,0.3351830840110779,0.4993361234664917,0.08562158793210983,31,0.08981436491012573,0.19977226853370667,0.09271162003278732,0.675
+attn_11_15,11,cal-en-2,1,0.5792163610458374,0.1499037742614746,0.1789509803056717,31,0.2536986768245697,0.41500288248062134,0.2618825435638428,0.55
+attn_11_15,11,cal-en-2,2,0.20357520878314972,0.4823607802391052,0.11572320759296417,31,0.16091257333755493,0.35024091601371765,0.1661033034324646,0.6000000000000001
+attn_11_15,11,cal-en-2,3,0.11477383971214294,0.4060356318950653,0.08803554624319077,31,0.14013110101222992,0.2660136818885803,0.14465147256851196,0.75
+attn_11_15,11,cal-en-2,4,0.1895177811384201,0.2564787268638611,0.1033802479505539,31,0.10677468031644821,0.19931447505950928,0.11021901667118073,0.7125
+attn_11_15,11,cal-en-2,5,0.2725520133972168,0.3527078926563263,0.1301337331533432,31,0.12496069073677063,0.22729209065437317,0.12899169325828552,0.6312500000000001
+attn_11_15,11,cal-en-2,6,0.17815515398979187,0.4918685555458069,0.08680415898561478,31,0.1395310014486313,0.22975514829158783,0.14403200149536133,0.68125
+attn_11_15,11,cal-en-2,7,0.1471988707780838,0.4305167496204376,0.15007153153419495,31,0.1933930218219757,0.31820762157440186,0.1996314972639084,0.60625
+attn_11_15,15,cal-zh-1,0,0.20194272696971893,0.35520243644714355,0.1205381229519844,31,0.13211002945899963,0.2777846157550812,0.1363716572523117,0.63125
+attn_11_15,15,cal-zh-1,1,0.23890593647956848,0.12377866357564926,0.21396537125110626,31,0.21463359892368317,0.41781342029571533,0.22155724465847015,0.6062500000000001
+attn_11_15,15,cal-zh-1,2,0.13280686736106873,0.27719253301620483,0.1377590000629425,31,0.14551271498203278,0.2994697093963623,0.15020668506622314,0.65
+attn_11_15,15,cal-zh-1,3,0.04135940596461296,0.23285867273807526,0.21063239872455597,31,0.24926505982875824,0.45159438252449036,0.2573058605194092,0.6937500000000001
+attn_11_15,15,cal-zh-1,4,0.07648462802171707,0.4026046395301819,0.1620333045721054,31,0.23594452440738678,0.4158128798007965,0.24355560541152954,0.6500000000000001
+attn_11_15,15,cal-zh-1,5,0.14696139097213745,0.41105949878692627,0.1549629122018814,31,0.18883343040943146,0.3439426124095917,0.19492481648921967,0.6187499999999999
+attn_11_15,15,cal-zh-1,6,0.0764666199684143,0.34140151739120483,0.11151057481765747,31,0.16509875655174255,0.31560462713241577,0.17042450606822968,0.6625
+attn_11_15,15,cal-zh-1,7,0.10250869393348694,0.5306729078292847,0.16751603782176971,31,0.25549793243408203,0.3858420252799988,0.26373982429504395,0.6062500000000001
+attn_11_15,15,cal-zh-2,0,0.22696512937545776,0.35936564207077026,0.14069660007953644,31,0.13378489017486572,0.28451812267303467,0.1381005346775055,0.65
+attn_11_15,15,cal-zh-2,1,0.25547248125076294,0.10696703940629959,0.23399728536605835,31,0.24166268110275269,0.39466142654418945,0.24945826828479767,0.5875
+attn_11_15,15,cal-zh-2,2,0.17822568118572235,0.2864110469818115,0.08877820521593094,31,0.10718775540590286,0.2511947453022003,0.11064542829990387,0.6937500000000001
+attn_11_15,15,cal-zh-2,3,0.04604611545801163,0.23604938387870789,0.22704336047172546,31,0.2641073167324066,0.4154779016971588,0.2726269066333771,0.70625
+attn_11_15,15,cal-zh-2,4,0.08757959306240082,0.2918083965778351,0.22551555931568146,31,0.24939756095409393,0.48551467061042786,0.2574426531791687,0.6000000000000001
+attn_11_15,15,cal-zh-2,5,0.11609973013401031,0.3603805601596832,0.21242640912532806,31,0.23965227603912354,0.4984361231327057,0.2473829984664917,0.51875
+attn_11_15,15,cal-zh-2,6,0.0702231377363205,0.29952526092529297,0.12570251524448395,31,0.17031171917915344,0.32249733805656433,0.17580564320087433,0.6125
+attn_11_15,15,cal-zh-2,7,0.0856853500008583,0.4777143895626068,0.1792273223400116,31,0.27432966232299805,0.47331610321998596,0.28317901492118835,0.65625
+attn_11_15,15,cal-en-1,0,0.2137582153081894,0.36318275332450867,0.121327243745327,31,0.12573851644992828,0.2549639940261841,0.12979458272457123,0.64375
+attn_11_15,15,cal-en-1,1,0.31107524037361145,0.1114300861954689,0.24769999086856842,31,0.28352850675582886,0.4685184955596924,0.29267460107803345,0.53125
+attn_11_15,15,cal-en-1,2,0.17990511655807495,0.29312068223953247,0.12414757162332535,31,0.12930452823638916,0.3116541802883148,0.1334756314754486,0.74375
+attn_11_15,15,cal-en-1,3,0.048217516392469406,0.23240914940834045,0.20335303246974945,31,0.2404388189315796,0.3818458914756775,0.2481948882341385,0.70625
+attn_11_15,15,cal-en-1,4,0.10281141847372055,0.38790467381477356,0.18024703860282898,31,0.22299107909202576,0.4459393620491028,0.23018434643745422,0.70625
+attn_11_15,15,cal-en-1,5,0.15091386437416077,0.3693487346172333,0.1916365921497345,31,0.21596358716487885,0.39266589283943176,0.2229301631450653,0.53125
+attn_11_15,15,cal-en-1,6,0.0790158361196518,0.32175618410110474,0.11456245183944702,31,0.16404178738594055,0.30913954973220825,0.16933345794677734,0.6875
+attn_11_15,15,cal-en-1,7,0.10605430603027344,0.5771000385284424,0.18373095989227295,31,0.28234609961509705,0.4291541278362274,0.2914540469646454,0.625
+attn_11_15,15,cal-en-2,0,0.251964807510376,0.31455427408218384,0.10559824854135513,31,0.11457027494907379,0.24910703301429749,0.11826609075069427,0.675
+attn_11_15,15,cal-en-2,1,0.31838059425354004,0.11868949234485626,0.257511168718338,31,0.2719646394252777,0.44276586174964905,0.2807376980781555,0.5687500000000001
+attn_11_15,15,cal-en-2,2,0.26772984862327576,0.33754318952560425,0.0924711525440216,31,0.09108234941959381,0.191604882478714,0.09402050077915192,0.7375
+attn_11_15,15,cal-en-2,3,0.06395735591650009,0.2933264374732971,0.20108558237552643,31,0.2424471080303192,0.4563605785369873,0.25026798248291016,0.66875
+attn_11_15,15,cal-en-2,4,0.08209535479545593,0.3278740644454956,0.1982748657464981,31,0.24514394998550415,0.3975651264190674,0.2530518174171448,0.56875
+attn_11_15,15,cal-en-2,5,0.09166160225868225,0.44504809379577637,0.17847460508346558,31,0.2476060837507248,0.3910382390022278,0.2555933892726898,0.4875
+attn_11_15,15,cal-en-2,6,0.07513932138681412,0.45416805148124695,0.12279381603002548,31,0.22479453682899475,0.34578394889831543,0.2320459932088852,0.59375
+attn_11_15,15,cal-en-2,7,0.1007680892944336,0.5731154084205627,0.18980540335178375,31,0.3043324649333954,0.41584962606430054,0.3141496479511261,0.5375000000000001
+attn_19_23,19,cal-zh-1,0,0.0955502837896347,0.03435931354761124,0.08661527186632156,31,0.10083474218845367,0.1792067289352417,0.10408748686313629,0.73125
+attn_19_23,19,cal-zh-1,1,0.10103033483028412,0.04235740005970001,0.16347354650497437,31,0.16801373660564423,0.37647515535354614,0.17343351244926453,0.6125
+attn_19_23,19,cal-zh-1,2,0.047133591026067734,0.03135885298252106,0.47210854291915894,31,0.46042412519454956,0.6502387523651123,0.47527655959129333,0.5375000000000001
+attn_19_23,19,cal-zh-1,3,0.07069262862205505,0.06008598208427429,0.12148955464363098,31,0.11885681003332138,0.26471832394599915,0.12269090116024017,0.6625000000000001
+attn_19_23,19,cal-zh-1,4,0.10007724165916443,0.044495031237602234,0.16073106229305267,31,0.16553831100463867,0.3016175925731659,0.17087827622890472,0.68125
+attn_19_23,19,cal-zh-1,5,0.06431633979082108,0.040679484605789185,0.0816011056303978,31,0.08355634659528732,0.1771341860294342,0.08625171333551407,0.8
+attn_19_23,19,cal-zh-1,6,0.08145896345376968,0.03135719522833824,0.4502938687801361,31,0.44716787338256836,0.6495819091796875,0.4615926444530487,0.5
+attn_19_23,19,cal-zh-1,7,0.08670078217983246,0.03888973593711853,0.09126521646976471,31,0.09830940514802933,0.18466481566429138,0.10148067027330399,0.70625
+attn_19_23,19,cal-zh-2,0,0.06336566060781479,0.03736908361315727,0.07764910906553268,31,0.08064413070678711,0.16660116612911224,0.08324555307626724,0.8187500000000001
+attn_19_23,19,cal-zh-2,1,0.0888693630695343,0.06530671566724777,0.1942276507616043,31,0.18916305899620056,0.33451300859451294,0.19526511430740356,0.6125
+attn_19_23,19,cal-zh-2,2,0.07010967284440994,0.031316906213760376,0.3875632584095001,31,0.3861002027988434,0.6248269081115723,0.3985550105571747,0.65
+attn_19_23,19,cal-zh-2,3,0.08096092939376831,0.08616353571414948,0.08690906316041946,31,0.08501926809549332,0.19463855028152466,0.08776183426380157,0.7937500000000001
+attn_19_23,19,cal-zh-2,4,0.0936945378780365,0.04400797560811043,0.11036926507949829,31,0.11584575474262238,0.2122059315443039,0.11958272755146027,0.8187500000000001
+attn_19_23,19,cal-zh-2,5,0.056567154824733734,0.03972743824124336,0.06345413625240326,31,0.06394530087709427,0.16273385286331177,0.066008061170578,0.7937500000000001
+attn_19_23,19,cal-zh-2,6,0.08575102686882019,0.031342219561338425,0.4025454819202423,31,0.4032740592956543,0.5980108976364136,0.4162829518318176,0.60625
+attn_19_23,19,cal-zh-2,7,0.10350999981164932,0.0646558329463005,0.10814103484153748,31,0.10938012599945068,0.23185187578201294,0.11290851980447769,0.7375
+attn_19_23,19,cal-en-1,0,0.22517549991607666,0.09088313579559326,0.11986515671014786,31,0.1366778463125229,0.2391858845949173,0.14108680188655853,0.74375
+attn_19_23,19,cal-en-1,1,0.23973168432712555,0.11891130357980728,0.17766058444976807,31,0.16800963878631592,0.2676515281200409,0.17342931032180786,0.66875
+attn_19_23,19,cal-en-1,2,0.21452879905700684,0.03307320922613144,0.3807314932346344,31,0.4102427363395691,0.5871645212173462,0.42347636818885803,0.66875
+attn_19_23,19,cal-en-1,3,0.21709948778152466,0.14377164840698242,0.0770971029996872,31,0.08144749701023102,0.15173088014125824,0.08407483249902725,0.8125
+attn_19_23,19,cal-en-1,4,0.23791193962097168,0.19146007299423218,0.12041598558425903,31,0.11442764848470688,0.22197961807250977,0.11811887472867966,0.66875
+attn_19_23,19,cal-en-1,5,0.2485853135585785,0.10372771322727203,0.08068431913852692,31,0.11313603073358536,0.23292472958564758,0.11678558588027954,0.7562500000000001
+attn_19_23,19,cal-en-1,6,0.31094634532928467,0.031405214220285416,0.39337676763534546,31,0.45084744691848755,0.6603637933731079,0.4653908908367157,0.55
+attn_19_23,19,cal-en-1,7,0.2927525043487549,0.15012584626674652,0.12347250431776047,31,0.12942291796207428,0.2467067539691925,0.13359786570072174,0.68125
+attn_19_23,19,cal-en-2,0,0.19307208061218262,0.055295493453741074,0.1307787448167801,31,0.15591834485530853,0.3169764280319214,0.1609479784965515,0.5875
+attn_19_23,19,cal-en-2,1,0.1491451859474182,0.10034040361642838,0.14188656210899353,31,0.1398725062608719,0.2532970607280731,0.14438451826572418,0.59375
+attn_19_23,19,cal-en-2,2,0.11011949181556702,0.033411312848329544,0.4344772696495056,31,0.4410479664802551,0.6305621266365051,0.4552752673625946,0.5375
+attn_19_23,19,cal-en-2,3,0.11554233729839325,0.08585710823535919,0.1385604739189148,31,0.13558831810951233,0.3369371294975281,0.13996215164661407,0.5874999999999999
+attn_19_23,19,cal-en-2,4,0.1721733808517456,0.10996092110872269,0.1335446536540985,31,0.13328826427459717,0.24826908111572266,0.13758789002895355,0.625
+attn_19_23,19,cal-en-2,5,0.18817278742790222,0.06915841996669769,0.11790769547224045,31,0.13643309473991394,0.2266169786453247,0.14083418250083923,0.65625
+attn_19_23,19,cal-en-2,6,0.2252667248249054,0.03137722611427307,0.45010486245155334,31,0.4878306984901428,0.6629999876022339,0.503567099571228,0.49375
+attn_19_23,19,cal-en-2,7,0.19858212769031525,0.07953452318906784,0.1386381983757019,31,0.15347915887832642,0.2583900988101959,0.15843011438846588,0.5687500000000001
+attn_19_23,23,cal-zh-1,0,0.18854272365570068,0.06906979531049728,0.08094117045402527,31,0.10190774500370026,0.19865523278713226,0.10519509017467499,0.61875
+attn_19_23,23,cal-zh-1,1,0.12637057900428772,0.094294473528862,0.11472687125205994,31,0.11000899970531464,0.20533926784992218,0.11355767399072647,0.60625
+attn_19_23,23,cal-zh-1,2,0.06566040217876434,0.0320066399872303,0.3936012387275696,31,0.3894338607788086,0.6095475554466248,0.40199628472328186,0.53125
+attn_19_23,23,cal-zh-1,3,0.10236738622188568,0.07628713548183441,0.0909285917878151,31,0.09045256674289703,0.16597077250480652,0.09337039291858673,0.65625
+attn_19_23,23,cal-zh-1,4,0.11772166937589645,0.09994609653949738,0.11158378422260284,31,0.1082884818315506,0.2333190143108368,0.11178164929151535,0.6000000000000001
+attn_19_23,23,cal-zh-1,5,0.1445631980895996,0.05731264501810074,0.06436286121606827,31,0.07913714647293091,0.15170255303382874,0.08168993890285492,0.6625000000000001
+attn_19_23,23,cal-zh-1,6,0.13692563772201538,0.03156186267733574,0.42265841364860535,31,0.43705910444259644,0.6045969128608704,0.45115774869918823,0.5249999999999999
+attn_19_23,23,cal-zh-1,7,0.17913174629211426,0.0774894654750824,0.055066999047994614,31,0.07003086805343628,0.11456935852766037,0.07228992879390717,0.6875
+attn_19_23,23,cal-zh-2,0,0.08711955696344376,0.08616629242897034,0.09776511043310165,31,0.09617017209529877,0.2418135702610016,0.09927243739366531,0.7125
+attn_19_23,23,cal-zh-2,1,0.1001269742846489,0.1142689436674118,0.11429562419652939,31,0.11118914932012558,0.19028690457344055,0.1147758960723877,0.6625000000000001
+attn_19_23,23,cal-zh-2,2,0.13165870308876038,0.03296923264861107,0.32159483432769775,31,0.34172311425209045,0.5507174134254456,0.35274645686149597,0.675
+attn_19_23,23,cal-zh-2,3,0.10557965934276581,0.08546337485313416,0.10869138687849045,31,0.10796000063419342,0.267814040184021,0.11144257336854935,0.71875
+attn_19_23,23,cal-zh-2,4,0.0753588005900383,0.0625741258263588,0.09596671164035797,31,0.09504459798336029,0.18062597513198853,0.09811054915189743,0.6937500000000001
+attn_19_23,23,cal-zh-2,5,0.0798945426940918,0.0568334236741066,0.07381810247898102,31,0.0762210339307785,0.13966575264930725,0.07867977768182755,0.74375
+attn_19_23,23,cal-zh-2,6,0.0759878009557724,0.031536251306533813,0.39428046345710754,31,0.3959207534790039,0.5929224491119385,0.408692330121994,0.6375
+attn_19_23,23,cal-zh-2,7,0.11138643324375153,0.07136214524507523,0.07186198234558105,31,0.07652410864830017,0.15722090005874634,0.07899262756109238,0.7312500000000001
+attn_19_23,23,cal-en-1,0,0.12444712221622467,0.0752042829990387,0.10921256989240646,31,0.10872898995876312,0.22416618466377258,0.11223636567592621,0.6875
+attn_19_23,23,cal-en-1,1,0.10719993710517883,0.08359251916408539,0.12698784470558167,31,0.12113931775093079,0.22020384669303894,0.12504702806472778,0.7000000000000001
+attn_19_23,23,cal-en-1,2,0.13445816934108734,0.03345245122909546,0.3665089011192322,31,0.3763067126274109,0.5767470002174377,0.3884456157684326,0.65625
+attn_19_23,23,cal-en-1,3,0.10536500066518784,0.07050738483667374,0.07684175670146942,31,0.07960161566734314,0.14911630749702454,0.08216940611600876,0.78125
+attn_19_23,23,cal-en-1,4,0.0911213681101799,0.17202866077423096,0.10298387706279755,31,0.10929879546165466,0.20484577119350433,0.11282456666231155,0.61875
+attn_19_23,23,cal-en-1,5,0.18703660368919373,0.12409132719039917,0.09463418275117874,31,0.09024524688720703,0.15500988066196442,0.09315639734268188,0.70625
+attn_19_23,23,cal-en-1,6,0.13999314606189728,0.03152035176753998,0.4262007474899292,31,0.43630746006965637,0.6232555508613586,0.450381875038147,0.5187499999999999
+attn_19_23,23,cal-en-1,7,0.11322036385536194,0.10936357825994492,0.07323051989078522,31,0.07213389873504639,0.1506829559803009,0.07446079701185226,0.6125
+attn_19_23,23,cal-en-2,0,0.11863207072019577,0.13078393042087555,0.09141960740089417,31,0.08733218908309937,0.1577986180782318,0.09014936536550522,0.59375
+attn_19_23,23,cal-en-2,1,0.07288858294487,0.1372600495815277,0.1089932844042778,31,0.11363835632801056,0.1698836386203766,0.11730410903692245,0.50625
+attn_19_23,23,cal-en-2,2,0.07288709282875061,0.04088134318590164,0.376287043094635,31,0.3685654401779175,0.5480253100395203,0.3804546594619751,0.5375
+attn_19_23,23,cal-en-2,3,0.06728185713291168,0.07526300847530365,0.09231049567461014,31,0.08894366025924683,0.15198129415512085,0.09181280434131622,0.6187499999999999
+attn_19_23,23,cal-en-2,4,0.09862713515758514,0.12817975878715515,0.09404788166284561,31,0.08950259536504745,0.16771778464317322,0.09238976985216141,0.5562499999999999
+attn_19_23,23,cal-en-2,5,0.07161959260702133,0.0855005532503128,0.10532733052968979,31,0.10144606977701187,0.1908321976661682,0.10471852868795395,0.5625
+attn_19_23,23,cal-en-2,6,0.08469876646995544,0.03168217092752457,0.4412229359149933,31,0.43726012110710144,0.6215424537658691,0.45136526226997375,0.5437500000000001
+attn_19_23,23,cal-en-2,7,0.10539951920509338,0.08276306837797165,0.07327506691217422,31,0.0738329142332077,0.13249149918556213,0.07621461898088455,0.63125

results/cpu-qwen35-2b/audit-validation.json

@@ -0,0 +1,12 @@
+{
+  "passed": true,
+  "validated_unix": 1794204667.0190518,
+  "checksummed_files": 83,
+  "proof_recomputed_from_retained_weights": true,
+  "heldout_nll_recomputed_from_saved_arrays": true,
+  "weights_present_and_retained": true,
+  "archive_excludes_weights_and_tokenizer": true,
+  "delivery_excludes_external_activation_caches": true,
+  "full_experiment_checksums": "checksums-full-server.json",
+  "figure_source_tables_and_token_nll_complete": true
+}

results/cpu-qwen35-2b/calibration-metrics.csv

@@ -0,0 +1,89 @@
+candidate,kind,source,sample_id,split,relative_l2,reverse_relative_l2,token_relative_p95,reverse_token_relative_p95,cosine,low_norm_tokens,first_key_mass_a,first_key_mass_b,js_without_first_key,without_first_key_valid_rows,js_mean,js_p95,js_without_first_query,top5_overlap
+attn_3_7,attn,3,cal-zh-1,calibration,1.75886973383732,1.2170194989630583,2.3326619869739242,1.3829599318796777,-0.001709405865837512,0,0.12333933264017105,0.5467976927757263,0.21484676003456116,248,0.2868691682815552,0.4794144630432129,0.2961229979991913,0.5515625
+attn_3_7,attn,3,cal-zh-2,calibration,2.0666622735799773,1.1397562572129543,2.6608388071181586,1.2319401946881372,0.004627105486643174,0,0.1346932351589203,0.6545802354812622,0.1990504413843155,248,0.3104545772075653,0.512997567653656,0.3204692304134369,0.55859375
+attn_3_7,attn,3,cal-en-1,calibration,1.9684586970451095,1.1687255334183004,2.5460869484043305,1.3209705033413297,-0.011291865355731309,0,0.1436830461025238,0.6550427675247192,0.2021898329257965,248,0.31806862354278564,0.5598381161689758,0.3283289074897766,0.56015625
+attn_3_7,attn,3,cal-en-2,calibration,2.1157833337453664,1.1413057345289206,2.896573536355606,1.279392467480941,-0.010752011897128681,0,0.1500767469406128,0.6574594974517822,0.2199564427137375,248,0.3112969994544983,0.5577057003974915,0.32133883237838745,0.5484375
+attn_3_7,attn,7,cal-zh-1,calibration,1.3340187814155438,1.42344327376934,1.4289276609634798,1.638356412458448,0.05265559345262384,0,0.07230515778064728,0.17881479859352112,0.144265815615654,248,0.15721087157726288,0.3267841339111328,0.16228218376636505,0.5890625
+attn_3_7,attn,7,cal-zh-2,calibration,1.3371723133481364,1.418275178551954,1.4336050300809013,1.6659403627988982,0.05349503065318622,0,0.081089086830616,0.1558690071105957,0.13882805407047272,248,0.1448846012353897,0.3248489499092102,0.14955829083919525,0.590625
+attn_3_7,attn,7,cal-en-1,calibration,1.405999268639987,1.3246539988272832,1.532711571208276,1.4346161336828325,0.07054514898566913,0,0.08104264736175537,0.16970808804035187,0.1672150194644928,248,0.17672982811927795,0.38887327909469604,0.18243078887462616,0.5859375
+attn_3_7,attn,7,cal-en-2,calibration,1.366473684595312,1.4009631878257847,1.4664096677417215,1.59944923669573,0.0431210172272288,0,0.07256138324737549,0.1774885654449463,0.1528259664773941,248,0.16717864573001862,0.3629882335662842,0.17257151007652283,0.5625
+attn_11_15,attn,11,cal-zh-1,calibration,2.7606058538571316,0.9703348793344666,2.4183673572578086,1.0857279648648543,0.2588922604193846,0,0.24492809176445007,0.4079643785953522,0.11288609355688095,248,0.1462744027376175,0.3241320848464966,0.15099292993545532,0.6625
+attn_11_15,attn,11,cal-zh-2,calibration,2.5044424857870546,0.9843061509576916,2.592796177795181,1.1478459097975344,0.23612970738935357,0,0.18575024604797363,0.3524782657623291,0.12114741653203964,248,0.16847708821296692,0.3656128942966461,0.17391183972358704,0.66328125
+attn_11_15,attn,11,cal-en-1,calibration,2.649979511635331,0.9795891219874814,2.4632888494810317,1.1094207865752836,0.23948147120552113,0,0.24272800981998444,0.39156001806259155,0.11876673251390457,248,0.15664874017238617,0.337306410074234,0.16170193254947662,0.6929687500000001
+attn_11_15,attn,11,cal-en-2,calibration,2.7191409308620766,0.9759398005295231,2.5703854656788505,1.1257459370791087,0.24568690002591262,0,0.2525215446949005,0.3836510181427002,0.11734011769294739,248,0.15115201473236084,0.3331157863140106,0.1560278832912445,0.65078125
+attn_11_15,attn,15,cal-zh-1,calibration,3.7267750977169203,0.9943195388381949,1.8571948100572258,1.4659544762265972,0.15463241076860404,0,0.12717953324317932,0.33434635400772095,0.15986472368240356,248,0.19836202263832092,0.40538591146469116,0.20476080477237701,0.6398437499999999
+attn_11_15,attn,15,cal-zh-2,calibration,3.484117411382163,0.9958762023786318,1.9555333671772042,1.5034310789069971,0.15731411020281283,0,0.13328716158866882,0.30227771401405334,0.17917340993881226,248,0.21005423367023468,0.41705968976020813,0.21683017909526825,0.628125
+attn_11_15,attn,15,cal-en-1,calibration,3.5714882610996845,0.9941664364521509,1.8393702843739006,1.5570787013529828,0.16007660497298756,0,0.14896893501281738,0.3320315480232239,0.17083808779716492,248,0.2080441117286682,0.4197816848754883,0.21475520730018616,0.646875
+attn_11_15,attn,15,cal-en-2,calibration,3.783713712256275,0.9885545400787986,2.0440754255967826,1.5107488942763196,0.17418988407724514,0,0.15646213293075562,0.35803988575935364,0.1682518720626831,248,0.2177426666021347,0.419617235660553,0.22476662695407867,0.6046875
+attn_19_23,attn,19,cal-zh-1,calibration,3.1071992972679547,1.080818751446137,3.9981135079246157,1.1800456131328154,-0.06780946108706973,0,0.08087001740932465,0.04044787213206291,0.20344728231430054,248,0.20533767342567444,0.5970298051834106,0.21196146309375763,0.65390625
+attn_19_23,attn,19,cal-zh-2,calibration,3.5223723842603682,1.066376496758468,4.7369591160509135,1.1555630164896329,-0.07515431162135316,0,0.08035354316234589,0.04998621344566345,0.17885737121105194,248,0.17917148768901825,0.5459575057029724,0.18495121598243713,0.72890625
+attn_19_23,attn,19,cal-en-1,calibration,3.7338041826396506,1.0390406833474568,4.724166114393082,1.1075162239519407,-0.0038920109670231525,0,0.24834147095680237,0.10791976749897003,0.18416298925876617,248,0.20052647590637207,0.5714813470840454,0.20699506998062134,0.69375
+attn_19_23,attn,19,cal-en-2,calibration,3.3431946049591024,1.0618542149778405,4.853161886124916,1.1462103851521934,-0.04195946512998055,0,0.169009268283844,0.07061692327260971,0.21073731780052185,248,0.22293230891227722,0.6151081323623657,0.23012366890907288,0.58125
+attn_19_23,attn,23,cal-zh-1,calibration,2.981238336078164,1.0546738615923779,3.342676305565292,1.0914181503369393,0.018114203094761517,0,0.1326604187488556,0.06724601984024048,0.1667337417602539,248,0.17328983545303345,0.5646554827690125,0.17887982726097107,0.6109375
+attn_19_23,attn,23,cal-zh-2,calibration,3.0395249971461062,1.0612041520781252,3.9775899363194043,1.111839148442333,-0.006099502317895858,0,0.09588906913995743,0.06764671951532364,0.15978427231311798,248,0.16259410977363586,0.5136446356773376,0.16783908009529114,0.6968749999999999
+attn_19_23,attn,23,cal-en-1,calibration,3.044278410194487,1.074729515389283,4.006931562831888,1.1811940201100057,-0.04307182449714221,0,0.12535521388053894,0.08747006952762604,0.1720750480890274,248,0.1742202639579773,0.5411514639854431,0.17984026670455933,0.66015625
+attn_19_23,attn,23,cal-en-2,calibration,2.9710652272259535,1.0799822705908066,3.68900214533588,1.1411985546671786,-0.0470831335221732,0,0.08650433272123337,0.08903923630714417,0.17286045849323273,248,0.17006517946720123,0.538245439529419,0.1755511462688446,0.56875
+mlp_5_6,mlp,5,cal-zh-1,calibration,1.2343440431264474,1.2448870136879528,1.4262659492232905,1.3330956688296154,0.23172673358878942,0,,,,,,,,
+mlp_5_6,mlp,5,cal-zh-2,calibration,1.2339654450576971,1.2535104800781527,1.32561843838562,1.329338987748564,0.22672917563938177,0,,,,,,,,
+mlp_5_6,mlp,5,cal-en-1,calibration,1.2263457192811411,1.2317267194400943,1.3474685872410954,1.321453653023078,0.2447481896291194,0,,,,,,,,
+mlp_5_6,mlp,5,cal-en-2,calibration,1.2077976165854318,1.2358845908873308,1.3068781752715062,1.3085733890221656,0.2539150140171328,0,,,,,,,,
+mlp_5_6,mlp,6,cal-zh-1,calibration,1.4662701939974856,1.3822306441896342,1.7481556583915787,1.7624732809638852,-0.011619417250216499,0,,,,,,,,
+mlp_5_6,mlp,6,cal-zh-2,calibration,1.4798445991262743,1.3637763123448376,1.6959117760237137,1.4894484186140093,-0.005750877367744503,0,,,,,,,,
+mlp_5_6,mlp,6,cal-en-1,calibration,1.464591579412775,1.3766590597797903,1.689543163635455,1.5032948778211077,-0.006204174235411236,0,,,,,,,,
+mlp_5_6,mlp,6,cal-en-2,calibration,1.4368842673613405,1.4008394919323996,1.6226235618647424,1.6896559214993705,-0.006099379702393146,0,,,,,,,,
+mlp_20_21,mlp,20,cal-zh-1,calibration,1.3631629826891347,1.3480373135423591,1.4508876157140895,1.4575901158218467,0.0812649689016906,0,,,,,,,,
+mlp_20_21,mlp,20,cal-zh-2,calibration,1.4305121300612613,1.314068906091469,1.5976490501921767,1.4075788062559882,0.06371076852261268,0,,,,,,,,
+mlp_20_21,mlp,20,cal-en-1,calibration,1.366192568032983,1.35226347926907,1.5789533371800022,1.5827466919073778,0.07632635231947486,0,,,,,,,,
+mlp_20_21,mlp,20,cal-en-2,calibration,1.4386161527552375,1.3312892565009653,1.5821573752114773,1.4214461315124072,0.045400137548793094,0,,,,,,,,
+mlp_20_21,mlp,21,cal-zh-1,calibration,1.389882929510158,1.3382556516005442,1.4917751564635304,1.4560073575192687,0.07070714893962673,0,,,,,,,,
+mlp_20_21,mlp,21,cal-zh-2,calibration,1.4299639779406776,1.3285677793009314,1.6132811807602088,1.41117608068315,0.052803821787486716,0,,,,,,,,
+mlp_20_21,mlp,21,cal-en-1,calibration,1.3453233062044447,1.4107978765350015,1.5399413812441296,1.7867758701030254,0.052139705952592985,0,,,,,,,,
+mlp_20_21,mlp,21,cal-en-2,calibration,1.4554092594533905,1.3180744101503734,1.5831740681011888,1.3865901398082663,0.045747088042376524,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-zh-1,calibration,0.22945597860473388,0.23195008932522895,0.24221562711880595,0.24569969899481256,0.973447272274363,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-zh-2,calibration,0.2290088724166579,0.23181448107574626,0.24542894690924066,0.24779202863840974,0.9735303499099761,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-en-1,calibration,0.23455492648208914,0.2376969550882275,0.24989139477418182,0.2529845119820021,0.972212040535478,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-en-2,calibration,0.22901516213757628,0.23221465328764443,0.24944281350466996,0.2558284584833273,0.9735059067182225,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-zh-1,calibration,0.19278924917702345,0.19044985719492102,0.21696829135487192,0.21613622029669263,0.9817161843232514,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-zh-2,calibration,0.1914737037557284,0.18949712918171538,0.22059518754606067,0.22263213250570094,0.98191197888193,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-en-1,calibration,0.19741689848184946,0.1948868605635172,0.25540160556856767,0.2544707934700031,0.9808462075453858,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-en-2,calibration,0.21059744905193553,0.20938264759903177,0.24188706740832974,0.24203851319432876,0.9779690077710401,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-zh-1,calibration,0.20847388940801903,0.20369709176427747,0.2170900495487928,0.21347731005429094,0.9790359000979272,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-zh-2,calibration,0.19935697980676284,0.19546641114417262,0.21545417284813181,0.20622001465458153,0.9807104227014257,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-en-1,calibration,0.20377890669282792,0.1992044511742432,0.2180618268213074,0.21066534853291913,0.9799609125477599,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-en-2,calibration,0.2049050681850306,0.20143020818672328,0.21865291387428726,0.20943815578722963,0.9795092387917065,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-zh-1,calibration,0.11058006301891404,0.10891256044231754,0.12484384056818072,0.12331584938512573,0.9940936589516192,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-zh-2,calibration,0.1025377174596076,0.10115362014926518,0.1190380009492926,0.11772165381976195,0.994906319610355,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-en-1,calibration,0.11061820718242668,0.10874403924660925,0.1246694172087446,0.12218803487494587,0.9941314653674382,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-en-2,calibration,0.1097129827694824,0.10824015260116268,0.12658812743768616,0.1257629217271933,0.9941536582549061,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-zh-1,calibration,0.13162251097797606,0.1336688160421926,0.14723992137490324,0.1503968566479842,0.9913220883632566,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-zh-2,calibration,0.1293519958781259,0.1314802843900349,0.15550840977823716,0.15903465263092675,0.9916295485881612,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-en-1,calibration,0.12647077725692374,0.12852946057966494,0.14803436556570124,0.1502912423902487,0.9920027531738658,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-en-2,calibration,0.1325318326855699,0.13489053569820655,0.15675391452711515,0.16029285586480232,0.9912169569277748,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-zh-1,calibration,0.19235633022629203,0.19080289992858507,0.2290122037106061,0.22871738277610806,0.9816818019088915,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-zh-2,calibration,0.20373761974053708,0.2026516990230578,0.22610646515999447,0.22641810348752162,0.9793703931467622,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-en-1,calibration,0.20072570704403014,0.1997337419816918,0.23210824690537907,0.232000375512668,0.9799664235115935,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-en-2,calibration,0.21292543194773159,0.2114735746425023,0.2540111544217496,0.253548340715113,0.9775093552656579,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-zh-1,calibration,0.15982234891287764,0.1668593847865133,0.1806429713396301,0.18971996556779416,0.9875945267208645,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-zh-2,calibration,0.15740332806536778,0.1630453842176067,0.17808143035469418,0.1865293914125642,0.987788243570594,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-en-1,calibration,0.16051291705164175,0.16606195948802505,0.18552144269008958,0.19215103895989022,0.9872500535949923,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-en-2,calibration,0.1646608571816797,0.17165005121760066,0.18445606954569943,0.19141765315408663,0.9867321292200623,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-zh-1,calibration,0.12891191745886205,0.12454504053986937,0.1385991791084394,0.1331904813118359,0.9925662013931714,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-zh-2,calibration,0.12161598809109211,0.11777013931789594,0.1368420974285716,0.1315377420603533,0.9933549654835544,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-en-1,calibration,0.12197566460111624,0.11819082207382924,0.135322184241321,0.13196489880351644,0.9932886289657402,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-en-2,calibration,0.12441025575886772,0.12064470117171276,0.1406080640306027,0.13504984145616641,0.9929676304409041,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-zh-1,calibration,0.35607606695379634,0.35506507510461943,0.4419932151127893,0.43353427677209583,0.9367889544420241,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-zh-2,calibration,0.3050017336245941,0.3063513991209177,0.35702892926241475,0.35499913749231526,0.9532908937392158,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-en-1,calibration,0.3004172668571062,0.30248734097466856,0.3551434800248225,0.3574678230240773,0.9545873680313379,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-en-2,calibration,0.332120216475189,0.33442277896716227,0.39234612989053624,0.3869494866775299,0.9444895843396817,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-zh-1,calibration,0.1494891933687725,0.1453919459232531,0.19873146648565732,0.1911951771845603,0.9895189305032556,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-zh-2,calibration,0.14909792210432973,0.14536281895078332,0.19307308117224836,0.18807685138750432,0.9894852004735976,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-en-1,calibration,0.1529571262250341,0.14878974314439253,0.19553431973346902,0.18740449249037416,0.9890023265976378,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-en-2,calibration,0.1608915089597201,0.15621975475373284,0.19992876686197072,0.1909273809291801,0.9878669550506567,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-zh-1,calibration,0.5472715903763039,0.4942115602598659,0.5959124252670717,0.5246077344510891,0.8699706486172409,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-zh-2,calibration,0.5066278025120974,0.4652273156892467,0.5698812253775749,0.505450409971441,0.8857874771365101,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-en-1,calibration,0.5396383470456222,0.4881273164223247,0.6104980922233472,0.5338668240636818,0.8733304573488526,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-en-2,calibration,0.5484944291943057,0.4948734608679084,0.6117866340678192,0.5354951226175159,0.8695786360146698,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-zh-1,calibration,0.1313144920837702,0.13005524261632884,0.18219490064170027,0.1762085222381826,0.9915073560564274,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-zh-2,calibration,0.12151622394280005,0.12124252528158447,0.1793544881596151,0.17483197862503747,0.9926360753692314,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-en-1,calibration,0.12730373699904377,0.12630314407300633,0.17809052202752992,0.17316093196082644,0.9919917024971001,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-en-2,calibration,0.15218433424490216,0.1492986535261055,0.1960834395064235,0.18960021577591676,0.9888227905277445,0,,,,,,,,

results/cpu-qwen35-2b/candidate-summary.json

@@ -0,0 +1,101 @@
+[
+  {
+    "id": "attn_3_7",
+    "kind": "attn",
+    "i": 3,
+    "j": 7,
+    "passed": false,
+    "maximum_symmetric_relative_error": 2.1157833337453664,
+    "donor_component_bytes": 29361152
+  },
+  {
+    "id": "attn_11_15",
+    "kind": "attn",
+    "i": 11,
+    "j": 15,
+    "passed": false,
+    "maximum_symmetric_relative_error": 3.783713712256275,
+    "donor_component_bytes": 29361152
+  },
+  {
+    "id": "attn_19_23",
+    "kind": "attn",
+    "i": 19,
+    "j": 23,
+    "passed": false,
+    "maximum_symmetric_relative_error": 3.7338041826396506,
+    "donor_component_bytes": 29361152
+  },
+  {
+    "id": "mlp_5_6",
+    "kind": "mlp",
+    "i": 5,
+    "j": 6,
+    "passed": false,
+    "maximum_symmetric_relative_error": 1.4798445991262743,
+    "donor_component_bytes": 75497472
+  },
+  {
+    "id": "mlp_20_21",
+    "kind": "mlp",
+    "i": 20,
+    "j": 21,
+    "passed": false,
+    "maximum_symmetric_relative_error": 1.4554092594533905,
+    "donor_component_bytes": 75497472
+  },
+  {
+    "id": "qk_norm_3",
+    "kind": "qk_norm",
+    "i": 3,
+    "j": 3,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.2376969550882275,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_7",
+    "kind": "qk_norm",
+    "i": 7,
+    "j": 7,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.20847388940801903,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_11",
+    "kind": "qk_norm",
+    "i": 11,
+    "j": 11,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.21292543194773159,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_15",
+    "kind": "qk_norm",
+    "i": 15,
+    "j": 15,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.17165005121760066,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_19",
+    "kind": "qk_norm",
+    "i": 19,
+    "j": 19,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.35607606695379634,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_23",
+    "kind": "qk_norm",
+    "i": 23,
+    "j": 23,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.5484944291943057,
+    "donor_component_bytes": 512
+  }
+]

results/cpu-qwen35-2b/checksums-full-server.json

@@ -0,0 +1,85 @@
+{
+  "FUNCTIONAL_REPORT.md": "6bfde7e8843e4c87186f8229bf15ab1db919f08ddd8b40fd04aecdec20c7390d",
+  "attention-head-metrics.csv": "732ada825914f8db49b72083cb129cf1305434c84c88253cc32b499ca1a9a119",
+  "baseline-activations.npz": "a851b489a8917a63b07cb6615a47fd1399b36e62f62d6140be618c4966d204ef",
+  "baseline-heldout-token-nll.npy": "303853deb86cb09848b63e2f8b62673db5e730c3b7984c7e65fef3f61c33cda1",
+  "calibration-metrics.csv": "fff8d1e638c6dcfaed88df580de348083fb01621bae7cda7ca2f04ec29636137",
+  "candidate-summary.json": "dc1d738760fa87ef681e33a72caf67cdb68db7413b66f364f80a67b0233fdbdc",
+  "compute-budget-ledger.json": "33dac0f3e9aaf6a82a01c67c22f9a755dff9be8092710f5d3fa8236ce551b255",
+  "download.log": "9f5b76650a94f15729e949605b50ab172ff0fc3a1d791b35dd2a88d4ace02dd0",
+  "experiment.json": "911ba54b489547373b724bd0e719717d8f836807d8fad8aa221eadb5a525d05a",
+  "figures/algebraic_gain_identity.pdf": "b9c44ed9f2df66f7b9172e9a1650db31ec3915290fef3d59b77320a0510ee6de",
+  "figures/algebraic_gain_identity.png": "2e637bb3362ac1592959fbe8f84e67ba06b1ea00ec71dfeec80c45a7ce7520b1",
+  "figures/algebraic_gain_identity.svg": "553ac72babfc005f67a91564f8b737b5edd279d62fb013b8d5d87f6884c4f1ac",
+  "figures/candidate_function_errors.pdf": "370f9b888a6cff64d69bb5f52541e71c66695139cc0b1a45ca8115eae44aae33",
+  "figures/candidate_function_errors.png": "a469f33e4f0a8ba81d4cbcaa333ddcf822f6fa2e4f33ea2c0a6d7da3cbd814b9",
+  "figures/candidate_function_errors.svg": "a647cfcf58bd5d96770daed29cf14ffd245aef3a0c463706f3724bf84b800469",
+  "figures/learned_norm_function_errors.pdf": "cba6296e9297d8f14b1f3346b4ab4015845a54bc880d0f153c8d4b586fe16f90",
+  "figures/learned_norm_function_errors.png": "85b1bce927e1c152778b814629a52996bfdb4a0828995492f56e1fb12f68648f",
+  "figures/learned_norm_function_errors.svg": "ce867a8b8aedfd941dde51ae53ad2541f4e407235c2572092dca6446e71d8f8c",
+  "figures/learned_norm_network_substitution.pdf": "6d3440323d276b4a70209f71cd31e016d8eb78d257bdaa41ffc0468d2813a428",
+  "figures/learned_norm_network_substitution.png": "e0dca3ce8082abf3e904aaa27aa581bd1c7ed1ff7a19285dde613dd88b3738ad",
+  "figures/learned_norm_network_substitution.svg": "7753abfa3f85cda50fbc3a00491f54a597cbbb54fdb75f0c0b2fe7a7fcc131c5",
+  "final-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "final-source-hashes.json": "52398645bce397827c36048c417f721a1cd8f76dc38eeaee530b8a5031b09e91",
+  "final-tests.txt": "3eb63c08f7b8de12c4198e79da9909582d5d64cb821ef1b08d38de7e4e43eaf5",
+  "functional-code-hashes.json": "3756c43b0904c53d9a3ea1fe92496cc9f890cc7b033a3c9577ef53cecd4d8536",
+  "functional-environment.json": "b9a2b4b850b3f26cfa96efc1ac6a9470d678f17197f96a01d9c3bdf5111b148a",
+  "functional-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "functional-run.log": "95684864b79b72dbefdf5485fca00bca3e80719c9e925a2a6fe1a6b0a07825d3",
+  "functional-status.json": "fdda303b7e520f9e0065c65281df5b4bd6e9277edec52f9e5818047598a2a2ea",
+  "gain-factorization-k_to_q-token-nll.npy": "767f2a885532e347a654a86ed0a7f19bd4938c52296dcc74db26876acb3fe6a1",
+  "gain-factorization-kernel-timing.csv": "7ee790a456d4b9dac9d4756b71288f4df165c0c6111fe94657fa6804f91154d1",
+  "gain-factorization-optimized-summary.json": "0e303074384a3933577b2c3c29a042fe00e64cf308eb8f6190db8f7e3db55e7b",
+  "gain-factorization-optimized-timing.csv": "b5c85e1f9bcfb49bf317090470530a44546079c5a79c1df2365c00c36144dc10",
+  "gain-factorization-optimized.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "gain-factorization-protocol.json": "7690e4cf5dec415e414813e5beb3434ae62d51d95e68ad5de48e698980e26cab",
+  "gain-factorization-q_to_k-token-nll.npy": "6ea27edea56593c7269403ce5a0f7eb8673b9d80d1e00806d38bc6a2a681fff8",
+  "gain-factorization-score-metrics.csv": "7cfcc997980cd0566721e529f6bb1bca2f95858da50d5cf533eb31742ab0c230",
+  "gain-factorization-summary.json": "3bfde7dba2abd19e6dc945c05ad924181bc0927e0b88806337c9bbab9db7b359",
+  "gain-factorization.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "install-torch-disk-tmp.log": "85ee085d264707b8fd0031876c3ae9cf4e36151db9cbf53edef979f10d144c13",
+  "install-torch.log": "e297f77b38de146e85e6159025ea1e0ec93622fe26ef46698df180c7c96378b8",
+  "manifest.json": "28efbaf94a5034333339a35f08ead151459a7d20c6c2d2f07a451e7f3bf154de",
+  "model_config.json": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4",
+  "neuron-reuse-activations.npz": "0ce0b7ba4efef0164a68d91595e8eb47a6347f1f364eeda764835cfa09b09448",
+  "neuron-reuse-protocol.json": "cc4fcf06ebd4ee5420933534021006df6e62259b00fd922f6e8fabada89472f7",
+  "neuron-reuse-search-summary.json": "30a5ef914c4ff766e9bde09e3784a229bdbe7a10cea871bef3be327df8047c25",
+  "neuron-reuse-selection.json": "afafc7f8d35c562940f5afe47c5edaf3c5b7ca97744152bb5f9988b9910797a4",
+  "neuron-reuse-summary.json": "f14999d51d262c67f90e2e37918701b6b56107c7d5c47fad6da1cdfeb35d62a2",
+  "neuron-reuse.log": "be387fa429688a6a34131f0a5899a81586a2bde85cf209253c2da977e625168c",
+  "norm-repetition-12-uses-13-token-nll.npy": "fb05b796fd40122aa0a1b05ce92585c980f47a1cb1c771df4fabc720ade4bbd8",
+  "norm-repetition-13-uses-12-token-nll.npy": "5470a87f22f100ed8e43caf90a05831453b4277707d6d7085eecc4bd43ae8cb6",
+  "norm-repetition-function-metrics.csv": "d5f4eed05a9f5c6e0f971331dd9d0b834d6141b4aeb9e3c725a5bee1f4314593",
+  "norm-repetition-input-profiles.npz": "7e3415b8f28d8844658facf2c37415cc0f3a4de8f6d39d3e0c4b01e1b3a4d7e0",
+  "norm-repetition-protocol.json": "ad8d2319572b4a1daa5d3ae227bd344b5f1db7941ee567b43d1f25dbd963f6a0",
+  "norm-repetition-selection.json": "4dae5a62da0889c0ed4de7813a8f720bb94991e468b9d6cdee8c5327d549ac55",
+  "norm-repetition-summary.json": "70b5067d0820b5a5bc7062c71b9f29c5590d9bb12bf538beb961135d1aab8834",
+  "norm-repetition-uniform-bounds.csv": "48e71a9cac9410d110483cef8356e02b2dcdeae1120bcb243ae86903852fb32a",
+  "norm-repetition.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "preflight-first-attempt.log": "1a336568f0ed99ac306ab1d690ff00f325c9c73ec6be84df4ace4374d490c650",
+  "preflight-second-attempt.json": "a6b68d31710e45cbf09ee100b490c781547334f76ecfbb3e09332779e953b30c",
+  "preflight-second-attempt.log": "42a07ef0a3fd8cb1a6475bec6e7958f754a24012fb8269073d8625b1b891594f",
+  "preflight.json": "86113ecbf499f4a84ea4d7fd7893e8c12f2784b6e4678b5cbcbb0e022e295e4b",
+  "preflight.log": "66dd7d1e0b146fc448a1064efbde5d693126d3f3cbb2fc9abdeb4485ead7c951",
+  "reference-check-first-attempt.log": "ed77d6619c19551401938a54fae0ef1aa50846b943ea1ba3321750c9cb985428",
+  "reference-check-second-attempt.log": "5923802faaba183a099d0eb313d5dc2de4825b1d52ec6ce0699a8201f68d99af",
+  "reference-check.json": "e82b0d8634ccae0671269587da960e06cce33e0a0e2ed15db15d946f6399d9ed",
+  "reference-check.log": "e4a1d55d309bff39535ecab272bd2aa6d8fe183656724b7731e4171d03150f11",
+  "retention-policy.json": "e3cdb6707985d2fa1fcb7f088f8f46f9eafb89eea13e37f51c554f57790acd6e",
+  "samples-tokenized.json": "2e56e14a4eefedd702bfe09d3edb2f17f360f53dd420e29fcf0c0ec1affc0184",
+  "selection.json": "c8c86d12fe7db6fb8fffca8078ec0ff5438b97998088c89862db8f9b6152daa3",
+  "source_metadata.json": "d3be2a8560f4270b8af27891d1076cb7cf80bd84c4c19fe7790d5fb06f3a342e",
+  "storage-accounting.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "storage-evidence.json": "c6968f4143910cef9d1359d3cd3342d289b9c791c6e2a6ab598fe235b105e0d9",
+  "supervisor-error.txt": "4cba26d15e963620745ab3d4625f4de967859a9c43993f78ca1ce2e5c06aeb42",
+  "supervisor-first-attempt-error.txt": "e689f94e983b84cfc4e6b372175826d77dcd2d5c9b8d166598bc8607227afbba",
+  "supervisor-first-attempt.json": "93d7185df1810aec2aa396fd8247e188b8a37a821731ef681e66389695bd9a8e",
+  "supervisor-second-attempt.json": "7333a3634f7d169bc78d9eeb5e47d8fbd2970c8401b4d6160b92038ad2a89f55",
+  "supervisor-status.json": "ae078d540c9c683bac468313a85754d8ecf1a88be482a2d5895bbfca0c610404",
+  "supervisor-third-attempt.json": "e86ce8e2f55f75ab3c61ffd7b4a1517e627cebbdb85ee7f966615c5d683722ed",
+  "supervisor.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "tokenizer-download.log": "1d9c8b608f3c79c9d7af93c593543ddd14db3e077e2c108738bccd518df9a79a",
+  "tokenizers-wheel.log": "f0b4f7c1360ff1a00e04fe443e619682c7d7b024df55545d31f54951b3001480",
+  "unit-tests.txt": "fa8ffc7277ea722b9261da462b797710ed4ae3eb8975ad51937c419b3052e3af"
+}

results/cpu-qwen35-2b/checksums-functional.json

@@ -0,0 +1,85 @@
+{
+  "DELIVERY.md": "8f4608add548dc4c6bc3785ccc262a4c695098c043563f43193edac904702cb1",
+  "FUNCTIONAL_REPORT.md": "6bfde7e8843e4c87186f8229bf15ab1db919f08ddd8b40fd04aecdec20c7390d",
+  "attention-head-metrics.csv": "732ada825914f8db49b72083cb129cf1305434c84c88253cc32b499ca1a9a119",
+  "baseline-heldout-token-nll.npy": "303853deb86cb09848b63e2f8b62673db5e730c3b7984c7e65fef3f61c33cda1",
+  "calibration-metrics.csv": "fff8d1e638c6dcfaed88df580de348083fb01621bae7cda7ca2f04ec29636137",
+  "candidate-summary.json": "dc1d738760fa87ef681e33a72caf67cdb68db7413b66f364f80a67b0233fdbdc",
+  "checksums-full-server.json": "152305bd23c849a75a0e0b5b686cadd93e5db017ea9f2733a52aa771353ed5f3",
+  "compute-budget-ledger.json": "33dac0f3e9aaf6a82a01c67c22f9a755dff9be8092710f5d3fa8236ce551b255",
+  "download.log": "9f5b76650a94f15729e949605b50ab172ff0fc3a1d791b35dd2a88d4ace02dd0",
+  "experiment.json": "911ba54b489547373b724bd0e719717d8f836807d8fad8aa221eadb5a525d05a",
+  "external-activation-cache.json": "ceb8ec04ac1ca6fe1a0466520b0c58180741339541b4ace4c5696c359ed25917",
+  "figures/algebraic_gain_identity.pdf": "b9c44ed9f2df66f7b9172e9a1650db31ec3915290fef3d59b77320a0510ee6de",
+  "figures/algebraic_gain_identity.png": "2e637bb3362ac1592959fbe8f84e67ba06b1ea00ec71dfeec80c45a7ce7520b1",
+  "figures/algebraic_gain_identity.svg": "553ac72babfc005f67a91564f8b737b5edd279d62fb013b8d5d87f6884c4f1ac",
+  "figures/candidate_function_errors.pdf": "370f9b888a6cff64d69bb5f52541e71c66695139cc0b1a45ca8115eae44aae33",
+  "figures/candidate_function_errors.png": "a469f33e4f0a8ba81d4cbcaa333ddcf822f6fa2e4f33ea2c0a6d7da3cbd814b9",
+  "figures/candidate_function_errors.svg": "a647cfcf58bd5d96770daed29cf14ffd245aef3a0c463706f3724bf84b800469",
+  "figures/learned_norm_function_errors.pdf": "cba6296e9297d8f14b1f3346b4ab4015845a54bc880d0f153c8d4b586fe16f90",
+  "figures/learned_norm_function_errors.png": "85b1bce927e1c152778b814629a52996bfdb4a0828995492f56e1fb12f68648f",
+  "figures/learned_norm_function_errors.svg": "ce867a8b8aedfd941dde51ae53ad2541f4e407235c2572092dca6446e71d8f8c",
+  "figures/learned_norm_network_substitution.pdf": "6d3440323d276b4a70209f71cd31e016d8eb78d257bdaa41ffc0468d2813a428",
+  "figures/learned_norm_network_substitution.png": "e0dca3ce8082abf3e904aaa27aa581bd1c7ed1ff7a19285dde613dd88b3738ad",
+  "figures/learned_norm_network_substitution.svg": "7753abfa3f85cda50fbc3a00491f54a597cbbb54fdb75f0c0b2fe7a7fcc131c5",
+  "final-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "final-source-hashes.json": "52398645bce397827c36048c417f721a1cd8f76dc38eeaee530b8a5031b09e91",
+  "final-tests.txt": "3eb63c08f7b8de12c4198e79da9909582d5d64cb821ef1b08d38de7e4e43eaf5",
+  "functional-code-hashes.json": "3756c43b0904c53d9a3ea1fe92496cc9f890cc7b033a3c9577ef53cecd4d8536",
+  "functional-environment.json": "b9a2b4b850b3f26cfa96efc1ac6a9470d678f17197f96a01d9c3bdf5111b148a",
+  "functional-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "functional-run.log": "95684864b79b72dbefdf5485fca00bca3e80719c9e925a2a6fe1a6b0a07825d3",
+  "functional-status.json": "fdda303b7e520f9e0065c65281df5b4bd6e9277edec52f9e5818047598a2a2ea",
+  "gain-factorization-k_to_q-token-nll.npy": "767f2a885532e347a654a86ed0a7f19bd4938c52296dcc74db26876acb3fe6a1",
+  "gain-factorization-kernel-timing.csv": "7ee790a456d4b9dac9d4756b71288f4df165c0c6111fe94657fa6804f91154d1",
+  "gain-factorization-optimized-summary.json": "0e303074384a3933577b2c3c29a042fe00e64cf308eb8f6190db8f7e3db55e7b",
+  "gain-factorization-optimized-timing.csv": "b5c85e1f9bcfb49bf317090470530a44546079c5a79c1df2365c00c36144dc10",
+  "gain-factorization-optimized.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "gain-factorization-protocol.json": "7690e4cf5dec415e414813e5beb3434ae62d51d95e68ad5de48e698980e26cab",
+  "gain-factorization-q_to_k-token-nll.npy": "6ea27edea56593c7269403ce5a0f7eb8673b9d80d1e00806d38bc6a2a681fff8",
+  "gain-factorization-score-metrics.csv": "7cfcc997980cd0566721e529f6bb1bca2f95858da50d5cf533eb31742ab0c230",
+  "gain-factorization-summary.json": "3bfde7dba2abd19e6dc945c05ad924181bc0927e0b88806337c9bbab9db7b359",
+  "gain-factorization.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "install-torch-disk-tmp.log": "85ee085d264707b8fd0031876c3ae9cf4e36151db9cbf53edef979f10d144c13",
+  "install-torch.log": "e297f77b38de146e85e6159025ea1e0ec93622fe26ef46698df180c7c96378b8",
+  "manifest.json": "28efbaf94a5034333339a35f08ead151459a7d20c6c2d2f07a451e7f3bf154de",
+  "model_config.json": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4",
+  "neuron-reuse-protocol.json": "cc4fcf06ebd4ee5420933534021006df6e62259b00fd922f6e8fabada89472f7",
+  "neuron-reuse-search-summary.json": "30a5ef914c4ff766e9bde09e3784a229bdbe7a10cea871bef3be327df8047c25",
+  "neuron-reuse-selection.json": "afafc7f8d35c562940f5afe47c5edaf3c5b7ca97744152bb5f9988b9910797a4",
+  "neuron-reuse-summary.json": "f14999d51d262c67f90e2e37918701b6b56107c7d5c47fad6da1cdfeb35d62a2",
+  "neuron-reuse.log": "be387fa429688a6a34131f0a5899a81586a2bde85cf209253c2da977e625168c",
+  "norm-repetition-12-uses-13-token-nll.npy": "fb05b796fd40122aa0a1b05ce92585c980f47a1cb1c771df4fabc720ade4bbd8",
+  "norm-repetition-13-uses-12-token-nll.npy": "5470a87f22f100ed8e43caf90a05831453b4277707d6d7085eecc4bd43ae8cb6",
+  "norm-repetition-function-metrics.csv": "d5f4eed05a9f5c6e0f971331dd9d0b834d6141b4aeb9e3c725a5bee1f4314593",
+  "norm-repetition-protocol.json": "ad8d2319572b4a1daa5d3ae227bd344b5f1db7941ee567b43d1f25dbd963f6a0",
+  "norm-repetition-selection.json": "4dae5a62da0889c0ed4de7813a8f720bb94991e468b9d6cdee8c5327d549ac55",
+  "norm-repetition-summary.json": "70b5067d0820b5a5bc7062c71b9f29c5590d9bb12bf538beb961135d1aab8834",
+  "norm-repetition-uniform-bounds.csv": "48e71a9cac9410d110483cef8356e02b2dcdeae1120bcb243ae86903852fb32a",
+  "norm-repetition.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "preflight-first-attempt.log": "1a336568f0ed99ac306ab1d690ff00f325c9c73ec6be84df4ace4374d490c650",
+  "preflight-second-attempt.json": "a6b68d31710e45cbf09ee100b490c781547334f76ecfbb3e09332779e953b30c",
+  "preflight-second-attempt.log": "42a07ef0a3fd8cb1a6475bec6e7958f754a24012fb8269073d8625b1b891594f",
+  "preflight.json": "86113ecbf499f4a84ea4d7fd7893e8c12f2784b6e4678b5cbcbb0e022e295e4b",
+  "preflight.log": "66dd7d1e0b146fc448a1064efbde5d693126d3f3cbb2fc9abdeb4485ead7c951",
+  "reference-check-first-attempt.log": "ed77d6619c19551401938a54fae0ef1aa50846b943ea1ba3321750c9cb985428",
+  "reference-check-second-attempt.log": "5923802faaba183a099d0eb313d5dc2de4825b1d52ec6ce0699a8201f68d99af",
+  "reference-check.json": "e82b0d8634ccae0671269587da960e06cce33e0a0e2ed15db15d946f6399d9ed",
+  "reference-check.log": "e4a1d55d309bff39535ecab272bd2aa6d8fe183656724b7731e4171d03150f11",
+  "retention-policy.json": "e3cdb6707985d2fa1fcb7f088f8f46f9eafb89eea13e37f51c554f57790acd6e",
+  "samples-tokenized.json": "2e56e14a4eefedd702bfe09d3edb2f17f360f53dd420e29fcf0c0ec1affc0184",
+  "selection.json": "c8c86d12fe7db6fb8fffca8078ec0ff5438b97998088c89862db8f9b6152daa3",
+  "source_metadata.json": "d3be2a8560f4270b8af27891d1076cb7cf80bd84c4c19fe7790d5fb06f3a342e",
+  "storage-accounting.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "storage-evidence.json": "c6968f4143910cef9d1359d3cd3342d289b9c791c6e2a6ab598fe235b105e0d9",
+  "supervisor-error.txt": "4cba26d15e963620745ab3d4625f4de967859a9c43993f78ca1ce2e5c06aeb42",
+  "supervisor-first-attempt-error.txt": "e689f94e983b84cfc4e6b372175826d77dcd2d5c9b8d166598bc8607227afbba",
+  "supervisor-first-attempt.json": "93d7185df1810aec2aa396fd8247e188b8a37a821731ef681e66389695bd9a8e",
+  "supervisor-second-attempt.json": "7333a3634f7d169bc78d9eeb5e47d8fbd2970c8401b4d6160b92038ad2a89f55",
+  "supervisor-status.json": "ae078d540c9c683bac468313a85754d8ecf1a88be482a2d5895bbfca0c610404",
+  "supervisor-third-attempt.json": "e86ce8e2f55f75ab3c61ffd7b4a1517e627cebbdb85ee7f966615c5d683722ed",
+  "supervisor.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "tokenizer-download.log": "1d9c8b608f3c79c9d7af93c593543ddd14db3e077e2c108738bccd518df9a79a",
+  "tokenizers-wheel.log": "f0b4f7c1360ff1a00e04fe443e619682c7d7b024df55545d31f54951b3001480",
+  "unit-tests.txt": "fa8ffc7277ea722b9261da462b797710ed4ae3eb8975ad51937c419b3052e3af"
+}

results/cpu-qwen35-2b/compute-budget-ledger.json

@@ -0,0 +1,54 @@
+{
+  "spent_seconds": 1167.355532555841,
+  "initial_reserve_reason": "Conservative reserve for early reference checks and stopped preflight, not a measured runtime",
+  "calls": [
+    {
+      "module": "functional.reference_check",
+      "seconds": 6.833657197188586
+    },
+    {
+      "module": "functional.preflight",
+      "seconds": 27.313251950312406
+    },
+    {
+      "module": "functional.report",
+      "seconds": 2.1849583038128912
+    },
+    {
+      "module": "functional.reference_check",
+      "seconds": 16.264290761202574
+    },
+    {
+      "module": "functional.preflight",
+      "seconds": 58.00242758495733
+    },
+    {
+      "module": "functional.run",
+      "seconds": 204.33689004788175
+    },
+    {
+      "module": "functional.storage",
+      "seconds": 1.2859071069397032
+    },
+    {
+      "module": "functional.report",
+      "seconds": 10.058376160915941
+    },
+    {
+      "module": "functional.gain_factorization",
+      "seconds": 232.08875080291182
+    },
+    {
+      "module": "functional.benchmark_factorization",
+      "seconds": 4.185962562914938
+    },
+    {
+      "module": "functional.neuron_reuse",
+      "seconds": 55.004304428584874
+    },
+    {
+      "module": "functional.norm_repetition",
+      "seconds": 369.7967556482181
+    }
+  ]
+}

results/cpu-qwen35-2b/download.log

@@ -0,0 +1,73 @@
+Downloading README.md, expected 62814 bytes
+Downloading config.json, expected 2908 bytes
+Downloading model.safetensors-00001-of-00001.safetensors, expected 4548221488 bytes
+Weight download 67108864/4548221488 bytes (1.5%)
+Weight download 134217728/4548221488 bytes (3.0%)
+Weight download 201326592/4548221488 bytes (4.4%)
+Weight download 268435456/4548221488 bytes (5.9%)
+Weight download 335544320/4548221488 bytes (7.4%)
+Weight download 402653184/4548221488 bytes (8.9%)
+Weight download 469762048/4548221488 bytes (10.3%)
+Weight download 536870912/4548221488 bytes (11.8%)
+Weight download 603979776/4548221488 bytes (13.3%)
+Weight download 671088640/4548221488 bytes (14.8%)
+Weight download 738197504/4548221488 bytes (16.2%)
+Weight download 805306368/4548221488 bytes (17.7%)
+Weight download 872415232/4548221488 bytes (19.2%)
+Weight download 939524096/4548221488 bytes (20.7%)
+Weight download 1006632960/4548221488 bytes (22.1%)
+Weight download 1073741824/4548221488 bytes (23.6%)
+Weight download 1140850688/4548221488 bytes (25.1%)
+Weight download 1207959552/4548221488 bytes (26.6%)
+Weight download 1275068416/4548221488 bytes (28.0%)
+Weight download 1342177280/4548221488 bytes (29.5%)
+Weight download 1409286144/4548221488 bytes (31.0%)
+Weight download 1476395008/4548221488 bytes (32.5%)
+Weight download 1543503872/4548221488 bytes (33.9%)
+Weight download 1610612736/4548221488 bytes (35.4%)
+Weight download 1677721600/4548221488 bytes (36.9%)
+Weight download 1744830464/4548221488 bytes (38.4%)
+Weight download 1811939328/4548221488 bytes (39.8%)
+Weight download 1879048192/4548221488 bytes (41.3%)
+Weight download 1946157056/4548221488 bytes (42.8%)
+Weight download 2013265920/4548221488 bytes (44.3%)
+Weight download 2080374784/4548221488 bytes (45.7%)
+Weight download 2147483648/4548221488 bytes (47.2%)
+Weight download 2214592512/4548221488 bytes (48.7%)
+Weight download 2281701376/4548221488 bytes (50.2%)
+Weight download 2348810240/4548221488 bytes (51.6%)
+Weight download 2415919104/4548221488 bytes (53.1%)
+Weight download 2483027968/4548221488 bytes (54.6%)
+Weight download 2550136832/4548221488 bytes (56.1%)
+Weight download 2617245696/4548221488 bytes (57.5%)
+Weight download 2684354560/4548221488 bytes (59.0%)
+Weight download 2751463424/4548221488 bytes (60.5%)
+Weight download 2818572288/4548221488 bytes (62.0%)
+Weight download 2885681152/4548221488 bytes (63.4%)
+Weight download 2952790016/4548221488 bytes (64.9%)
+Weight download 3019898880/4548221488 bytes (66.4%)
+Weight download 3087007744/4548221488 bytes (67.9%)
+Weight download 3154116608/4548221488 bytes (69.3%)
+Weight download 3221225472/4548221488 bytes (70.8%)
+Weight download 3288334336/4548221488 bytes (72.3%)
+Weight download 3355443200/4548221488 bytes (73.8%)
+Weight download 3422552064/4548221488 bytes (75.3%)
+Weight download 3489660928/4548221488 bytes (76.7%)
+Weight download 3556769792/4548221488 bytes (78.2%)
+Weight download 3623878656/4548221488 bytes (79.7%)
+Weight download 3690987520/4548221488 bytes (81.2%)
+Weight download 3758096384/4548221488 bytes (82.6%)
+Weight download 3825205248/4548221488 bytes (84.1%)
+Weight download 3892314112/4548221488 bytes (85.6%)
+Weight download 3959422976/4548221488 bytes (87.1%)
+Weight download 4026531840/4548221488 bytes (88.5%)
+Weight download 4093640704/4548221488 bytes (90.0%)
+Weight download 4160749568/4548221488 bytes (91.5%)
+Weight download 4227858432/4548221488 bytes (93.0%)
+Weight download 4294967296/4548221488 bytes (94.4%)
+Weight download 4346894896/4548221488 bytes (95.6%)
+Weight download 4414003760/4548221488 bytes (97.0%)
+Weight download 4481112624/4548221488 bytes (98.5%)
+Weight download 4548221488/4548221488 bytes (100.0%)
+Downloading model.safetensors.index.json, expected 64460 bytes
+{"revision": "15852e8c16360a2fea060d615a32b45270f8a8fc", "bytes": 4548351670}

results/cpu-qwen35-2b/experiment.json

@@ -0,0 +1,49 @@
+{
+  "revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "attention_pairs": [
+    [
+      3,
+      7
+    ],
+    [
+      11,
+      15
+    ],
+    [
+      19,
+      23
+    ]
+  ],
+  "mlp_pairs": [
+    [
+      5,
+      6
+    ],
+    [
+      20,
+      21
+    ]
+  ],
+  "norm_layers": [
+    3,
+    7,
+    11,
+    15,
+    19,
+    23
+  ],
+  "seconds": 6849,
+  "output_relative_l2_limit": 0.05,
+  "token_p95_limit": 0.1,
+  "attention_js_limit": 0.01,
+  "attention_top5_min": 0.8,
+  "heldout_nll_mean_delta_limit": 0.01,
+  "heldout_nll_sequence_p95_delta_limit": 0.05,
+  "max_intervention_candidates": 1,
+  "directions_per_selected_candidate": 2,
+  "training": false,
+  "search_R": false,
+  "automatic_weight_cleanup": false,
+  "dtype": "original weights decoded to FP32; not a BF16 backend benchmark",
+  "blas_threads": "1"
+}

results/cpu-qwen35-2b/external-activation-cache.json

@@ -0,0 +1,31 @@
+{
+  "host": "8.211.154.134",
+  "files": [
+    {
+      "name": "baseline-activations.npz",
+      "server_path": "/opt/exp-model-parameter-similarity/runs/cpu-qwen35-2b/baseline-activations.npz",
+      "bytes": 38802964,
+      "sha256": "a851b489a8917a63b07cb6615a47fd1399b36e62f62d6140be618c4966d204ef",
+      "retained": true,
+      "reproducible_by": "functional.run / functional.neuron_reuse / functional.norm_repetition"
+    },
+    {
+      "name": "neuron-reuse-activations.npz",
+      "server_path": "/opt/exp-model-parameter-similarity/runs/cpu-qwen35-2b/neuron-reuse-activations.npz",
+      "bytes": 25166826,
+      "sha256": "0ce0b7ba4efef0164a68d91595e8eb47a6347f1f364eeda764835cfa09b09448",
+      "retained": true,
+      "reproducible_by": "functional.run / functional.neuron_reuse / functional.norm_repetition"
+    },
+    {
+      "name": "norm-repetition-input-profiles.npz",
+      "server_path": "/opt/exp-model-parameter-similarity/runs/cpu-qwen35-2b/norm-repetition-input-profiles.npz",
+      "bytes": 4194818,
+      "sha256": "7e3415b8f28d8844658facf2c37415cc0f3a4de8f6d39d3e0c4b01e1b3a4d7e0",
+      "retained": true,
+      "reproducible_by": "functional.run / functional.neuron_reuse / functional.norm_repetition"
+    }
+  ],
+  "not_model_weights": true,
+  "reason": "Large intermediate caches retained on the authorized server; all figure source tables and token NLL arrays are delivered in Git."
+}

results/cpu-qwen35-2b/figures/algebraic_gain_identity.svg

@@ -0,0 +1,581 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="712.424062pt" height="285.518438pt" viewBox="0 0 712.424062 285.518438" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:11:02.311213</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 285.518438 
+L 712.424062 285.518438 
+L 712.424062 -0 
+L 0 -0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 50.466406 247.317656 
+L 353.864062 247.317656 
+L 353.864062 45.880469 
+L 50.466406 45.880469 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="m50faf94306" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m50faf94306" x="64.257209" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="64.257209" y="261.915313" transform="rotate(-0 64.257209 261.915313)">3</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#m50faf94306" x="119.420419" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="119.420419" y="261.915313" transform="rotate(-0 119.420419 261.915313)">7</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#m50faf94306" x="174.583629" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="174.583629" y="261.915313" transform="rotate(-0 174.583629 261.915313)">11</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#m50faf94306" x="229.746839" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="229.746839" y="261.915313" transform="rotate(-0 229.746839 261.915313)">15</text>
+     </g>
+    </g>
+    <g id="xtick_5">
+     <g id="line2d_5">
+      <g>
+       <use xlink:href="#m50faf94306" x="284.91005" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_5">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="284.91005" y="261.915313" transform="rotate(-0 284.91005 261.915313)">19</text>
+     </g>
+    </g>
+    <g id="xtick_6">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#m50faf94306" x="340.07326" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="340.07326" y="261.915313" transform="rotate(-0 340.07326 261.915313)">23</text>
+     </g>
+    </g>
+    <g id="text_7">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="202.165234" y="275.916094" transform="rotate(-0 202.165234 275.916094)">Full-attention layer index</text>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_7">
+      <defs>
+       <path id="m9a2e91a9aa" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="229.265954" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="233.064782" transform="rotate(-0 43.466406 233.064782)">2.00</text>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="205.263431" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="209.06226" transform="rotate(-0 43.466406 209.06226)">2.25</text>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_9">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="181.260909" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="185.059737" transform="rotate(-0 43.466406 185.059737)">2.50</text>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="157.258386" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="161.057214" transform="rotate(-0 43.466406 161.057214)">2.75</text>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_11">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="133.255863" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_12">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="137.054691" transform="rotate(-0 43.466406 137.054691)">3.00</text>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_12">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="109.25334" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_13">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="113.052169" transform="rotate(-0 43.466406 113.052169)">3.25</text>
+     </g>
+    </g>
+    <g id="ytick_7">
+     <g id="line2d_13">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="85.250818" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_14">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="89.049646" transform="rotate(-0 43.466406 89.049646)">3.50</text>
+     </g>
+    </g>
+    <g id="ytick_8">
+     <g id="line2d_14">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="61.248295" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_15">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="65.047123" transform="rotate(-0 43.466406 65.047123)">3.75</text>
+     </g>
+    </g>
+    <g id="text_16">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="146.599063" transform="rotate(-90 14.798438 146.599063)">Maximum absolute score error</text>
+    </g>
+    <g id="text_17">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="50.466406" y="42.880469" transform="rotate(-0 50.466406 42.880469)">1e−6</text>
+    </g>
+   </g>
+   <g id="line2d_15">
+    <path d="M 64.257209 238.16142 
+L 119.420419 238.16142 
+L 174.583629 238.16142 
+L 229.746839 146.599063 
+L 284.91005 55.036705 
+L 340.07326 55.036705 
+" clip-path="url(#p0542a2a98b)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m9b4cc9d8d2" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #1f77b4"/>
+    </defs>
+    <g clip-path="url(#p0542a2a98b)">
+     <use xlink:href="#m9b4cc9d8d2" x="64.257209" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="119.420419" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="174.583629" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="229.746839" y="146.599063" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="284.91005" y="55.036705" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="340.07326" y="55.036705" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_16">
+    <path d="M 64.257209 238.16142 
+L 119.420419 238.16142 
+L 174.583629 146.599063 
+L 229.746839 238.16142 
+L 284.91005 238.16142 
+L 340.07326 55.036705 
+" clip-path="url(#p0542a2a98b)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m556f16804f" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #ff7f0e"/>
+    </defs>
+    <g clip-path="url(#p0542a2a98b)">
+     <use xlink:href="#m556f16804f" x="64.257209" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="119.420419" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="174.583629" y="146.599063" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="229.746839" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="284.91005" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="340.07326" y="55.036705" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="patch_3">
+    <path d="M 50.466406 247.317656 
+L 50.466406 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 353.864062 247.317656 
+L 353.864062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 50.466406 247.317656 
+L 353.864062 247.317656 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 50.466406 45.880469 
+L 353.864062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="legend_1">
+    <g id="patch_7">
+     <path d="M 56.066406 76.281719 
+L 107.408906 76.281719 
+Q 109.008906 76.281719 109.008906 74.681719 
+L 109.008906 51.480469 
+Q 109.008906 49.880469 107.408906 49.880469 
+L 56.066406 49.880469 
+Q 54.466406 49.880469 54.466406 51.480469 
+L 54.466406 74.681719 
+Q 54.466406 76.281719 56.066406 76.281719 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_17">
+     <path d="M 57.666406 56.359219 
+L 65.666406 56.359219 
+L 73.666406 56.359219 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m9b4cc9d8d2" x="65.666406" y="56.359219" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_18">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="59.159219" transform="rotate(-0 80.066406 59.159219)">q_to_k</text>
+    </g>
+    <g id="line2d_18">
+     <path d="M 57.666406 68.359844 
+L 65.666406 68.359844 
+L 73.666406 68.359844 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m556f16804f" x="65.666406" y="68.359844" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_19">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="71.159844" transform="rotate(-0 80.066406 71.159844)">k_to_q</text>
+    </g>
+   </g>
+  </g>
+  <g id="axes_2">
+   <g id="patch_8">
+    <path d="M 401.826406 247.317656 
+L 705.224062 247.317656 
+L 705.224062 45.880469 
+L 401.826406 45.880469 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_3">
+    <g id="xtick_7">
+     <g id="line2d_19">
+      <g>
+       <use xlink:href="#m50faf94306" x="415.617209" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_20">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="415.617209" y="261.915313" transform="rotate(-0 415.617209 261.915313)">3</text>
+     </g>
+    </g>
+    <g id="xtick_8">
+     <g id="line2d_20">
+      <g>
+       <use xlink:href="#m50faf94306" x="470.780419" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_21">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="470.780419" y="261.915313" transform="rotate(-0 470.780419 261.915313)">7</text>
+     </g>
+    </g>
+    <g id="xtick_9">
+     <g id="line2d_21">
+      <g>
+       <use xlink:href="#m50faf94306" x="525.943629" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_22">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="525.943629" y="261.915313" transform="rotate(-0 525.943629 261.915313)">11</text>
+     </g>
+    </g>
+    <g id="xtick_10">
+     <g id="line2d_22">
+      <g>
+       <use xlink:href="#m50faf94306" x="581.106839" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_23">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="581.106839" y="261.915313" transform="rotate(-0 581.106839 261.915313)">15</text>
+     </g>
+    </g>
+    <g id="xtick_11">
+     <g id="line2d_23">
+      <g>
+       <use xlink:href="#m50faf94306" x="636.27005" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_24">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="636.27005" y="261.915313" transform="rotate(-0 636.27005 261.915313)">19</text>
+     </g>
+    </g>
+    <g id="xtick_12">
+     <g id="line2d_24">
+      <g>
+       <use xlink:href="#m50faf94306" x="691.43326" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_25">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="691.43326" y="261.915313" transform="rotate(-0 691.43326 261.915313)">23</text>
+     </g>
+    </g>
+    <g id="text_26">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="553.525234" y="275.916094" transform="rotate(-0 553.525234 275.916094)">Full-attention layer index</text>
+    </g>
+   </g>
+   <g id="matplotlib.axis_4">
+    <g id="ytick_9">
+     <g id="line2d_25">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="242.241146" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_27">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="246.039974" transform="rotate(-0 394.826406 246.039974)">5.0</text>
+     </g>
+    </g>
+    <g id="ytick_10">
+     <g id="line2d_26">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="211.517917" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_28">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="215.316745" transform="rotate(-0 394.826406 215.316745)">5.5</text>
+     </g>
+    </g>
+    <g id="ytick_11">
+     <g id="line2d_27">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="180.794688" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_29">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="184.593516" transform="rotate(-0 394.826406 184.593516)">6.0</text>
+     </g>
+    </g>
+    <g id="ytick_12">
+     <g id="line2d_28">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="150.071459" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_30">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="153.870287" transform="rotate(-0 394.826406 153.870287)">6.5</text>
+     </g>
+    </g>
+    <g id="ytick_13">
+     <g id="line2d_29">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="119.34823" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_31">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="123.147058" transform="rotate(-0 394.826406 123.147058)">7.0</text>
+     </g>
+    </g>
+    <g id="ytick_14">
+     <g id="line2d_30">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="88.625" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_32">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="92.423829" transform="rotate(-0 394.826406 92.423829)">7.5</text>
+     </g>
+    </g>
+    <g id="ytick_15">
+     <g id="line2d_31">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="57.901771" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_33">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="61.700599" transform="rotate(-0 394.826406 61.700599)">8.0</text>
+     </g>
+    </g>
+    <g id="text_34">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="372.520938" y="146.599063" transform="rotate(-90 372.520938 146.599063)">Maximum absolute probability error</text>
+    </g>
+    <g id="text_35">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="401.826406" y="42.880469" transform="rotate(-0 401.826406 42.880469)">1e−7</text>
+    </g>
+   </g>
+   <g id="line2d_32">
+    <path d="M 415.617209 238.16142 
+L 470.780419 55.036705 
+L 525.943629 146.599063 
+L 581.106839 109.974119 
+L 636.27005 183.224006 
+L 691.43326 219.848949 
+" clip-path="url(#p601cd532af)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <g clip-path="url(#p601cd532af)">
+     <use xlink:href="#m9b4cc9d8d2" x="415.617209" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="470.780419" y="55.036705" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="525.943629" y="146.599063" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="581.106839" y="109.974119" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="636.27005" y="183.224006" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="691.43326" y="219.848949" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_33">
+    <path d="M 415.617209 219.848949 
+L 470.780419 183.224006 
+L 525.943629 128.286591 
+L 581.106839 109.974119 
+L 636.27005 183.224006 
+L 691.43326 146.599063 
+" clip-path="url(#p601cd532af)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <g clip-path="url(#p601cd532af)">
+     <use xlink:href="#m556f16804f" x="415.617209" y="219.848949" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="470.780419" y="183.224006" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="525.943629" y="128.286591" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="581.106839" y="109.974119" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="636.27005" y="183.224006" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="691.43326" y="146.599063" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="patch_9">
+    <path d="M 401.826406 247.317656 
+L 401.826406 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_10">
+    <path d="M 705.224062 247.317656 
+L 705.224062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_11">
+    <path d="M 401.826406 247.317656 
+L 705.224062 247.317656 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_12">
+    <path d="M 401.826406 45.880469 
+L 705.224062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="legend_2">
+    <g id="patch_13">
+     <path d="M 407.426406 76.281719 
+L 458.768906 76.281719 
+Q 460.368906 76.281719 460.368906 74.681719 
+L 460.368906 51.480469 
+Q 460.368906 49.880469 458.768906 49.880469 
+L 407.426406 49.880469 
+Q 405.826406 49.880469 405.826406 51.480469 
+L 405.826406 74.681719 
+Q 405.826406 76.281719 407.426406 76.281719 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_34">
+     <path d="M 409.026406 56.359219 
+L 417.026406 56.359219 
+L 425.026406 56.359219 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m9b4cc9d8d2" x="417.026406" y="56.359219" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_36">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="431.426406" y="59.159219" transform="rotate(-0 431.426406 59.159219)">q_to_k</text>
+    </g>
+    <g id="line2d_35">
+     <path d="M 409.026406 68.359844 
+L 417.026406 68.359844 
+L 425.026406 68.359844 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m556f16804f" x="417.026406" y="68.359844" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_37">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="431.426406" y="71.159844" transform="rotate(-0 431.426406 71.159844)">k_to_q</text>
+    </g>
+   </g>
+  </g>
+  <g id="text_38">
+   <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="356.024063" y="16.318125" transform="rotate(-0 356.024063 16.318125)">Supplement: algebraic reparameterization, separate from learned-function repetition</text>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="p0542a2a98b">
+   <rect x="50.466406" y="45.880469" width="303.397656" height="201.437187"/>
+  </clipPath>
+  <clipPath id="p601cd532af">
+   <rect x="401.826406" y="45.880469" width="303.397656" height="201.437187"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/figures/candidate_function_errors.svg

@@ -0,0 +1,467 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="712.218437pt" height="280.996571pt" viewBox="0 0 712.218437 280.996571" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:10:51.395327</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 280.996571 
+L 712.218437 280.996571 
+L 712.218437 0 
+L 0 0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 51.700781 195.511975 
+L 705.018437 195.511975 
+L 705.018437 37.922578 
+L 51.700781 37.922578 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="patch_3">
+    <path d="M 81.397038 76335.575007 
+L 125.391493 76335.575007 
+L 125.391493 64.33392 
+L 81.397038 64.33392 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 136.390107 76335.575007 
+L 180.384562 76335.575007 
+L 180.384562 45.085733 
+L 136.390107 45.085733 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 191.383176 76335.575007 
+L 235.377631 76335.575007 
+L 235.377631 45.525425 
+L 191.383176 45.525425 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 246.376244 76335.575007 
+L 290.370699 76335.575007 
+L 290.370699 76.171564 
+L 246.376244 76.171564 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_7">
+    <path d="M 301.369313 76335.575007 
+L 345.363768 76335.575007 
+L 345.363768 76.722901 
+L 301.369313 76.722901 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_8">
+    <path d="M 356.362382 76335.575007 
+L 400.356837 76335.575007 
+L 400.356837 136.725903 
+L 356.362382 136.725903 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_9">
+    <path d="M 411.355451 76335.575007 
+L 455.349906 76335.575007 
+L 455.349906 141.06981 
+L 411.355451 141.06981 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_10">
+    <path d="M 466.348519 76335.575007 
+L 510.342974 76335.575007 
+L 510.342974 140.370183 
+L 466.348519 140.370183 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_11">
+    <path d="M 521.341588 76335.575007 
+L 565.336043 76335.575007 
+L 565.336043 147.505598 
+L 521.341588 147.505598 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_12">
+    <path d="M 576.334657 76335.575007 
+L 620.329112 76335.575007 
+L 620.329112 123.343194 
+L 576.334657 123.343194 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_13">
+    <path d="M 631.327725 76335.575007 
+L 675.32218 76335.575007 
+L 675.32218 109.03712 
+L 631.327725 109.03712 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="mc7fdb9fc78" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="103.394266" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(71.267864 238.312018) rotate(-45)">attn_3_7</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="158.387335" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(117.262999 247.309952) rotate(-45)">attn_11_15</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="213.380403" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(172.256067 247.309952) rotate(-45)">attn_19_23</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="268.373472" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(237.264641 237.295) rotate(-45)">mlp_5_6</text>
+     </g>
+    </g>
+    <g id="xtick_5">
+     <g id="line2d_5">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="323.366541" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_5">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(283.259775 246.292933) rotate(-45)">mlp_20_21</text>
+     </g>
+    </g>
+    <g id="xtick_6">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="378.359609" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(338.028559 246.517219) rotate(-45)">qk_norm_3</text>
+     </g>
+    </g>
+    <g id="xtick_7">
+     <g id="line2d_7">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="433.352678" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_7">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(393.021627 246.517219) rotate(-45)">qk_norm_7</text>
+     </g>
+    </g>
+    <g id="xtick_8">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="488.345747" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(443.515729 251.016186) rotate(-45)">qk_norm_11</text>
+     </g>
+    </g>
+    <g id="xtick_9">
+     <g id="line2d_9">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="543.338815" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(498.508798 251.016186) rotate(-45)">qk_norm_15</text>
+     </g>
+    </g>
+    <g id="xtick_10">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="598.331884" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(553.501867 251.016186) rotate(-45)">qk_norm_19</text>
+     </g>
+    </g>
+    <g id="xtick_11">
+     <g id="line2d_11">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="653.324953" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(608.494935 251.016186) rotate(-45)">qk_norm_23</text>
+     </g>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_12">
+      <defs>
+       <path id="mdcab3d4c70" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#mdcab3d4c70" x="51.700781" y="165.39636" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_12">
+      <!-- $\mathdefault{10^{-1}}$ -->
+      <g transform="translate(21.200781 170.04636)">
+       <text>
+        <tspan x="0" y="-0.762187" style="font-size: 10px; font-family: 'DejaVu Sans'">1</tspan>
+        <tspan x="6.362305" y="-0.762187" style="font-size: 10px; font-family: 'DejaVu Sans'">0</tspan>
+        <tspan x="12.820312" y="-4.892187" style="font-size: 7px; font-family: 'DejaVu Sans'">−</tspan>
+        <tspan x="18.685547" y="-4.892187" style="font-size: 7px; font-family: 'DejaVu Sans'">1</tspan>
+       </text>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_13">
+      <g>
+       <use xlink:href="#mdcab3d4c70" x="51.700781" y="89.149935" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_13">
+      <!-- $\mathdefault{10^{0}}$ -->
+      <g transform="translate(27.100781 93.849935)">
+       <text>
+        <tspan x="0" y="-0.674688" style="font-size: 10px; font-family: 'DejaVu Sans'">1</tspan>
+        <tspan x="6.362305" y="-0.674688" style="font-size: 10px; font-family: 'DejaVu Sans'">0</tspan>
+        <tspan x="12.820312" y="-4.804688" style="font-size: 7px; font-family: 'DejaVu Sans'">0</tspan>
+       </text>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_14">
+      <defs>
+       <path id="md42a30ea1c" d="M 0 0 
+L -2 0 
+" style="stroke: #000000; stroke-width: 0.6"/>
+      </defs>
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="188.348821" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_15">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="182.311534" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_16">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="177.20708" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_17">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="172.785402" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_7">
+     <g id="line2d_18">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="168.885205" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_8">
+     <g id="line2d_19">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="142.443899" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_9">
+     <g id="line2d_20">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="129.01757" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_10">
+     <g id="line2d_21">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="119.491438" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_11">
+     <g id="line2d_22">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="112.102396" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_12">
+     <g id="line2d_23">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="106.065109" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_13">
+     <g id="line2d_24">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="100.960655" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_14">
+     <g id="line2d_25">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="96.538977" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_15">
+     <g id="line2d_26">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="92.63878" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_16">
+     <g id="line2d_27">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="66.197474" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_17">
+     <g id="line2d_28">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="52.771145" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_18">
+     <g id="line2d_29">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="43.245012" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="text_14">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="116.717277" transform="rotate(-90 14.798438 116.717277)">Worst calibration directional relative L2</text>
+    </g>
+   </g>
+   <g id="line2d_30">
+    <path d="M 51.700781 188.348821 
+L 705.018437 188.348821 
+" clip-path="url(#pf9e7526e4c)" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+   </g>
+   <g id="patch_14">
+    <path d="M 51.700781 195.511975 
+L 51.700781 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_15">
+    <path d="M 705.018437 195.511975 
+L 705.018437 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_16">
+    <path d="M 51.700781 195.511975 
+L 705.018437 195.511975 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_17">
+    <path d="M 51.700781 37.922578 
+L 705.018437 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="text_15">
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(226.773359 17.519297)">Fixed candidates; same-input function comparison</text>
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(254.193359 31.922578)">4 calibration texts, 32 tokens; no training</text>
+   </g>
+   <g id="legend_1">
+    <g id="patch_18">
+     <path d="M 57.300781 273.796571 
+L 280.582031 273.796571 
+Q 282.182031 273.796571 282.182031 272.196571 
+L 282.182031 260.995946 
+Q 282.182031 259.395946 280.582031 259.395946 
+L 57.300781 259.395946 
+Q 55.700781 259.395946 55.700781 260.995946 
+L 55.700781 272.196571 
+Q 55.700781 273.796571 57.300781 273.796571 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_31">
+     <path d="M 58.900781 265.874696 
+L 66.900781 265.874696 
+L 74.900781 265.874696 
+" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+    </g>
+    <g id="text_16">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="81.300781" y="268.674696" transform="rotate(-0 81.300781 268.674696)">Relative L2 screen = 0.05; other gates also apply</text>
+    </g>
+   </g>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="pf9e7526e4c">
+   <rect x="51.700781" y="37.922578" width="653.317656" height="157.589397"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/figures/learned_norm_function_errors.svg

@@ -0,0 +1,388 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="640.424062pt" height="312.347793pt" viewBox="0 0 640.424062 312.347793" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:10:55.472730</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 312.347793 
+L 640.424062 312.347793 
+L 640.424062 0 
+L 0 0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 50.466406 208.18075 
+L 633.224062 208.18075 
+L 633.224062 37.922578 
+L 50.466406 37.922578 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="mb7bbc15d82" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="76.955391" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(41.486676 241.545565) rotate(-30)">cal-zh-1</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="152.638203" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(117.169489 241.545565) rotate(-30)">cal-zh-2</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="228.321016" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(192.068819 241.997909) rotate(-30)">cal-en-1</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="304.003828" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(267.751631 241.997909) rotate(-30)">cal-en-2</text>
+     </g>
+    </g>
+    <g id="xtick_5">
+     <g id="line2d_5">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="379.686641" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_5">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(340.062357 243.944784) rotate(-30)">test-zh-1</text>
+     </g>
+    </g>
+    <g id="xtick_6">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="455.369453" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(415.74517 243.944784) rotate(-30)">test-zh-2</text>
+     </g>
+    </g>
+    <g id="xtick_7">
+     <g id="line2d_7">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="531.052266" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_7">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(490.6445 244.396451) rotate(-30)">test-en-1</text>
+     </g>
+    </g>
+    <g id="xtick_8">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="606.735078" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(566.327312 244.396451) rotate(-30)">test-en-2</text>
+     </g>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_9">
+      <defs>
+       <path id="ma0713fe695" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="204.137497" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="207.936325" transform="rotate(-0 43.466406 207.936325)">0.04</text>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="182.54738" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="186.346208" transform="rotate(-0 43.466406 186.346208)">0.05</text>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_11">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="160.957263" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="164.756091" transform="rotate(-0 43.466406 164.756091)">0.06</text>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_12">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="139.367146" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_12">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="143.165974" transform="rotate(-0 43.466406 143.165974)">0.07</text>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_13">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="117.777029" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_13">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="121.575857" transform="rotate(-0 43.466406 121.575857)">0.08</text>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_14">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="96.186911" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_14">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="99.98574" transform="rotate(-0 43.466406 99.98574)">0.09</text>
+     </g>
+    </g>
+    <g id="ytick_7">
+     <g id="line2d_15">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="74.596794" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_15">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="78.395623" transform="rotate(-0 43.466406 78.395623)">0.10</text>
+     </g>
+    </g>
+    <g id="ytick_8">
+     <g id="line2d_16">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="53.006677" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_16">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="56.805505" transform="rotate(-0 43.466406 56.805505)">0.11</text>
+     </g>
+    </g>
+    <g id="text_17">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="123.051664" transform="rotate(-90 14.798438 123.051664)">Worst of two directional relative L2 errors</text>
+    </g>
+   </g>
+   <g id="line2d_17">
+    <path d="M 76.955391 197.451805 
+L 152.638203 192.877368 
+L 228.321016 193.610655 
+L 304.003828 199.099794 
+L 379.686641 196.376861 
+L 455.369453 192.726888 
+L 531.052266 194.835109 
+L 606.735078 198.558148 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m8708773af9" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #1f77b4"/>
+    </defs>
+    <g clip-path="url(#pc5e8d93515)">
+     <use xlink:href="#m8708773af9" x="76.955391" y="197.451805" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="152.638203" y="192.877368" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="228.321016" y="193.610655" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="304.003828" y="199.099794" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="379.686641" y="196.376861" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="455.369453" y="192.726888" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="531.052266" y="194.835109" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="606.735078" y="198.558148" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_18">
+    <path d="M 76.955391 198.934893 
+L 152.638203 194.123521 
+L 228.321016 195.5305 
+L 304.003828 200.441742 
+L 379.686641 197.894474 
+L 455.369453 194.413382 
+L 531.052266 196.725517 
+L 606.735078 199.872446 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m5a845d07a0" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #ff7f0e"/>
+    </defs>
+    <g clip-path="url(#pc5e8d93515)">
+     <use xlink:href="#m5a845d07a0" x="76.955391" y="198.934893" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="152.638203" y="194.123521" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="228.321016" y="195.5305" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="304.003828" y="200.441742" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="379.686641" y="197.894474" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="455.369453" y="194.413382" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="531.052266" y="196.725517" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="606.735078" y="199.872446" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="line2d_19">
+    <path d="M 50.466406 182.54738 
+L 633.224062 182.54738 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+   </g>
+   <g id="line2d_20">
+    <path d="M 50.466406 45.661586 
+L 633.224062 45.661586 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #6a51a3; stroke-width: 1.5"/>
+   </g>
+   <g id="line2d_21">
+    <path d="M 341.845234 208.18075 
+L 341.845234 37.922578 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke: #808080; stroke-width: 0.8; stroke-linecap: square"/>
+   </g>
+   <g id="patch_3">
+    <path d="M 50.466406 208.18075 
+L 50.466406 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 633.224062 208.18075 
+L 633.224062 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 50.466406 208.18075 
+L 633.224063 208.18075 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 50.466406 37.922578 
+L 633.224063 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="text_18">
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(126.937422 17.519297)">Learned input RMSNorm at layer indices 12/13: same-input comparison</text>
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(137.227422 31.922578)">Calibration (left) / independent heldout texts (right), 32 tokens each</text>
+   </g>
+   <g id="legend_1">
+    <g id="patch_7">
+     <path d="M 56.066406 305.147793 
+L 211.338906 305.147793 
+Q 212.938906 305.147793 212.938906 303.547793 
+L 212.938906 256.345293 
+Q 212.938906 254.745293 211.338906 254.745293 
+L 56.066406 254.745293 
+Q 54.466406 254.745293 54.466406 256.345293 
+L 54.466406 303.547793 
+Q 54.466406 305.147793 56.066406 305.147793 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_22">
+     <path d="M 57.666406 261.224043 
+L 65.666406 261.224043 
+L 73.666406 261.224043 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m8708773af9" x="65.666406" y="261.224043" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_19">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="264.024043" transform="rotate(-0 80.066406 264.024043)">Input captured at layer 12</text>
+    </g>
+    <g id="line2d_23">
+     <path d="M 57.666406 273.224668 
+L 65.666406 273.224668 
+L 73.666406 273.224668 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m5a845d07a0" x="65.666406" y="273.224668" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_20">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="276.024668" transform="rotate(-0 80.066406 276.024668)">Input captured at layer 13</text>
+    </g>
+    <g id="line2d_24">
+     <path d="M 57.666406 285.225293 
+L 65.666406 285.225293 
+L 73.666406 285.225293 
+" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+    </g>
+    <g id="text_21">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="288.025293" transform="rotate(-0 80.066406 288.025293)">Heldout empirical screen = 0.05</text>
+    </g>
+    <g id="line2d_25">
+     <path d="M 57.666406 297.225918 
+L 65.666406 297.225918 
+L 73.666406 297.225918 
+" style="fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #6a51a3; stroke-width: 1.5"/>
+    </g>
+    <g id="text_22">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="300.025918" transform="rotate(-0 80.066406 300.025918)">Proven uniform all-input bound</text>
+    </g>
+   </g>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="pc5e8d93515">
+   <rect x="50.466406" y="37.922578" width="582.757656" height="170.258172"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/figures/learned_norm_network_substitution.svg

@@ -0,0 +1,286 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="567.92875pt" height="308.816598pt" viewBox="0 0 567.92875 308.816598" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:10:58.596152</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 308.816598 
+L 567.92875 308.816598 
+L 567.92875 0 
+L 0 0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 71.571094 233.247975 
+L 560.72875 233.247975 
+L 560.72875 22.318125 
+L 71.571094 22.318125 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="m4de31d702a" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m4de31d702a" x="93.805533" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="93.805533" y="247.846412" transform="rotate(-0 93.805533 247.846412)">zh1</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#m4de31d702a" x="242.035125" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="242.035125" y="247.846412" transform="rotate(-0 242.035125 247.846412)">zh2</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#m4de31d702a" x="390.264718" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="390.264718" y="247.845631" transform="rotate(-0 390.264718 247.845631)">en1</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#m4de31d702a" x="538.494311" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="538.494311" y="247.845631" transform="rotate(-0 538.494311 247.845631)">en2</text>
+     </g>
+    </g>
+    <g id="text_5">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="316.149922" y="261.847194" transform="rotate(-0 316.149922 261.847194)">Independent heldout text (32 tokens)</text>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_5">
+      <defs>
+       <path id="m32017c2f5a" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="231.508072" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="235.3069" transform="rotate(-0 64.571094 235.3069)">−0.0015</text>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="192.395623" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_7">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="196.194451" transform="rotate(-0 64.571094 196.194451)">−0.0010</text>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_7">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="153.283173" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="157.082001" transform="rotate(-0 64.571094 157.082001)">−0.0005</text>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="114.170724" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="117.969552" transform="rotate(-0 64.571094 117.969552)">0.0000</text>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_9">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="75.058274" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="78.857102" transform="rotate(-0 64.571094 78.857102)">0.0005</text>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="35.945825" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="39.744653" transform="rotate(-0 64.571094 39.744653)">0.0010</text>
+     </g>
+    </g>
+    <g id="text_12">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="127.78305" transform="rotate(-90 14.798438 127.78305)">Mean token NLL change (nat/token)</text>
+    </g>
+   </g>
+   <g id="line2d_11">
+    <path d="M 93.805533 31.905845 
+L 242.035125 223.660255 
+L 390.264718 124.077707 
+L 538.494311 66.84947 
+" clip-path="url(#p78d521230c)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="ma59ebf376b" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #1f77b4"/>
+    </defs>
+    <g clip-path="url(#p78d521230c)">
+     <use xlink:href="#ma59ebf376b" x="93.805533" y="31.905845" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#ma59ebf376b" x="242.035125" y="223.660255" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#ma59ebf376b" x="390.264718" y="124.077707" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#ma59ebf376b" x="538.494311" y="66.84947" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_12">
+    <path d="M 93.805533 45.225019 
+L 242.035125 215.36524 
+L 390.264718 134.743944 
+L 538.494311 175.235695 
+" clip-path="url(#p78d521230c)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m2ce5b34917" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #ff7f0e"/>
+    </defs>
+    <g clip-path="url(#p78d521230c)">
+     <use xlink:href="#m2ce5b34917" x="93.805533" y="45.225019" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m2ce5b34917" x="242.035125" y="215.36524" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m2ce5b34917" x="390.264718" y="134.743944" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m2ce5b34917" x="538.494311" y="175.235695" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="line2d_13">
+    <path d="M 71.571094 114.170724 
+L 560.72875 114.170724 
+" clip-path="url(#p78d521230c)" style="fill: none; stroke: #808080; stroke-width: 0.8; stroke-linecap: square"/>
+   </g>
+   <g id="patch_3">
+    <path d="M 71.571094 233.247975 
+L 71.571094 22.318125 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 560.72875 233.247975 
+L 560.72875 22.318125 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 71.571094 233.247975 
+L 560.72875 233.247975 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 71.571094 22.318125 
+L 560.72875 22.318125 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="text_13">
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="316.149922" y="16.318125" transform="rotate(-0 316.149922 16.318125)">Bidirectional single-site substitution of existing learned RMSNorm modules</text>
+   </g>
+   <g id="legend_1">
+    <g id="patch_7">
+     <path d="M 77.171094 301.616598 
+L 257.259844 301.616598 
+Q 258.859844 301.616598 258.859844 300.016598 
+L 258.859844 276.815348 
+Q 258.859844 275.215348 257.259844 275.215348 
+L 77.171094 275.215348 
+Q 75.571094 275.215348 75.571094 276.815348 
+L 75.571094 300.016598 
+Q 75.571094 301.616598 77.171094 301.616598 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_14">
+     <path d="M 78.771094 281.694098 
+L 86.771094 281.694098 
+L 94.771094 281.694098 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#ma59ebf376b" x="86.771094" y="281.694098" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_14">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="101.171094" y="284.494098" transform="rotate(-0 101.171094 284.494098)">Layer 12 uses 13; mean=3.2565e-05</text>
+    </g>
+    <g id="line2d_15">
+     <path d="M 78.771094 293.694723 
+L 86.771094 293.694723 
+L 94.771094 293.694723 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m2ce5b34917" x="86.771094" y="293.694723" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_15">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="101.171094" y="296.494723" transform="rotate(-0 101.171094 296.494723)">Layer 13 uses 12; mean=-0.00036397</text>
+    </g>
+   </g>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="p78d521230c">
+   <rect x="71.571094" y="22.318125" width="489.157656" height="210.92985"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/final-source-hashes.json

@@ -0,0 +1,18 @@
+{
+  "functional/__init__.py": "e572d3a9040dfb440b92e969df495d2a9e0edbb1bfe1318e5e66841cc6144067",
+  "functional/audit.py": "5e10c858fe574f4c9a51cdb9c61a94932d9e4bb6dac82082a94c2ef7d57a693b",
+  "functional/benchmark_factorization.py": "135759dbe1af48b86e327cf23a1ef656d000dfef83dea2dbf48d2514356ea729",
+  "functional/gain_factorization.py": "33f3620027c50bce913ad07ed12f42fad83d69a811652174c803467c295e77c9",
+  "functional/model.py": "48a2a97d0445dc6e08ac7f27a935637a7553417d0982eb94122d6cb02bbb499c",
+  "functional/neuron_reuse.py": "d408bc14242c7950b846b6d91870d88374cdb8ece7fc111a2ccdc25e725ca19c",
+  "functional/norm_repetition.py": "aac0c44904c2b059510a754f2f0b1247ffb4b2a1715ecee6281b5ed2b43612b3",
+  "functional/preflight.py": "87c13d21512e8cbb82bc8a8aeb614a3be06bddbb8489c52eb2bf316480b0f20c",
+  "functional/reference.json": "af6bf3040369580de3dff4a6733576a75c61bcbd7e94201d309cf28e9b75e3d6",
+  "functional/reference_check.py": "422cd70c4f1276590402b10261aff0fc744a656387eb260b54bac303353c277c",
+  "functional/report.py": "ca74122de373e80429fab0f8c2e2d0dba3f8f7c6ce046510e5bf28fa7b8d827d",
+  "functional/run.py": "89d3abe098c240ec29c3f6684a94e24b951c372e1393c8db5a48f79ef141aa48",
+  "functional/storage.py": "8387fac8c192fe050e6d275dff6fe55a405332ee18c09f3e3b5d4d537a3db59e",
+  "functional/supervise.py": "44c01fd9ba989ed68ef58313e490602a49f5da26aeb918ee0b6545aff766144e",
+  "pilot/tensors.py": "2e620849501b6bc98a59cbf9c39495326b0c85530e18e6166aa9edb68ac2423e",
+  "tests/test_functional.py": "82c8585f6a9f4e6323cd978621639163298a369bdf298dfa5103c020c3e71f38"
+}

results/cpu-qwen35-2b/final-tests.txt

@@ -0,0 +1,23 @@
+test_causal_convolution_and_norm_boundary (test_functional.FunctionalTests.test_causal_convolution_and_norm_boundary) ... ok
+test_causality_and_identical_function_metrics (test_functional.FunctionalTests.test_causality_and_identical_function_metrics) ... ok
+test_neuron_function_metrics_and_targeted_replacement (test_functional.FunctionalTests.test_neuron_function_metrics_and_targeted_replacement) ... ok
+test_nonrotary_gain_identity_and_rotary_negative_control (test_functional.FunctionalTests.test_nonrotary_gain_identity_and_rotary_negative_control) ... ok
+test_streamed_projection_embedding_and_output_head (test_functional.FunctionalTests.test_streamed_projection_embedding_and_output_head) ... ok
+test_uniform_rmsnorm_function_bound_is_tight (test_functional.FunctionalTests.test_uniform_rmsnorm_function_bound_is_tight) ... ok
+test_3d_reader_and_exact_block (test_pilot.PilotTests.test_3d_reader_and_exact_block) ... ok
+test_bfloat16 (test_pilot.PilotTests.test_bfloat16) ... ok
+test_directional_replacement_error_and_scaling (test_pilot.PilotTests.test_directional_replacement_error_and_scaling) ... ok
+test_end_to_end_synthetic_groups_and_savings (test_pilot.PilotTests.test_end_to_end_synthetic_groups_and_savings) ... Inventory: 4 tensors, 33792 bytes
+tensor: sampling 4 objects
+block: sampling 22 objects
+block: verified 100/109
+{"model": "synthetic", "revision": "test", "tensor_count": 4, "parameter_count": 8448, "model_weight_bytes": 33792, "ndim_counts": {"2": 2, "3": 2}, "elapsed_seconds": 0.1379160056822002, "analysis_budget_seconds": 60, "peak_rss_kib": 53592, "python": "3.14.4", "numpy": "2.5.4", "seed": 20261109, "limitations": ["No inference or functional equivalence measured", "No exhaustive pair search", "Block search uses per-role/shape/scale reservoirs of 256", "Different block scales overlap; savings must not be added", "Block savings count only verified disjoint sampled blocks, no extrapolation", "Spectral probes use aligned bounded subvolumes, not whole tensors", "Shared encoding is theoretical, not a runnable compressed checkpoint", "No cross-role grouping, neuron permutation alignment, or training"], "completed": true}
+ok
+test_no_transitive_chaining (test_pilot.PilotTests.test_no_transitive_chaining) ... ok
+test_ranged_download_preserves_prefix (test_pilot.PilotTests.test_ranged_download_preserves_prefix) ... Weight download 36/36 bytes (100.0%)
+ok
+
+----------------------------------------------------------------------
+Ran 12 tests in 40.946s
+
+OK

results/cpu-qwen35-2b/functional-code-hashes.json

@@ -0,0 +1,12 @@
+{
+  "functional/__init__.py": "e572d3a9040dfb440b92e969df495d2a9e0edbb1bfe1318e5e66841cc6144067",
+  "functional/model.py": "48a2a97d0445dc6e08ac7f27a935637a7553417d0982eb94122d6cb02bbb499c",
+  "functional/preflight.py": "87c13d21512e8cbb82bc8a8aeb614a3be06bddbb8489c52eb2bf316480b0f20c",
+  "functional/reference.json": "af6bf3040369580de3dff4a6733576a75c61bcbd7e94201d309cf28e9b75e3d6",
+  "functional/reference_check.py": "422cd70c4f1276590402b10261aff0fc744a656387eb260b54bac303353c277c",
+  "functional/report.py": "e52201fa910a062b8521b4b1b2ea7ac80d276dca06fa8a322b9d3cb926ce100d",
+  "functional/run.py": "89d3abe098c240ec29c3f6684a94e24b951c372e1393c8db5a48f79ef141aa48",
+  "functional/samples.json": "bccb6ed515a8703c6ab2fc29b5227e1d11823b2ac94c95495ecf24a0340ba103",
+  "functional/storage.py": "8387fac8c192fe050e6d275dff6fe55a405332ee18c09f3e3b5d4d537a3db59e",
+  "functional/supervise.py": "44c01fd9ba989ed68ef58313e490602a49f5da26aeb918ee0b6545aff766144e"
+}

results/cpu-qwen35-2b/functional-environment.json

@@ -0,0 +1,34 @@
+{
+  "python": "3.14.4",
+  "platform": "Linux-7.0.0-31-generic-x86_64-with-glibc2.43",
+  "numpy": "2.5.4",
+  "packages": [
+    "contourpy==1.4.0",
+    "cycler==0.12.1",
+    "filelock==4.2.4",
+    "fonttools==4.68.0",
+    "fsspec==2026.9.0",
+    "Jinja2==3.1.6",
+    "kiwisolver==1.5.1",
+    "MarkupSafe==3.0.3",
+    "matplotlib==3.11.2",
+    "mpmath==1.3.0",
+    "networkx==3.7",
+    "numpy==2.5.4",
+    "packaging==26.3",
+    "pillow==13.0.0",
+    "pyparsing==3.3.3",
+    "python-dateutil==4.0.0",
+    "setuptools==78.1.0",
+    "sympy==1.14.0",
+    "tokenizers==0.23.3",
+    "torch==2.15.0+cpu",
+    "typing_extensions==4.16.0",
+    "tzdata==2026.6"
+  ],
+  "weights_retained": true,
+  "no_automatic_cleanup": true,
+  "ledger_seconds_including_initial_reserve": 1167.355532555841,
+  "final_tests_report_audit_additional_budget_upper_reserve_seconds": 600,
+  "historical_error_logs": "Early reference adapter errors and preflight pressure stops; latest summaries/tests are authoritative."
+}

results/cpu-qwen35-2b/functional-run.log

@@ -0,0 +1,24 @@
+{"phase": "baseline", "layer": 0, "elapsed": 6.704378366470337}
+{"phase": "baseline", "layer": 1, "elapsed": 12.428538799285889}
+{"phase": "baseline", "layer": 2, "elapsed": 18.311316967010498}
+{"phase": "baseline", "layer": 3, "elapsed": 22.585159063339233}
+{"phase": "baseline", "layer": 4, "elapsed": 27.968208074569702}
+{"phase": "baseline", "layer": 5, "elapsed": 34.463871240615845}
+{"phase": "baseline", "layer": 6, "elapsed": 41.34764099121094}
+{"phase": "baseline", "layer": 7, "elapsed": 44.77713179588318}
+{"phase": "baseline", "layer": 8, "elapsed": 50.20741152763367}
+{"phase": "baseline", "layer": 9, "elapsed": 55.92143273353577}
+{"phase": "baseline", "layer": 10, "elapsed": 62.15613389015198}
+{"phase": "baseline", "layer": 11, "elapsed": 65.01726865768433}
+{"phase": "baseline", "layer": 12, "elapsed": 70.20210671424866}
+{"phase": "baseline", "layer": 13, "elapsed": 76.30333161354065}
+{"phase": "baseline", "layer": 14, "elapsed": 81.29059743881226}
+{"phase": "baseline", "layer": 15, "elapsed": 85.60254788398743}
+{"phase": "baseline", "layer": 16, "elapsed": 91.78613877296448}
+{"phase": "baseline", "layer": 17, "elapsed": 98.67645645141602}
+{"phase": "baseline", "layer": 18, "elapsed": 106.65264892578125}
+{"phase": "baseline", "layer": 19, "elapsed": 109.9351601600647}
+{"phase": "baseline", "layer": 20, "elapsed": 116.18031859397888}
+{"phase": "baseline", "layer": 21, "elapsed": 122.5530002117157}
+{"phase": "baseline", "layer": 22, "elapsed": 128.36239457130432}
+{"phase": "baseline", "layer": 23, "elapsed": 131.58471012115479}

results/cpu-qwen35-2b/functional-status.json

@@ -0,0 +1,11 @@
+{
+  "state": "completed",
+  "started_unix": 1794202338.6853228,
+  "phase": "done",
+  "last_completed_layer": 23,
+  "elapsed_seconds": 196.65513348579407,
+  "selected_candidate": [],
+  "positive_bidirectional_quality": false,
+  "peak_rss_kib": 162372,
+  "claim_limit": "Small text-only heldout pilot; no global interchangeability proof, no measured GPU speedup; component storage savings require explicit layout accounting."
+}

results/cpu-qwen35-2b/gain-factorization-kernel-timing.csv

@@ -0,0 +1,21 @@
+repeat,implementation,seconds_per_call
+0,baseline,0.0036114158341661096
+0,factored,0.003925090050324797
+1,factored,0.0032154972897842525
+1,baseline,0.0025859603192657232
+2,baseline,0.0023861153982579707
+2,factored,0.0025824901415035127
+3,factored,0.001997539587318897
+3,baseline,0.0016429506475105881
+4,baseline,0.0017620180500671268
+4,factored,0.0027307189535349607
+5,factored,0.002951206616126001
+5,baseline,0.0020800257567316293
+6,baseline,0.0018509919987991451
+6,factored,0.0024777360958978534
+7,factored,0.0025671867886558177
+7,baseline,0.002200991357676685
+8,baseline,0.0020182608393952252
+8,factored,0.002254301542416215
+9,factored,0.0025048630544915796
+9,baseline,0.002296067262068391

results/cpu-qwen35-2b/gain-factorization-optimized-summary.json

@@ -0,0 +1,102 @@
+{
+  "state": "completed",
+  "bitwise_checks": [
+    {
+      "layer": 3,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 3,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 7,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 7,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 11,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 11,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 15,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 15,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 19,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 19,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 23,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 23,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    }
+  ],
+  "medians_seconds": {
+    "baseline_legacy": 0.0005341526120901108,
+    "baseline_inplace": 0.00034534828271716835,
+    "factored_inplace": 0.00048380128573626284
+  },
+  "paired_baseline_over_factored_median": 0.6901853298227001,
+  "paired_ratio_p10": 0.38049978760565983,
+  "paired_ratio_p90": 1.0876508522302268,
+  "fraction_blocks_factored_faster": 0.13333333333333333,
+  "descriptive_local_gain": false,
+  "claim_limit": "Hot-array CPU normalization microbenchmark against equally allocation-optimized baseline; no full-model speedup or independent statistical significance claim.",
+  "elapsed_seconds": 4.185962562914938,
+  "peak_rss_kib": 62168
+}

results/cpu-qwen35-2b/gain-factorization-optimized-timing.csv

@@ -0,0 +1,91 @@
+repeat,implementation,seconds_per_call,calls_per_block
+0,baseline_legacy,0.0005406431574374438,50
+0,baseline_inplace,0.00030974450521171093,50
+0,factored_inplace,0.0004578591044992208,50
+1,baseline_inplace,0.0003033110871911049,50
+1,factored_inplace,0.0005383838433772326,50
+1,baseline_legacy,0.0005124531593173742,50
+2,factored_inplace,0.0004439983796328306,50
+2,baseline_legacy,0.0005330004263669252,50
+2,baseline_inplace,0.00034275135956704614,50
+3,baseline_legacy,0.00048629873432219027,50
+3,baseline_inplace,0.00030624629929661753,50
+3,factored_inplace,0.0004589695204049349,50
+4,baseline_inplace,0.0003086038026958704,50
+4,factored_inplace,0.0005195329245179891,50
+4,baseline_legacy,0.000474175363779068,50
+5,factored_inplace,0.00045118503272533416,50
+5,baseline_legacy,0.00048273258842527865,50
+5,baseline_inplace,0.0003844233974814415,50
+6,baseline_legacy,0.0006650591176003218,50
+6,baseline_inplace,0.0007703853026032448,50
+6,factored_inplace,0.0004403704963624477,50
+7,baseline_inplace,0.00038892081938683986,50
+7,factored_inplace,0.0004549186024814844,50
+7,baseline_legacy,0.0005137955769896508,50
+8,factored_inplace,0.0004692125599831343,50
+8,baseline_legacy,0.0005263624526560307,50
+8,baseline_inplace,0.000349312499165535,50
+9,baseline_legacy,0.0004857080150395632,50
+9,baseline_inplace,0.0012390289548784494,50
+9,factored_inplace,0.0018109613563865423,50
+10,baseline_inplace,0.0013814480416476727,50
+10,factored_inplace,0.0013840428367257118,50
+10,baseline_legacy,0.001071744617074728,50
+11,factored_inplace,0.0016566905751824379,50
+11,baseline_legacy,0.001358420280739665,50
+11,baseline_inplace,0.0003716108575463295,50
+12,baseline_legacy,0.0008325341437011957,50
+12,baseline_inplace,0.0003479452058672905,50
+12,factored_inplace,0.0004391701612621546,50
+13,baseline_inplace,0.0005428373627364636,50
+13,factored_inplace,0.0015458692144602537,50
+13,baseline_legacy,0.0013308324199169874,50
+14,factored_inplace,0.0014238305203616618,50
+14,baseline_legacy,0.0006564708799123764,50
+14,baseline_inplace,0.0003204820398241281,50
+15,baseline_legacy,0.0005279563181102276,50
+15,baseline_inplace,0.00030857514590024946,50
+15,factored_inplace,0.0005265675205737352,50
+16,baseline_inplace,0.000458676153793931,50
+16,factored_inplace,0.00045228853821754455,50
+16,baseline_legacy,0.0004912188835442066,50
+17,factored_inplace,0.0004910450987517834,50
+17,baseline_legacy,0.0005007587838917971,50
+17,baseline_inplace,0.0003247600421309471,50
+18,baseline_legacy,0.0004860841576009989,50
+18,baseline_inplace,0.00032584095373749733,50
+18,factored_inplace,0.0004711645469069481,50
+19,baseline_inplace,0.0003332006186246872,50
+19,factored_inplace,0.0004859814327210188,50
+19,baseline_legacy,0.0006259976234287023,50
+20,factored_inplace,0.0005134599283337593,50
+20,baseline_legacy,0.0006658115796744823,50
+20,baseline_inplace,0.0003131803590804338,50
+21,baseline_legacy,0.00048669124022126197,50
+21,baseline_inplace,0.0003487722761929035,50
+21,factored_inplace,0.00048093565739691256,50
+22,baseline_inplace,0.000310970526188612,50
+22,factored_inplace,0.0004317485820502043,50
+22,baseline_legacy,0.0005281844362616539,50
+23,factored_inplace,0.0004816211387515068,50
+23,baseline_legacy,0.0005441690050065518,50
+23,baseline_inplace,0.0012436784617602826,50
+24,baseline_legacy,0.0013620165828615426,50
+24,baseline_inplace,0.000990196019411087,50
+24,factored_inplace,0.0010849225427955389,50
+25,baseline_inplace,0.0019172614254057408,50
+25,factored_inplace,0.0009820526745170355,50
+25,baseline_legacy,0.0009186737984418869,50
+26,factored_inplace,0.0013663648813962937,50
+26,baseline_legacy,0.0009482460748404264,50
+26,baseline_inplace,0.0005478111188858747,50
+27,baseline_legacy,0.000548997139558196,50
+27,baseline_inplace,0.0003215675055980682,50
+27,factored_inplace,0.0004668480064719915,50
+28,baseline_inplace,0.0003313558641821146,50
+28,factored_inplace,0.00047064563259482386,50
+28,baseline_legacy,0.0005353047978132963,50
+29,factored_inplace,0.0006785523053258657,50
+29,baseline_legacy,0.000497497720643878,50
+29,baseline_inplace,0.000260401563718915,50

results/cpu-qwen35-2b/gain-factorization-protocol.json

@@ -0,0 +1,23 @@
+{
+  "fixed_intervention_layer": 3,
+  "directions": [
+    "q_to_k",
+    "k_to_q"
+  ],
+  "diagnostic_layers": [
+    3,
+    7,
+    11,
+    15,
+    19,
+    23
+  ],
+  "no_selection": true,
+  "no_training": true,
+  "precision": "FP32 product; no BF16 requantization",
+  "quality_gate": {
+    "mean_nll_delta": 0.01,
+    "sequence_p95_nll_delta": 0.05
+  },
+  "registered_unix": 1794202505.9768724
+}

results/cpu-qwen35-2b/gain-factorization-score-metrics.csv

@@ -0,0 +1,97 @@
+layer,direction,sample_id,split,score_max_abs_error,score_relative_l2,probability_max_abs_error,first_key_mass_a,first_key_mass_b,js_without_first_key,without_first_key_valid_rows,js_mean,js_p95,js_without_first_query,top5_overlap
+3,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.101167139945906e-08,3.5762786865234375e-07,0.12333933264017105,0.12333933264017105,6.144742492608657e-09,248,6.2515415066854985e-09,3.3901077500786414e-08,6.453204193235251e-09,1.0
+3,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.091678723372976e-08,5.066394805908203e-07,0.1346932351589203,0.1346932053565979,6.3971117292283e-09,248,5.0883031121884414e-09,3.329163433818394e-08,5.252442036862703e-09,1.0
+3,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.108396212348145e-08,3.725290298461914e-07,0.1436830461025238,0.1436830461025238,6.142506059347852e-09,248,5.038649941724316e-09,2.9201892459695955e-08,5.2011870366186486e-09,1.0
+3,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.1701194575127235e-08,3.8743019104003906e-07,0.1500767469406128,0.1500767469406128,6.898510651609513e-09,248,5.6701336959008586e-09,3.627561184771366e-08,5.853041162851014e-09,1.0
+3,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.220060152896206e-08,4.172325134277344e-07,0.09449940919876099,0.09449941664934158,4.331650149680399e-09,248,5.2700910302405646e-09,3.2252330584015e-08,5.440094152930897e-09,1.0
+3,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.0557365349437256e-08,3.5762786865234375e-07,0.12233515083789825,0.12233515828847885,4.755000837519674e-09,248,5.405037306616123e-09,2.982928037909005e-08,5.579393391741405e-09,1.0
+3,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.036456447011736e-08,4.76837158203125e-07,0.16284234821796417,0.16284234821796417,3.5884786253603806e-09,248,6.1204539214543274e-09,3.24787201577692e-08,6.317887990547888e-09,1.0
+3,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.110061520200993e-08,4.76837158203125e-07,0.1765710860490799,0.1765711009502411,4.401169650947168e-09,248,4.359010929988472e-09,3.009398241715644e-08,4.4996242287709265e-09,1.0
+3,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.1102714936484615e-08,3.2782554626464844e-07,0.12333933264017105,0.12333932518959045,5.54685852804937e-09,248,6.077549130623083e-09,3.364394274285587e-08,6.273598973649541e-09,1.0
+3,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.047857377998922e-08,5.066394805908203e-07,0.1346932351589203,0.1346932202577591,5.5039839352843956e-09,248,5.080743381569164e-09,3.255715697036976e-08,5.244638501267218e-09,1.0
+3,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.134420550367216e-08,4.172325134277344e-07,0.1436830461025238,0.1436830461025238,5.426067595237782e-09,248,6.617717041024207e-09,3.062589826186013e-08,6.831191612377552e-09,1.0
+3,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.226853937237364e-08,5.364418029785156e-07,0.1500767469406128,0.1500767469406128,7.92923859904704e-09,248,3.863607211940234e-09,2.7573278771342302e-08,3.988239516417025e-09,1.0
+3,k_to_q,test-zh-1,heldout,1.9073486328125e-06,5.123835134005697e-08,4.76837158203125e-07,0.09449940919876099,0.09449940919876099,3.631874356813114e-09,248,5.632322164217385e-09,2.992773318055697e-08,5.814010162197292e-09,1.0
+3,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.0224548540677525e-08,4.172325134277344e-07,0.12233515083789825,0.12233515083789825,5.022106286389771e-09,248,4.692121358118584e-09,3.066226383907633e-08,4.843480283511781e-09,1.0
+3,k_to_q,test-en-1,heldout,1.9073486328125e-06,4.9758794253491996e-08,4.76837158203125e-07,0.16284234821796417,0.16284234821796417,3.619864186177324e-09,248,6.1619935820544924e-09,3.4145102745242184e-08,6.360767468294171e-09,1.0
+3,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.142298277550361e-08,2.980232238769531e-07,0.1765710860490799,0.1765710860490799,4.500570138787907e-09,248,5.100083466658134e-09,3.277366289466954e-08,5.264602087606818e-09,1.0
+7,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.222536694121921e-08,4.172325134277344e-07,0.17881479859352112,0.17881479859352112,5.7563616095990255e-09,248,5.532780456007913e-09,3.966695061308201e-08,5.711257244911394e-09,1.0
+7,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.170127993939104e-08,4.470348358154297e-07,0.1558690071105957,0.1558690071105957,4.651088403306858e-09,248,5.351785681284582e-09,4.158310318302938e-08,5.524424029346164e-09,1.0
+7,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.378773497110193e-08,8.046627044677734e-07,0.16970808804035187,0.16970807313919067,7.066096152641421e-09,248,4.739160619493532e-09,3.499418710362079e-08,4.892036553627577e-09,1.0
+7,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.317526550862134e-08,4.76837158203125e-07,0.1774885654449463,0.1774885654449463,2.2907555852214045e-09,248,4.155392474558539e-09,3.55765088499993e-08,4.289437249838102e-09,1.0
+7,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.211860231803155e-08,5.364418029785156e-07,0.13551732897758484,0.13551732897758484,3.869626841179752e-09,248,4.760383198743057e-09,3.7595043522742344e-08,4.91394391843869e-09,1.0
+7,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.3270193367964134e-08,3.8743019104003906e-07,0.16214892268180847,0.16214892268180847,5.834590144360163e-09,248,3.3417555389547715e-09,3.232060308278051e-08,3.4495541978429856e-09,1.0
+7,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.1701003812639285e-08,5.364418029785156e-07,0.16952991485595703,0.16952991485595703,4.620759330720148e-09,248,3.3314184744170916e-09,3.060024056367183e-08,3.4388836223087083e-09,1.0
+7,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.3793891417825e-08,5.364418029785156e-07,0.20624876022338867,0.20624877512454987,5.72924729880242e-09,248,5.260036406440349e-09,3.098259782063906e-08,5.429714899918281e-09,1.0
+7,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.2622550638818123e-08,5.066394805908203e-07,0.17881479859352112,0.17881478369235992,5.618802756401919e-09,248,3.0388158656080577e-09,3.318642782801362e-08,3.1368421193889162e-09,1.0
+7,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.2178797749660394e-08,4.76837158203125e-07,0.1558690071105957,0.1558690220117569,5.6034279438677e-09,248,6.106143590756119e-09,3.9555089870191296e-08,6.3031158070714355e-09,1.0
+7,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.331118520036339e-08,5.364418029785156e-07,0.16970808804035187,0.16970807313919067,4.860980507004342e-09,248,3.6745022580220166e-09,3.220852917706907e-08,3.7930347751569116e-09,1.0
+7,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.3509503240043504e-08,4.172325134277344e-07,0.1774885654449463,0.1774885505437851,2.684577671274724e-09,248,3.914189861120576e-09,3.4250152936010636e-08,4.040454193443566e-09,1.0
+7,k_to_q,test-zh-1,heldout,1.9073486328125e-06,5.186216628978779e-08,5.364418029785156e-07,0.13551732897758484,0.13551732897758484,7.021717873811895e-09,248,3.1829396895943773e-09,3.381542512670421e-08,3.285615113313156e-09,1.0
+7,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.435688576791881e-08,5.960464477539062e-07,0.16214892268180847,0.16214892268180847,4.47947234860635e-09,248,5.434995564712608e-09,3.8530473034370516e-08,5.610317987958524e-09,1.0
+7,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.182678341307826e-08,4.76837158203125e-07,0.16952991485595703,0.16952991485595703,5.902526023504606e-09,248,3.811826410071717e-09,3.153426675339688e-08,3.934788495030261e-09,1.0
+7,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.389036154327309e-08,5.364418029785156e-07,0.20624876022338867,0.20624877512454987,4.393694741366971e-09,248,6.306869693162298e-09,3.442805152076289e-08,6.510317174246438e-09,1.0
+11,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.3373433612649885e-08,4.76837158203125e-07,0.24492809176445007,0.24492809176445007,5.236138189701478e-09,248,4.897512173585028e-09,3.2079633172088506e-08,5.055496465899978e-09,1.0
+11,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.3680472349671996e-08,4.76837158203125e-07,0.18575024604797363,0.18575023114681244,4.706864675796396e-09,248,6.692440379651998e-09,3.841551432515189e-08,6.908325467236409e-09,1.0
+11,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.413658613357613e-08,6.556510925292969e-07,0.24272800981998444,0.24272799491882324,5.221439280944651e-09,248,5.445071948884106e-09,3.742365706216333e-08,5.620719445431632e-09,1.0
+11,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.431145274674744e-08,6.258487701416016e-07,0.2525215446949005,0.2525215148925781,4.965174049686993e-09,248,3.2684353001855015e-09,3.038027784896258e-08,3.373868739942054e-09,1.0
+11,q_to_k,test-zh-1,heldout,1.9073486328125e-06,4.975615695445315e-08,4.76837158203125e-07,0.20683012902736664,0.20683012902736664,4.743543779994752e-09,248,5.727621932294369e-09,3.784743896062537e-08,5.912383915784858e-09,1.0
+11,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.341885392276808e-08,5.066394805908203e-07,0.2263796031475067,0.2263796031475067,5.8260534174792156e-09,248,4.675535514309104e-09,3.7589622081668495e-08,4.826359312204431e-09,1.0
+11,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.3474540038830716e-08,4.76837158203125e-07,0.242807075381279,0.242807075381279,5.4214881473058085e-09,248,3.967941974991618e-09,2.985759550711009e-08,4.095940031589862e-09,1.0
+11,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.447992218148785e-08,5.960464477539062e-07,0.2930097281932831,0.2930097281932831,6.2322125238267745e-09,248,3.5252292196474855e-09,3.577358498318972e-08,3.6389462554353713e-09,1.0
+11,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.28470006223089e-08,4.470348358154297e-07,0.24492809176445007,0.24492809176445007,5.811663150723234e-09,248,5.133418579106319e-09,3.507095414079231e-08,5.29901278412126e-09,1.0
+11,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.340567432956712e-08,5.066394805908203e-07,0.18575024604797363,0.18575024604797363,2.0245674026142524e-09,248,5.099674460495862e-09,3.472874965382289e-08,5.26418020285746e-09,1.0
+11,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.450061260733951e-08,5.066394805908203e-07,0.24272800981998444,0.24272799491882324,6.1068083923032646e-09,248,6.209086134134623e-09,3.4522884106991114e-08,6.409379249561198e-09,1.0
+11,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.419026691337401e-08,5.960464477539062e-07,0.2525215446949005,0.2525215446949005,6.355903803267893e-09,248,4.629203242956237e-09,3.0078894042162574e-08,4.778532236571209e-09,1.0
+11,k_to_q,test-zh-1,heldout,2.86102294921875e-06,5.101806004082587e-08,4.76837158203125e-07,0.20683012902736664,0.20683011412620544,4.4362411522058665e-09,248,5.5794813214049555e-09,3.09307068846465e-08,5.759464460908248e-09,1.0
+11,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.354370300835508e-08,5.364418029785156e-07,0.2263796031475067,0.22637958824634552,5.91522208992501e-09,248,5.174931594353893e-09,3.6706762074345534e-08,5.341864728336532e-09,1.0
+11,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.454591675148741e-08,5.066394805908203e-07,0.242807075381279,0.24280709028244019,4.439097978092832e-09,248,5.318464335601902e-09,3.0657005822831707e-08,5.490027543686438e-09,1.0
+11,k_to_q,test-en-2,heldout,2.86102294921875e-06,5.424732981759204e-08,6.854534149169922e-07,0.2930097281932831,0.2930097281932831,6.705069388601714e-09,248,4.751804283387173e-09,3.399427228600871e-08,4.90508833550507e-09,1.0
+15,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.3514833863035024e-08,4.470348358154297e-07,0.33434635400772095,0.33434635400772095,4.285159338479616e-09,248,4.447109791527737e-09,3.414147897728981e-08,4.590564817164022e-09,1.0
+15,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.218853708885142e-08,4.76837158203125e-07,0.30227771401405334,0.30227774381637573,5.229982669163746e-09,248,7.101956356336814e-09,4.079232240883357e-08,7.3310517656466345e-09,1.0
+15,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.4103478102894914e-08,4.76837158203125e-07,0.3320315480232239,0.3320315480232239,5.162816396619974e-09,248,4.775797535216952e-09,3.1169314240742096e-08,4.929855634827618e-09,1.0
+15,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.31591652511587e-08,3.5762786865234375e-07,0.35803988575935364,0.35803985595703125,5.665685698375e-09,248,3.281495741802587e-09,3.316486996141066e-08,3.3873504001746824e-09,1.0
+15,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.2584229204196615e-08,5.960464477539062e-07,0.3134230375289917,0.3134230375289917,4.611414805566483e-09,248,3.2052387410885785e-09,3.137078863346687e-08,3.3086335893273144e-09,1.0
+15,q_to_k,test-zh-2,heldout,2.86102294921875e-06,5.334021097494736e-08,7.152557373046875e-07,0.31523996591567993,0.31523996591567993,6.134295738036144e-09,248,3.099508649739846e-09,3.3302633539733506e-08,3.1994928928469335e-09,1.0
+15,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.277330399580438e-08,5.662441253662109e-07,0.35721129179000854,0.35721129179000854,3.875603393765914e-09,248,6.00105520831562e-09,3.1643388354041235e-08,6.194637691692151e-09,1.0
+15,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.4245038413836726e-08,5.364418029785156e-07,0.3785627782344818,0.3785627782344818,5.588637108644434e-09,248,5.273080638801275e-09,3.732484543661485e-08,5.443180128850145e-09,1.0
+15,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.370409043564381e-08,4.76837158203125e-07,0.33434635400772095,0.33434635400772095,5.413625991934623e-09,248,7.632287690739759e-09,3.8797516310751234e-08,7.878490748680633e-09,1.0
+15,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.2645265789812776e-08,5.066394805908203e-07,0.30227771401405334,0.30227774381637573,6.007307984390309e-09,248,4.376814466411361e-09,3.4955100147726625e-08,4.518001972542152e-09,1.0
+15,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.3713584045477537e-08,5.364418029785156e-07,0.3320315480232239,0.3320315480232239,3.789843550094929e-09,248,4.10175182707917e-09,3.3631373241860274e-08,4.2340664307971565e-09,1.0
+15,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.329913616712736e-08,4.470348358154297e-07,0.35803988575935364,0.35803985595703125,5.377542411366676e-09,248,3.882600019267102e-09,3.18341726313065e-08,4.007845166853485e-09,1.0
+15,k_to_q,test-zh-1,heldout,1.9073486328125e-06,5.1740198395570186e-08,5.960464477539062e-07,0.3134230375289917,0.3134230077266693,5.392958968286621e-09,248,3.644287094317633e-09,2.9920162347707446e-08,3.761844613592302e-09,1.0
+15,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.17587827155604e-08,7.152557373046875e-07,0.31523996591567993,0.31523996591567993,6.041951827739922e-09,248,4.992775970436014e-09,3.2279736217333266e-08,5.1538333600831265e-09,1.0
+15,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.262549006299803e-08,6.258487701416016e-07,0.35721129179000854,0.35721129179000854,4.4030894486013494e-09,248,4.485939619769397e-09,3.2921569470545364e-08,4.630647421066669e-09,1.0
+15,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.460989035027149e-08,5.662441253662109e-07,0.3785627782344818,0.3785627782344818,6.211682723744616e-09,248,3.997459696591932e-09,3.3428371182253613e-08,4.1264098804560945e-09,1.0
+19,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,4.990160271767093e-08,5.364418029785156e-07,0.08087001740932465,0.08087002485990524,4.9786765821124845e-09,248,4.393741370734006e-09,3.151729544015325e-08,4.535475106592912e-09,1.0
+19,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,4.9742994871294574e-08,4.76837158203125e-07,0.08035354316234589,0.08035353571176529,5.756638277176762e-09,248,4.049901303204706e-09,3.086056210577226e-08,4.180543466958397e-09,1.0
+19,q_to_k,cal-en-1,calibration,3.814697265625e-06,5.087667850358471e-08,5.066394805908203e-07,0.24834147095680237,0.24834147095680237,4.231536010479431e-09,248,4.469023373587788e-09,2.7867795182601185e-08,4.613185389246155e-09,1.0
+19,q_to_k,cal-en-2,calibration,2.86102294921875e-06,5.025801246712497e-08,4.470348358154297e-07,0.169009268283844,0.169009268283844,5.525796265004601e-09,248,4.058557934172313e-09,2.8106194704946574e-08,4.1894789859497905e-09,1.0
+19,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.0221761295632774e-08,4.172325134277344e-07,0.06396828591823578,0.06396827846765518,3.353163524622005e-09,248,4.251817564693283e-09,2.829117207170384e-08,4.3889731848878455e-09,1.0
+19,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.0605528455235685e-08,5.960464477539062e-07,0.07303687185049057,0.07303687930107117,5.027091631859548e-09,248,3.928162239930089e-09,3.122080727280263e-08,4.054877322801076e-09,1.0
+19,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.211114332289351e-08,5.364418029785156e-07,0.2881055176258087,0.2881055176258087,3.5609850623785633e-09,248,4.370760198213475e-09,3.161588324473996e-08,4.511752305091932e-09,1.0
+19,q_to_k,test-en-2,heldout,3.814697265625e-06,5.1614248682850084e-08,4.172325134277344e-07,0.08680859953165054,0.08680859953165054,4.462207936484219e-09,248,4.6642192330637045e-09,3.021447270157296e-08,4.814677989628535e-09,1.0
+19,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.113471077024349e-08,3.8743019104003906e-07,0.08087001740932465,0.08087001740932465,4.6697485878155476e-09,248,5.351755927307522e-09,3.250153568501446e-08,5.524393387190685e-09,1.0
+19,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.029181744672548e-08,5.662441253662109e-07,0.08035354316234589,0.08035354316234589,4.907285688915408e-09,248,5.877964781575429e-09,3.508033685761802e-08,6.06757666332669e-09,1.0
+19,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.1180455196819575e-08,5.960464477539062e-07,0.24834147095680237,0.24834147095680237,5.958978199771536e-09,248,3.994703678955602e-09,2.9186557171101413e-08,4.123565044977795e-09,1.0
+19,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.087171697055175e-08,5.662441253662109e-07,0.169009268283844,0.1690092831850052,6.885077397100758e-09,248,4.357643579311343e-09,2.7700329141566726e-08,4.498212913262023e-09,1.0
+19,k_to_q,test-zh-1,heldout,1.9073486328125e-06,4.9891522075043136e-08,4.172325134277344e-07,0.06396828591823578,0.06396828591823578,3.918689817083987e-09,248,6.348101155850827e-09,3.146694638189729e-08,6.552878684118468e-09,1.0
+19,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.0542783963068e-08,5.066394805908203e-07,0.07303687185049057,0.07303687185049057,6.0575562343956335e-09,248,3.8445029382216944e-09,2.87921402275515e-08,3.968519290964423e-09,1.0
+19,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.202611404804706e-08,5.364418029785156e-07,0.2881055176258087,0.28810548782348633,4.291980548742913e-09,248,5.839524419570807e-09,2.908876339802191e-08,6.027895960158958e-09,1.0
+19,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.31186969928777e-08,4.172325134277344e-07,0.08680859953165054,0.08680859208106995,5.296885152716868e-09,248,6.1070810630781125e-09,2.844184265882177e-08,6.304083477459699e-09,1.0
+23,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,4.932807759644251e-08,4.470348358154297e-07,0.06724601984024048,0.06724601984024048,4.144036225284253e-09,248,4.558249777630863e-09,2.929247777672117e-08,4.705289935458268e-09,1.0
+23,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,4.9029117626786305e-08,4.76837158203125e-07,0.06764671951532364,0.06764672696590424,3.972651985151288e-09,248,4.91240381705893e-09,3.200177900453127e-08,5.070868613898938e-09,1.0
+23,q_to_k,cal-en-1,calibration,3.814697265625e-06,4.95385074720159e-08,3.8743019104003906e-07,0.08747006952762604,0.08747006952762604,3.265131720553427e-09,248,3.707365081595526e-09,3.0631660763447144e-08,3.826957417629728e-09,1.0
+23,q_to_k,cal-en-2,calibration,1.9073486328125e-06,4.922304127933099e-08,4.76837158203125e-07,0.08903923630714417,0.08903922885656357,5.270068825780072e-09,248,3.997369546482332e-09,2.7743276120872906e-08,4.126317065811236e-09,1.0
+23,q_to_k,test-zh-1,heldout,1.9073486328125e-06,4.8364042774388367e-08,3.5762786865234375e-07,0.05931711941957474,0.05931711569428444,3.816367222242434e-09,248,3.653502167466627e-09,2.7387025980374347e-08,3.771357004467291e-09,1.0
+23,q_to_k,test-zh-2,heldout,3.814697265625e-06,4.96448251794115e-08,5.364418029785156e-07,0.08312603086233139,0.08312603831291199,5.321914020584018e-09,248,3.6995391194949434e-09,2.9038851323548442e-08,3.818878990813346e-09,1.0
+23,q_to_k,test-en-1,heldout,3.814697265625e-06,5.034978202226261e-08,4.023313522338867e-07,0.1146349161863327,0.1146349087357521,4.471692793828197e-09,248,3.4011016225576896e-09,2.928134534840865e-08,3.5108145279849623e-09,1.0
+23,q_to_k,test-en-2,heldout,3.814697265625e-06,4.994535705946019e-08,4.470348358154297e-07,0.08886055648326874,0.08886055648326874,4.2449421755463845e-09,248,3.4883309574240684e-09,2.892927319919636e-08,3.600857834129556e-09,1.0
+23,k_to_q,cal-zh-1,calibration,2.86102294921875e-06,4.8831644467980875e-08,4.76837158203125e-07,0.06724601984024048,0.06724601984024048,3.977453033598977e-09,248,4.496496952555162e-09,2.8416268449404924e-08,4.64154537027639e-09,1.0
+23,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,4.9029117626786305e-08,4.470348358154297e-07,0.06764671951532364,0.06764672696590424,4.682830567759311e-09,248,2.651090902361375e-09,2.4917778063127116e-08,2.736610049680621e-09,1.0
+23,k_to_q,cal-en-1,calibration,3.814697265625e-06,5.024324385126141e-08,3.8743019104003906e-07,0.08747006952762604,0.08747006952762604,3.7081571147012937e-09,248,4.977271039763309e-09,3.31379155227296e-08,5.13782838496013e-09,1.0
+23,k_to_q,cal-en-2,calibration,1.9073486328125e-06,4.8518648048109764e-08,2.980232238769531e-07,0.08903923630714417,0.08903922885656357,3.85932619195728e-09,248,4.874880499272649e-09,2.9354460195918364e-08,5.0321347089266055e-09,1.0
+23,k_to_q,test-zh-1,heldout,3.814697265625e-06,4.978168384729527e-08,4.172325134277344e-07,0.05931711941957474,0.05931711941957474,4.285771737499999e-09,248,4.133625886026948e-09,2.540235399806079e-08,4.266968556265738e-09,1.0
+23,k_to_q,test-zh-2,heldout,1.9073486328125e-06,4.943963198590656e-08,6.556510925292969e-07,0.08312603086233139,0.08312603086233139,3.7222060988995054e-09,248,4.004627740528122e-09,2.6530234009669584e-08,4.133809294870616e-09,1.0
+23,k_to_q,test-en-1,heldout,3.814697265625e-06,4.9439166765060464e-08,4.023313522338867e-07,0.1146349161863327,0.1146349161863327,6.534782048817078e-09,248,4.612270565473864e-09,2.6872147174117345e-08,4.7610533293607205e-09,1.0
+23,k_to_q,test-en-2,heldout,1.9073486328125e-06,4.8735765106561524e-08,4.172325134277344e-07,0.08886055648326874,0.08886055648326874,4.1334708988927105e-09,248,4.0138719015203606e-09,2.8323970724386527e-08,4.1433514397226645e-09,1.0

results/cpu-qwen35-2b/gain-factorization-summary.json

@@ -0,0 +1,46 @@
+{
+  "state": "completed",
+  "records": [],
+  "interventions": [
+    {
+      "layer": 3,
+      "direction": "q_to_k",
+      "sequence_nll_deltas": [
+        3.114725206643243e-07,
+        -5.23134924712565e-07,
+        1.7465536439331686e-07,
+        1.733758607791458e-07
+      ],
+      "mean_nll_delta": 3.40922052810555e-08,
+      "sequence_p95_delta": 2.9094994722367315e-07,
+      "passed": true
+    },
+    {
+      "layer": 3,
+      "direction": "k_to_q",
+      "sequence_nll_deltas": [
+        4.0256782884995346e-07,
+        -8.45824208006718e-07,
+        2.1304516684139685e-07,
+        -3.860568877911419e-07
+      ],
+      "mean_nll_delta": -1.540670250266274e-07,
+      "sequence_p95_delta": 3.741394295486699e-07,
+      "passed": true
+    }
+  ],
+  "maximum_score_abs_error": 3.814697265625e-06,
+  "maximum_probability_abs_error": 8.046627044677734e-07,
+  "bidirectional_heldout_pass": true,
+  "elementwise_gain_multiplies_saved_per_token_layer": {
+    "q_to_k": 1536,
+    "k_to_q": 384
+  },
+  "original_bf16_payload_bytes": 1024,
+  "factored_lossless_mixed_payload_bytes": 1024,
+  "fp32_cached_gain_payload_before": 2048,
+  "fp32_cached_gain_payload_after": 1280,
+  "claim_limit": "Exact real-arithmetic identity only on non-RoPE coordinates; empirical FP32/NLL checks on tiny text pilot. Not evidence that two existing full layers interchange, not model-wide speedup.",
+  "elapsed_seconds": 232.08875080291182,
+  "peak_rss_kib": 83704
+}

results/cpu-qwen35-2b/install-torch-disk-tmp.log

@@ -0,0 +1,39 @@
+Looking in indexes: https://download.pytorch.org/whl/cpu
+Collecting torch
+  Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (35 kB)
+Collecting filelock (from torch)
+  Downloading filelock-4.2.4-py3-none-any.whl.metadata (2.0 kB)
+Collecting typing-extensions>=4.13.0 (from torch)
+  Downloading typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
+Collecting setuptools>=77.0.3 (from torch)
+  Downloading https://download.pytorch.org/whl/setuptools-78.1.0-py3-none-any.whl.metadata (6.6 kB)
+Collecting sympy>=1.13.3 (from torch)
+  Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
+Collecting networkx>=2.5.1 (from torch)
+  Downloading networkx-3.7-py3-none-any.whl.metadata (6.7 kB)
+Collecting jinja2 (from torch)
+  Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
+Collecting fsspec>=0.8.5 (from torch)
+  Downloading fsspec-2026.9.0-py3-none-any.whl.metadata (10 kB)
+Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
+  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
+Collecting MarkupSafe>=2.0 (from jinja2->torch)
+  Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
+Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl (209.5 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.5/209.5 MB 235.8 kB/s eta 0:00:00
+Downloading fsspec-2026.9.0-py3-none-any.whl (221 kB)
+Downloading networkx-3.7-py3-none-any.whl (2.1 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 143.1 kB/s eta 0:00:00
+Downloading https://download.pytorch.org/whl/setuptools-78.1.0-py3-none-any.whl (1.3 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 368.1 kB/s eta 0:00:00
+Downloading sympy-1.14.0-py3-none-any.whl (6.3 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 27.0 kB/s eta 0:00:00
+Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 19.7 kB/s eta 0:00:00
+Downloading typing_extensions-4.16.0-py3-none-any.whl (45 kB)
+Downloading filelock-4.2.4-py3-none-any.whl (162 kB)
+Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl (134 kB)
+Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (23 kB)
+Installing collected packages: mpmath, typing-extensions, sympy, setuptools, networkx, MarkupSafe, fsspec, filelock, jinja2, torch
+
+Successfully installed MarkupSafe-3.0.3 filelock-4.2.4 fsspec-2026.9.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.7 setuptools-78.1.0 sympy-1.14.0 torch-2.15.0+cpu typing-extensions-4.16.0

results/cpu-qwen35-2b/install-torch.log

@@ -0,0 +1,25 @@
+Looking in indexes: https://download.pytorch.org/whl/cpu
+Collecting torch
+  Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (35 kB)
+Collecting filelock (from torch)
+  Downloading filelock-4.2.4-py3-none-any.whl.metadata (2.0 kB)
+Collecting typing-extensions>=4.13.0 (from torch)
+  Downloading typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
+Collecting setuptools>=77.0.3 (from torch)
+  Downloading https://download.pytorch.org/whl/setuptools-78.1.0-py3-none-any.whl.metadata (6.6 kB)
+Collecting sympy>=1.13.3 (from torch)
+  Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
+Collecting networkx>=2.5.1 (from torch)
+  Downloading networkx-3.7-py3-none-any.whl.metadata (6.7 kB)
+Collecting jinja2 (from torch)
+  Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
+Collecting fsspec>=0.8.5 (from torch)
+  Downloading fsspec-2026.9.0-py3-none-any.whl.metadata (10 kB)
+Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
+  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
+Collecting MarkupSafe>=2.0 (from jinja2->torch)
+  Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
+Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl (209.5 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━             145.8/209.5 MB 447.0 kB/s eta 0:02:23
+ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device
+

results/cpu-qwen35-2b/manifest.json

@@ -0,0 +1,34 @@
+{
+  "repo": "Qwen/Qwen3.5-2B",
+  "revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "expected_bytes": 4548351670,
+  "started_unix": 1794169446.8545816,
+  "files": [
+    {
+      "name": "README.md",
+      "bytes": 62814,
+      "sha256": "c0e83a849c776e6fa843d011f023132d942a0f0140d903205bf1c363adad2275",
+      "upstream_sha256": null
+    },
+    {
+      "name": "config.json",
+      "bytes": 2908,
+      "sha256": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4",
+      "upstream_sha256": null
+    },
+    {
+      "name": "model.safetensors-00001-of-00001.safetensors",
+      "bytes": 4548221488,
+      "sha256": "aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1",
+      "upstream_sha256": "aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1"
+    },
+    {
+      "name": "model.safetensors.index.json",
+      "bytes": 64460,
+      "sha256": "aca8afed9da75b0f050b408d270766fd77627f1af401e240f61c3b47d0db02f9",
+      "upstream_sha256": null
+    }
+  ],
+  "license": "apache-2.0",
+  "completed_unix": 1794173525.0766253
+}

results/cpu-qwen35-2b/model_config.json

@@ -0,0 +1,96 @@
+{
+    "architectures": [
+        "Qwen3_5ForConditionalGeneration"
+    ],
+    "image_token_id": 248056,
+    "model_type": "qwen3_5",
+    "text_config": {
+        "attention_bias": false,
+        "attention_dropout": 0.0,
+        "attn_output_gate": true,
+        "dtype": "bfloat16",
+        "eos_token_id": 248044,
+        "full_attention_interval": 4,
+        "head_dim": 256,
+        "hidden_act": "silu",
+        "hidden_size": 2048,
+        "initializer_range": 0.02,
+        "intermediate_size": 6144,
+        "layer_types": [
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention"
+        ],
+        "linear_conv_kernel_dim": 4,
+        "linear_key_head_dim": 128,
+        "linear_num_key_heads": 16,
+        "linear_num_value_heads": 16,
+        "linear_value_head_dim": 128,
+        "max_position_embeddings": 262144,
+        "mlp_only_layers": [],
+        "model_type": "qwen3_5_text",
+        "mtp_num_hidden_layers": 1,
+        "mtp_use_dedicated_embeddings": false,
+        "num_attention_heads": 8,
+        "num_hidden_layers": 24,
+        "num_key_value_heads": 2,
+        "rms_norm_eps": 1e-06,
+        "tie_word_embeddings": true,
+        "use_cache": true,
+        "vocab_size": 248320,
+        "mamba_ssm_dtype": "float32",
+        "rope_parameters": {
+            "mrope_interleaved": true,
+            "mrope_section": [
+                11,
+                11,
+                10
+            ],
+            "rope_type": "default",
+            "rope_theta": 10000000,
+            "partial_rotary_factor": 0.25
+        }
+    },
+    "tie_word_embeddings": true,
+    "transformers_version": "4.57.0.dev0",
+    "video_token_id": 248057,
+    "vision_config": {
+        "deepstack_visual_indexes": [],
+        "depth": 24,
+        "hidden_act": "gelu_pytorch_tanh",
+        "hidden_size": 1024,
+        "in_channels": 3,
+        "initializer_range": 0.02,
+        "intermediate_size": 4096,
+        "model_type": "qwen3_5",
+        "num_heads": 16,
+        "num_position_embeddings": 2304,
+        "out_hidden_size": 2048,
+        "patch_size": 16,
+        "spatial_merge_size": 2,
+        "temporal_patch_size": 2
+    },
+    "vision_end_token_id": 248054,
+    "vision_start_token_id": 248053
+}

results/cpu-qwen35-2b/neuron-reuse-protocol.json

@@ -0,0 +1,29 @@
+{
+  "layers": [
+    5,
+    6,
+    20,
+    21
+  ],
+  "unit": "post_silu_gate_times_up_scalar_function; keep each output column",
+  "thresholds": [
+    0.05,
+    0.1,
+    0.2
+  ],
+  "maximum_directional_relative_l2": 0.2,
+  "cosine_min": 0.98,
+  "pearson_min": 0.95,
+  "rms_floor": "at least within-layer median calibration neuron RMS",
+  "std_over_rms_min": 0.25,
+  "effective_energy_tokens_min": 16,
+  "max_selected_pairs": 1,
+  "selection": "smallest worst directional calibration error; then layer/id",
+  "heldout_per_text_error_max": 0.35,
+  "heldout_quality_mean_nll_delta_max": 0.01,
+  "heldout_quality_sequence_p95_delta_max": 0.05,
+  "training": false,
+  "scale_fitting": false,
+  "cost_required": false,
+  "registered_unix": 1794203413.872667
+}

results/cpu-qwen35-2b/neuron-reuse-search-summary.json

@@ -0,0 +1,54 @@
+[
+  {
+    "layer": 5,
+    "neurons": 6144,
+    "eligible_neurons": 2111,
+    "eligible_unique_pairs_checked": 2227105,
+    "rms_floor": 0.05843841928142779,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  },
+  {
+    "layer": 6,
+    "neurons": 6144,
+    "eligible_neurons": 2090,
+    "eligible_unique_pairs_checked": 2183005,
+    "rms_floor": 0.05875614556680463,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  },
+  {
+    "layer": 20,
+    "neurons": 6144,
+    "eligible_neurons": 1699,
+    "eligible_unique_pairs_checked": 1442451,
+    "rms_floor": 0.1179593664401982,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  },
+  {
+    "layer": 21,
+    "neurons": 6144,
+    "eligible_neurons": 1788,
+    "eligible_unique_pairs_checked": 1597578,
+    "rms_floor": 0.12231686838397056,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  }
+]

results/cpu-qwen35-2b/neuron-reuse-selection.json

@@ -0,0 +1,4 @@
+{
+  "selected": null,
+  "selection_uses_heldout": false
+}

results/cpu-qwen35-2b/neuron-reuse-summary.json

@@ -0,0 +1,10 @@
+{
+  "state": "completed",
+  "selected": null,
+  "function_pass": false,
+  "interventions": [],
+  "empirical_repeated_function_positive": false,
+  "claim_limit": "Distinct learned scalar MLP subfunctions on the specified short-text distribution; not whole-layer duplication, global equivalence or broad task-capability proof. No performance requirement.",
+  "elapsed_seconds": 55.004304428584874,
+  "peak_rss_kib": 185036
+}

results/cpu-qwen35-2b/neuron-reuse.log

@@ -0,0 +1,4 @@
+{"layer": 5, "neurons": 6144, "eligible_neurons": 2111, "eligible_unique_pairs_checked": 2227105, "rms_floor": 0.05843841928142779, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}
+{"layer": 6, "neurons": 6144, "eligible_neurons": 2090, "eligible_unique_pairs_checked": 2183005, "rms_floor": 0.05875614556680463, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}
+{"layer": 20, "neurons": 6144, "eligible_neurons": 1699, "eligible_unique_pairs_checked": 1442451, "rms_floor": 0.1179593664401982, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}
+{"layer": 21, "neurons": 6144, "eligible_neurons": 1788, "eligible_unique_pairs_checked": 1597578, "rms_floor": 0.12231686838397056, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}

results/cpu-qwen35-2b/norm-repetition-function-metrics.csv

@@ -0,0 +1,17 @@
+source_layer,role,a,b,sample_id,split,relative_l2,reverse_relative_l2,token_relative_p95,reverse_token_relative_p95,cosine,low_norm_tokens
+12,input_layernorm,12,13,cal-zh-1,calibration,0.04185799286154599,0.04309664461005982,0.044446126787094885,0.04595947974911848,0.9995232826957284,0
+12,input_layernorm,12,13,cal-zh-2,calibration,0.04383456526622353,0.04521540878040512,0.046012856287805166,0.047545525578214155,0.9994900129478096,0
+12,input_layernorm,12,13,cal-en-1,calibration,0.04347907732513803,0.04487576882582582,0.04643377233703883,0.048238404047826186,0.9995243164548383,0
+12,input_layernorm,12,13,cal-en-2,calibration,0.041111186492761374,0.04233333726141896,0.04398539072853077,0.04542340159464054,0.9995589314599114,0
+12,input_layernorm,12,13,test-zh-1,heldout,0.04230986484298778,0.04359453156137845,0.04521637327744173,0.046674126177753086,0.9995251413848774,0
+12,input_layernorm,12,13,test-zh-2,heldout,0.043902033219287144,0.04528510757179746,0.047435978938916296,0.04912927000592789,0.9994870299724158,0
+12,input_layernorm,12,13,test-en-1,heldout,0.042965663401383455,0.04430863234604914,0.04650584597139243,0.04815192582322126,0.9995218127753321,0
+12,input_layernorm,12,13,test-en-2,heldout,0.04135010433212149,0.042584214188789195,0.046368214514353176,0.047927790990014825,0.9995520355780412,0
+13,input_layernorm,12,13,cal-zh-1,calibration,0.041182645479044225,0.04240971565525536,0.04352880565374195,0.045001128308403464,0.9995577794005558,0
+13,input_layernorm,12,13,cal-zh-2,calibration,0.04327063750976668,0.04463822187523808,0.045272158865488125,0.04679408388648026,0.999518385770098,0
+13,input_layernorm,12,13,cal-en-1,calibration,0.04262779877016355,0.04398654464567335,0.045497400019515064,0.04723836815505396,0.9995547796247127,0
+13,input_layernorm,12,13,cal-en-2,calibration,0.040518961963038086,0.04171178072648754,0.043328686448901595,0.04470198445787224,0.9995758632252388,0
+13,input_layernorm,12,13,test-zh-1,heldout,0.04162196134899919,0.04289161132734105,0.04428805531751076,0.045755187013951454,0.9995588682914622,0
+13,input_layernorm,12,13,test-zh-2,heldout,0.043144621015003826,0.04450396583648517,0.045958426245692637,0.04754094135863426,0.9995211225735321,0
+13,input_layernorm,12,13,test-en-1,heldout,0.042127619483222106,0.04343304280377689,0.045197718466321105,0.04683576056261672,0.999550812480402,0
+13,input_layernorm,12,13,test-en-2,heldout,0.04076045660321344,0.04197546447097981,0.045561271843477975,0.04710887060271581,0.9995759438544739,0

results/cpu-qwen35-2b/norm-repetition-protocol.json

@@ -0,0 +1,44 @@
+{
+  "roles": [
+    "input_layernorm",
+    "post_attention_layernorm"
+  ],
+  "layers": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23
+  ],
+  "same_role_only": true,
+  "same_eps": 1e-06,
+  "uniform_worst_error_limit": 0.2,
+  "empirical_function_worst_relative_error_limit": 0.05,
+  "selection": "smallest maximum two-direction uniform bound, then role/layer; no input data used",
+  "minimum_gain_standard_deviation": 0.001,
+  "max_selected_pairs": 1,
+  "heldout_nll_mean_delta_limit": 0.01,
+  "heldout_nll_sequence_p95_delta_limit": 0.05,
+  "training": false,
+  "cost_required": false,
+  "registered_unix": 1794203690.521922
+}

results/cpu-qwen35-2b/norm-repetition-selection.json

@@ -0,0 +1,23 @@
+{
+  "selected": [
+    {
+      "role": "input_layernorm",
+      "a": 12,
+      "b": 13,
+      "a_to_b_uniform_bound": 0.10185185185185185,
+      "b_to_a_uniform_bound": 0.1134020618556701,
+      "worst_uniform_bound": 0.1134020618556701,
+      "eligible_nonconstant_gains": true,
+      "gain_std_a": 0.1900545060634613,
+      "gain_std_b": 0.16846242547035217,
+      "gain_min_a": 0.5546875,
+      "gain_min_b": 0.5703125
+    }
+  ],
+  "uniform_bound_counts": {
+    "0.05": 0,
+    "0.1": 0,
+    "0.2": 18
+  },
+  "selection_reads_no_input_data": true
+}

results/cpu-qwen35-2b/norm-repetition-summary.json

@@ -0,0 +1,59 @@
+{
+  "state": "completed",
+  "selected": {
+    "role": "input_layernorm",
+    "a": 12,
+    "b": 13,
+    "a_to_b_uniform_bound": 0.10185185185185185,
+    "b_to_a_uniform_bound": 0.1134020618556701,
+    "worst_uniform_bound": 0.1134020618556701,
+    "eligible_nonconstant_gains": true,
+    "gain_std_a": 0.1900545060634613,
+    "gain_std_b": 0.16846242547035217,
+    "gain_min_a": 0.5546875,
+    "gain_min_b": 0.5703125
+  },
+  "interventions": [
+    {
+      "target": 12,
+      "donor": 13,
+      "sequence_nll_deltas": [
+        0.0010516456964495546,
+        -0.0013996762182379996,
+        -0.00012664743940504763,
+        0.0006049385086996928
+      ],
+      "mean_nll_delta": 3.256513687655005e-05,
+      "sequence_p95_nll_delta": 0.0009846396182870752,
+      "passed_quality": true
+    },
+    {
+      "target": 13,
+      "donor": 12,
+      "sequence_nll_deltas": [
+        0.0008813779918703801,
+        -0.0012936356267054805,
+        -0.0002630009203641315,
+        -0.0007806334344149235
+      ],
+      "mean_nll_delta": -0.00036397299740353883,
+      "sequence_p95_nll_delta": 0.000709721155035203,
+      "passed_quality": true
+    }
+  ],
+  "uniform_bound_counts": {
+    "0.05": 0,
+    "0.1": 0,
+    "0.2": 18
+  },
+  "uniform_function_similarity_proven": true,
+  "heldout_function_pass": true,
+  "worst_heldout_relative_l2": 0.04528510757179746,
+  "worst_heldout_cosine": 0.9994870299724158,
+  "untouched_full_forward_bitwise_matches_baseline": true,
+  "learned_function_repetition_positive": true,
+  "bidirectional_network_quality_pass": true,
+  "claim_limit": "Existing learned RMSNorm module functions with identical eps and shape; uniform real-arithmetic relative bound and tiny real-text probes. Does not establish attention/MLP-body duplication or broad task capability.",
+  "elapsed_seconds": 369.7967556482181,
+  "peak_rss_kib": 71596
+}

results/cpu-qwen35-2b/norm-repetition-uniform-bounds.csv

@@ -0,0 +1,553 @@
+role,a,b,a_to_b_uniform_bound,b_to_a_uniform_bound,worst_uniform_bound,eligible_nonconstant_gains,gain_std_a,gain_std_b,gain_min_a,gain_min_b
+input_layernorm,0,1,0.6032863849765259,1.5207100591715976,1.5207100591715976,True,0.11805251985788345,0.0930413156747818,0.794921875,0.607421875
+input_layernorm,0,2,0.5692488262910798,1.3215258855585832,1.3215258855585832,True,0.11805251985788345,0.10473614931106567,0.794921875,0.6484375
+input_layernorm,0,3,0.5469483568075117,1.2072538860103628,1.2072538860103628,True,0.11805251985788345,0.07621725648641586,0.794921875,0.7265625
+input_layernorm,0,4,0.5751173708920188,1.3535911602209945,1.3535911602209945,True,0.11805251985788345,0.10847259312868118,0.794921875,0.662109375
+input_layernorm,0,5,0.6396713615023474,1.775244299674267,1.775244299674267,True,0.11805251985788345,0.13199757039546967,0.794921875,0.5234375
+input_layernorm,0,6,0.6643192488262911,1.979020979020979,1.979020979020979,True,0.11805251985788345,0.1498883068561554,0.794921875,0.47265625
+input_layernorm,0,7,0.6830985915492958,2.1555555555555554,2.1555555555555554,True,0.11805251985788345,0.095920130610466,0.794921875,0.46484375
+input_layernorm,0,8,0.6913145539906104,2.2395437262357416,2.2395437262357416,True,0.11805251985788345,0.20054922997951508,0.794921875,0.41796875
+input_layernorm,0,9,0.7136150234741784,2.4918032786885247,2.4918032786885247,True,0.11805251985788345,0.22649425268173218,0.794921875,0.42578125
+input_layernorm,0,10,0.7230046948356808,2.610169491525424,2.610169491525424,True,0.11805251985788345,0.2204512506723404,0.794921875,0.3828125
+input_layernorm,0,11,0.7347417840375586,2.769911504424779,2.769911504424779,True,0.11805251985788345,0.16603894531726837,0.794921875,0.34375
+input_layernorm,0,12,0.6338028169014085,1.7307692307692308,1.7307692307692308,True,0.11805251985788345,0.1900545060634613,0.794921875,0.5546875
+input_layernorm,0,13,0.6349765258215962,1.7395498392282958,1.7395498392282958,True,0.11805251985788345,0.16846242547035217,0.794921875,0.5703125
+input_layernorm,0,14,0.5516431924882629,1.2303664921465969,1.2303664921465969,True,0.11805251985788345,0.11825777590274811,0.794921875,0.6875
+input_layernorm,0,15,0.49018199233716475,0.9614842649131048,0.9614842649131048,True,0.11805251985788345,0.06029079854488373,0.794921875,0.779296875
+input_layernorm,0,16,0.5117370892018779,1.0480769230769231,1.0480769230769231,True,0.11805251985788345,0.09195870161056519,0.794921875,0.7763671875
+input_layernorm,0,17,0.6232876712328768,0.7185185185185186,0.7185185185185186,True,0.11805251985788345,0.06483674794435501,0.794921875,1.0012359619140625
+input_layernorm,0,18,0.8402948402948403,0.6637450199203188,0.8402948402948403,True,0.11805251985788345,0.06785038113594055,0.794921875,1.1640625
+input_layernorm,0,19,1.0393120393120394,1.1248445660283513,1.1248445660283513,True,0.11805251985788345,0.19409698247909546,0.794921875,0.884765625
+input_layernorm,0,20,1.113022113022113,0.9869156655168312,1.113022113022113,True,0.11805251985788345,0.08625888079404831,0.794921875,1.0262451171875
+input_layernorm,0,21,1.1867321867321867,0.9545986426398315,1.1867321867321867,True,0.11805251985788345,0.07095508277416229,0.794921875,1.043212890625
+input_layernorm,0,22,0.9656019656019657,0.9514018691588785,0.9656019656019657,True,0.11805251985788345,0.07214703410863876,0.794921875,1.044921875
+input_layernorm,0,23,0.941031941031941,0.9568884723523898,0.9568884723523898,True,0.11805251985788345,0.15369786322116852,0.794921875,0.98577880859375
+input_layernorm,1,2,0.16765578635014836,0.17593150850114556,0.17593150850114556,True,0.0930413156747818,0.10473614931106567,0.607421875,0.6484375
+input_layernorm,1,3,0.28125,0.34967320261437906,0.34967320261437906,True,0.0930413156747818,0.07621725648641586,0.607421875,0.7265625
+input_layernorm,1,4,0.1678622668579627,0.15668366741786266,0.1678622668579627,True,0.0930413156747818,0.10847259312868118,0.607421875,0.662109375
+input_layernorm,1,5,0.23018147086914995,0.2990074441687345,0.2990074441687345,True,0.0930413156747818,0.13199757039546967,0.607421875,0.5234375
+input_layernorm,1,6,0.34367141659681477,0.5236270753512133,0.5236270753512133,True,0.0930413156747818,0.1498883068561554,0.607421875,0.47265625
+input_layernorm,1,7,0.3996670702179177,0.6657423745903706,0.6657423745903706,True,0.0930413156747818,0.095920130610466,0.607421875,0.46484375
+input_layernorm,1,8,0.4585726004922067,0.8469696969696969,0.8469696969696969,True,0.0930413156747818,0.20054922997951508,0.607421875,0.41796875
+input_layernorm,1,9,0.5138306789606035,1.056896551724138,1.056896551724138,True,0.0930413156747818,0.22649425268173218,0.607421875,0.42578125
+input_layernorm,1,10,0.4786253143336127,0.9180064308681672,0.9180064308681672,True,0.0930413156747818,0.2204512506723404,0.607421875,0.3828125
+input_layernorm,1,11,0.49209302325581394,0.9688644688644689,0.9688644688644689,True,0.0930413156747818,0.16603894531726837,0.607421875,0.34375
+input_layernorm,1,12,0.3838141025641026,0.6228868660598179,0.6228868660598179,True,0.0930413156747818,0.1900545060634613,0.607421875,0.5546875
+input_layernorm,1,13,0.39622641509433965,0.65625,0.65625,True,0.0930413156747818,0.16846242547035217,0.607421875,0.5703125
+input_layernorm,1,14,0.27009646302250806,0.28465608465608466,0.28465608465608466,True,0.0930413156747818,0.11825777590274811,0.607421875,0.6875
+input_layernorm,1,15,0.9292604501607717,0.5223386651958081,0.9292604501607717,True,0.0930413156747818,0.06029079854488373,0.607421875,0.779296875
+input_layernorm,1,16,0.36012861736334406,0.2647754137115839,0.36012861736334406,True,0.0930413156747818,0.09195870161056519,0.607421875,0.7763671875
+input_layernorm,1,17,0.7958860759493671,0.4431718061674009,0.7958860759493671,True,0.0930413156747818,0.06483674794435501,0.607421875,1.0012359619140625
+input_layernorm,1,18,1.1044303797468353,0.524812030075188,1.1044303797468353,True,0.0930413156747818,0.06785038113594055,0.607421875,1.1640625
+input_layernorm,1,19,1.3858520900321543,0.5808625336927223,1.3858520900321543,True,0.0930413156747818,0.19409698247909546,0.607421875,0.884765625
+input_layernorm,1,20,1.206896551724138,0.546875,1.206896551724138,True,0.0930413156747818,0.08625888079404831,0.607421875,1.0262451171875
+input_layernorm,1,21,1.630225080385852,0.6198044009779952,1.630225080385852,True,0.0930413156747818,0.07095508277416229,0.607421875,1.043212890625
+input_layernorm,1,22,1.4043887147335423,0.5840938722294654,1.4043887147335423,True,0.0930413156747818,0.07214703410863876,0.607421875,1.044921875
+input_layernorm,1,23,1.9453376205787782,0.6604803493449781,1.9453376205787782,True,0.0930413156747818,0.15369786322116852,0.607421875,0.98577880859375
+input_layernorm,2,3,0.2867132867132867,0.4019607843137255,0.4019607843137255,True,0.10473614931106567,0.07621725648641586,0.6484375,0.7265625
+input_layernorm,2,4,0.10583941605839416,0.11836734693877551,0.11836734693877551,True,0.10473614931106567,0.10847259312868118,0.6484375,0.662109375
+input_layernorm,2,5,0.23737099402498643,0.31125356125356124,0.31125356125356124,True,0.10473614931106567,0.13199757039546967,0.6484375,0.5234375
+input_layernorm,2,6,0.31070496083550914,0.45075757575757575,0.45075757575757575,True,0.10473614931106567,0.1498883068561554,0.6484375,0.47265625
+input_layernorm,2,7,0.42205710955710957,0.7302747668263171,0.7302747668263171,True,0.10473614931106567,0.095920130610466,0.6484375,0.46484375
+input_layernorm,2,8,0.4018264840182648,0.6717557251908397,0.6717557251908397,True,0.10473614931106567,0.20054922997951508,0.6484375,0.41796875
+input_layernorm,2,9,0.4722187382415653,0.8947243346007605,0.8947243346007605,True,0.10473614931106567,0.22649425268173218,0.6484375,0.42578125
+input_layernorm,2,10,0.46013404206147446,0.8523116438356164,0.8523116438356164,True,0.10473614931106567,0.2204512506723404,0.6484375,0.3828125
+input_layernorm,2,11,0.5159817351598174,1.0660377358490567,1.0660377358490567,True,0.10473614931106567,0.16603894531726837,0.6484375,0.34375
+input_layernorm,2,12,0.30617848970251715,0.4412928759894459,0.4412928759894459,True,0.10473614931106567,0.1900545060634613,0.6484375,0.5546875
+input_layernorm,2,13,0.2900036175087423,0.4084578804347826,0.4084578804347826,True,0.10473614931106567,0.16846242547035217,0.6484375,0.5703125
+input_layernorm,2,14,0.19501226492232215,0.24225495175215844,0.24225495175215844,True,0.10473614931106567,0.11825777590274811,0.6484375,0.6875
+input_layernorm,2,15,0.7957142857142857,0.5678391959798995,0.7957142857142857,True,0.10473614931106567,0.06029079854488373,0.6484375,0.779296875
+input_layernorm,2,16,0.2740963855421687,0.21513002364066194,0.2740963855421687,True,0.10473614931106567,0.09195870161056519,0.6484375,0.7763671875
+input_layernorm,2,17,0.7093373493975904,0.4149779735682819,0.7093373493975904,True,0.10473614931106567,0.06483674794435501,0.6484375,1.0012359619140625
+input_layernorm,2,18,1.0030120481927711,0.5007518796992482,1.0030120481927711,True,0.10473614931106567,0.06785038113594055,0.6484375,1.1640625
+input_layernorm,2,19,1.2680722891566265,0.5590969455511288,1.2680722891566265,True,0.10473614931106567,0.19409698247909546,0.6484375,0.884765625
+input_layernorm,2,20,1.0873493975903614,0.5209235209235209,1.0873493975903614,True,0.10473614931106567,0.08625888079404831,0.6484375,1.0262451171875
+input_layernorm,2,21,1.463855421686747,0.5941320293398533,1.463855421686747,True,0.10473614931106567,0.07095508277416229,0.6484375,1.043212890625
+input_layernorm,2,22,1.249266862170088,0.5554106910039114,1.249266862170088,True,0.10473614931106567,0.07214703410863876,0.6484375,1.044921875
+input_layernorm,2,23,1.7448680351906158,0.6356837606837606,1.7448680351906158,True,0.10473614931106567,0.15369786322116852,0.6484375,0.98577880859375
+input_layernorm,3,4,0.40522875816993464,0.28837209302325584,0.40522875816993464,True,0.07621725648641586,0.10847259312868118,0.7265625,0.662109375
+input_layernorm,3,5,0.3660130718954248,0.39925373134328357,0.39925373134328357,True,0.07621725648641586,0.13199757039546967,0.7265625,0.5234375
+input_layernorm,3,6,0.35609756097560974,0.553030303030303,0.553030303030303,True,0.07621725648641586,0.1498883068561554,0.7265625,0.47265625
+input_layernorm,3,7,0.39575530586766544,0.6549586776859504,0.6549586776859504,True,0.07621725648641586,0.095920130610466,0.7265625,0.46484375
+input_layernorm,3,8,0.42473118279569894,0.7383177570093458,0.7383177570093458,True,0.07621725648641586,0.20054922997951508,0.7265625,0.41796875
+input_layernorm,3,9,0.49180741114192084,0.9677579365079365,0.9677579365079365,True,0.07621725648641586,0.22649425268173218,0.7265625,0.42578125
+input_layernorm,3,10,0.47804878048780486,0.9158878504672897,0.9158878504672897,True,0.07621725648641586,0.2204512506723404,0.7265625,0.3828125
+input_layernorm,3,11,0.545338441890166,1.199438202247191,1.199438202247191,True,0.07621725648641586,0.16603894531726837,0.7265625,0.34375
+input_layernorm,3,12,0.40460669274228594,0.5290237467018469,0.5290237467018469,True,0.07621725648641586,0.1900545060634613,0.7265625,0.5546875
+input_layernorm,3,13,0.3559322033898305,0.5441576086956522,0.5441576086956522,True,0.07621725648641586,0.16846242547035217,0.7265625,0.5703125
+input_layernorm,3,14,0.28804587560652845,0.22363013698630138,0.28804587560652845,True,0.07621725648641586,0.11825777590274811,0.7265625,0.6875
+input_layernorm,3,15,0.5853333333333334,0.36921783010933557,0.5853333333333334,True,0.07621725648641586,0.06029079854488373,0.7265625,0.779296875
+input_layernorm,3,16,0.43790849673202614,0.30454545454545456,0.43790849673202614,True,0.07621725648641586,0.09195870161056519,0.7265625,0.7763671875
+input_layernorm,3,17,0.5490196078431373,0.35443037974683544,0.5490196078431373,True,0.07621725648641586,0.06483674794435501,0.7265625,1.0012359619140625
+input_layernorm,3,18,0.7876344086021505,0.4406015037593985,0.7876344086021505,True,0.07621725648641586,0.06785038113594055,0.7265625,1.1640625
+input_layernorm,3,19,1.0241935483870968,0.5059760956175299,1.0241935483870968,True,0.07621725648641586,0.19409698247909546,0.7265625,0.884765625
+input_layernorm,3,20,0.9153567110036276,0.4779040404040404,0.9153567110036276,True,0.07621725648641586,0.08625888079404831,0.7265625,1.0262451171875
+input_layernorm,3,21,1.1706666666666667,0.5393120393120393,1.1706666666666667,True,0.07621725648641586,0.07095508277416229,0.7265625,1.043212890625
+input_layernorm,3,22,1.0453333333333332,0.5110821382007823,1.0453333333333332,True,0.07621725648641586,0.07214703410863876,0.7265625,1.044921875
+input_layernorm,3,23,1.496,0.5993589743589743,1.496,True,0.07621725648641586,0.15369786322116852,0.7265625,0.98577880859375
+input_layernorm,4,5,0.21176470588235294,0.26865671641791045,0.26865671641791045,True,0.10847259312868118,0.13199757039546967,0.662109375,0.5234375
+input_layernorm,4,6,0.2985507246376812,0.4256198347107438,0.4256198347107438,True,0.10847259312868118,0.1498883068561554,0.662109375,0.47265625
+input_layernorm,4,7,0.42340116279069767,0.7343080413410638,0.7343080413410638,True,0.10847259312868118,0.095920130610466,0.662109375,0.46484375
+input_layernorm,4,8,0.37971014492753624,0.6121495327102804,0.6121495327102804,True,0.10847259312868118,0.20054922997951508,0.662109375,0.41796875
+input_layernorm,4,9,0.4610917537746806,0.8556034482758621,0.8556034482758621,True,0.10847259312868118,0.22649425268173218,0.662109375,0.42578125
+input_layernorm,4,10,0.4318840579710145,0.7602040816326531,0.7602040816326531,True,0.10847259312868118,0.2204512506723404,0.662109375,0.3828125
+input_layernorm,4,11,0.5041140976412507,1.0165929203539823,1.0165929203539823,True,0.10847259312868118,0.16603894531726837,0.662109375,0.34375
+input_layernorm,4,12,0.31369924141008476,0.45708712613784136,0.45708712613784136,True,0.10847259312868118,0.1900545060634613,0.662109375,0.5546875
+input_layernorm,4,13,0.3016249555212905,0.4318953804347826,0.4318953804347826,True,0.10847259312868118,0.16846242547035217,0.662109375,0.5703125
+input_layernorm,4,14,0.177762982689747,0.21619433198380567,0.21619433198380567,True,0.10847259312868118,0.11825777590274811,0.662109375,0.6875
+input_layernorm,4,15,0.799410029498525,0.5714938328003655,0.799410029498525,True,0.10847259312868118,0.06029079854488373,0.662109375,0.779296875
+input_layernorm,4,16,0.2727919383521043,0.21432563338301044,0.2727919383521043,True,0.10847259312868118,0.09195870161056519,0.662109375,0.7763671875
+input_layernorm,4,17,0.644927536231884,0.3920704845814978,0.644927536231884,True,0.10847259312868118,0.06483674794435501,0.662109375,1.0012359619140625
+input_layernorm,4,18,0.927536231884058,0.48120300751879697,0.927536231884058,True,0.10847259312868118,0.06785038113594055,0.662109375,1.1640625
+input_layernorm,4,19,1.191740412979351,0.5437415881561238,1.191740412979351,True,0.10847259312868118,0.19409698247909546,0.662109375,0.884765625
+input_layernorm,4,20,1.0705882352941176,0.5170454545454546,1.0705882352941176,True,0.10847259312868118,0.08625888079404831,0.662109375,1.0262451171875
+input_layernorm,4,21,1.4129793510324484,0.5855745721271394,1.4129793510324484,True,0.10847259312868118,0.07095508277416229,0.662109375,1.043212890625
+input_layernorm,4,22,1.2558823529411764,0.5567144719687093,1.2558823529411764,True,0.10847259312868118,0.07214703410863876,0.662109375,1.044921875
+input_layernorm,4,23,1.7529411764705882,0.6367521367521367,1.7529411764705882,True,0.10847259312868118,0.15369786322116852,0.662109375,0.98577880859375
+input_layernorm,5,6,0.21281039684381528,0.27034198113207547,0.27034198113207547,True,0.13199757039546967,0.1498883068561554,0.5234375,0.47265625
+input_layernorm,5,7,0.40684808612440193,0.6859087471641039,0.6859087471641039,True,0.13199757039546967,0.095920130610466,0.5234375,0.46484375
+input_layernorm,5,8,0.3513911921872121,0.5417613636363636,0.5417613636363636,True,0.13199757039546967,0.20054922997951508,0.5234375,0.41796875
+input_layernorm,5,9,0.41179234202490017,0.7000798722044729,0.7000798722044729,True,0.13199757039546967,0.22649425268173218,0.5234375,0.42578125
+input_layernorm,5,10,0.35843218153687467,0.5586816720257235,0.5586816720257235,True,0.13199757039546967,0.2204512506723404,0.5234375,0.3828125
+input_layernorm,5,11,0.4428827751196172,0.7949543746645196,0.7949543746645196,True,0.13199757039546967,0.16603894531726837,0.5234375,0.34375
+input_layernorm,5,12,0.2774254169603007,0.3839401820546164,0.3839401820546164,True,0.13199757039546967,0.1900545060634613,0.5234375,0.5546875
+input_layernorm,5,13,0.2767029052269517,0.3825577445652174,0.3825577445652174,True,0.13199757039546967,0.16846242547035217,0.5234375,0.5703125
+input_layernorm,5,14,0.4141791044776119,0.2928759894459103,0.4141791044776119,True,0.13199757039546967,0.11825777590274811,0.5234375,0.6875
+input_layernorm,5,15,1.2182835820895523,0.5492010092514719,1.2182835820895523,True,0.13199757039546967,0.06029079854488373,0.5234375,0.779296875
+input_layernorm,5,16,0.5316051136363636,0.34709019244145606,0.5316051136363636,True,0.13199757039546967,0.09195870161056519,0.5234375,0.7763671875
+input_layernorm,5,17,1.0124113475177305,0.5030837004405286,1.0124113475177305,True,0.13199757039546967,0.06483674794435501,0.5234375,1.0012359619140625
+input_layernorm,5,18,1.3581560283687943,0.5759398496240602,1.3581560283687943,True,0.13199757039546967,0.06785038113594055,0.5234375,1.1640625
+input_layernorm,5,19,1.7798507462686568,0.640268456375839,1.7798507462686568,True,0.13199757039546967,0.19409698247909546,0.5234375,0.884765625
+input_layernorm,5,20,1.626865671641791,0.6193181818181818,1.626865671641791,True,0.13199757039546967,0.08625888079404831,0.5234375,1.0262451171875
+input_layernorm,5,21,2.037313432835821,0.6707616707616708,2.037313432835821,True,0.13199757039546967,0.07095508277416229,0.5234375,1.043212890625
+input_layernorm,5,22,1.8619402985074627,0.6505867014341591,1.8619402985074627,True,0.13199757039546967,0.07214703410863876,0.5234375,1.044921875
+input_layernorm,5,23,2.4925373134328357,0.7136752136752137,2.4925373134328357,True,0.13199757039546967,0.15369786322116852,0.5234375,0.98577880859375
+input_layernorm,6,7,0.37231012658227847,0.5931434333249307,0.5931434333249307,True,0.1498883068561554,0.095920130610466,0.47265625,0.46484375
+input_layernorm,6,8,0.1901840490797546,0.23484848484848486,0.23484848484848486,True,0.1498883068561554,0.20054922997951508,0.47265625,0.41796875
+input_layernorm,6,9,0.26697892271662765,0.36421725239616615,0.36421725239616615,True,0.1498883068561554,0.22649425268173218,0.47265625,0.42578125
+input_layernorm,6,10,0.2579185520361991,0.3475609756097561,0.3475609756097561,True,0.1498883068561554,0.2204512506723404,0.47265625,0.3828125
+input_layernorm,6,11,0.41044303797468357,0.6961889425657541,0.6961889425657541,True,0.1498883068561554,0.16603894531726837,0.47265625,0.34375
+input_layernorm,6,12,0.2694422623723488,0.21225247524752475,0.2694422623723488,True,0.1498883068561554,0.1900545060634613,0.47265625,0.5546875
+input_layernorm,6,13,0.2518703241895262,0.22759714512291832,0.2518703241895262,True,0.1498883068561554,0.16846242547035217,0.47265625,0.5703125
+input_layernorm,6,14,0.5578512396694215,0.35809018567639256,0.5578512396694215,True,0.1498883068561554,0.11825777590274811,0.47265625,0.6875
+input_layernorm,6,15,1.3971774193548387,0.5828427249789739,1.3971774193548387,True,0.1498883068561554,0.06029079854488373,0.47265625,0.779296875
+input_layernorm,6,16,0.7217261904761905,0.4191875540190147,0.7217261904761905,True,0.1498883068561554,0.09195870161056519,0.47265625,0.7763671875
+input_layernorm,6,17,1.3450413223140496,0.573568281938326,1.3450413223140496,True,0.1498883068561554,0.06483674794435501,0.47265625,1.0012359619140625
+input_layernorm,6,18,1.7479338842975207,0.6360902255639098,1.7479338842975207,True,0.1498883068561554,0.06785038113594055,0.47265625,1.1640625
+input_layernorm,6,19,2.1115702479338845,0.6786188579017264,2.1115702479338845,True,0.1498883068561554,0.19409698247909546,0.47265625,0.884765625
+input_layernorm,6,20,1.8636363636363635,0.6507936507936508,1.8636363636363635,True,0.1498883068561554,0.08625888079404831,0.47265625,1.0262451171875
+input_layernorm,6,21,2.282258064516129,0.6953316953316954,2.282258064516129,True,0.1498883068561554,0.07095508277416229,0.47265625,1.043212890625
+input_layernorm,6,22,2.092741935483871,0.6766623207301173,2.092741935483871,True,0.1498883068561554,0.07214703410863876,0.47265625,1.044921875
+input_layernorm,6,23,2.774193548387097,0.7350427350427351,2.774193548387097,True,0.1498883068561554,0.15369786322116852,0.47265625,0.98577880859375
+input_layernorm,7,8,0.564910511721704,0.6390151515151515,0.6390151515151515,True,0.095920130610466,0.20054922997951508,0.46484375,0.41796875
+input_layernorm,7,9,0.4822283841693975,0.8075657894736842,0.8075657894736842,True,0.095920130610466,0.22649425268173218,0.46484375,0.42578125
+input_layernorm,7,10,0.4459289135366776,0.6383139067524116,0.6383139067524116,True,0.095920130610466,0.2204512506723404,0.46484375,0.3828125
+input_layernorm,7,11,0.3941018766756032,0.6504424778761062,0.6504424778761062,True,0.095920130610466,0.16603894531726837,0.46484375,0.34375
+input_layernorm,7,12,0.5676959619952494,0.449868073878628,0.5676959619952494,True,0.095920130610466,0.1900545060634613,0.46484375,0.5546875
+input_layernorm,7,13,0.5866983372921615,0.46976902173913043,0.5866983372921615,True,0.095920130610466,0.16846242547035217,0.46484375,0.5703125
+input_layernorm,7,14,0.630188679245283,0.38657407407407407,0.630188679245283,True,0.095920130610466,0.11825777590274811,0.46484375,0.6875
+input_layernorm,7,15,1.4814049586776858,0.5970024979184013,1.4814049586776858,True,0.095920130610466,0.06029079854488373,0.46484375,0.779296875
+input_layernorm,7,16,0.9477434679334917,0.4865853658536585,0.9477434679334917,True,0.095920130610466,0.09195870161056519,0.46484375,0.7763671875
+input_layernorm,7,17,1.2394957983193278,0.5534709193245778,1.2394957983193278,True,0.095920130610466,0.06483674794435501,0.46484375,1.0012359619140625
+input_layernorm,7,18,1.5630081300813008,0.6098334655035687,1.5630081300813008,True,0.095920130610466,0.06785038113594055,0.46484375,1.1640625
+input_layernorm,7,19,2.19327731092437,0.6868421052631579,2.19327731092437,True,0.095920130610466,0.19409698247909546,0.46484375,0.884765625
+input_layernorm,7,20,1.9201680672268908,0.6575539568345323,1.9201680672268908,True,0.095920130610466,0.08625888079404831,0.46484375,1.0262451171875
+input_layernorm,7,21,2.3445378151260505,0.7010050251256281,2.3445378151260505,True,0.095920130610466,0.07095508277416229,0.46484375,1.043212890625
+input_layernorm,7,22,2.1178861788617884,0.6792698826597132,2.1178861788617884,True,0.095920130610466,0.07214703410863876,0.46484375,1.044921875
+input_layernorm,7,23,2.8048780487804876,0.7371794871794872,2.8048780487804876,True,0.095920130610466,0.15369786322116852,0.46484375,0.98577880859375
+input_layernorm,8,9,0.2168141592920354,0.1781818181818182,0.2168141592920354,True,0.20054922997951508,0.22649425268173218,0.41796875,0.42578125
+input_layernorm,8,10,0.19393939393939394,0.1813186813186813,0.19393939393939394,True,0.20054922997951508,0.2204512506723404,0.41796875,0.3828125
+input_layernorm,8,11,0.39980670103092786,0.666129898013956,0.666129898013956,True,0.20054922997951508,0.16603894531726837,0.41796875,0.34375
+input_layernorm,8,12,0.3672566371681416,0.2686084142394822,0.3672566371681416,True,0.20054922997951508,0.1900545060634613,0.41796875,0.5546875
+input_layernorm,8,13,0.3644859813084112,0.2671232876712329,0.3644859813084112,True,0.20054922997951508,0.16846242547035217,0.41796875,0.5703125
+input_layernorm,8,14,0.7616822429906542,0.4323607427055703,0.7616822429906542,True,0.20054922997951508,0.11825777590274811,0.41796875,0.6875
+input_layernorm,8,15,1.677927927927928,0.6265769554247267,1.677927927927928,True,0.20054922997951508,0.06029079854488373,0.41796875,0.779296875
+input_layernorm,8,16,0.9478114478114478,0.48660328435609335,0.9478114478114478,True,0.20054922997951508,0.09195870161056519,0.41796875,0.7763671875
+input_layernorm,8,17,1.6518691588785046,0.6229074889867842,1.6518691588785046,True,0.20054922997951508,0.06483674794435501,0.41796875,1.0012359619140625
+input_layernorm,8,18,2.107476635514019,0.6781954887218045,2.107476635514019,True,0.20054922997951508,0.06785038113594055,0.41796875,1.1640625
+input_layernorm,8,19,2.5186915887850465,0.7158034528552457,2.5186915887850465,True,0.20054922997951508,0.19409698247909546,0.41796875,0.884765625
+input_layernorm,8,20,2.282442748091603,0.6953488372093023,2.282442748091603,True,0.20054922997951508,0.08625888079404831,0.41796875,1.0262451171875
+input_layernorm,8,21,2.6666666666666665,0.7272727272727273,2.6666666666666665,True,0.20054922997951508,0.07095508277416229,0.41796875,1.043212890625
+input_layernorm,8,22,2.467289719626168,0.7115902964959568,2.467289719626168,True,0.20054922997951508,0.07214703410863876,0.41796875,1.044921875
+input_layernorm,8,23,3.2162162162162162,0.7628205128205128,3.2162162162162162,True,0.20054922997951508,0.15369786322116852,0.41796875,0.98577880859375
+input_layernorm,9,10,0.31333333333333335,0.3008849557522124,0.31333333333333335,True,0.22649425268173218,0.2204512506723404,0.42578125,0.3828125
+input_layernorm,9,11,0.5083333333333333,0.5780998389694042,0.5780998389694042,True,0.22649425268173218,0.16603894531726837,0.42578125,0.34375
+input_layernorm,9,12,0.452991452991453,0.31176470588235294,0.452991452991453,True,0.22649425268173218,0.1900545060634613,0.42578125,0.5546875
+input_layernorm,9,13,0.4921259842519685,0.32981530343007914,0.4921259842519685,True,0.22649425268173218,0.16846242547035217,0.42578125,0.5703125
+input_layernorm,9,14,0.8134920634920635,0.4485776805251641,0.8134920634920635,True,0.22649425268173218,0.11825777590274811,0.42578125,0.6875
+input_layernorm,9,15,1.7727272727272727,0.639344262295082,1.7727272727272727,True,0.22649425268173218,0.06029079854488373,0.42578125,0.779296875
+input_layernorm,9,16,1.2080152671755726,0.547104580812446,1.2080152671755726,True,0.22649425268173218,0.09195870161056519,0.42578125,0.7763671875
+input_layernorm,9,17,1.7058823529411764,0.6304347826086957,1.7058823529411764,True,0.22649425268173218,0.06483674794435501,0.42578125,1.0012359619140625
+input_layernorm,9,18,2.1470588235294117,0.6822429906542056,2.1470588235294117,True,0.22649425268173218,0.06785038113594055,0.42578125,1.1640625
+input_layernorm,9,19,2.4873949579831933,0.7132530120481928,2.4873949579831933,True,0.22649425268173218,0.19409698247909546,0.42578125,0.884765625
+input_layernorm,9,20,2.6134453781512605,0.7232558139534884,2.6134453781512605,True,0.22649425268173218,0.08625888079404831,0.42578125,1.0262451171875
+input_layernorm,9,21,2.7394957983193278,0.7325842696629213,2.7394957983193278,True,0.22649425268173218,0.07095508277416229,0.42578125,1.043212890625
+input_layernorm,9,22,2.403669724770642,0.706199460916442,2.403669724770642,True,0.22649425268173218,0.07214703410863876,0.42578125,1.044921875
+input_layernorm,9,23,2.990990990990991,0.7494356659142212,2.990990990990991,True,0.22649425268173218,0.15369786322116852,0.42578125,0.98577880859375
+input_layernorm,10,11,0.350418410041841,0.5394524959742351,0.5394524959742351,True,0.2204512506723404,0.16603894531726837,0.3828125,0.34375
+input_layernorm,10,12,0.4489795918367347,0.30985915492957744,0.4489795918367347,True,0.2204512506723404,0.1900545060634613,0.3828125,0.5546875
+input_layernorm,10,13,0.4897959183673469,0.3287671232876712,0.4897959183673469,True,0.2204512506723404,0.16846242547035217,0.3828125,0.5703125
+input_layernorm,10,14,0.923469387755102,0.48010610079575594,0.923469387755102,True,0.2204512506723404,0.11825777590274811,0.3828125,0.6875
+input_layernorm,10,15,1.8762755102040816,0.6523281596452328,1.8762755102040816,True,0.2204512506723404,0.06029079854488373,0.3828125,0.779296875
+input_layernorm,10,16,1.1346863468634687,0.5315471045808124,1.1346863468634687,True,0.2204512506723404,0.09195870161056519,0.3828125,0.7763671875
+input_layernorm,10,17,1.8954081632653061,0.654625550660793,1.8954081632653061,True,0.2204512506723404,0.06483674794435501,0.3828125,1.0012359619140625
+input_layernorm,10,18,2.392857142857143,0.7052631578947368,2.392857142857143,True,0.2204512506723404,0.06785038113594055,0.3828125,1.1640625
+input_layernorm,10,19,2.8418367346938775,0.7397078353253652,2.8418367346938775,True,0.2204512506723404,0.19409698247909546,0.3828125,0.884765625
+input_layernorm,10,20,2.5357142857142856,0.7171717171717171,2.5357142857142856,True,0.2204512506723404,0.08625888079404831,0.3828125,1.0262451171875
+input_layernorm,10,21,2.9285714285714284,0.7454545454545455,2.9285714285714284,True,0.2204512506723404,0.07095508277416229,0.3828125,1.043212890625
+input_layernorm,10,22,2.7857142857142856,0.7358490566037735,2.7857142857142856,True,0.2204512506723404,0.07214703410863876,0.3828125,1.044921875
+input_layernorm,10,23,3.3333333333333335,0.7692307692307693,3.3333333333333335,True,0.2204512506723404,0.15369786322116852,0.3828125,0.98577880859375
+input_layernorm,11,12,0.7078651685393258,0.4144736842105263,0.7078651685393258,True,0.16603894531726837,0.1900545060634613,0.34375,0.5546875
+input_layernorm,11,13,0.7368421052631579,0.42424242424242425,0.7368421052631579,True,0.16603894531726837,0.16846242547035217,0.34375,0.5703125
+input_layernorm,11,14,1.1420454545454546,0.53315649867374,1.1420454545454546,True,0.16603894531726837,0.11825777590274811,0.34375,0.6875
+input_layernorm,11,15,2.3516483516483517,0.7016393442622951,2.3516483516483517,True,0.16603894531726837,0.06029079854488373,0.34375,0.779296875
+input_layernorm,11,16,1.414745145631068,0.5858776228169368,1.414745145631068,True,0.16603894531726837,0.09195870161056519,0.34375,0.7763671875
+input_layernorm,11,17,2.2244318181818183,0.6898678414096916,2.2244318181818183,True,0.16603894531726837,0.06483674794435501,0.34375,1.0012359619140625
+input_layernorm,11,18,2.778409090909091,0.7353383458646616,2.778409090909091,True,0.16603894531726837,0.06785038113594055,0.34375,1.1640625
+input_layernorm,11,19,3.278409090909091,0.7662682602921647,3.278409090909091,True,0.16603894531726837,0.19409698247909546,0.34375,0.884765625
+input_layernorm,11,20,3.056603773584906,0.7534883720930232,3.056603773584906,True,0.16603894531726837,0.08625888079404831,0.34375,1.0262451171875
+input_layernorm,11,21,3.4831460674157304,0.7769423558897243,3.4831460674157304,True,0.16603894531726837,0.07095508277416229,0.34375,1.043212890625
+input_layernorm,11,22,3.215909090909091,0.7628032345013477,3.215909090909091,True,0.16603894531726837,0.07214703410863876,0.34375,1.044921875
+input_layernorm,11,23,4.142857142857143,0.8055555555555556,4.142857142857143,True,0.16603894531726837,0.15369786322116852,0.34375,0.98577880859375
+input_layernorm,12,13,0.10185185185185185,0.1134020618556701,0.1134020618556701,True,0.1900545060634613,0.16846242547035217,0.5546875,0.5703125
+input_layernorm,12,14,0.3644343302990897,0.26709554443650224,0.3644343302990897,True,0.1900545060634613,0.11825777590274811,0.5546875,0.6875
+input_layernorm,12,15,1.0359589041095891,0.5088309503784693,1.0359589041095891,True,0.1900545060634613,0.06029079854488373,0.5546875,0.779296875
+input_layernorm,12,16,0.6295774647887324,0.3863439930855661,0.6295774647887324,True,0.1900545060634613,0.09195870161056519,0.5546875,0.7763671875
+input_layernorm,12,17,0.9982394366197183,0.4995594713656388,0.9982394366197183,True,0.1900545060634613,0.06483674794435501,0.5546875,1.0012359619140625
+input_layernorm,12,18,1.341549295774648,0.5729323308270676,1.341549295774648,True,0.1900545060634613,0.06785038113594055,0.5546875,1.1640625
+input_layernorm,12,19,1.6514084507042253,0.6228419654714475,1.6514084507042253,True,0.1900545060634613,0.19409698247909546,0.5546875,0.884765625
+input_layernorm,12,20,1.5294117647058822,0.6046511627906976,1.5294117647058822,True,0.1900545060634613,0.08625888079404831,0.5546875,1.0262451171875
+input_layernorm,12,21,1.7876712328767124,0.6412776412776413,1.7876712328767124,True,0.1900545060634613,0.07095508277416229,0.5546875,1.043212890625
+input_layernorm,12,22,1.6267123287671232,0.6192959582790091,1.6267123287671232,True,0.1900545060634613,0.07214703410863876,0.5546875,1.044921875
+input_layernorm,12,23,2.2054794520547945,0.688034188034188,2.2054794520547945,True,0.1900545060634613,0.15369786322116852,0.5546875,0.98577880859375
+input_layernorm,13,14,0.3580163043478261,0.263631815907954,0.3580163043478261,True,0.16846242547035217,0.11825777590274811,0.5703125,0.6875
+input_layernorm,13,15,1.0890410958904109,0.521311475409836,1.0890410958904109,True,0.16846242547035217,0.06029079854488373,0.5703125,0.779296875
+input_layernorm,13,16,0.6413043478260869,0.39072847682119205,0.6413043478260869,True,0.16846242547035217,0.09195870161056519,0.5703125,0.7763671875
+input_layernorm,13,17,0.9434931506849316,0.48546255506607927,0.9434931506849316,True,0.16846242547035217,0.06483674794435501,0.5703125,1.0012359619140625
+input_layernorm,13,18,1.2773972602739727,0.5609022556390978,1.2773972602739727,True,0.16846242547035217,0.06785038113594055,0.5703125,1.1640625
+input_layernorm,13,19,1.5787671232876712,0.6122177954847278,1.5787671232876712,True,0.16846242547035217,0.19409698247909546,0.5703125,0.884765625
+input_layernorm,13,20,1.457142857142857,0.5930232558139535,1.457142857142857,True,0.16846242547035217,0.08625888079404831,0.5703125,1.0262451171875
+input_layernorm,13,21,1.759322033898305,0.6375921375921376,1.759322033898305,True,0.16846242547035217,0.07095508277416229,0.5703125,1.043212890625
+input_layernorm,13,22,1.6,0.6153846153846154,1.6,True,0.16846242547035217,0.07214703410863876,0.5703125,1.044921875
+input_layernorm,13,23,2.1728813559322036,0.6848290598290598,2.1728813559322036,True,0.16846242547035217,0.15369786322116852,0.5703125,0.98577880859375
+input_layernorm,14,15,0.6661931818181818,0.667953667953668,0.667953667953668,True,0.11825777590274811,0.06029079854488373,0.6875,0.779296875
+input_layernorm,14,16,0.4027630180658874,0.2871212121212121,0.4027630180658874,True,0.11825777590274811,0.09195870161056519,0.6875,0.7763671875
+input_layernorm,14,17,0.640807651434644,0.3905440414507772,0.640807651434644,True,0.11825777590274811,0.06483674794435501,0.6875,1.0012359619140625
+input_layernorm,14,18,0.8575982996811902,0.46167048054919907,0.8575982996811902,True,0.11825777590274811,0.06785038113594055,0.6875,1.1640625
+input_layernorm,14,19,1.1590909090909092,0.5774647887323944,1.1590909090909092,True,0.11825777590274811,0.19409698247909546,0.6875,0.884765625
+input_layernorm,14,20,1.107843137254902,0.5255813953488372,1.107843137254902,True,0.11825777590274811,0.08625888079404831,0.6875,1.0262451171875
+input_layernorm,14,21,1.2613636363636365,0.5577889447236181,1.2613636363636365,True,0.11825777590274811,0.07095508277416229,0.6875,1.043212890625
+input_layernorm,14,22,1.0710227272727273,0.5171467764060357,1.0710227272727273,True,0.11825777590274811,0.07214703410863876,0.6875,1.044921875
+input_layernorm,14,23,1.5056818181818181,0.6009070294784581,1.5056818181818181,True,0.11825777590274811,0.15369786322116852,0.6875,0.98577880859375
+input_layernorm,15,16,0.8797573083287369,0.4993757802746567,0.8797573083287369,True,0.06029079854488373,0.09195870161056519,0.779296875,0.7763671875
+input_layernorm,15,17,0.732297852900868,0.42273206751054854,0.732297852900868,True,0.06029079854488373,0.06483674794435501,0.779296875,1.0012359619140625
+input_layernorm,15,18,0.8297872340425532,0.45348837209302323,0.8297872340425532,True,0.06029079854488373,0.06785038113594055,0.779296875,1.1640625
+input_layernorm,15,19,1.0300751879699248,0.5074074074074074,1.0300751879699248,True,0.06029079854488373,0.19409698247909546,0.779296875,0.884765625
+input_layernorm,15,20,1.0451127819548873,0.5110294117647058,1.0451127819548873,True,0.06029079854488373,0.08625888079404831,0.779296875,1.0262451171875
+input_layernorm,15,21,1.150375939849624,0.534965034965035,1.150375939849624,True,0.06029079854488373,0.07095508277416229,0.779296875,1.043212890625
+input_layernorm,15,22,1.005012531328321,0.50125,1.005012531328321,True,0.06029079854488373,0.07214703410863876,0.779296875,1.044921875
+input_layernorm,15,23,1.100250626566416,0.5238663484486874,1.100250626566416,True,0.06029079854488373,0.15369786322116852,0.779296875,0.98577880859375
+input_layernorm,16,17,0.3909313725490196,0.28105726872246695,0.3909313725490196,True,0.09195870161056519,0.06483674794435501,0.7763671875,1.0012359619140625
+input_layernorm,16,18,0.6299019607843137,0.38646616541353385,0.6299019607843137,True,0.09195870161056519,0.06785038113594055,0.7763671875,1.1640625
+input_layernorm,16,19,0.8952618453865336,0.7777777777777778,0.8952618453865336,True,0.09195870161056519,0.19409698247909546,0.7763671875,0.884765625
+input_layernorm,16,20,0.7578027465667915,0.43110795454545453,0.7578027465667915,True,0.09195870161056519,0.08625888079404831,0.7763671875,1.0262451171875
+input_layernorm,16,21,1.0324594257178528,0.507985257985258,1.0324594257178528,True,0.09195870161056519,0.07095508277416229,0.7763671875,1.043212890625
+input_layernorm,16,22,0.9151061173533084,0.47783572359843546,0.9151061173533084,True,0.09195870161056519,0.07214703410863876,0.7763671875,1.044921875
+input_layernorm,16,23,1.3370786516853932,0.5721153846153846,1.3370786516853932,True,0.09195870161056519,0.15369786322116852,0.7763671875,0.98577880859375
+input_layernorm,17,18,0.20625,0.17098445595854922,0.20625,True,0.06483674794435501,0.06785038113594055,1.0012359619140625,1.1640625
+input_layernorm,17,19,0.4493804959080726,0.6191966614501826,0.6191966614501826,True,0.06483674794435501,0.19409698247909546,1.0012359619140625,0.884765625
+input_layernorm,17,20,0.35731807031888796,0.2632530120481928,0.35731807031888796,True,0.06483674794435501,0.08625888079404831,1.0012359619140625,1.0262451171875
+input_layernorm,17,21,0.5803900259614804,0.367244804400978,0.5803900259614804,True,0.06483674794435501,0.07095508277416229,1.0012359619140625,1.043212890625
+input_layernorm,17,22,0.4475112054729889,0.30915906127770537,0.4475112054729889,True,0.06483674794435501,0.07214703410863876,1.0012359619140625,1.044921875
+input_layernorm,17,23,0.7790511605224256,0.45824255628177196,0.7790511605224256,True,0.06483674794435501,0.15369786322116852,1.0012359619140625,0.98577880859375
+input_layernorm,18,19,0.40243142144638405,0.6734480959833072,0.6734480959833072,True,0.06785038113594055,0.19409698247909546,1.1640625,0.884765625
+input_layernorm,18,20,0.17917304747320062,0.19578313253012047,0.19578313253012047,True,0.06785038113594055,0.08625888079404831,1.1640625,1.0262451171875
+input_layernorm,18,21,0.3624161073825503,0.2660098522167488,0.3624161073825503,True,0.06785038113594055,0.07095508277416229,1.1640625,1.043212890625
+input_layernorm,18,22,0.22378199834847234,0.27217125382262997,0.27217125382262997,True,0.06785038113594055,0.07214703410863876,1.1640625,1.044921875
+input_layernorm,18,23,0.4929343308395677,0.5599088838268793,0.5599088838268793,True,0.06785038113594055,0.15369786322116852,1.1640625,0.98577880859375
+input_layernorm,19,20,0.702803274621682,0.4127331002331002,0.702803274621682,True,0.19409698247909546,0.08625888079404831,0.884765625,1.0262451171875
+input_layernorm,19,21,0.8282009724473258,0.4530141843971631,0.8282009724473258,True,0.19409698247909546,0.07095508277416229,0.884765625,1.043212890625
+input_layernorm,19,22,0.6790923824959482,0.40444015444015446,0.6790923824959482,True,0.19409698247909546,0.07214703410863876,0.884765625,1.044921875
+input_layernorm,19,23,0.34451054624121147,0.5234624145785877,0.5234624145785877,True,0.19409698247909546,0.15369786322116852,0.884765625,0.98577880859375
+input_layernorm,20,21,0.2359208523592085,0.19088669950738915,0.2359208523592085,True,0.08625888079404831,0.07095508277416229,1.0262451171875,1.043212890625
+input_layernorm,20,22,0.18092909535452323,0.2208955223880597,0.2208955223880597,True,0.08625888079404831,0.07214703410863876,1.0262451171875,1.044921875
+input_layernorm,20,23,0.3733133433283358,0.5809627347949508,0.5809627347949508,True,0.08625888079404831,0.15369786322116852,1.0262451171875,0.98577880859375
+input_layernorm,21,22,0.14822546972860126,0.17401960784313725,0.17401960784313725,True,0.07095508277416229,0.07214703410863876,1.043212890625,1.044921875
+input_layernorm,21,23,0.3882978723404255,0.6347826086956522,0.6347826086956522,True,0.07095508277416229,0.15369786322116852,1.043212890625,0.98577880859375
+input_layernorm,22,23,0.333976833976834,0.5014492753623189,0.5014492753623189,True,0.07214703410863876,0.15369786322116852,1.044921875,0.98577880859375
+post_attention_layernorm,0,1,129.5,0.9923371647509579,129.5,True,0.108433797955513,0.09410307556390762,0.00390625,0.45703125
+post_attention_layernorm,0,2,154.0,0.9935483870967742,154.0,True,0.108433797955513,0.10385182499885559,0.00390625,0.533203125
+post_attention_layernorm,0,3,165.5,0.993993993993994,165.5,True,0.108433797955513,0.11546041816473007,0.00390625,0.48828125
+post_attention_layernorm,0,4,181.5,0.9945205479452055,181.5,True,0.108433797955513,0.11874310672283173,0.00390625,0.4765625
+post_attention_layernorm,0,5,194.25,0.9948783610755442,194.25,True,0.108433797955513,0.14567376673221588,0.00390625,0.421875
+post_attention_layernorm,0,6,221.0,0.9954954954954955,221.0,True,0.108433797955513,0.13559450209140778,0.00390625,0.43359375
+post_attention_layernorm,0,7,188.5,0.9947229551451188,188.5,True,0.108433797955513,0.17147497832775116,0.00390625,0.375
+post_attention_layernorm,0,8,187.0,0.9946808510638298,187.0,True,0.108433797955513,0.18255221843719482,0.00390625,0.33984375
+post_attention_layernorm,0,9,185.0,0.9946236559139785,185.0,True,0.108433797955513,0.17869991064071655,0.00390625,0.3515625
+post_attention_layernorm,0,10,215.75,0.9953863898500577,215.75,True,0.108433797955513,0.16611140966415405,0.00390625,0.36328125
+post_attention_layernorm,0,11,195.75,0.9949174078780177,195.75,True,0.108433797955513,0.1528426557779312,0.00390625,0.41796875
+post_attention_layernorm,0,12,188.0,0.9947089947089947,188.0,True,0.108433797955513,0.12497646361589432,0.00390625,0.5234375
+post_attention_layernorm,0,13,199.25,0.9950062421972534,199.25,True,0.108433797955513,0.09632012248039246,0.00390625,0.619140625
+post_attention_layernorm,0,14,228.875,0.9956498096791735,228.875,True,0.108433797955513,0.053455036133527756,0.00390625,0.87646484375
+post_attention_layernorm,0,15,197.25,0.9949558638083228,197.25,True,0.108433797955513,0.06801187992095947,0.00390625,0.736328125
+post_attention_layernorm,0,16,189.5,0.994750656167979,189.5,True,0.108433797955513,0.08680811524391174,0.00390625,0.72265625
+post_attention_layernorm,0,17,181.5,0.9945205479452055,181.5,True,0.108433797955513,0.09242258220911026,0.00390625,0.712890625
+post_attention_layernorm,0,18,224.875,0.9955727725511898,224.875,True,0.108433797955513,0.09676424413919449,0.00390625,0.8095703125
+post_attention_layernorm,0,19,233.5,0.9957356076759062,233.5,True,0.108433797955513,0.07613449543714523,0.00390625,0.81640625
+post_attention_layernorm,0,20,209.5,0.995249406175772,209.5,True,0.108433797955513,0.07808392494916916,0.00390625,0.822265625
+post_attention_layernorm,0,21,220.0,0.995475113122172,220.0,True,0.108433797955513,0.06570179015398026,0.00390625,0.86328125
+post_attention_layernorm,0,22,248.5625,0.995992987728525,248.5625,True,0.108433797955513,0.05012959614396095,0.00390625,0.9713134765625
+post_attention_layernorm,0,23,258.484375,0.9961462034082014,258.484375,True,0.108433797955513,0.06521295011043549,0.00390625,1.01361083984375
+post_attention_layernorm,1,2,0.42512908777969016,0.29830917874396135,0.42512908777969016,True,0.09410307556390762,0.10385182499885559,0.45703125,0.533203125
+post_attention_layernorm,1,3,0.27586206896551724,0.26858457997698504,0.27586206896551724,True,0.09410307556390762,0.11546041816473007,0.45703125,0.48828125
+post_attention_layernorm,1,4,0.39846743295019155,0.28493150684931506,0.39846743295019155,True,0.09410307556390762,0.11874310672283173,0.45703125,0.4765625
+post_attention_layernorm,1,5,0.49616858237547895,0.33162612035851474,0.49616858237547895,True,0.09410307556390762,0.14567376673221588,0.45703125,0.421875
+post_attention_layernorm,1,6,0.7011494252873564,0.41216216216216217,0.7011494252873564,True,0.09410307556390762,0.13559450209140778,0.45703125,0.43359375
+post_attention_layernorm,1,7,0.4521072796934866,0.45161290322580644,0.4521072796934866,True,0.09410307556390762,0.17147497832775116,0.45703125,0.375
+post_attention_layernorm,1,8,0.5296052631578947,0.594488188976378,0.594488188976378,True,0.09410307556390762,0.18255221843719482,0.45703125,0.33984375
+post_attention_layernorm,1,9,0.42528735632183906,0.5133079847908745,0.5133079847908745,True,0.09410307556390762,0.17869991064071655,0.45703125,0.3515625
+post_attention_layernorm,1,10,0.6609195402298851,0.5073260073260073,0.6609195402298851,True,0.09410307556390762,0.16611140966415405,0.45703125,0.36328125
+post_attention_layernorm,1,11,0.5076628352490421,0.3785807291666667,0.5076628352490421,True,0.09410307556390762,0.1528426557779312,0.45703125,0.41796875
+post_attention_layernorm,1,12,0.7397203947368421,0.4274732620320856,0.7397203947368421,True,0.09410307556390762,0.12497646361589432,0.45703125,0.5234375
+post_attention_layernorm,1,13,0.8725328947368421,0.46596398770311814,0.8725328947368421,True,0.09410307556390762,0.09632012248039246,0.45703125,0.619140625
+post_attention_layernorm,1,14,1.08739406779661,0.5209337731540218,1.08739406779661,True,0.09410307556390762,0.053455036133527756,0.45703125,0.87646484375
+post_attention_layernorm,1,15,1.7415254237288136,0.6352395672333848,1.7415254237288136,True,0.09410307556390762,0.06801187992095947,0.45703125,0.736328125
+post_attention_layernorm,1,16,0.9901315789473685,0.4975206611570248,0.9901315789473685,True,0.09410307556390762,0.08680811524391174,0.45703125,0.72265625
+post_attention_layernorm,1,17,1.009733606557377,0.5024216161101198,1.009733606557377,True,0.09410307556390762,0.09242258220911026,0.45703125,0.712890625
+post_attention_layernorm,1,18,1.194078947368421,0.5442278860569715,1.194078947368421,True,0.09410307556390762,0.09676424413919449,0.45703125,0.8095703125
+post_attention_layernorm,1,19,1.5277777777777777,0.6043956043956044,1.5277777777777777,True,0.09410307556390762,0.07613449543714523,0.45703125,0.81640625
+post_attention_layernorm,1,20,1.3644067796610169,0.5770609318996416,1.3644067796610169,True,0.09410307556390762,0.07808392494916916,0.45703125,0.822265625
+post_attention_layernorm,1,21,1.7166666666666666,0.6319018404907976,1.7166666666666666,True,0.09410307556390762,0.06570179015398026,0.45703125,0.86328125
+post_attention_layernorm,1,22,2.377049180327869,0.7038834951456311,2.377049180327869,True,0.09410307556390762,0.05012959614396095,0.45703125,0.9713134765625
+post_attention_layernorm,1,23,2.6324786324786325,0.7247058823529412,2.6324786324786325,True,0.09410307556390762,0.06521295011043549,0.45703125,1.01361083984375
+post_attention_layernorm,2,3,0.3058252427184466,0.4405594405594406,0.4405594405594406,True,0.10385182499885559,0.11546041816473007,0.533203125,0.48828125
+post_attention_layernorm,2,4,0.3094660194174757,0.44815465729349735,0.44815465729349735,True,0.10385182499885559,0.11874310672283173,0.533203125,0.4765625
+post_attention_layernorm,2,5,0.3109903381642512,0.45135845749342685,0.45135845749342685,True,0.10385182499885559,0.14567376673221588,0.533203125,0.421875
+post_attention_layernorm,2,6,0.432258064516129,0.5354659248956884,0.5354659248956884,True,0.10385182499885559,0.13559450209140778,0.533203125,0.43359375
+post_attention_layernorm,2,7,0.3689613526570048,0.584688995215311,0.584688995215311,True,0.10385182499885559,0.17147497832775116,0.533203125,0.375
+post_attention_layernorm,2,8,0.42462600690448793,0.738,0.738,True,0.10385182499885559,0.18255221843719482,0.533203125,0.33984375
+post_attention_layernorm,2,9,0.42002301495972383,0.7242063492063492,0.7242063492063492,True,0.10385182499885559,0.17869991064071655,0.533203125,0.3515625
+post_attention_layernorm,2,10,0.414368739953011,0.7075591216216216,0.7075591216216216,True,0.10385182499885559,0.16611140966415405,0.533203125,0.36328125
+post_attention_layernorm,2,11,0.37333937198067635,0.5957600578173934,0.5957600578173934,True,0.10385182499885559,0.1528426557779312,0.533203125,0.41796875
+post_attention_layernorm,2,12,0.46098066298342544,0.5644780349551252,0.5644780349551252,True,0.10385182499885559,0.12497646361589432,0.533203125,0.5234375
+post_attention_layernorm,2,13,0.5725138121546961,0.52416014726185,0.5725138121546961,True,0.10385182499885559,0.09632012248039246,0.533203125,0.619140625
+post_attention_layernorm,2,14,0.8044871794871795,0.4500659920809503,0.8044871794871795,True,0.10385182499885559,0.053455036133527756,0.533203125,0.87646484375
+post_attention_layernorm,2,15,1.36996336996337,0.5780525502318392,1.36996336996337,True,0.10385182499885559,0.06801187992095947,0.533203125,0.736328125
+post_attention_layernorm,2,16,0.6712707182320442,0.47909888895709285,0.6712707182320442,True,0.10385182499885559,0.08680811524391174,0.533203125,0.72265625
+post_attention_layernorm,2,17,0.7513489208633094,0.4290115532734275,0.7513489208633094,True,0.10385182499885559,0.09242258220911026,0.533203125,0.712890625
+post_attention_layernorm,2,18,0.8425414364640884,0.4572713643178411,0.8425414364640884,True,0.10385182499885559,0.09676424413919449,0.533203125,0.8095703125
+post_attention_layernorm,2,19,1.1666666666666667,0.5384615384615384,1.1666666666666667,True,0.10385182499885559,0.07613449543714523,0.533203125,0.81640625
+post_attention_layernorm,2,20,1.043956043956044,0.510752688172043,1.043956043956044,True,0.10385182499885559,0.07808392494916916,0.533203125,0.822265625
+post_attention_layernorm,2,21,1.381294964028777,0.5800604229607251,1.381294964028777,True,0.10385182499885559,0.06570179015398026,0.533203125,0.86328125
+post_attention_layernorm,2,22,1.9640287769784173,0.662621359223301,1.9640287769784173,True,0.10385182499885559,0.05012959614396095,0.533203125,0.9713134765625
+post_attention_layernorm,2,23,2.1135531135531136,0.6788235294117647,2.1135531135531136,True,0.10385182499885559,0.06521295011043549,0.533203125,1.01361083984375
+post_attention_layernorm,3,4,0.1832089552238806,0.22430333485609869,0.22430333485609869,True,0.11546041816473007,0.11874310672283173,0.48828125,0.4765625
+post_attention_layernorm,3,5,0.1939149560117302,0.2405638926784902,0.2405638926784902,True,0.11546041816473007,0.14567376673221588,0.48828125,0.421875
+post_attention_layernorm,3,6,0.3333333333333333,0.25,0.3333333333333333,True,0.11546041816473007,0.13559450209140778,0.48828125,0.43359375
+post_attention_layernorm,3,7,0.2893772893772894,0.4072164948453608,0.4072164948453608,True,0.11546041816473007,0.17147497832775116,0.48828125,0.375
+post_attention_layernorm,3,8,0.34798534798534797,0.5337078651685393,0.5337078651685393,True,0.11546041816473007,0.18255221843719482,0.48828125,0.33984375
+post_attention_layernorm,3,9,0.34065934065934067,0.5166666666666667,0.5166666666666667,True,0.11546041816473007,0.17869991064071655,0.48828125,0.3515625
+post_attention_layernorm,3,10,0.3333333333333333,0.5,0.5,True,0.11546041816473007,0.16611140966415405,0.48828125,0.36328125
+post_attention_layernorm,3,11,0.2514124293785311,0.28756009615384615,0.28756009615384615,True,0.11546041816473007,0.1528426557779312,0.48828125,0.41796875
+post_attention_layernorm,3,12,0.43326558265582654,0.3787001287001287,0.43326558265582654,True,0.11546041816473007,0.12497646361589432,0.48828125,0.5234375
+post_attention_layernorm,3,13,0.5426829268292683,0.35177865612648224,0.5426829268292683,True,0.11546041816473007,0.09632012248039246,0.48828125,0.619140625
+post_attention_layernorm,3,14,0.902027027027027,0.47424511545293074,0.902027027027027,True,0.11546041816473007,0.053455036133527756,0.48828125,0.87646484375
+post_attention_layernorm,3,15,1.56,0.609375,1.56,True,0.11546041816473007,0.06801187992095947,0.48828125,0.736328125
+post_attention_layernorm,3,16,0.8092643051771117,0.44728915662650603,0.8092643051771117,True,0.11546041816473007,0.08680811524391174,0.48828125,0.72265625
+post_attention_layernorm,3,17,0.8933397683397684,0.47183278103492227,0.8933397683397684,True,0.11546041816473007,0.09242258220911026,0.48828125,0.712890625
+post_attention_layernorm,3,18,0.9215,0.47957325006505336,0.9215,True,0.11546041816473007,0.09676424413919449,0.48828125,0.8095703125
+post_attention_layernorm,3,19,1.366,0.577345731191885,1.366,True,0.11546041816473007,0.07613449543714523,0.48828125,0.81640625
+post_attention_layernorm,3,20,1.169,0.5389580451821115,1.169,True,0.11546041816473007,0.07808392494916916,0.48828125,0.822265625
+post_attention_layernorm,3,21,1.5396825396825398,0.60625,1.5396825396825398,True,0.11546041816473007,0.06570179015398026,0.48828125,0.86328125
+post_attention_layernorm,3,22,2.2698412698412698,0.6941747572815534,2.2698412698412698,True,0.11546041816473007,0.05012959614396095,0.48828125,0.9713134765625
+post_attention_layernorm,3,23,2.4,0.7058823529411765,2.4,True,0.11546041816473007,0.06521295011043549,0.48828125,1.01361083984375
+post_attention_layernorm,4,5,0.16475095785440613,0.19724770642201836,0.19724770642201836,True,0.11874310672283173,0.14567376673221588,0.4765625,0.421875
+post_attention_layernorm,4,6,0.2548342541436464,0.2030820033021464,0.2548342541436464,True,0.11874310672283173,0.13559450209140778,0.4765625,0.43359375
+post_attention_layernorm,4,7,0.2785923753665689,0.3861788617886179,0.3861788617886179,True,0.11874310672283173,0.17147497832775116,0.4765625,0.375
+post_attention_layernorm,4,8,0.3253731343283582,0.4823008849557522,0.4823008849557522,True,0.11874310672283173,0.18255221843719482,0.4765625,0.33984375
+post_attention_layernorm,4,9,0.3128491620111732,0.45528455284552843,0.45528455284552843,True,0.11874310672283173,0.17869991064071655,0.4765625,0.3515625
+post_attention_layernorm,4,10,0.30434782608695654,0.4375,0.4375,True,0.11874310672283173,0.16611140966415405,0.4765625,0.36328125
+post_attention_layernorm,4,11,0.2160188457008245,0.27554086538461536,0.27554086538461536,True,0.11874310672283173,0.1528426557779312,0.4765625,0.41796875
+post_attention_layernorm,4,12,0.3737012987012987,0.3658301158301158,0.3737012987012987,True,0.11874310672283173,0.12497646361589432,0.4765625,0.5234375
+post_attention_layernorm,4,13,0.4785714285714286,0.32367149758454106,0.4785714285714286,True,0.11874310672283173,0.09632012248039246,0.4765625,0.619140625
+post_attention_layernorm,4,14,0.9615384615384616,0.49019607843137253,0.9615384615384616,True,0.11874310672283173,0.053455036133527756,0.4765625,0.87646484375
+post_attention_layernorm,4,15,1.6598360655737705,0.6240369799691834,1.6598360655737705,True,0.11874310672283173,0.06801187992095947,0.4765625,0.736328125
+post_attention_layernorm,4,16,0.7994579945799458,0.4442771084337349,0.7994579945799458,True,0.11874310672283173,0.08680811524391174,0.4765625,0.72265625
+post_attention_layernorm,4,17,0.9569672131147541,0.48900523560209425,0.9569672131147541,True,0.11874310672283173,0.09242258220911026,0.4765625,0.712890625
+post_attention_layernorm,4,18,1.0363729508196722,0.5089308176100629,1.0363729508196722,True,0.11874310672283173,0.09676424413919449,0.4765625,0.8095703125
+post_attention_layernorm,4,19,1.4246031746031746,0.5875613747954174,1.4246031746031746,True,0.11874310672283173,0.07613449543714523,0.4765625,0.81640625
+post_attention_layernorm,4,20,1.2930327868852458,0.5638963360142984,1.2930327868852458,True,0.11874310672283173,0.07808392494916916,0.4765625,0.822265625
+post_attention_layernorm,4,21,1.5806451612903225,0.6125,1.5806451612903225,True,0.11874310672283173,0.06570179015398026,0.4765625,0.86328125
+post_attention_layernorm,4,22,2.3225806451612905,0.6990291262135923,2.3225806451612905,True,0.11874310672283173,0.05012959614396095,0.4765625,0.9713134765625
+post_attention_layernorm,4,23,2.373015873015873,0.7035294117647058,2.373015873015873,True,0.11874310672283173,0.06521295011043549,0.4765625,1.01361083984375
+post_attention_layernorm,5,6,0.32515337423312884,0.24537037037037038,0.32515337423312884,True,0.14567376673221588,0.13559450209140778,0.421875,0.43359375
+post_attention_layernorm,5,7,0.1608832807570978,0.19172932330827067,0.19172932330827067,True,0.14567376673221588,0.17147497832775116,0.421875,0.375
+post_attention_layernorm,5,8,0.2413793103448276,0.3181818181818182,0.3181818181818182,True,0.14567376673221588,0.18255221843719482,0.421875,0.33984375
+post_attention_layernorm,5,9,0.24968314321926488,0.3327702702702703,0.3327702702702703,True,0.14567376673221588,0.17869991064071655,0.421875,0.3515625
+post_attention_layernorm,5,10,0.2541436464088398,0.30067567567567566,0.30067567567567566,True,0.14567376673221588,0.16611140966415405,0.421875,0.36328125
+post_attention_layernorm,5,11,0.214,0.23786808894230768,0.23786808894230768,True,0.14567376673221588,0.1528426557779312,0.421875,0.41796875
+post_attention_layernorm,5,12,0.4103333333333333,0.3254906692406692,0.4103333333333333,True,0.14567376673221588,0.12497646361589432,0.421875,0.5234375
+post_attention_layernorm,5,13,0.555045871559633,0.35693215339233036,0.555045871559633,True,0.14567376673221588,0.09632012248039246,0.421875,0.619140625
+post_attention_layernorm,5,14,1.275092936802974,0.5604575163398693,1.275092936802974,True,0.14567376673221588,0.053455036133527756,0.421875,0.87646484375
+post_attention_layernorm,5,15,1.962962962962963,0.6625,1.962962962962963,True,0.14567376673221588,0.06801187992095947,0.421875,0.736328125
+post_attention_layernorm,5,16,1.063197026022305,0.5153153153153153,1.063197026022305,True,0.14567376673221588,0.08680811524391174,0.421875,0.72265625
+post_attention_layernorm,5,17,1.2333715596330275,0.5522464698331194,1.2333715596330275,True,0.14567376673221588,0.09242258220911026,0.421875,0.712890625
+post_attention_layernorm,5,18,1.2573327850877194,0.5569993017395792,1.2573327850877194,True,0.14567376673221588,0.09676424413919449,0.421875,0.8095703125
+post_attention_layernorm,5,19,1.738532110091743,0.6348408710217756,1.738532110091743,True,0.14567376673221588,0.07613449543714523,0.421875,0.81640625
+post_attention_layernorm,5,20,1.5136363636363637,0.6021699819168174,1.5136363636363637,True,0.14567376673221588,0.07808392494916916,0.421875,0.822265625
+post_attention_layernorm,5,21,2.036697247706422,0.6706948640483383,2.036697247706422,True,0.14567376673221588,0.06570179015398026,0.421875,0.86328125
+post_attention_layernorm,5,22,2.779816513761468,0.7354368932038835,2.779816513761468,True,0.14567376673221588,0.05012959614396095,0.421875,0.9713134765625
+post_attention_layernorm,5,23,2.935185185185185,0.7458823529411764,2.935185185185185,True,0.14567376673221588,0.06521295011043549,0.421875,1.01361083984375
+post_attention_layernorm,6,7,0.3519163763066202,0.543010752688172,0.543010752688172,True,0.13559450209140778,0.17147497832775116,0.43359375,0.375
+post_attention_layernorm,6,8,0.4122905027932961,0.7015209125475285,0.7015209125475285,True,0.13559450209140778,0.18255221843719482,0.43359375,0.33984375
+post_attention_layernorm,6,9,0.35195530726256985,0.5431034482758621,0.5431034482758621,True,0.13559450209140778,0.17869991064071655,0.43359375,0.3515625
+post_attention_layernorm,6,10,0.3743016759776536,0.5982142857142857,0.5982142857142857,True,0.13559450209140778,0.16611140966415405,0.43359375,0.36328125
+post_attention_layernorm,6,11,0.2961672473867596,0.4207920792079208,0.4207920792079208,True,0.13559450209140778,0.1528426557779312,0.43359375,0.41796875
+post_attention_layernorm,6,12,0.3181818181818182,0.29812834224598933,0.3181818181818182,True,0.13559450209140778,0.12497646361589432,0.43359375,0.5234375
+post_attention_layernorm,6,13,0.5526315789473685,0.3559322033898305,0.5526315789473685,True,0.13559450209140778,0.09632012248039246,0.43359375,0.619140625
+post_attention_layernorm,6,14,1.1714801444043321,0.5394846217788861,1.1714801444043321,True,0.13559450209140778,0.053455036133527756,0.43359375,0.87646484375
+post_attention_layernorm,6,15,1.927927927927928,0.6584615384615384,1.927927927927928,True,0.13559450209140778,0.06801187992095947,0.43359375,0.736328125
+post_attention_layernorm,6,16,0.9859913793103449,0.4964731416169289,0.9859913793103449,True,0.13559450209140778,0.08680811524391174,0.43359375,0.72265625
+post_attention_layernorm,6,17,1.2088963963963963,0.5472852408870762,1.2088963963963963,True,0.13559450209140778,0.09242258220911026,0.43359375,0.712890625
+post_attention_layernorm,6,18,1.2401463963963963,0.5536006032424281,1.2401463963963963,True,0.13559450209140778,0.09676424413919449,0.43359375,0.8095703125
+post_attention_layernorm,6,19,1.6644144144144144,0.6246830092983939,1.6644144144144144,True,0.13559450209140778,0.07613449543714523,0.43359375,0.81640625
+post_attention_layernorm,6,20,1.5202702702702702,0.6032171581769437,1.5202702702702702,True,0.13559450209140778,0.07808392494916916,0.43359375,0.822265625
+post_attention_layernorm,6,21,1.8782608695652174,0.6525679758308157,1.8782608695652174,True,0.13559450209140778,0.06570179015398026,0.43359375,0.86328125
+post_attention_layernorm,6,22,2.6140350877192984,0.7233009708737864,2.6140350877192984,True,0.13559450209140778,0.05012959614396095,0.43359375,0.9713134765625
+post_attention_layernorm,6,23,2.828828828828829,0.7388235294117647,2.828828828828829,True,0.13559450209140778,0.06521295011043549,0.43359375,1.01361083984375
+post_attention_layernorm,7,8,0.19383825417201542,0.16236559139784945,0.19383825417201542,True,0.17147497832775116,0.18255221843719482,0.375,0.33984375
+post_attention_layernorm,7,9,0.12010443864229765,0.13649851632047477,0.13649851632047477,True,0.17147497832775116,0.17869991064071655,0.375,0.3515625
+post_attention_layernorm,7,10,0.15468549422336328,0.1339633129516398,0.15468549422336328,True,0.17147497832775116,0.16611140966415405,0.375,0.36328125
+post_attention_layernorm,7,11,0.23232323232323232,0.1885245901639344,0.23232323232323232,True,0.17147497832775116,0.1528426557779312,0.375,0.41796875
+post_attention_layernorm,7,12,0.5047169811320755,0.335423197492163,0.5047169811320755,True,0.17147497832775116,0.12497646361589432,0.375,0.5234375
+post_attention_layernorm,7,13,0.7424242424242424,0.4260869565217391,0.7424242424242424,True,0.17147497832775116,0.09632012248039246,0.375,0.619140625
+post_attention_layernorm,7,14,1.5932203389830508,0.6143790849673203,1.5932203389830508,True,0.17147497832775116,0.053455036133527756,0.375,0.87646484375
+post_attention_layernorm,7,15,2.3645833333333335,0.7027863777089783,2.3645833333333335,True,0.17147497832775116,0.06801187992095947,0.375,0.736328125
+post_attention_layernorm,7,16,1.3799283154121864,0.5798192771084337,1.3799283154121864,True,0.17147497832775116,0.08680811524391174,0.375,0.72265625
+post_attention_layernorm,7,17,1.5019132653061225,0.6003058883507519,1.5019132653061225,True,0.17147497832775116,0.09242258220911026,0.375,0.712890625
+post_attention_layernorm,7,18,1.579954954954955,0.6123963334788302,1.579954954954955,True,0.17147497832775116,0.09676424413919449,0.375,0.8095703125
+post_attention_layernorm,7,19,2.085858585858586,0.6759410801963993,2.085858585858586,True,0.17147497832775116,0.07613449543714523,0.375,0.81640625
+post_attention_layernorm,7,20,1.884020618556701,0.6532618409294012,1.884020618556701,True,0.17147497832775116,0.07808392494916916,0.375,0.822265625
+post_attention_layernorm,7,21,2.377551020408163,0.7039274924471299,2.377551020408163,True,0.17147497832775116,0.06570179015398026,0.375,0.86328125
+post_attention_layernorm,7,22,3.2916666666666665,0.7669902912621359,3.2916666666666665,True,0.17147497832775116,0.05012959614396095,0.375,0.9713134765625
+post_attention_layernorm,7,23,3.381443298969072,0.7717647058823529,3.381443298969072,True,0.17147497832775116,0.06521295011043549,0.375,1.01361083984375
+post_attention_layernorm,8,9,0.1510791366906475,0.13125,0.1510791366906475,True,0.18255221843719482,0.17869991064071655,0.33984375,0.3515625
+post_attention_layernorm,8,10,0.17464424320827943,0.1486784140969163,0.17464424320827943,True,0.18255221843719482,0.16611140966415405,0.33984375,0.36328125
+post_attention_layernorm,8,11,0.28421052631578947,0.22131147540983606,0.28421052631578947,True,0.18255221843719482,0.1528426557779312,0.33984375,0.41796875
+post_attention_layernorm,8,12,0.6377551020408163,0.3894080996884735,0.6377551020408163,True,0.18255221843719482,0.12497646361589432,0.33984375,0.5234375
+post_attention_layernorm,8,13,0.8956043956043956,0.47246376811594204,0.8956043956043956,True,0.18255221843719482,0.09632012248039246,0.33984375,0.619140625
+post_attention_layernorm,8,14,1.7567567567567568,0.6372549019607843,1.7567567567567568,True,0.18255221843719482,0.053455036133527756,0.33984375,0.87646484375
+post_attention_layernorm,8,15,2.7011494252873565,0.7298136645962733,2.7011494252873565,True,0.18255221843719482,0.06801187992095947,0.33984375,0.736328125
+post_attention_layernorm,8,16,1.6141732283464567,0.6174698795180723,1.6141732283464567,True,0.18255221843719482,0.08680811524391174,0.33984375,0.72265625
+post_attention_layernorm,8,17,1.7862215909090908,0.6410910017843487,1.7862215909090908,True,0.18255221843719482,0.09242258220911026,0.33984375,0.712890625
+post_attention_layernorm,8,18,1.8256392045454546,0.6460977755435465,1.8256392045454546,True,0.18255221843719482,0.09676424413919449,0.33984375,0.8095703125
+post_attention_layernorm,8,19,2.357142857142857,0.7021276595744681,2.357142857142857,True,0.18255221843719482,0.07613449543714523,0.33984375,0.81640625
+post_attention_layernorm,8,20,2.143258426966292,0.6818588025022342,2.143258426966292,True,0.18255221843719482,0.07808392494916916,0.33984375,0.822265625
+post_attention_layernorm,8,21,2.67816091954023,0.728125,2.67816091954023,True,0.18255221843719482,0.06570179015398026,0.33984375,0.86328125
+post_attention_layernorm,8,22,3.735632183908046,0.7888349514563107,3.735632183908046,True,0.18255221843719482,0.05012959614396095,0.33984375,0.9713134765625
+post_attention_layernorm,8,23,3.7222222222222223,0.788235294117647,3.7222222222222223,True,0.18255221843719482,0.06521295011043549,0.33984375,1.01361083984375
+post_attention_layernorm,9,10,0.16532258064516128,0.14186851211072665,0.16532258064516128,True,0.17869991064071655,0.16611140966415405,0.3515625,0.36328125
+post_attention_layernorm,9,11,0.2708333333333333,0.21311475409836064,0.2708333333333333,True,0.17869991064071655,0.1528426557779312,0.3515625,0.41796875
+post_attention_layernorm,9,12,0.5735294117647058,0.3644859813084112,0.5735294117647058,True,0.17869991064071655,0.12497646361589432,0.3515625,0.5234375
+post_attention_layernorm,9,13,0.8166666666666667,0.44954128440366975,0.8166666666666667,True,0.17869991064071655,0.09632012248039246,0.3515625,0.619140625
+post_attention_layernorm,9,14,1.6645299145299146,0.6246992782678428,1.6645299145299146,True,0.17869991064071655,0.053455036133527756,0.3515625,0.87646484375
+post_attention_layernorm,9,15,2.6055555555555556,0.7226502311248074,2.6055555555555556,True,0.17869991064071655,0.06801187992095947,0.3515625,0.736328125
+post_attention_layernorm,9,16,1.4316239316239316,0.5887521968365553,1.4316239316239316,True,0.17869991064071655,0.08680811524391174,0.3515625,0.72265625
+post_attention_layernorm,9,17,1.6527777777777777,0.6230366492146597,1.6527777777777777,True,0.17869991064071655,0.09242258220911026,0.3515625,0.712890625
+post_attention_layernorm,9,18,1.7604166666666667,0.6377358490566037,1.7604166666666667,True,0.17869991064071655,0.09676424413919449,0.3515625,0.8095703125
+post_attention_layernorm,9,19,2.1822916666666665,0.6857610474631751,2.1822916666666665,True,0.17869991064071655,0.07613449543714523,0.3515625,0.81640625
+post_attention_layernorm,9,20,2.1083333333333334,0.67828418230563,2.1083333333333334,True,0.17869991064071655,0.07808392494916916,0.3515625,0.822265625
+post_attention_layernorm,9,21,2.4916666666666667,0.7136038186157518,2.4916666666666667,True,0.17869991064071655,0.06570179015398026,0.3515625,0.86328125
+post_attention_layernorm,9,22,3.382978723404255,0.7718446601941747,3.382978723404255,True,0.17869991064071655,0.05012959614396095,0.3515625,0.9713134765625
+post_attention_layernorm,9,23,3.473684210526316,0.7764705882352941,3.473684210526316,True,0.17869991064071655,0.06521295011043549,0.3515625,1.01361083984375
+post_attention_layernorm,10,11,0.23232323232323232,0.1885245901639344,0.23232323232323232,True,0.16611140966415405,0.1528426557779312,0.36328125,0.41796875
+post_attention_layernorm,10,12,0.510752688172043,0.33807829181494664,0.510752688172043,True,0.16611140966415405,0.12497646361589432,0.36328125,0.5234375
+post_attention_layernorm,10,13,0.796875,0.4434782608695652,0.796875,True,0.16611140966415405,0.09632012248039246,0.36328125,0.619140625
+post_attention_layernorm,10,14,1.7079646017699115,0.630718954248366,1.7079646017699115,True,0.16611140966415405,0.053455036133527756,0.36328125,0.87646484375
+post_attention_layernorm,10,15,2.4946236559139785,0.7138461538461538,2.4946236559139785,True,0.16611140966415405,0.06801187992095947,0.36328125,0.736328125
+post_attention_layernorm,10,16,1.4557522123893805,0.5927927927927928,1.4557522123893805,True,0.16611140966415405,0.08680811524391174,0.36328125,0.72265625
+post_attention_layernorm,10,17,1.6364247311827957,0.6206984450675503,1.6364247311827957,True,0.16611140966415405,0.09242258220911026,0.36328125,0.712890625
+post_attention_layernorm,10,18,1.67372311827957,0.6259896946085208,1.67372311827957,True,0.16611140966415405,0.09676424413919449,0.36328125,0.8095703125
+post_attention_layernorm,10,19,2.146276595744681,0.6821639898562976,2.146276595744681,True,0.16611140966415405,0.07613449543714523,0.36328125,0.81640625
+post_attention_layernorm,10,20,1.976063829787234,0.6639857015192135,1.976063829787234,True,0.16611140966415405,0.07808392494916916,0.36328125,0.822265625
+post_attention_layernorm,10,21,2.4123711340206184,0.7069486404833837,2.4123711340206184,True,0.16611140966415405,0.06570179015398026,0.36328125,0.86328125
+post_attention_layernorm,10,22,3.247422680412371,0.7645631067961165,3.247422680412371,True,0.16611140966415405,0.05012959614396095,0.36328125,0.9713134765625
+post_attention_layernorm,10,23,3.521276595744681,0.7788235294117647,3.521276595744681,True,0.16611140966415405,0.06521295011043549,0.36328125,1.01361083984375
+post_attention_layernorm,11,12,0.3375,0.2523364485981308,0.3375,True,0.1528426557779312,0.12497646361589432,0.41796875,0.5234375
+post_attention_layernorm,11,13,0.5131578947368421,0.3391304347826087,0.5131578947368421,True,0.1528426557779312,0.09632012248039246,0.41796875,0.619140625
+post_attention_layernorm,11,14,1.2612781954887218,0.5577722360764755,1.2612781954887218,True,0.1528426557779312,0.053455036133527756,0.41796875,0.87646484375
+post_attention_layernorm,11,15,1.9906542056074767,0.665625,1.9906542056074767,True,0.1528426557779312,0.06801187992095947,0.41796875,0.736328125
+post_attention_layernorm,11,16,1.1914191419141915,0.5436746987951807,1.1914191419141915,True,0.1528426557779312,0.08680811524391174,0.41796875,0.72265625
+post_attention_layernorm,11,17,1.249426605504587,0.5554422635737956,1.249426605504587,True,0.1528426557779312,0.09242258220911026,0.41796875,0.712890625
+post_attention_layernorm,11,18,1.28125,0.5616438356164384,1.28125,True,0.1528426557779312,0.09676424413919449,0.41796875,0.8095703125
+post_attention_layernorm,11,19,1.764018691588785,0.6382079459002535,1.764018691588785,True,0.1528426557779312,0.07613449543714523,0.41796875,0.81640625
+post_attention_layernorm,11,20,1.5344036697247707,0.6054298642533936,1.5344036697247707,True,0.1528426557779312,0.07808392494916916,0.41796875,0.822265625
+post_attention_layernorm,11,21,1.9369158878504673,0.6595067621320605,1.9369158878504673,True,0.1528426557779312,0.06570179015398026,0.41796875,0.86328125
+post_attention_layernorm,11,22,2.7454545454545456,0.7330097087378641,2.7454545454545456,True,0.1528426557779312,0.05012959614396095,0.41796875,0.9713134765625
+post_attention_layernorm,11,23,2.97196261682243,0.7482352941176471,2.97196261682243,True,0.1528426557779312,0.06521295011043549,0.41796875,1.01361083984375
+post_attention_layernorm,12,13,0.26492537313432835,0.20943952802359883,0.26492537313432835,True,0.12497646361589432,0.09632012248039246,0.5234375,0.619140625
+post_attention_layernorm,12,14,0.9213836477987422,0.4795417348608838,0.9213836477987422,True,0.12497646361589432,0.053455036133527756,0.5234375,0.87646484375
+post_attention_layernorm,12,15,1.3861940298507462,0.5809225957779516,1.3861940298507462,True,0.12497646361589432,0.06801187992095947,0.5234375,0.736328125
+post_attention_layernorm,12,16,0.9358600583090378,0.48343373493975905,0.9358600583090378,True,0.12497646361589432,0.08680811524391174,0.5234375,0.72265625
+post_attention_layernorm,12,17,0.9096209912536443,0.4763358778625954,0.9096209912536443,True,0.12497646361589432,0.09242258220911026,0.5234375,0.712890625
+post_attention_layernorm,12,18,0.9358600583090378,0.48343373493975905,0.9358600583090378,True,0.12497646361589432,0.09676424413919449,0.5234375,0.8095703125
+post_attention_layernorm,12,19,1.2276119402985075,0.5510887772194305,1.2276119402985075,True,0.12497646361589432,0.07613449543714523,0.5234375,0.81640625
+post_attention_layernorm,12,20,1.056985294117647,0.5138516532618409,1.056985294117647,True,0.12497646361589432,0.07808392494916916,0.5234375,0.822265625
+post_attention_layernorm,12,21,1.4701492537313432,0.595166163141994,1.4701492537313432,True,0.12497646361589432,0.06570179015398026,0.5234375,0.86328125
+post_attention_layernorm,12,22,2.0294117647058822,0.6699029126213593,2.0294117647058822,True,0.12497646361589432,0.05012959614396095,0.5234375,0.9713134765625
+post_attention_layernorm,12,23,2.1481481481481484,0.6823529411764706,2.1481481481481484,True,0.12497646361589432,0.06521295011043549,0.5234375,1.01361083984375
+post_attention_layernorm,13,14,0.6276595744680851,0.38562091503267976,0.6276595744680851,True,0.09632012248039246,0.053455036133527756,0.619140625,0.87646484375
+post_attention_layernorm,13,15,1.0504731861198737,0.5123076923076924,1.0504731861198737,True,0.09632012248039246,0.06801187992095947,0.619140625,0.736328125
+post_attention_layernorm,13,16,0.5697399527186762,0.3629518072289157,0.5697399527186762,True,0.09632012248039246,0.08680811524391174,0.619140625,0.72265625
+post_attention_layernorm,13,17,0.5619930475086906,0.35979228486646886,0.5619930475086906,True,0.09632012248039246,0.09242258220911026,0.619140625,0.712890625
+post_attention_layernorm,13,18,0.657010428736964,0.3965034965034965,0.657010428736964,True,0.09632012248039246,0.09676424413919449,0.619140625,0.8095703125
+post_attention_layernorm,13,19,0.8659305993690851,0.4640743871513102,0.8659305993690851,True,0.09632012248039246,0.07613449543714523,0.619140625,0.81640625
+post_attention_layernorm,13,20,0.7811594202898551,0.4385679414157852,0.7811594202898551,True,0.09632012248039246,0.07808392494916916,0.619140625,0.822265625
+post_attention_layernorm,13,21,0.9826498422712934,0.49562450278440734,0.9826498422712934,True,0.09632012248039246,0.06570179015398026,0.619140625,0.86328125
+post_attention_layernorm,13,22,1.5432098765432098,0.6067961165048543,1.5432098765432098,True,0.09632012248039246,0.05012959614396095,0.619140625,0.9713134765625
+post_attention_layernorm,13,23,1.6813880126182965,0.6270588235294118,1.6813880126182965,True,0.09632012248039246,0.06521295011043549,0.619140625,1.01361083984375
+post_attention_layernorm,14,15,0.4135667396061269,0.29256965944272445,0.4135667396061269,True,0.053455036133527756,0.06801187992095947,0.87646484375,0.736328125
+post_attention_layernorm,14,16,0.2551505546751189,0.2418918918918919,0.2551505546751189,True,0.053455036133527756,0.08680811524391174,0.87646484375,0.72265625
+post_attention_layernorm,14,17,0.28498372849837283,0.2595890410958904,0.28498372849837283,True,0.053455036133527756,0.09242258220911026,0.87646484375,0.712890625
+post_attention_layernorm,14,18,0.39717282261741904,0.28426892950391647,0.39717282261741904,True,0.053455036133527756,0.09676424413919449,0.87646484375,0.8095703125
+post_attention_layernorm,14,19,0.3989968080255358,0.28520208604954367,0.3989968080255358,True,0.053455036133527756,0.07613449543714523,0.87646484375,0.81640625
+post_attention_layernorm,14,20,0.4737802097583219,0.3214727722772277,0.4737802097583219,True,0.053455036133527756,0.07808392494916916,0.87646484375,0.822265625
+post_attention_layernorm,14,21,0.47515257192676547,0.32210401891252954,0.47515257192676547,True,0.053455036133527756,0.06570179015398026,0.87646484375,0.86328125
+post_attention_layernorm,14,22,0.7402323125659979,0.4253640776699029,0.7402323125659979,True,0.053455036133527756,0.05012959614396095,0.87646484375,0.9713134765625
+post_attention_layernorm,14,23,0.8220793140407289,0.4511764705882353,0.8220793140407289,True,0.053455036133527756,0.06521295011043549,0.87646484375,1.01361083984375
+post_attention_layernorm,15,16,0.3656832298136646,0.576499388004896,0.576499388004896,True,0.06801187992095947,0.08680811524391174,0.736328125,0.72265625
+post_attention_layernorm,15,17,0.29876160990712075,0.4260485651214128,0.4260485651214128,True,0.06801187992095947,0.09242258220911026,0.736328125,0.712890625
+post_attention_layernorm,15,18,0.30124223602484473,0.4311111111111111,0.4311111111111111,True,0.06801187992095947,0.09676424413919449,0.736328125,0.8095703125
+post_attention_layernorm,15,19,0.2840549102428722,0.22121710526315788,0.2840549102428722,True,0.06801187992095947,0.07613449543714523,0.736328125,0.81640625
+post_attention_layernorm,15,20,0.33996683250414594,0.2537128712871287,0.33996683250414594,True,0.06801187992095947,0.07808392494916916,0.736328125,0.822265625
+post_attention_layernorm,15,21,0.4206008583690987,0.29607250755287007,0.4206008583690987,True,0.06801187992095947,0.06570179015398026,0.736328125,0.86328125
+post_attention_layernorm,15,22,0.45278969957081544,0.31166912850812406,0.45278969957081544,True,0.06801187992095947,0.05012959614396095,0.736328125,0.9713134765625
+post_attention_layernorm,15,23,0.815378923352038,0.4491508152173913,0.815378923352038,True,0.06801187992095947,0.06521295011043549,0.736328125,1.01361083984375
+post_attention_layernorm,16,17,0.17602657004830918,0.1496790757381258,0.17602657004830918,True,0.08680811524391174,0.09242258220911026,0.72265625,0.712890625
+post_attention_layernorm,16,18,0.2154215976331361,0.17724022516354784,0.2154215976331361,True,0.08680811524391174,0.09676424413919449,0.72265625,0.8095703125
+post_attention_layernorm,16,19,0.4420289855072464,0.3065326633165829,0.4420289855072464,True,0.08680811524391174,0.07613449543714523,0.72265625,0.81640625
+post_attention_layernorm,16,20,0.33108935128518974,0.24873563218390804,0.33108935128518974,True,0.08680811524391174,0.07808392494916916,0.72265625,0.822265625
+post_attention_layernorm,16,21,0.5990338164251208,0.37462235649546827,0.5990338164251208,True,0.08680811524391174,0.06570179015398026,0.72265625,0.86328125
+post_attention_layernorm,16,22,1.0171358629130967,0.504247572815534,1.0171358629130967,True,0.08680811524391174,0.05012959614396095,0.72265625,0.9713134765625
+post_attention_layernorm,16,23,1.0506634499396863,0.5123529411764706,1.0506634499396863,True,0.08680811524391174,0.06521295011043549,0.72265625,1.01361083984375
+post_attention_layernorm,17,18,0.23767123287671232,0.19203099059214168,0.23767123287671232,True,0.09242258220911026,0.09676424413919449,0.712890625,0.8095703125
+post_attention_layernorm,17,19,0.29360306178239476,0.22696534234995774,0.29360306178239476,True,0.09242258220911026,0.07613449543714523,0.712890625,0.81640625
+post_attention_layernorm,17,20,0.20969441517386722,0.17334494773519163,0.20969441517386722,True,0.09242258220911026,0.07808392494916916,0.712890625,0.822265625
+post_attention_layernorm,17,21,0.40350877192982454,0.2875,0.40350877192982454,True,0.09242258220911026,0.06570179015398026,0.712890625,0.86328125
+post_attention_layernorm,17,22,0.8070175438596491,0.44660194174757284,0.8070175438596491,True,0.09242258220911026,0.05012959614396095,0.712890625,0.9713134765625
+post_attention_layernorm,17,23,0.8589393110989612,0.46205882352941174,0.8589393110989612,True,0.09242258220911026,0.06521295011043549,0.712890625,1.01361083984375
+post_attention_layernorm,18,19,0.27958115183246074,0.21849427168576105,0.27958115183246074,True,0.09676424413919449,0.07613449543714523,0.8095703125,0.81640625
+post_attention_layernorm,18,20,0.20833333333333334,0.1724137931034483,0.20833333333333334,True,0.09676424413919449,0.07808392494916916,0.8095703125,0.822265625
+post_attention_layernorm,18,21,0.4222222222222222,0.296875,0.4222222222222222,True,0.09676424413919449,0.06570179015398026,0.8095703125,0.86328125
+post_attention_layernorm,18,22,0.8311111111111111,0.4538834951456311,0.8311111111111111,True,0.09676424413919449,0.05012959614396095,0.8095703125,0.9713134765625
+post_attention_layernorm,18,23,0.8,0.4444444444444444,0.8,True,0.09676424413919449,0.06521295011043549,0.8095703125,1.01361083984375
+post_attention_layernorm,19,20,0.151033386327504,0.1441947565543071,0.151033386327504,True,0.07613449543714523,0.07808392494916916,0.81640625,0.822265625
+post_attention_layernorm,19,21,0.1678832116788321,0.14375,0.1678832116788321,True,0.07613449543714523,0.06570179015398026,0.81640625,0.86328125
+post_attention_layernorm,19,22,0.5036496350364964,0.33495145631067963,0.5036496350364964,True,0.07613449543714523,0.05012959614396095,0.81640625,0.9713134765625
+post_attention_layernorm,19,23,0.48267526188557613,0.34782608695652173,0.48267526188557613,True,0.07613449543714523,0.06521295011043549,0.81640625,1.01361083984375
+post_attention_layernorm,20,21,0.2091324200913242,0.1729607250755287,0.2091324200913242,True,0.07808392494916916,0.06570179015398026,0.822265625,0.86328125
+post_attention_layernorm,20,22,0.5154022988505748,0.3401092233009709,0.5154022988505748,True,0.07808392494916916,0.05012959614396095,0.822265625,0.9713134765625
+post_attention_layernorm,20,23,0.5675426463808206,0.4219858156028369,0.5675426463808206,True,0.07808392494916916,0.06521295011043549,0.822265625,1.01361083984375
+post_attention_layernorm,21,22,0.2875,0.22330097087378642,0.2875,True,0.06570179015398026,0.05012959614396095,0.86328125,0.9713134765625
+post_attention_layernorm,21,23,0.3960546282245827,0.4148936170212766,0.4148936170212766,True,0.06570179015398026,0.06521295011043549,0.86328125,1.01361083984375
+post_attention_layernorm,22,23,0.3124108416547789,0.34263295553618134,0.34263295553618134,True,0.05012959614396095,0.06521295011043549,0.9713134765625,1.01361083984375