40 lines
3.3 KiB
Markdown
40 lines
3.3 KiB
Markdown
# Cross-layer QKV CPU study
|
||
|
||
Primary protocol and exploratory addendum are reported separately; no post-hoc success relabeling.
|
||
- Primary state: completed; selected candidate: [].
|
||
- Candidate counts: {'full_attention': 15, 'qkv_head': 960, 'qk_map': 30, 'qk_head': 960, 'v_group': 60}.
|
||
- Passing counts (approximate tier, all gates): {'full_attention': 0, 'qkv_head': 0, 'qk_map': 0, 'qk_head': 0, 'v_group': 0}.
|
||
- Primary computation seconds: 74.78326310915872; peak RSS KiB: 172056.
|
||
|
||
## What was measured
|
||
|
||
Six full-attention layers, indices 3/7/11/15/19/23; all 15 cross-layer pairs. No within-layer Q/K/V interchange.
|
||
Same normalized hidden inputs from both source layers; frozen Q/K norm, partial RoPE, GQA, V, output gate and O.
|
||
QK head probabilities, whole-map injection in original/frozen matched head order, direct V functions, complete head contributions in residual coordinates, and complete branches were separated.
|
||
Primary metrics use queries 7–31. Discrete matching uses calibration only; no continuous alignment, scale fitting or training.
|
||
No primary candidate passed, so primary validation/fresh/network intervention stages were not entered.
|
||
|
||
- full_attention: n=15, best worst output error=1.8534952718777882, minimum worst-text mean JS=None, best worst direct V error=None.
|
||
- qkv_head: n=960, best worst output error=1.3442678606109162, minimum worst-text mean JS=0.08609102699674716, best worst direct V error=None.
|
||
- qk_map: n=30, best worst output error=0.7028184861876081, minimum worst-text mean JS=0.1696159433927246, best worst direct V error=None.
|
||
- qk_head: n=960, best worst output error=None, minimum worst-text mean JS=0.08609102699674716, best worst direct V error=None.
|
||
- v_group: n=60, best worst output error=None, minimum worst-text mean JS=None, best worst direct V error=1.503141145681371.
|
||
|
||
## Exploratory metric audit: frozen one-source candidate
|
||
|
||
After inspecting primary calibration results, one candidate was frozen: layer3 head7 vs layer23 head6 on H23.
|
||
This addendum audited the instability of unweighted top-k tail ranking; original primary results were not changed.
|
||
Four old validation and four previously unused fresh texts were evaluated with frozen extra gates and no candidate fallback.
|
||
- Addendum fresh function pass: False.
|
||
- Overall addendum state: completed; primary unchanged/negative: True.
|
||
Fresh probability-mass and fixed V/gate/O readout measurements show conditional behavioral overlap; strict fresh mean-JS, row-JS-P95 and row-TV-P95 limits still failed.
|
||
No full-network substitution was run after that failure. V similarity, bidirectionality and complete QKV-head equivalence are not established.
|
||
The actual-H3 cache probability is a separate different-input diagnostic, not the same-input function comparison.
|
||
|
||
## Limits and retention
|
||
|
||
This is a short text-only pilot, not proof of absence of repetition elsewhere. Exact arithmetic, matrix coordinates, retrieval behavior and final residual output are distinct questions.
|
||
Centered value Gram CKA can be high even for independent Gaussian projections of common inputs; it is not substitution evidence.
|
||
All weights and large profile caches remain on the authorized server. Git delivery includes all plot tables, probabilities, protocols and cache location/hash metadata.
|
||
No GPU, new checkpoint download, model training, continuous R fitting or automatic weight/cache deletion.
|