2.8 KiB
Paired task-finetuning comparison (registered before reading finetuned effects)
Primary checkpoint: Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo, HF revision038d8be9bd0edd3883fa019651b02bdf65609468, pure lora/seed0, step150 after shared40-step warmup. GSM8K GRPO, not SFT. Selected by smallest seed index before observing concentration, not by benchmark score or our metric. No gate01/gate02 arm or architecture changes. Exact base revision matches retained Qwen/Qwen3.5-2B15852e8c16360a2fea060d615a32b45270f8a8fc.
Download existing18.9MB F32 LoRA tensors, verify official file SHA. All144 tensors must form72 A/B pairs on the4 declared GDN projections of18 linear-attention layers, rank16 alpha32. Apply FP32 effective W +2BA in streamed row chunks, also when decomposing source output heads. Do not quantize merged weights, mutate or save over base weights, execute foreign .pt files, train or use GPU. Validate against separate base-output +2B(Ax) forward and check head sums/null masks.
Primary comparison exactly matches the reported8.931822/16 statistic: all six ordinary attention receivers3/7/11/15/19/23, previous-layer16 donor output heads, all32 tokens computed, query7..31, original4 plain-text inputs and same token IDs. Reuse corresponding immutable base single-ablation arrays, recompute FT baselines and all16 single donor removals, source MLP/residual/recipient normalization exact. Use the same sqrt(JS) distance influence, normalized shares and N_eff=1/sum(p²). Preserve JS divergence as a sensitivity row; no change of ranking or threshold. All4800 per-token/query-head observations are paired. No new joint-mask search: the requested statistic depends on single removals, not the previous joint tests.
Secondary task-conditioned comparison: first4 original GSM8K test questions with at least32 base-tokenizer tokens (length selection only, before model effects), question-only plain text, first32 tokens. Freeze original author repository commit, JSONL SHA, item indices, question hashes and exact token IDs. No answers, generation, chat-template change or accuracy claim. Compute both base and FT on these same questions; never compare old general-text base to new GSM8K FT as if paired.
Report overall median/P10/P90, paired differences, per receiver layer and per text, strong/weak-quartile shares, and dominant donor identity agreement. No iid-token p-values, no claim over training seeds (one fixed trained seed), no proof of GSM8K accuracy improvement or long-context behavior. Positive, negative and near zero changes all count as results. Source cohort remains4+4 short texts.
All model work/tests/plots on authorized CPU server8.211.154.134, bounded7200s. Preserve all base/FT weights, adapters and caches; no automatic cleanup. Save raw arrays, source provenance, SHA manifests, fixed chart gallery and reviewable PR.