Statistical head influence dominance (CPU)
This study answers how concentrated the distribution of measured upstream-head influences is, without assuming that dominant head identities are fixed across texts, positions or recipient heads. Read registered protocol.
core.py: normalized effective Q/K/V extraction for both attention types, source projected head decomposition, exact nonlinear downstream recomputation, coherent masks, continuous concentration and JS.run.py: four fixed prior texts, all23 adjacent connections, single-head and weak/strong/random-half/all-head interventions, per-transition checkpointing, canonical-forward/null controls, preserved baseline caches.report.py: full saved-data distributions, per-transition CSV, continuous effective head plots, influence rank curves, joint tests and Chinese report.audit.py: independently recomputes concentration/masks from raw effects, validates scope/null/head sums, and hashes retained caches, source and results.tests/test_dominance.py: continuous limiting cases, undefined zero-total, token-dependent masks, JS identity, both canonical mixed-layer paths, and joint interventions on a small checkpoint with known arrays.
Influence for Q/K/V is relative L2 change after deleting one whole donor attention head (at all sequence positions), preserving the prior residual and recomputing source MLP + recipient normalization. Source output/value heads are the units: 16 for linear attention,8 for full attention. These are not arbitrary individual Q/K/V weight blocks. For a recipient query-head at one token, normalize donor influences to p. Primary continuous summary is N_eff=1/sum(p²), and N_eff/H. C=(sum(p²)-1/H)/(1-1/H) is a related0..1 concentration score. These are not independent confirmations. No RMS near-zero threshold or R fitting is used.
Q/K/V are separate primary scales; balanced QKV score is RMS of their three relative changes. It is a declared summary, not a norm in a single physical space. Zero total implies undefined concentration, not a uniform distribution. Single-head relative effects before head-share normalization and true joint effects are retained as well as shares. The saved metric named max_absolute is the largest unsigned relative-L2 effect, not an unnormalized hidden-space norm. The total field sums individual relative effects; it is not the measured joint effect of removing the whole attention branch.
Full-attention recipients additionally have real token-key probability JS changes; linear attention does not have an ordinary softmax attention probability matrix. Full-attention K/V repeat their GQA groups to query-head slots for aligned joint experiments, not as extra independent KV observations. Every headline statistic is descriptive: query/token/head observations are dependent, and four short prior texts do not establish universal dominance or independent generalization.
Joint masks vary per source token and recipient head. Recompute a coherent full sequence for each recipient head, scoring that intended head afterward. Rank by single QKV influence, remove weakest half/strongest half/a fixed-seed random half, or all attention heads. This investigates interactions and cross-token effects; it is not a validated free selector, fixed removable head subset or LM pruning.
Only run model work and scientific plotting on the authorized CPU server:
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_dominance -v
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m dominance.run --source runs/cpu-qwen35-2b --out runs/NEW_DOMINANCE_DIR --seconds 7200
venv/bin/python -B -m dominance.report runs/NEW_DOMINANCE_DIR
venv/bin/python -B -m dominance.audit runs/NEW_DOMINANCE_DIR runs/cpu-qwen35-2b
No training, GPU, new model download or automatic deletion. Raw intervention arrays and masks are delivered to Git; large baseline caches stay on the server with path/size/SHA metadata. Existing checkpoints/caches are never removed.
The saved sum of selected individual effects is only a query-wise mask-weighted reference: individual ablations remove a head at all positions, whereas joint masks can vary across positions. Their difference cannot isolate pure nonlinear interaction, since temporal intervention supports also differ. Use the measured weak/strong/random joint effects directly; do not call this reference an additive causal decomposition or a Shapley attribution.
Scale sensitivity: after reviewing the first complete summaries, add sqrt(JS) (Jensen–Shannon distance) alongside primary JS divergence. Divergence is roughly quadratic for small changes, so shares and effective head counts depend on this scale. All original primary statistics and raw interventions remain unchanged; primary-analysis-summary.json preserves the pre-sensitivity aggregate for exact audit comparison. Retrieval-scale plots and source tables show both definitions.