Measure parameter shell geometry in paired Qwen3.5-2B checkpoints

model_shell/PROTOCOL.md

@@ -0,0 +1,83 @@
+# Actual-model parameter clouds: frozen grouping and null protocol
+
+Assets: retained Qwen3.5-2B original15852e8c16360a2fea060d615a32b45270f8a8fc,
+and pure GSM8K GRPO LoRA seed0 at HF038d8be9bd0edd3883fa019651b02bdf65609468.
+Use the established read-only FP32 W+2BA wrapper. No training, GPU, new models,
+weight mutation, inference requirement or deletion. Every retained asset stays.
+
+## What constitutes a distribution
+
+A model has no unique canonical number of parameter distributions. Enumerate a
+fixed family, not "all possible distributions":
+1. Each ndim>=2 tensor flattened as [axis0, product(other axes)]: row vectors
+   and column vectors, when point count>=16 and feature dimension>=32.
+2. Decoder language/MTP full-attention Q and interleaved gate rows per query
+   head; K/V rows per KV head. GDN concatenated Q/K/V rows per corresponding head.
+   No claim that these groups are disjoint or iid; whole-matrix views overlap.
+3. Primary sample-size stratum has >=64 available points;16..63 is small-sample
+   descriptive only. One-dimensional norm/bias vectors have no cloud of vectors;
+   GDN kernel row dimension4 is low dimensional and excluded with explicit reason.
+   Vision parameters are covered by matrix/flattened views, not vision-head splits.
+
+Include all checkpoint components (language, vision and MTP). Per view choose
+up to512 point indices without replacement, split deterministically into calibration
+and held-out halves. Use full vector coordinates up to dimension8192; only larger
+vectors use a fixed random coordinate subset, labelled projected rather than full.
+Use identical indices, coordinates and null random draws for original/FT.
+No overlap-group parameter savings or total distribution counts are additive.
+
+## Centers and radius/scaling
+
+Analyze origin and calibration-mean centers separately. Fit R_max using calibration
+norms only; do not silently clip held-out points outside R_max. Also fit calibration
+RMS radius. Store held-out maximum, outside-radius mass and radial spread.
+
+Null A: uniform d-dimensional ball. Simulate the SAME center-fitting and calibration
+maximum/RMS fitting before held-out evaluation. For origin, also retain exact
+expected max-fitted null CDF a^d*n_cal/(n_cal+1), a<=1; log probabilities remain
+available and zero Monte Carlo hits are not exact zeros.
+Null B: independent coordinate-uniform box bounded by calibration min/max,
+including asymmetric coordinates. Fit its center/radii using its own calibration
+sample by exactly the same procedure. Record energy/range mismatch and held-out
+coordinate violations; this null is conditional on fitted bounds, not a statement
+that all real vectors lie inside that box.
+Null C: independent coordinate-uniform box with calibration mean mu and standard
+deviation sigma, bounds mu +/- sqrt(3)*sigma (sample variance ddof1). This matches
+first and diagonal second moments in its generating law. Use its own calibration
+and held-out center/max/RMS fits too; RMS-normalized comparisons target shape
+rather than a change of global scale. Matching diagonal moments is not covariance
+matching and does not preserve correlations.
+
+Use512 validation null draws (not iid model observations), same calibration count
+as each data cloud. Reuse isotropic-ball nulls only for identical dimension/count
+and center-fitting rules. Monte Carlo resolution/error must be stated. Fit both
+centers from the same raw draws where applicable; paired FT uses common randomness.
+
+## Metrics and interpretation
+
+Retain measured norm arrays, empirical CDF grids, max-normalized interior fractions
+at0.5/0.75/0.9/0.95/0.99 and outside1, RMS-normalized CDF from0.5 to1.5,
+median radial ratios and radial spread. Prefer probability-point differences over
+astronomical ratios against tiny a^d. Inspect positive AND negative CDF gaps:
+more inner tail plus more outer tail is not global inward stochastic dominance.
+
+Record exact zeros, coordinate energy participation (sum squares)^2/sum fourths,
+coordinate-variance participation, empirical covariance/second-moment participation
+via calibration Gram matrices. Label sample rank ceiling n_cal-1 (mean centered)
+or n_cal (origin); compare to equally sampled nulls, do not interpret the raw
+finite-sample participation as true intrinsic dimension.
+
+Report both full-radius and moment-controlled results for each center, whole-view
+and head subgroups, primary versus small sample/projection strata, original and
+changed72-LoRA-target groups. Unchanged parameter views can reuse identical results
+with explicit references rather than re-read/recompute. Results describe parameters,
+not activations, model quality, functional duplicates, or pruning gains.
+
+Execution budget: one CPU, 512 MiB RSS ceiling, 7200 seconds per invocation;
+checkpoint every group, resume without replacing completed groups. Any incomplete
+coverage is reported before interpretation. No numerical analysis runs locally.
+
+Illustrative radial CDFs use these fixed original/origin views, selected by module
+role before the census completes, not by the resulting similarity/interior score:
+language layer0 linear-attention Q head0, layer3 full-attention Q head0, and layer0
+MLP up-projection rows. Their plots are illustrations, not independent replications.

model_shell/README.md

@@ -0,0 +1,42 @@
+# Actual-model shell census
+
+Read [PROTOCOL.md](PROTOCOL.md) before interpreting the results. This analyzes
+retained parameter arrays, not model activations or task accuracy.
+
+Run only on the authorized CPU server from the repository root:
+
+```sh
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m unittest model_shell.test_analysis -v
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.report
+```
+
+Inputs are the already retained `runs/cpu-qwen35-2b/weights` and
+`runs/finetune-head-comparison/adapter.safetensors` with their existing JSON
+configuration and metadata. The adapter is evaluated as FP32 W+2BA. No weights
+are modified, no training or inference is run, and no files are deleted.
+
+`runs/model-shell/checkpoints` is a resumable group ledger. Re-running skips
+completed groups; do not reuse a ledger after changing its protocol/code/inputs.
+One invocation stops at 7,200 seconds, RSS above 512 MiB or available memory below
+128 MiB. Resume after addressing a resource stop; do not relax guards silently.
+
+`results/model-shell` contains the report, group inventory, per-profile metric
+CSV, compressed complete CDF/control summaries, and measured calibration/held-out
+radii. The NPZ keys are `GROUPINDEX_MODEL_CENTER_cal` / `_eval`, with zero-based
+group indices into `inventory.json`. Unchanged fine-tune profiles explicitly
+reference the original profiles in `profiles.json.gz`; the CSV expands both.
+Raw null draws are reproducible from the recorded deterministic seeds; null CDFs,
+scales and participation summaries are stored. `audit.json` verifies coverage,
+recomputes empirical CDFs from the saved norms, checks finite-sample rank limits,
+verifies the original asset SHA256 values, and hashes the output artifacts.
+`example_radii.npz` also saves calibration/held-out radii for the three fixed role
+examples and their regenerated nulls; their grid CDFs are checked against the
+original ledger before plotting full empirical step curves from radius zero.
+
+The count of views is defined by the protocol and is not a unique number of
+distributions inherent in a neural network. These overlapping views are equally
+weighted for description; they are not independent replicates. The box nulls
+match fitted marginal ranges or moments, not correlations. The maximum fitted
+radius is not the unknown support boundary. Quantities near numerical or Monte
+Carlo resolution cannot support a definitive population statement.

model_shell/analyze.py

@@ -0,0 +1,124 @@
+"""CPU-only, read-only parameter cloud census with paired uniform references."""
+import argparse, hashlib, json, math, os, time
+from pathlib import Path
+from types import SimpleNamespace
+import numpy as np
+from pilot.tensors import load_tensors
+from finetune_compare.lora import install
+from model_shell.groups import enumerate_groups
+
+MAX_GRID=np.array([.5,.75,.9,.95,.99,1.,1.05,1.1,1.25])
+RMS_GRID=np.linspace(.5,1.5,41)
+
+def seed(key):return int.from_bytes(hashlib.sha256(key.encode()).digest()[:8],'little')
+
+def guard(start):
+    mem=dict((s.split(':')[0],int(s.split()[1])) for s in Path('/proc/meminfo').read_text().splitlines() if s.startswith('MemAvailable:'))
+    rss=int(Path('/proc/self/statm').read_text().split()[1])*os.sysconf('SC_PAGE_SIZE')
+    if rss>512*1024**2 or mem['MemAvailable']<128*1024 or time.monotonic()-start>7200:
+        raise RuntimeError(f'Resource guard: RSS={rss}, available_kB={mem}, elapsed={time.monotonic()-start:.0f}')
+
+def sample(t,g):
+    rng=np.random.default_rng(seed(g['id']+'/indices'))
+    n=min(512,g['population']);d=min(8192,g['dimension'])
+    points=np.sort(rng.choice(g['population'],n,replace=False))
+    features=np.sort(rng.choice(g['dimension'],d,replace=False)) if d<g['dimension'] else np.arange(d)
+    width=math.prod(t.shape[1:]);out=np.empty((n,d),np.float32)
+    if g['orientation']=='rows':
+        for j,i in enumerate(points):
+            row=i+g['start'];out[j]=t.read(slice(row*width,(row+1)*width))[features]
+            if j%32==0:t.evict()
+    else:
+        for j,i in enumerate(features):
+            out[:,j]=t.read(slice(i*width,(i+1)*width))[points]
+            if j%32==0:t.evict()
+    t.evict();out=out[rng.permutation(n)]
+    digest=hashlib.sha256(points.tobytes()+features.tobytes()).hexdigest()
+    return out,digest
+
+def participation(y):
+    gram=y@y.T;energy=float(np.trace(gram));den=float(np.square(gram).sum())
+    return energy**2/den if den else 0.
+
+def geometry(x,ncal,center,raw=False):
+    # Float64 arithmetic; bounded sample matrix, no model-wide conversion.
+    z=x.astype(np.float64);mu=z[:ncal].mean(0) if center=='mean' else np.zeros(z.shape[1])
+    y=z-mu;norm=np.linalg.norm(y,axis=1);cal=norm[:ncal];r=norm[ncal:]
+    maximum=float(cal.max());rms=float(np.sqrt(np.mean(cal**2)))
+    valid=maximum>0 and rms>0
+    a=r/maximum if maximum else np.zeros_like(r);b=r/rms if rms else np.zeros_like(r)
+    coordvar=z[:ncal].var(0,ddof=1);cv=float(coordvar.sum()**2/np.square(coordvar).sum()) if coordvar.any() else 0.
+    e=np.square(y[ncal:]);den=np.square(e).sum(1)
+    pe=np.divide(e.sum(1)**2,den,out=np.zeros(len(e)),where=den>0)
+    out=dict(valid=bool(valid),ncal=ncal,neval=len(r),d=z.shape[1],rmax=maximum,rms=rms,
+        median_max=float(np.median(a)),median_rms=float(np.median(b)),
+        p10_rms=float(np.quantile(b,.1)),p90_rms=float(np.quantile(b,.9)),
+        outside=float(np.mean(a>1)),cdf_max=[float(np.mean(a<=v)) for v in MAX_GRID],
+        cdf_rms=[float(np.mean(b<=v)) for v in RMS_GRID],
+        covariance_participation=participation(y[:ncal]),rank_ceiling=ncal-(center=='mean'),
+        coordinate_variance_participation=cv,point_energy_participation_median=float(np.median(pe)),
+        zero_fraction=float(np.mean(z==0)))
+    if raw:out.update(cal_norm=cal.tolist(),eval_norm=r.tolist())
+    return out
+
+BALL_CACHE={}
+def references(x,ncal,key):
+    d=x.shape[1];cachekey=(d,ncal)
+    if cachekey not in BALL_CACHE:
+        rng=np.random.default_rng(seed(f'ball/{d}/{ncal}'))
+        y=rng.standard_normal((ncal+512,d),dtype=np.float32)
+        y*=((rng.random(len(y))**(1/d))/np.linalg.norm(y,axis=1))[:,None]
+        BALL_CACHE[cachekey]={c:geometry(y,ncal,c) for c in ('origin','mean')}
+    cal=x[:ncal].astype(np.float64);lo=cal.min(0);hi=cal.max(0);mu=cal.mean(0);sd=cal.std(0,ddof=1)
+    rng=np.random.default_rng(seed(key+'/uniform'))
+    u=rng.random((ncal+512,d),dtype=np.float32)
+    refs={'ball':BALL_CACHE[cachekey]}
+    for name,left,right in [('range_box',lo,hi),('moment_box',mu-np.sqrt(3)*sd,mu+np.sqrt(3)*sd)]:
+        y=(left+(right-left)*u).astype(np.float32)
+        refs[name]={c:geometry(y,ncal,c) for c in ('origin','mean')}
+    return refs,dict(heldout_outside_coordinate_range=float(np.mean((x[ncal:]<lo)|(x[ncal:]>hi))),
+                     range_box_expected_energy=float(np.sum((lo**2+lo*hi+hi**2)/3)),
+                     observed_cal_energy=float(np.mean(np.sum(cal**2,axis=1))))
+
+def analyze(x,key):
+    ncal=len(x)//2;refs,extra=references(x,ncal,key);out={}
+    for center in ('origin','mean'):
+        p=geometry(x,ncal,center,True);p['references']=refs; p.update(extra)
+        p['comparisons']={}
+        for name,bycenter in refs.items():
+            ref=bycenter[center];diff=np.array(p['cdf_rms'])-ref['cdf_rms']
+            p['comparisons'][name]=dict(interior90_gap=p['cdf_max'][2]-ref['cdf_max'][2],
+                interior95_gap=p['cdf_max'][3]-ref['cdf_max'][3],
+                rms_cdf_positive_gap=float(max(0,diff.max())),rms_cdf_negative_gap=float(max(0,-diff.min())),
+                median_rms_delta=p['median_rms']-ref['median_rms'],
+                participation_ratio=p['covariance_participation']/ref['covariance_participation'] if ref['covariance_participation'] else None)
+        p['ball_origin_log_expected_cdf90']=float(x.shape[1]*np.log(.9)+np.log(ncal/(ncal+1)))
+        out[center]=p
+    return out
+
+def main():
+    ap=argparse.ArgumentParser();ap.add_argument('--root',default='runs');ap.add_argument('--out',default='runs/model-shell');args=ap.parse_args()
+    root=Path(args.root);out=Path(args.out);out.mkdir(exist_ok=True,parents=True);(out/'checkpoints').mkdir(exist_ok=True)
+    cfg=json.loads((root/'cpu-qwen35-2b/model_config.json').read_text())['text_config']
+    base=load_tensors(root/'cpu-qwen35-2b/weights');ft=SimpleNamespace(tensors=dict(base),config=cfg)
+    meta=json.loads((root/'finetune-head-comparison/adapter_meta.json').read_text())
+    pairs,scale=install(ft,root/'finetune-head-comparison',meta)
+    groups,skips=enumerate_groups(base,cfg)
+    manifest=dict(groups=groups,skips=skips,total_tensors=len(base),parameters=sum(t.numel for t in base.values()),changed_tensors=sorted(pairs),adapter_meta=meta,
+                  max_grid=MAX_GRID.tolist(),rms_grid=RMS_GRID.tolist(),null_eval=512,point_cap=512,feature_cap=8192)
+    (out/'inventory.json').write_text(json.dumps(manifest,indent=2));start=time.monotonic()
+    for i,g in enumerate(groups):
+        target=out/'checkpoints'/f'{i:04d}.json'
+        if target.exists():continue
+        guard(start);x,digest=sample(base[g['tensor']],g);p=analyze(x,g['id']);del x
+        changed=g['tensor'] in pairs
+        if changed:
+            guard(start);x,digest2=sample(ft.tensors[g['tensor']],g);assert digest==digest2
+            q=analyze(x,g['id']);del x
+        else:q=None
+        record=dict(group=g,sampling_sha256=digest,changed=changed,original=p,finetuned=q,finetuned_reuses_original=not changed)
+        temp=target.with_suffix('.tmp');temp.write_text(json.dumps(record,allow_nan=False));temp.replace(target)
+        if i%10==0:print(f'{i+1}/{len(groups)} elapsed={time.monotonic()-start:.1f}s',flush=True)
+    print('COMPLETE',len(groups),flush=True)
+
+if __name__=='__main__':main()

model_shell/groups.py

@@ -0,0 +1,29 @@
+"""Deterministic finite family of matrix/vector clouds; no data-dependent choice."""
+import math,re
+
+def enumerate_groups(tensors,cfg):
+    groups=[];skips=[]
+    for name,t in sorted(tensors.items()):
+        if len(t.shape)<2:skips.append({'name':name,'reason':'one vector, no vector cloud','shape':t.shape});continue
+        rows=t.shape[0];cols=math.prod(t.shape[1:]);count=0
+        for axis,n,d in [('rows',rows,cols),('columns',cols,rows)]:
+            if n>=16 and d>=32:
+                groups.append({'id':name+'::'+axis,'tensor':name,'kind':'matrix','orientation':axis,'start':0,'stop':rows,'population':n,'dimension':d,'stratum':'primary' if n>=64 else 'small_sample'});count+=1
+        if not count:skips.append({'name':name,'reason':'insufficient points or dimension <32','shape':t.shape})
+        if not (name.startswith('model.language_model.layers.') or name.startswith('mtp.layers.')):continue
+        segments=[]
+        if name.endswith('.linear_attn.in_proj_qkv.weight'):
+            kh,kd,vh,vd=[cfg[k] for k in ('linear_num_key_heads','linear_key_head_dim','linear_num_value_heads','linear_value_head_dim')]
+            for role,num,dim,offset in [('Q',kh,kd,0),('K',kh,kd,kh*kd),('V',vh,vd,2*kh*kd)]:
+                for h in range(num):segments.append((role,h,offset+h*dim,offset+(h+1)*dim))
+        elif name.endswith('.self_attn.q_proj.weight'):
+            dim=cfg['head_dim'];assert rows==cfg['num_attention_heads']*2*dim
+            for h in range(cfg['num_attention_heads']):
+                for role,offset in [('Q',0),('gate',dim)]:segments.append((role,h,h*2*dim+offset,h*2*dim+offset+dim))
+        elif name.endswith(('.self_attn.k_proj.weight','.self_attn.v_proj.weight')):
+            role='K' if '.k_proj.' in name else 'V';dim=cfg['head_dim'];assert rows==cfg['num_key_value_heads']*dim
+            for h in range(cfg['num_key_value_heads']):segments.append((role,h,h*dim,(h+1)*dim))
+        for role,h,start,stop in segments:
+            assert stop<=rows
+            groups.append({'id':name+f'::{role}_head{h}','tensor':name,'kind':'head_rows','orientation':'rows','start':start,'stop':stop,'population':stop-start,'dimension':cols,'stratum':'primary' if stop-start>=64 else 'small_sample'})
+    return groups,skips

model_shell/report.py

@@ -0,0 +1,176 @@
+"""Validate complete checkpoints; write compact scientific data, figures and report."""
+import argparse,csv,gzip,hashlib,json,platform,sys
+from pathlib import Path
+import numpy as np
+import matplotlib
+matplotlib.use('Agg')
+import matplotlib.pyplot as plt
+from model_shell.analyze import MAX_GRID,RMS_GRID,sample,seed
+from pilot.tensors import load_tensors
+
+def sha(p):
+    h=hashlib.sha256()
+    with p.open('rb') as f:
+        for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
+    return h.hexdigest()
+
+def main():
+    ap=argparse.ArgumentParser();ap.add_argument('--input',default='runs/model-shell');ap.add_argument('--out',default='results/model-shell');a=ap.parse_args()
+    source=Path(a.input);out=Path(a.out);out.mkdir(parents=True,exist_ok=True);figdir=out/'figures';figdir.mkdir(exist_ok=True)
+    inventory=json.loads((source/'inventory.json').read_text());groups=inventory['groups'];records=[];rows=[];radii={}
+    for i,g in enumerate(groups):
+        r=json.loads((source/'checkpoints'/f'{i:04d}.json').read_text());assert r['group']==g
+        assert r['changed']==(g['tensor'] in inventory['changed_tensors'])
+        for model in ('original','finetuned'):
+            profiles=r[model] if r[model] is not None else r['original']
+            for center,p in profiles.items():
+                if center=='origin':
+                    p['ball_log_expected_max_cdf']=[float(p['d']*np.log(z)+np.log(p['ncal']/(p['ncal']+1))) for z in MAX_GRID if z<=1]
+                assert p['valid'] and p['d']==min(8192,g['dimension'])
+                assert p['covariance_participation']<=p['rank_ceiling']+1e-6
+                v=np.asarray(p['eval_norm']);cal=np.asarray(p['cal_norm'])
+                assert abs(p['rmax']-cal.max())<1e-12
+                assert abs(p['outside']-np.mean(v>p['rmax']))<1e-12
+                np.testing.assert_allclose(p['cdf_max'],[np.mean(v<=z*p['rmax']) for z in MAX_GRID])
+                np.testing.assert_allclose(p['cdf_rms'],[np.mean(v<=z*p['rms']) for z in RMS_GRID])
+                key=f'{i}_{model}_{center}';radii[key+'_cal']=cal;radii[key+'_eval']=v
+                row=dict(group_index=i,group=g['id'],model=model,center=center,kind=g['kind'],stratum=g['stratum'],changed=r['changed'],projected=g['dimension']>8192,
+                         dimension=g['dimension'],analyzed_dimension=p['d'],population=g['population'],ncal=p['ncal'],neval=p['neval'],sampling_sha256=r['sampling_sha256'])
+                for k in ('median_max','median_rms','p10_rms','p90_rms','outside','rmax','rms','covariance_participation','rank_ceiling','coordinate_variance_participation','point_energy_participation_median','zero_fraction','heldout_outside_coordinate_range'):
+                    row[k]=p[k]
+                for name,c in p['comparisons'].items():
+                    for k,vv in c.items():row[name+'_'+k]=vv
+                rows.append(row)
+        # Compact profiles keep CDF and controls; raw measured radii live in NPZ.
+        for model in ('original','finetuned'):
+            if r[model] is not None:
+                for p in r[model].values():p.pop('cal_norm');p.pop('eval_norm')
+        records.append(r)
+    with gzip.open(out/'profiles.json.gz','wt',encoding='utf-8') as f:json.dump(records,f,allow_nan=False)
+    np.savez_compressed(out/'measured_radii.npz',**radii)
+    with (out/'metrics.csv').open('w',newline='') as f:
+        w=csv.DictWriter(f,fieldnames=list(rows[0]));w.writeheader();w.writerows(rows)
+    (out/'inventory.json').write_text(json.dumps(inventory,indent=2))
+    def subset(model='original',center='origin',kind=None,changed=None):
+        return [r for r in rows if r['model']==model and r['center']==center and r['stratum']=='primary' and not r['projected'] and (kind is None or r['kind']==kind) and (changed is None or r['changed']==changed)]
+    summary={}
+    for center in ('origin','mean'):
+        for kind in ('matrix','head_rows'):
+            rs=subset(center=center,kind=kind);s={'n':len(rs),
+                'median_zero_fraction':float(np.median([r['zero_fraction'] for r in rs])),
+                'median_coordinate_energy_participation_fraction':float(np.median([r['point_energy_participation_median']/r['analyzed_dimension'] for r in rs])),
+                'median_coordinate_variance_participation_fraction':float(np.median([r['coordinate_variance_participation']/r['analyzed_dimension'] for r in rs]))}
+            for ref in ('ball','range_box','moment_box'):
+                gap=np.array([r[ref+'_interior90_gap'] for r in rs]);pos=np.array([r[ref+'_rms_cdf_positive_gap'] for r in rs]);neg=np.array([r[ref+'_rms_cdf_negative_gap'] for r in rs])
+                s[ref]=dict(median_interior90_gap=float(np.median(gap)),fraction_gap_gt10pp=float(np.mean(gap>.1)),median_positive_rms_gap=float(np.median(pos)),median_negative_rms_gap=float(np.median(neg)),
+                            fraction_two_sided_rms_gt10pp=float(np.mean((pos>.1)&(neg>.1))),median_participation_ratio=float(np.median([r[ref+'_participation_ratio'] for r in rs])))
+            summary[center+'/'+kind]=s
+    paired={}
+    for center in ('origin','mean'):
+        b=subset(center=center,changed=True);t=subset(model='finetuned',center=center,changed=True);assert [r['group'] for r in b]==[r['group'] for r in t]
+        paired[center]={'n':len(b)}
+        for field in ('median_max','median_rms','covariance_participation','moment_box_interior90_gap'):
+            delta=np.array([q[field]-p[field] for p,q in zip(b,t)])
+            paired[center][field]=dict(median=float(np.median(delta)),p10=float(np.quantile(delta,.1)),p90=float(np.quantile(delta,.9)),max_abs=float(np.max(abs(delta))))
+    secondary=[]
+    for model in ('original','finetuned'):
+        for center in ('origin','mean'):
+            for label in ('small_sample','projected'):
+                ss=[r for r in rows if r['model']==model and r['center']==center and (r['stratum']=='small_sample' if label=='small_sample' else r['projected'])]
+                secondary.append(dict(model=model,center=center,stratum=label,n=len(ss),median_max=float(np.median([r['median_max'] for r in ss])),moment_interior90_gap=float(np.median([r['moment_box_interior90_gap'] for r in ss]))))
+    (out/'summary.json').write_text(json.dumps(dict(strata=summary,paired_changed=paired,secondary=secondary),indent=2))
+    plt.rcParams.update({'font.size':10,'axes.spines.top':False,'axes.spines.right':False})
+    def save(fig,name):
+        fig.tight_layout()
+        for ext in ('png','pdf','svg'):fig.savefig(figdir/(name+'.'+ext),dpi=180,bbox_inches='tight')
+        plt.close(fig)
+    fig,axs=plt.subplots(2,2,figsize=(11,7),sharex=True,sharey=True)
+    for i,center in enumerate(('origin','mean')):
+        for j,kind in enumerate(('matrix','head_rows')):
+            rs=subset(center=center,kind=kind);ax=axs[i,j]
+            for ref,label in [('ball','Uniform ball'),('range_box','Range-matched box'),('moment_box','Moment-matched box')]:
+                v=np.sort([r[ref+'_interior90_gap'] for r in rs]);ax.plot(v,np.arange(1,len(v)+1)/len(v),label=label)
+            ax.axvline(0,color='gray',lw=.8);ax.set_title(f'{center} center / {kind}, n={len(rs)}');ax.set_xlabel('F(data r/Rcal <= 0.9) - F(null)');ax.set_ylabel('Fraction of sampled groups')
+    axs[0,0].legend(fontsize=8);fig.suptitle('Actual Qwen3.5-2B: interior mass depends on the reference');save(fig,'interior_reference_sensitivity')
+    fig,axs=plt.subplots(1,2,figsize=(11,4.3))
+    for ax,center in zip(axs,('origin','mean')):
+        for kind,marker in [('matrix','o'),('head_rows','.')]:
+            rs=subset(center=center,kind=kind);ax.scatter([r['moment_box_rms_cdf_positive_gap'] for r in rs],[r['moment_box_rms_cdf_negative_gap'] for r in rs],s=10,alpha=.25,label=kind,marker=marker)
+        ax.axvline(.1,color='gray',lw=.7);ax.axhline(.1,color='gray',lw=.7);ax.set(xlabel='Largest positive CDF gap (more inner tail)',ylabel='Largest negative CDF gap (more outer tail)',title=f'{center} / moment-matched box',xlim=(-.02,1),ylim=(-.02,1));ax.legend()
+    fig.suptitle('RMS-normalized radial shape: both tails can grow together');save(fig,'radial_shape_two_tails')
+    fig,axs=plt.subplots(1,2,figsize=(11,4.3))
+    for ax,center in zip(axs,('origin','mean')):
+        b=subset(center=center,changed=True);t=subset(model='finetuned',center=center,changed=True)
+        x=[r['median_max'] for r in b];y=[r['median_max'] for r in t];ax.scatter(x,y,s=10,alpha=.3);low=min(x+y);high=max(x+y);ax.plot([low,high],[low,high],color='gray',lw=1)
+        ax.set(xlabel='Original median r/Rcal',ylabel='GRPO-LoRA median r/Rcal',title=f'{center}, changed primary groups n={len(b)}')
+    fig.suptitle('Paired same-index comparison; unchanged groups excluded');save(fig,'finetune_paired_radii')
+    fig,axs=plt.subplots(1,2,figsize=(11,4.3))
+    for ax,center in zip(axs,('origin','mean')):
+        for kind in ('matrix','head_rows'):
+            rs=subset(center=center,kind=kind);v=np.sort([r['moment_box_participation_ratio'] for r in rs]);ax.plot(v,np.arange(1,len(v)+1)/len(v),label=kind)
+        ax.set(xlabel='Empirical participation / equal-sample moment-box reference',ylabel='Fraction of groups',title=center);ax.axvline(1,color='gray',lw=.8);ax.legend()
+    fig.suptitle('Finite-sample participation is not the true intrinsic dimension');save(fig,'participation_reference')
+    examples=[('model.language_model.layers.0.linear_attn.in_proj_qkv.weight::Q_head0','L0 GDN Q head0'),
+              ('model.language_model.layers.3.self_attn.q_proj.weight::Q_head0','L3 full-attention Q head0'),
+              ('model.language_model.layers.0.mlp.up_proj.weight::rows','L0 MLP up rows')]
+    fig,axs=plt.subplots(2,3,figsize=(13,7),sharey=True)
+    byid={r['group']['id']:r for r in records}
+    indices={g['id']:i for i,g in enumerate(groups)}
+    base=load_tensors(Path('runs/cpu-qwen35-2b/weights'));example_radii={}
+    for j,(gid,label) in enumerate(examples):
+        p=byid[gid]['original']['origin']
+        g=byid[gid]['group'];x,digest=sample(base[g['tensor']],g);assert digest==byid[gid]['sampling_sha256']
+        ncal=len(x)//2;d=x.shape[1];cal=x[:ncal].astype(np.float64)
+        rng=np.random.default_rng(seed(gid+'/uniform'));u=rng.random((ncal+512,d),dtype=np.float32)
+        mu=cal.mean(0);sd=cal.std(0,ddof=1);lo=mu-np.sqrt(3)*sd;hi=mu+np.sqrt(3)*sd
+        moment=(lo+(hi-lo)*u).astype(np.float32)
+        rng=np.random.default_rng(seed(f'ball/{d}/{ncal}'));ball=rng.standard_normal((ncal+512,d),dtype=np.float32)
+        ball*=((rng.random(len(ball))**(1/d))/np.linalg.norm(ball,axis=1))[:,None]
+        arrays={'Actual weights':(radii[f'{indices[gid]}_original_origin_cal'],radii[f'{indices[gid]}_original_origin_eval'])}
+        for name,ref,raw in [('Uniform ball','ball',ball),('Moment box','moment_box',moment)]:
+            nr=np.linalg.norm(raw.astype(np.float64),axis=1);arrays[name]=(nr[:ncal],nr[ncal:])
+            np.testing.assert_allclose([np.mean(nr[ncal:]/nr[:ncal].max()<=z) for z in MAX_GRID],p['references'][ref]['origin']['cdf_max'])
+            np.testing.assert_allclose([np.mean(nr[ncal:]/np.sqrt(np.mean(nr[:ncal]**2))<=z) for z in RMS_GRID],p['references'][ref]['origin']['cdf_rms'])
+        for name,(nc,ne) in arrays.items():example_radii[f'example{j}_{name.replace(" ","_")}_cal']=nc;example_radii[f'example{j}_{name.replace(" ","_")}_eval']=ne
+        for i,(field,grid) in enumerate([('cdf_max',MAX_GRID),('cdf_rms',RMS_GRID)]):
+            ax=axs[i,j]
+            for name,(nc,ne) in arrays.items():
+                scale=nc.max() if i==0 else np.sqrt(np.mean(nc**2));v=np.sort(ne/scale)
+                ax.step(np.r_[0,v, max(1.5,float(v[-1]))],np.r_[0,np.arange(1,len(v)+1)/len(v),1],where='post',label=name,lw=1.8,ls='-' if name=='Actual weights' else '--')
+            ax.set(title=label,xlabel='r / calibration '+('maximum' if i==0 else 'RMS'),ylabel='Held-out CDF',ylim=(-.03,1.03),xlim=(0,1.5));ax.legend(fontsize=8)
+    np.savez_compressed(out/'example_radii.npz',**example_radii)
+    fig.suptitle('Fixed role examples: maximum-radius and RMS normalization');save(fig,'fixed_role_radial_cdfs')
+    lines=['# 两个实际2B检查点：参数云的球壳检验','',f'完整覆盖：{len(groups)}组/模型，{len(rows)}个模型×中心组合。632个参数张量，2,274,069,824个参数（包含语言、视觉和MTP）。',
+           '','## 分布计数与范围','', '固定规则给出554个矩阵行／列视图、1,004个语言/MTP头视图；视图互相重叠，不能相加成参数节省量。1,522组有至少64个可用点，36组只有16个点；1组高维嵌入列视图使用8,192个抽样坐标。主表只汇总样本充足且没有投影的1,521组。319个一维张量和18个低维卷积核不构成此规则下的高维样本云。',
+           '','## 原模型结果（逐组等权，非独立观测）','', '|中心|视图|组数|相对均匀球：90%半径内质量差中位数|相对矩匹配均匀盒：质量差中位数|矩匹配且RMS归一后，内外两尾差均>10个百分点的组占比|','|---|---|---:|---:|---:|---:|']
+    for key,s in summary.items():
+        c,k=key.split('/');lines.append(f'|{c}|{k}|{s["n"]}|{s["ball"]["median_interior90_gap"]:.3f}|{s["moment_box"]["median_interior90_gap"]:.3f}|{s["moment_box"]["fraction_two_sided_rms_gt10pp"]:.1%}|')
+    lines+=['','这些是描述性质量差，不是显著性检验。正差表示所指定半径以内有更多样本；不能将其自动解释为整条径向分布向内平移。若RMS归一后内外尾均变多，更准确的解释是径向分布更宽。',
+            '', '|中心/视图|逐组零值率中位数|点坐标能量参与率÷维度，中位数|坐标方差参与率÷维度，中位数|','|---|---:|---:|---:|']
+    for key,s in summary.items():lines.append(f'|{key}|{s["median_zero_fraction"]:.6g}|{s["median_coordinate_energy_participation_fraction"]:.4f}|{s["median_coordinate_variance_participation_fraction"]:.4f}|')
+    lines+=['','能量参与率衡量坐标能量集中程度，依赖坐标基；有限样本Gram参与率反映样本间几何，另见图。两者均不是精确非零坐标计数，也不是模型功能的有效维度。',
+            '','## 微调前后','', '原模型是官方Qwen3.5-2B；第二个模型是在同一revision上进行GSM8K GRPO的已有纯LoRA检查点，不是另一个完整模型。72个投影矩阵使用只读W+2BA展开，其余参数共享。972个视图受影响；其中936个属于主分析。未改变视图复用完全相同的分析结果。','', '|中心|指标|配对变化中位数|P10|P90|最大绝对变化|','|---|---|---:|---:|---:|---:|']
+    for c,p in paired.items():
+        for k,v in p.items():
+            if k!='n':lines.append(f'|{c}|{k}|{v["median"]:.6g}|{v["p10"]:.6g}|{v["p90"]:.6g}|{v["max_abs"]:.6g}|')
+    lines+=['','## 单独描述的小样本与坐标抽样组','', '|模型|中心|类别|组数|中位r/Rcal的组间中位数|相对矩匹配盒的90%半径质量差中位数|','|---|---|---|---:|---:|---:|']
+    for s in secondary:lines.append(f'|{s["model"]}|{s["center"]}|{s["stratum"]}|{s["n"]}|{s["median_max"]:.4f}|{s["moment_interior90_gap"]:.4f}|')
+    lines+=['','## 如何读图','', '1. `interior_reference_sensitivity`：不同均匀参照下，90%校准半径内质量差的组间CDF。横轴向右表示更多内侧质量。','2. `radial_shape_two_tails`：匹配矩且按各自校准RMS归一后的最大正／负CDF差。右上角同时增加两尾，并非单纯向内。','3. `finetune_paired_radii`：同一视图的微调前后中位半径；对角线表示不变。','4. `participation_reference`：Gram矩阵参与率相对同样本量参照的比值。受样本秩上限影响，不能当作真实内在维度。',
+            '5. `fixed_role_radial_cdfs`：预先按角色固定的GDN Q、全注意力Q和MLP上投影示例，上排按校准最大半径、下排按RMS归一化。不是按结果挑选最明显的组。',
+            '','## 方法与限制','', '每组最多512个不同参数向量，一半校准、一半留出；模型固定，留出指未用于拟合中心/半径的参数向量，不是独立训练模型。对照每组512个验证向量，最坏单项二项标准误约2.21个百分点；模型留出样本的误差还取决于样本数，且参数向量不保证独立。10个百分点仅是描述性筛选尺度，没有多重检验或总体显著性结论。',
+            '','均匀球、坐标范围均匀盒、匹配均值和逐坐标方差的均匀盒都按同一规则拟合自身中心、最大半径及RMS。均匀球原点中心的精确拟合半径CDF为a^d*ncal/(ncal+1)，保留其对数以避免把模拟零命中误报成零概率。盒对照不是协方差匹配。校准最大半径不是已知总体支撑边界；留出越界率已记录。',
+            '','`origin`保留坐标均值贡献；`mean`使用校准向量均值。没有使用所有数据来选边界、训练或推理；没有删除模型。坐标稀疏性、有限样本参与率与径向内侧质量分别测量，不能相互代替。该实验不能证明更好的模型更靠里，也不能证明功能重复、可剪枝或可复用。',
+            '','复现：`OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze`，完成后 `... python -m model_shell.report`。只在已授权CPU服务器运行。分组协议见 `model_shell/PROTOCOL.md`；全量指标在metrics.csv，完整CDF/参照在profiles.json.gz，实测范数在measured_radii.npz。']
+    (out/'REPORT.md').write_text('\n'.join(lines)+'\n')
+    sources={'base':Path('runs/cpu-qwen35-2b/weights/model.safetensors-00001-of-00001.safetensors'),'adapter':Path('runs/finetune-head-comparison/adapter.safetensors'),
+             'config':Path('runs/cpu-qwen35-2b/model_config.json'),'adapter_meta':Path('runs/finetune-head-comparison/adapter_meta.json')}
+    sourcehash={k:dict(path=str(p),bytes=p.stat().st_size,sha256=sha(p)) for k,p in sources.items()}
+    (out/'environment.json').write_text(json.dumps(dict(python=sys.version,platform=platform.platform(),numpy=np.__version__,matplotlib=matplotlib.__version__,
+        source_code_sha256={str(p):sha(p) for p in sorted(Path('model_shell').glob('*.py'))},protocol_sha256=sha(Path('model_shell/PROTOCOL.md')),seed='first 8 SHA256 bytes, little endian; group id plus purpose suffix',
+        thread_limit=1,server='8.211.154.134',training=False,inference=False,gpu=False),indent=2))
+    assert sourcehash['base']['sha256']=='aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1'
+    assert sourcehash['adapter']['sha256']=='93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6'
+    audit=dict(complete_groups=len(groups),profile_rows=len(rows),all_finite=True,cdf_recomputed=True,sample_rank_checked=True,sources=sourcehash,files={str(p.relative_to(out)):sha(p) for p in sorted(out.rglob('*')) if p.is_file() and p.name!='audit.json'})
+    (out/'audit.json').write_text(json.dumps(audit,indent=2));print(json.dumps(dict(summary=summary,paired=paired),indent=2))
+
+if __name__=='__main__':main()

model_shell/test_analysis.py

@@ -0,0 +1,49 @@
+"""Run on the authorized CPU server: python -m unittest model_shell.test_analysis."""
+import unittest
+import numpy as np
+from model_shell.analyze import sample,geometry,analyze,seed
+from finetune_compare.lora import EffectiveWeight
+
+class ArrayTensor:
+    def __init__(self,x):self.x=x;self.shape=x.shape;self.numel=x.size;self.dtype='F32'
+    def read(self,s=slice(None)):return self.x.reshape(-1)[s].copy()
+    def evict(self):pass
+
+class Checks(unittest.TestCase):
+    def test_sampling_and_lora(self):
+        rng=np.random.default_rng(17);w=rng.normal(size=(80,64)).astype('f');a=rng.normal(size=(2,64)).astype('f');b=rng.normal(size=(80,2)).astype('f')
+        effective=EffectiveWeight(ArrayTensor(w),a,b,2);explicit=ArrayTensor(w+2*b@a)
+        for axis,n,d,start,stop in [('rows',80,64,0,80),('columns',64,80,0,80),('rows',32,64,16,48)]:
+            g=dict(id=axis+str(start),orientation=axis,population=n,dimension=d,start=start,stop=stop)
+            x,h=sample(effective,g);y,k=sample(explicit,g);np.testing.assert_allclose(x,y,atol=2e-6,rtol=1e-5);self.assertEqual(h,k)
+            self.assertEqual(x.shape,(n,d))
+            rr=np.random.default_rng(seed(g['id']+'/indices'));indices=np.sort(rr.choice(n,n,replace=False));order=rr.permutation(n)
+            cloud=explicit.x[start:stop] if axis=='rows' else explicit.x.T
+            np.testing.assert_allclose(y,cloud[indices][order],atol=0,rtol=0)
+    def test_center_scale_and_rank(self):
+        rng=np.random.default_rng(5);x=rng.normal(size=(128,96)).astype('f')
+        for center in ('origin','mean'):
+            p=geometry(x,64,center);q=geometry(x*3,64,center)
+            np.testing.assert_allclose(p['cdf_rms'],q['cdf_rms']);self.assertLessEqual(p['covariance_participation'],p['rank_ceiling']+1e-6)
+        p=geometry(x,64,'mean');q=geometry(x+4,64,'mean');np.testing.assert_allclose(p['median_rms'],q['median_rms'],rtol=1e-6)
+        self.assertFalse(geometry(np.zeros((32,64)),16,'mean')['valid'])
+    def test_ball_fit_and_sparse_counterexample(self):
+        rng=np.random.default_rng(33);d=32;ncal=256
+        # Analytic expectation over independent calibration maxima.
+        c=rng.random((20000,ncal))**(1/d);v=rng.random(20000)**(1/d)
+        observed=np.mean(v<=.99*c.max(1));expected=.99**d*ncal/(ncal+1)
+        self.assertLess(abs(observed-expected),.015)
+        radii=rng.random(512)**(1/d);x=np.zeros((512,d),dtype='f');x[:,0]=radii
+        p=geometry(x,256,'origin');self.assertGreater(p['median_max'],.9)
+        # Sparse support alone retains the full-dimensional radial law.
+        self.assertLess(p['cdf_max'][2],.08)
+        x[:,0]=rng.random(512)**.5
+        self.assertGreater(geometry(x,256,'origin')['cdf_max'][2],.65)
+        sphere=rng.normal(size=(512,d));sphere/=np.linalg.norm(sphere,axis=1)[:,None]
+        self.assertEqual(geometry(sphere,256,'origin')['cdf_max'][2],0.)
+    def test_reproducible_controls(self):
+        x=np.random.default_rng(7).normal(size=(32,40)).astype('f');a=analyze(x,'test');b=analyze(x,'test')
+        self.assertEqual(a,b)
+        self.assertTrue(a['mean']['valid'])
+
+if __name__=='__main__':unittest.main()