Merge pull request '实测两个Qwen3.5-2B检查点的参数球壳与微调变化' (#9) from codex/model-shell into main
Reviewed-on: #9
This commit was merged in pull request #9.
This commit is contained in:
@@ -2,6 +2,8 @@
|
||||
|
||||
**[研究图册与固定在线副本](docs/FIGURE_GALLERY.md)**：连续分布、联合消融及原始PDF/SVG。
|
||||
|
||||
[两个实际2B检查点的参数球壳结论](docs/MODEL_SHELL_FINDINGS.md) · [完整数值报告](results/model-shell/REPORT.md) · [固定分组和均匀参照协议](model_shell/PROTOCOL.md)。
|
||||
|
||||
Research on repeated learned functions in **Qwen3.5-2B**: a completed static weight baseline
|
||||
and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm
|
||||
functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks.
|
||||
|
||||
+14
-2
@@ -1,6 +1,6 @@
|
||||
# 研究图册：Qwen3.5-2B 头贡献集中度
|
||||
# 研究图册：Qwen3.5-2B 参数几何与头贡献
|
||||
|
||||
这份图册与原始PNG/PDF/SVG一同保存在Git中。图片文件已随PR #6合并到main；以下固定提交链接不依赖聊天附件或本机路径。
|
||||
这份图册与原始PNG/PDF/SVG一同保存在Git中。头贡献基线图片已随PR #6合并到main；其固定提交链接不依赖聊天附件或本机路径。后续实验分别列于下方，可通过图册所在的Git提交查看对应版本。
|
||||
|
||||
原始结果提交：`098ea8650c7d0e06339c66e3a0509e38b85b3d7d`；模型revision：`15852e8c16360a2fea060d615a32b45270f8a8fc`。
|
||||
|
||||
@@ -43,3 +43,15 @@
|
||||
|
||||
|
||||
[条件定理、反例与数据说明](../results/sparse-shell/REPORT.md) · [PDF](../results/sparse-shell/sparse_shell_controls.pdf) · [SVG](../results/sparse-shell/sparse_shell_controls.svg)
|
||||
|
||||
## 两个实际2B检查点的参数球壳检验
|
||||
|
||||
[结论与解释](MODEL_SHELL_FINDINGS.md) · [实测报告、分组数量与限制](../results/model-shell/REPORT.md) · [固定协议](../model_shell/PROTOCOL.md) · [指标源表](../results/model-shell/metrics.csv) · [校验清单](../results/model-shell/audit.json)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
[微调前后配对图](../results/model-shell/figures/finetune_paired_radii.png) · [有限样本参与率](../results/model-shell/figures/participation_reference.png)。每幅图同目录均保存PDF/SVG，完整CDF及参照数据保存在`profiles.json.gz`，实测范数保存在`measured_radii.npz`；均随本次成果进入Git。
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
# 实际2B参数：比均匀参照更靠里吗？
|
||||
|
||||
**在明确的“校准最大半径以内”口径下，是；但数据同时显示更宽的径向分布，不能概括为整团参数向内收缩。此次GSM8K微调几乎没有改变这些几何指标。**
|
||||
|
||||
这轮直接读取了保留的Qwen3.5-2B权重和同一底座的已有纯LoRA GRPO检查点，在8.211.154.134上使用CPU完成。没有训练、推理或删除权重。
|
||||
|
||||
## 一个模型有多少组高维分布？
|
||||
|
||||
按事先固定的参数矩阵行／列与语言注意力头视图规则，每个模型有 **1,558组**：554个矩阵视图、1,004个按头视图。它们有重叠，不是模型固有的唯一分布数量。
|
||||
|
||||
主要汇总使用1,521个样本充足、未抽取坐标的视图。另有36个小样本视图和1个坐标抽样视图，全部单列并保留数据。整个检查点包括语言、视觉与MTP，共632个参数张量；319个一维张量和18个低维卷积核不满足这次“高维向量云”的规则。
|
||||
|
||||
## 有更多内侧质量，但也有更长的外尾
|
||||
|
||||
每组最多512个参数向量，一半校准中心及半径，另一半测量。对照也使用同样的校准规则；没有用所有向量的最大范数事后包住全部数据。
|
||||
|
||||
以原点为中心，比较归一化半径 `r / R_cal <= 0.9`：
|
||||
|
||||
|视图|组数|相对均匀球的内侧质量差中位数|相对匹配均值、逐坐标方差的均匀盒的质量差中位数|
|
||||
|---|---:|---:|---:|
|
||||
|矩阵行／列|517|+98.0个百分点|+97.7个百分点|
|
||||
|按头|1,004|+97.3个百分点|+97.3个百分点|
|
||||
|
||||
改用校准均值作为中心，结果仍有约96.9–97.7个百分点的差异。因此，这个现象并非仅来自中心放在原点。
|
||||
|
||||
但最大半径很容易受到少数大范数向量影响。换成各自的校准RMS尺度，并比较完整CDF后，**93.8%的矩阵视图和87.8%的头视图同时存在超过10个百分点的内尾、外尾差异**。去均值后分别为94.4%和86.4%。这说明两尾同时增加是普遍现象，比“所有向量整体更靠内”更符合这些数据。
|
||||
|
||||
|
||||
|
||||
这三组示例按模块角色固定：GDN的Q头、全注意力的Q头、MLP上投影行。上排用校准最大半径，下排用RMS；它们只是帮助读图，不代替全量统计。
|
||||
|
||||
## 是否说明稀疏或低维？
|
||||
|
||||
还不能如此推断。采样视图的逐组精确零值率中位数为0；点坐标能量参与率除以维度的中位数约0.31。径向内侧质量、坐标能量集中和样本间的有效维数是不同的测量对象。
|
||||
|
||||
本轮同时记录了有限样本Gram参与率，并与同样本数的均匀参照比较。它受校准样本秩上限限制，不能作为真实内在维度的直接估计。盒参照也只匹配边际范围或逐坐标矩，并未匹配相关结构。因此，本轮没有单独识别“稀疏支持”导致的效应。
|
||||
|
||||
## 微调前后改变了吗？
|
||||
|
||||
LoRA实际改变72个投影矩阵，影响972个视图，其中936个属于主要汇总。只对这些受影响的主要视图配对，以免大量未改动参数稀释差异。
|
||||
|
||||
原点中心下，逐组中位 `r/R_cal` 的配对变化中位数为 **6.30×10⁻⁷**，最大绝对变化 **4.03×10⁻⁴**。去均值后的结论相近。90%半径质量差在原点中心口径下未跨越当前经验CDF的分辨率；这不表示两套权重完全相同。
|
||||
|
||||
因此，这两个检查点不能支持“微调后、更好的模型更靠内”的结论；本轮也没有测任务质量。参数几何中的宽径向结构可以继续挖掘，但它本身还不是功能重复或可互相替代的证据。
|
||||
|
||||
## 数据与核验
|
||||
|
||||
[完整数值报告](../results/model-shell/REPORT.md) · [冻结协议](../model_shell/PROTOCOL.md) · [复现方式](../model_shell/README.md) · [指标CSV](../results/model-shell/metrics.csv) · [摘要JSON](../results/model-shell/summary.json) · [审计SHA](../results/model-shell/audit.json)
|
||||
|
||||
6,232个模型×中心×视图组合全部核验；CDF从保留的校准／留出范数复算，检查有限样本秩上限，原模型和适配器SHA均与既有记录一致。远程4项测试通过，覆盖索引、LoRA读取、中心与尺度、均匀球校准和稀疏贴壳反例。图表、完整CDF及对照摘要、实测范数一同保存。结论为描述性统计：视图相关且重叠，没有将其当作独立重复实验进行显著性检验。
|
||||
@@ -0,0 +1,83 @@
|
||||
# Actual-model parameter clouds: frozen grouping and null protocol
|
||||
|
||||
Assets: retained Qwen3.5-2B original15852e8c16360a2fea060d615a32b45270f8a8fc,
|
||||
and pure GSM8K GRPO LoRA seed0 at HF038d8be9bd0edd3883fa019651b02bdf65609468.
|
||||
Use the established read-only FP32 W+2BA wrapper. No training, GPU, new models,
|
||||
weight mutation, inference requirement or deletion. Every retained asset stays.
|
||||
|
||||
## What constitutes a distribution
|
||||
|
||||
A model has no unique canonical number of parameter distributions. Enumerate a
|
||||
fixed family, not "all possible distributions":
|
||||
1. Each ndim>=2 tensor flattened as [axis0, product(other axes)]: row vectors
|
||||
and column vectors, when point count>=16 and feature dimension>=32.
|
||||
2. Decoder language/MTP full-attention Q and interleaved gate rows per query
|
||||
head; K/V rows per KV head. GDN concatenated Q/K/V rows per corresponding head.
|
||||
No claim that these groups are disjoint or iid; whole-matrix views overlap.
|
||||
3. Primary sample-size stratum has >=64 available points;16..63 is small-sample
|
||||
descriptive only. One-dimensional norm/bias vectors have no cloud of vectors;
|
||||
GDN kernel row dimension4 is low dimensional and excluded with explicit reason.
|
||||
Vision parameters are covered by matrix/flattened views, not vision-head splits.
|
||||
|
||||
Include all checkpoint components (language, vision and MTP). Per view choose
|
||||
up to512 point indices without replacement, split deterministically into calibration
|
||||
and held-out halves. Use full vector coordinates up to dimension8192; only larger
|
||||
vectors use a fixed random coordinate subset, labelled projected rather than full.
|
||||
Use identical indices, coordinates and null random draws for original/FT.
|
||||
No overlap-group parameter savings or total distribution counts are additive.
|
||||
|
||||
## Centers and radius/scaling
|
||||
|
||||
Analyze origin and calibration-mean centers separately. Fit R_max using calibration
|
||||
norms only; do not silently clip held-out points outside R_max. Also fit calibration
|
||||
RMS radius. Store held-out maximum, outside-radius mass and radial spread.
|
||||
|
||||
Null A: uniform d-dimensional ball. Simulate the SAME center-fitting and calibration
|
||||
maximum/RMS fitting before held-out evaluation. For origin, also retain exact
|
||||
expected max-fitted null CDF a^d*n_cal/(n_cal+1), a<=1; log probabilities remain
|
||||
available and zero Monte Carlo hits are not exact zeros.
|
||||
Null B: independent coordinate-uniform box bounded by calibration min/max,
|
||||
including asymmetric coordinates. Fit its center/radii using its own calibration
|
||||
sample by exactly the same procedure. Record energy/range mismatch and held-out
|
||||
coordinate violations; this null is conditional on fitted bounds, not a statement
|
||||
that all real vectors lie inside that box.
|
||||
Null C: independent coordinate-uniform box with calibration mean mu and standard
|
||||
deviation sigma, bounds mu +/- sqrt(3)*sigma (sample variance ddof1). This matches
|
||||
first and diagonal second moments in its generating law. Use its own calibration
|
||||
and held-out center/max/RMS fits too; RMS-normalized comparisons target shape
|
||||
rather than a change of global scale. Matching diagonal moments is not covariance
|
||||
matching and does not preserve correlations.
|
||||
|
||||
Use512 validation null draws (not iid model observations), same calibration count
|
||||
as each data cloud. Reuse isotropic-ball nulls only for identical dimension/count
|
||||
and center-fitting rules. Monte Carlo resolution/error must be stated. Fit both
|
||||
centers from the same raw draws where applicable; paired FT uses common randomness.
|
||||
|
||||
## Metrics and interpretation
|
||||
|
||||
Retain measured norm arrays, empirical CDF grids, max-normalized interior fractions
|
||||
at0.5/0.75/0.9/0.95/0.99 and outside1, RMS-normalized CDF from0.5 to1.5,
|
||||
median radial ratios and radial spread. Prefer probability-point differences over
|
||||
astronomical ratios against tiny a^d. Inspect positive AND negative CDF gaps:
|
||||
more inner tail plus more outer tail is not global inward stochastic dominance.
|
||||
|
||||
Record exact zeros, coordinate energy participation (sum squares)^2/sum fourths,
|
||||
coordinate-variance participation, empirical covariance/second-moment participation
|
||||
via calibration Gram matrices. Label sample rank ceiling n_cal-1 (mean centered)
|
||||
or n_cal (origin); compare to equally sampled nulls, do not interpret the raw
|
||||
finite-sample participation as true intrinsic dimension.
|
||||
|
||||
Report both full-radius and moment-controlled results for each center, whole-view
|
||||
and head subgroups, primary versus small sample/projection strata, original and
|
||||
changed72-LoRA-target groups. Unchanged parameter views can reuse identical results
|
||||
with explicit references rather than re-read/recompute. Results describe parameters,
|
||||
not activations, model quality, functional duplicates, or pruning gains.
|
||||
|
||||
Execution budget: one CPU, 512 MiB RSS ceiling, 7200 seconds per invocation;
|
||||
checkpoint every group, resume without replacing completed groups. Any incomplete
|
||||
coverage is reported before interpretation. No numerical analysis runs locally.
|
||||
|
||||
Illustrative radial CDFs use these fixed original/origin views, selected by module
|
||||
role before the census completes, not by the resulting similarity/interior score:
|
||||
language layer0 linear-attention Q head0, layer3 full-attention Q head0, and layer0
|
||||
MLP up-projection rows. Their plots are illustrations, not independent replications.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Actual-model shell census
|
||||
|
||||
Read [PROTOCOL.md](PROTOCOL.md) before interpreting the results. This analyzes
|
||||
retained parameter arrays, not model activations or task accuracy.
|
||||
|
||||
Run only on the authorized CPU server from the repository root:
|
||||
|
||||
```sh
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m unittest model_shell.test_analysis -v
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.report
|
||||
```
|
||||
|
||||
Inputs are the already retained `runs/cpu-qwen35-2b/weights` and
|
||||
`runs/finetune-head-comparison/adapter.safetensors` with their existing JSON
|
||||
configuration and metadata. The adapter is evaluated as FP32 W+2BA. No weights
|
||||
are modified, no training or inference is run, and no files are deleted.
|
||||
|
||||
`runs/model-shell/checkpoints` is a resumable group ledger. Re-running skips
|
||||
completed groups; do not reuse a ledger after changing its protocol/code/inputs.
|
||||
One invocation stops at 7,200 seconds, RSS above 512 MiB or available memory below
|
||||
128 MiB. Resume after addressing a resource stop; do not relax guards silently.
|
||||
|
||||
`results/model-shell` contains the report, group inventory, per-profile metric
|
||||
CSV, compressed complete CDF/control summaries, and measured calibration/held-out
|
||||
radii. The NPZ keys are `GROUPINDEX_MODEL_CENTER_cal` / `_eval`, with zero-based
|
||||
group indices into `inventory.json`. Unchanged fine-tune profiles explicitly
|
||||
reference the original profiles in `profiles.json.gz`; the CSV expands both.
|
||||
Raw null draws are reproducible from the recorded deterministic seeds; null CDFs,
|
||||
scales and participation summaries are stored. `audit.json` verifies coverage,
|
||||
recomputes empirical CDFs from the saved norms, checks finite-sample rank limits,
|
||||
verifies the original asset SHA256 values, and hashes the output artifacts.
|
||||
`example_radii.npz` also saves calibration/held-out radii for the three fixed role
|
||||
examples and their regenerated nulls; their grid CDFs are checked against the
|
||||
original ledger before plotting full empirical step curves from radius zero.
|
||||
|
||||
The count of views is defined by the protocol and is not a unique number of
|
||||
distributions inherent in a neural network. These overlapping views are equally
|
||||
weighted for description; they are not independent replicates. The box nulls
|
||||
match fitted marginal ranges or moments, not correlations. The maximum fitted
|
||||
radius is not the unknown support boundary. Quantities near numerical or Monte
|
||||
Carlo resolution cannot support a definitive population statement.
|
||||
@@ -0,0 +1,124 @@
|
||||
"""CPU-only, read-only parameter cloud census with paired uniform references."""
|
||||
import argparse, hashlib, json, math, os, time
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
import numpy as np
|
||||
from pilot.tensors import load_tensors
|
||||
from finetune_compare.lora import install
|
||||
from model_shell.groups import enumerate_groups
|
||||
|
||||
MAX_GRID=np.array([.5,.75,.9,.95,.99,1.,1.05,1.1,1.25])
|
||||
RMS_GRID=np.linspace(.5,1.5,41)
|
||||
|
||||
def seed(key):return int.from_bytes(hashlib.sha256(key.encode()).digest()[:8],'little')
|
||||
|
||||
def guard(start):
|
||||
mem=dict((s.split(':')[0],int(s.split()[1])) for s in Path('/proc/meminfo').read_text().splitlines() if s.startswith('MemAvailable:'))
|
||||
rss=int(Path('/proc/self/statm').read_text().split()[1])*os.sysconf('SC_PAGE_SIZE')
|
||||
if rss>512*1024**2 or mem['MemAvailable']<128*1024 or time.monotonic()-start>7200:
|
||||
raise RuntimeError(f'Resource guard: RSS={rss}, available_kB={mem}, elapsed={time.monotonic()-start:.0f}')
|
||||
|
||||
def sample(t,g):
|
||||
rng=np.random.default_rng(seed(g['id']+'/indices'))
|
||||
n=min(512,g['population']);d=min(8192,g['dimension'])
|
||||
points=np.sort(rng.choice(g['population'],n,replace=False))
|
||||
features=np.sort(rng.choice(g['dimension'],d,replace=False)) if d<g['dimension'] else np.arange(d)
|
||||
width=math.prod(t.shape[1:]);out=np.empty((n,d),np.float32)
|
||||
if g['orientation']=='rows':
|
||||
for j,i in enumerate(points):
|
||||
row=i+g['start'];out[j]=t.read(slice(row*width,(row+1)*width))[features]
|
||||
if j%32==0:t.evict()
|
||||
else:
|
||||
for j,i in enumerate(features):
|
||||
out[:,j]=t.read(slice(i*width,(i+1)*width))[points]
|
||||
if j%32==0:t.evict()
|
||||
t.evict();out=out[rng.permutation(n)]
|
||||
digest=hashlib.sha256(points.tobytes()+features.tobytes()).hexdigest()
|
||||
return out,digest
|
||||
|
||||
def participation(y):
|
||||
gram=y@y.T;energy=float(np.trace(gram));den=float(np.square(gram).sum())
|
||||
return energy**2/den if den else 0.
|
||||
|
||||
def geometry(x,ncal,center,raw=False):
|
||||
# Float64 arithmetic; bounded sample matrix, no model-wide conversion.
|
||||
z=x.astype(np.float64);mu=z[:ncal].mean(0) if center=='mean' else np.zeros(z.shape[1])
|
||||
y=z-mu;norm=np.linalg.norm(y,axis=1);cal=norm[:ncal];r=norm[ncal:]
|
||||
maximum=float(cal.max());rms=float(np.sqrt(np.mean(cal**2)))
|
||||
valid=maximum>0 and rms>0
|
||||
a=r/maximum if maximum else np.zeros_like(r);b=r/rms if rms else np.zeros_like(r)
|
||||
coordvar=z[:ncal].var(0,ddof=1);cv=float(coordvar.sum()**2/np.square(coordvar).sum()) if coordvar.any() else 0.
|
||||
e=np.square(y[ncal:]);den=np.square(e).sum(1)
|
||||
pe=np.divide(e.sum(1)**2,den,out=np.zeros(len(e)),where=den>0)
|
||||
out=dict(valid=bool(valid),ncal=ncal,neval=len(r),d=z.shape[1],rmax=maximum,rms=rms,
|
||||
median_max=float(np.median(a)),median_rms=float(np.median(b)),
|
||||
p10_rms=float(np.quantile(b,.1)),p90_rms=float(np.quantile(b,.9)),
|
||||
outside=float(np.mean(a>1)),cdf_max=[float(np.mean(a<=v)) for v in MAX_GRID],
|
||||
cdf_rms=[float(np.mean(b<=v)) for v in RMS_GRID],
|
||||
covariance_participation=participation(y[:ncal]),rank_ceiling=ncal-(center=='mean'),
|
||||
coordinate_variance_participation=cv,point_energy_participation_median=float(np.median(pe)),
|
||||
zero_fraction=float(np.mean(z==0)))
|
||||
if raw:out.update(cal_norm=cal.tolist(),eval_norm=r.tolist())
|
||||
return out
|
||||
|
||||
BALL_CACHE={}
|
||||
def references(x,ncal,key):
|
||||
d=x.shape[1];cachekey=(d,ncal)
|
||||
if cachekey not in BALL_CACHE:
|
||||
rng=np.random.default_rng(seed(f'ball/{d}/{ncal}'))
|
||||
y=rng.standard_normal((ncal+512,d),dtype=np.float32)
|
||||
y*=((rng.random(len(y))**(1/d))/np.linalg.norm(y,axis=1))[:,None]
|
||||
BALL_CACHE[cachekey]={c:geometry(y,ncal,c) for c in ('origin','mean')}
|
||||
cal=x[:ncal].astype(np.float64);lo=cal.min(0);hi=cal.max(0);mu=cal.mean(0);sd=cal.std(0,ddof=1)
|
||||
rng=np.random.default_rng(seed(key+'/uniform'))
|
||||
u=rng.random((ncal+512,d),dtype=np.float32)
|
||||
refs={'ball':BALL_CACHE[cachekey]}
|
||||
for name,left,right in [('range_box',lo,hi),('moment_box',mu-np.sqrt(3)*sd,mu+np.sqrt(3)*sd)]:
|
||||
y=(left+(right-left)*u).astype(np.float32)
|
||||
refs[name]={c:geometry(y,ncal,c) for c in ('origin','mean')}
|
||||
return refs,dict(heldout_outside_coordinate_range=float(np.mean((x[ncal:]<lo)|(x[ncal:]>hi))),
|
||||
range_box_expected_energy=float(np.sum((lo**2+lo*hi+hi**2)/3)),
|
||||
observed_cal_energy=float(np.mean(np.sum(cal**2,axis=1))))
|
||||
|
||||
def analyze(x,key):
|
||||
ncal=len(x)//2;refs,extra=references(x,ncal,key);out={}
|
||||
for center in ('origin','mean'):
|
||||
p=geometry(x,ncal,center,True);p['references']=refs; p.update(extra)
|
||||
p['comparisons']={}
|
||||
for name,bycenter in refs.items():
|
||||
ref=bycenter[center];diff=np.array(p['cdf_rms'])-ref['cdf_rms']
|
||||
p['comparisons'][name]=dict(interior90_gap=p['cdf_max'][2]-ref['cdf_max'][2],
|
||||
interior95_gap=p['cdf_max'][3]-ref['cdf_max'][3],
|
||||
rms_cdf_positive_gap=float(max(0,diff.max())),rms_cdf_negative_gap=float(max(0,-diff.min())),
|
||||
median_rms_delta=p['median_rms']-ref['median_rms'],
|
||||
participation_ratio=p['covariance_participation']/ref['covariance_participation'] if ref['covariance_participation'] else None)
|
||||
p['ball_origin_log_expected_cdf90']=float(x.shape[1]*np.log(.9)+np.log(ncal/(ncal+1)))
|
||||
out[center]=p
|
||||
return out
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser();ap.add_argument('--root',default='runs');ap.add_argument('--out',default='runs/model-shell');args=ap.parse_args()
|
||||
root=Path(args.root);out=Path(args.out);out.mkdir(exist_ok=True,parents=True);(out/'checkpoints').mkdir(exist_ok=True)
|
||||
cfg=json.loads((root/'cpu-qwen35-2b/model_config.json').read_text())['text_config']
|
||||
base=load_tensors(root/'cpu-qwen35-2b/weights');ft=SimpleNamespace(tensors=dict(base),config=cfg)
|
||||
meta=json.loads((root/'finetune-head-comparison/adapter_meta.json').read_text())
|
||||
pairs,scale=install(ft,root/'finetune-head-comparison',meta)
|
||||
groups,skips=enumerate_groups(base,cfg)
|
||||
manifest=dict(groups=groups,skips=skips,total_tensors=len(base),parameters=sum(t.numel for t in base.values()),changed_tensors=sorted(pairs),adapter_meta=meta,
|
||||
max_grid=MAX_GRID.tolist(),rms_grid=RMS_GRID.tolist(),null_eval=512,point_cap=512,feature_cap=8192)
|
||||
(out/'inventory.json').write_text(json.dumps(manifest,indent=2));start=time.monotonic()
|
||||
for i,g in enumerate(groups):
|
||||
target=out/'checkpoints'/f'{i:04d}.json'
|
||||
if target.exists():continue
|
||||
guard(start);x,digest=sample(base[g['tensor']],g);p=analyze(x,g['id']);del x
|
||||
changed=g['tensor'] in pairs
|
||||
if changed:
|
||||
guard(start);x,digest2=sample(ft.tensors[g['tensor']],g);assert digest==digest2
|
||||
q=analyze(x,g['id']);del x
|
||||
else:q=None
|
||||
record=dict(group=g,sampling_sha256=digest,changed=changed,original=p,finetuned=q,finetuned_reuses_original=not changed)
|
||||
temp=target.with_suffix('.tmp');temp.write_text(json.dumps(record,allow_nan=False));temp.replace(target)
|
||||
if i%10==0:print(f'{i+1}/{len(groups)} elapsed={time.monotonic()-start:.1f}s',flush=True)
|
||||
print('COMPLETE',len(groups),flush=True)
|
||||
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Deterministic finite family of matrix/vector clouds; no data-dependent choice."""
|
||||
import math,re
|
||||
|
||||
def enumerate_groups(tensors,cfg):
|
||||
groups=[];skips=[]
|
||||
for name,t in sorted(tensors.items()):
|
||||
if len(t.shape)<2:skips.append({'name':name,'reason':'one vector, no vector cloud','shape':t.shape});continue
|
||||
rows=t.shape[0];cols=math.prod(t.shape[1:]);count=0
|
||||
for axis,n,d in [('rows',rows,cols),('columns',cols,rows)]:
|
||||
if n>=16 and d>=32:
|
||||
groups.append({'id':name+'::'+axis,'tensor':name,'kind':'matrix','orientation':axis,'start':0,'stop':rows,'population':n,'dimension':d,'stratum':'primary' if n>=64 else 'small_sample'});count+=1
|
||||
if not count:skips.append({'name':name,'reason':'insufficient points or dimension <32','shape':t.shape})
|
||||
if not (name.startswith('model.language_model.layers.') or name.startswith('mtp.layers.')):continue
|
||||
segments=[]
|
||||
if name.endswith('.linear_attn.in_proj_qkv.weight'):
|
||||
kh,kd,vh,vd=[cfg[k] for k in ('linear_num_key_heads','linear_key_head_dim','linear_num_value_heads','linear_value_head_dim')]
|
||||
for role,num,dim,offset in [('Q',kh,kd,0),('K',kh,kd,kh*kd),('V',vh,vd,2*kh*kd)]:
|
||||
for h in range(num):segments.append((role,h,offset+h*dim,offset+(h+1)*dim))
|
||||
elif name.endswith('.self_attn.q_proj.weight'):
|
||||
dim=cfg['head_dim'];assert rows==cfg['num_attention_heads']*2*dim
|
||||
for h in range(cfg['num_attention_heads']):
|
||||
for role,offset in [('Q',0),('gate',dim)]:segments.append((role,h,h*2*dim+offset,h*2*dim+offset+dim))
|
||||
elif name.endswith(('.self_attn.k_proj.weight','.self_attn.v_proj.weight')):
|
||||
role='K' if '.k_proj.' in name else 'V';dim=cfg['head_dim'];assert rows==cfg['num_key_value_heads']*dim
|
||||
for h in range(cfg['num_key_value_heads']):segments.append((role,h,h*dim,(h+1)*dim))
|
||||
for role,h,start,stop in segments:
|
||||
assert stop<=rows
|
||||
groups.append({'id':name+f'::{role}_head{h}','tensor':name,'kind':'head_rows','orientation':'rows','start':start,'stop':stop,'population':stop-start,'dimension':cols,'stratum':'primary' if stop-start>=64 else 'small_sample'})
|
||||
return groups,skips
|
||||
@@ -0,0 +1,176 @@
|
||||
"""Validate complete checkpoints; write compact scientific data, figures and report."""
|
||||
import argparse,csv,gzip,hashlib,json,platform,sys
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
import matplotlib
|
||||
matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
from model_shell.analyze import MAX_GRID,RMS_GRID,sample,seed
|
||||
from pilot.tensors import load_tensors
|
||||
|
||||
def sha(p):
|
||||
h=hashlib.sha256()
|
||||
with p.open('rb') as f:
|
||||
for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
|
||||
return h.hexdigest()
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser();ap.add_argument('--input',default='runs/model-shell');ap.add_argument('--out',default='results/model-shell');a=ap.parse_args()
|
||||
source=Path(a.input);out=Path(a.out);out.mkdir(parents=True,exist_ok=True);figdir=out/'figures';figdir.mkdir(exist_ok=True)
|
||||
inventory=json.loads((source/'inventory.json').read_text());groups=inventory['groups'];records=[];rows=[];radii={}
|
||||
for i,g in enumerate(groups):
|
||||
r=json.loads((source/'checkpoints'/f'{i:04d}.json').read_text());assert r['group']==g
|
||||
assert r['changed']==(g['tensor'] in inventory['changed_tensors'])
|
||||
for model in ('original','finetuned'):
|
||||
profiles=r[model] if r[model] is not None else r['original']
|
||||
for center,p in profiles.items():
|
||||
if center=='origin':
|
||||
p['ball_log_expected_max_cdf']=[float(p['d']*np.log(z)+np.log(p['ncal']/(p['ncal']+1))) for z in MAX_GRID if z<=1]
|
||||
assert p['valid'] and p['d']==min(8192,g['dimension'])
|
||||
assert p['covariance_participation']<=p['rank_ceiling']+1e-6
|
||||
v=np.asarray(p['eval_norm']);cal=np.asarray(p['cal_norm'])
|
||||
assert abs(p['rmax']-cal.max())<1e-12
|
||||
assert abs(p['outside']-np.mean(v>p['rmax']))<1e-12
|
||||
np.testing.assert_allclose(p['cdf_max'],[np.mean(v<=z*p['rmax']) for z in MAX_GRID])
|
||||
np.testing.assert_allclose(p['cdf_rms'],[np.mean(v<=z*p['rms']) for z in RMS_GRID])
|
||||
key=f'{i}_{model}_{center}';radii[key+'_cal']=cal;radii[key+'_eval']=v
|
||||
row=dict(group_index=i,group=g['id'],model=model,center=center,kind=g['kind'],stratum=g['stratum'],changed=r['changed'],projected=g['dimension']>8192,
|
||||
dimension=g['dimension'],analyzed_dimension=p['d'],population=g['population'],ncal=p['ncal'],neval=p['neval'],sampling_sha256=r['sampling_sha256'])
|
||||
for k in ('median_max','median_rms','p10_rms','p90_rms','outside','rmax','rms','covariance_participation','rank_ceiling','coordinate_variance_participation','point_energy_participation_median','zero_fraction','heldout_outside_coordinate_range'):
|
||||
row[k]=p[k]
|
||||
for name,c in p['comparisons'].items():
|
||||
for k,vv in c.items():row[name+'_'+k]=vv
|
||||
rows.append(row)
|
||||
# Compact profiles keep CDF and controls; raw measured radii live in NPZ.
|
||||
for model in ('original','finetuned'):
|
||||
if r[model] is not None:
|
||||
for p in r[model].values():p.pop('cal_norm');p.pop('eval_norm')
|
||||
records.append(r)
|
||||
with gzip.open(out/'profiles.json.gz','wt',encoding='utf-8') as f:json.dump(records,f,allow_nan=False)
|
||||
np.savez_compressed(out/'measured_radii.npz',**radii)
|
||||
with (out/'metrics.csv').open('w',newline='') as f:
|
||||
w=csv.DictWriter(f,fieldnames=list(rows[0]));w.writeheader();w.writerows(rows)
|
||||
(out/'inventory.json').write_text(json.dumps(inventory,indent=2))
|
||||
def subset(model='original',center='origin',kind=None,changed=None):
|
||||
return [r for r in rows if r['model']==model and r['center']==center and r['stratum']=='primary' and not r['projected'] and (kind is None or r['kind']==kind) and (changed is None or r['changed']==changed)]
|
||||
summary={}
|
||||
for center in ('origin','mean'):
|
||||
for kind in ('matrix','head_rows'):
|
||||
rs=subset(center=center,kind=kind);s={'n':len(rs),
|
||||
'median_zero_fraction':float(np.median([r['zero_fraction'] for r in rs])),
|
||||
'median_coordinate_energy_participation_fraction':float(np.median([r['point_energy_participation_median']/r['analyzed_dimension'] for r in rs])),
|
||||
'median_coordinate_variance_participation_fraction':float(np.median([r['coordinate_variance_participation']/r['analyzed_dimension'] for r in rs]))}
|
||||
for ref in ('ball','range_box','moment_box'):
|
||||
gap=np.array([r[ref+'_interior90_gap'] for r in rs]);pos=np.array([r[ref+'_rms_cdf_positive_gap'] for r in rs]);neg=np.array([r[ref+'_rms_cdf_negative_gap'] for r in rs])
|
||||
s[ref]=dict(median_interior90_gap=float(np.median(gap)),fraction_gap_gt10pp=float(np.mean(gap>.1)),median_positive_rms_gap=float(np.median(pos)),median_negative_rms_gap=float(np.median(neg)),
|
||||
fraction_two_sided_rms_gt10pp=float(np.mean((pos>.1)&(neg>.1))),median_participation_ratio=float(np.median([r[ref+'_participation_ratio'] for r in rs])))
|
||||
summary[center+'/'+kind]=s
|
||||
paired={}
|
||||
for center in ('origin','mean'):
|
||||
b=subset(center=center,changed=True);t=subset(model='finetuned',center=center,changed=True);assert [r['group'] for r in b]==[r['group'] for r in t]
|
||||
paired[center]={'n':len(b)}
|
||||
for field in ('median_max','median_rms','covariance_participation','moment_box_interior90_gap'):
|
||||
delta=np.array([q[field]-p[field] for p,q in zip(b,t)])
|
||||
paired[center][field]=dict(median=float(np.median(delta)),p10=float(np.quantile(delta,.1)),p90=float(np.quantile(delta,.9)),max_abs=float(np.max(abs(delta))))
|
||||
secondary=[]
|
||||
for model in ('original','finetuned'):
|
||||
for center in ('origin','mean'):
|
||||
for label in ('small_sample','projected'):
|
||||
ss=[r for r in rows if r['model']==model and r['center']==center and (r['stratum']=='small_sample' if label=='small_sample' else r['projected'])]
|
||||
secondary.append(dict(model=model,center=center,stratum=label,n=len(ss),median_max=float(np.median([r['median_max'] for r in ss])),moment_interior90_gap=float(np.median([r['moment_box_interior90_gap'] for r in ss]))))
|
||||
(out/'summary.json').write_text(json.dumps(dict(strata=summary,paired_changed=paired,secondary=secondary),indent=2))
|
||||
plt.rcParams.update({'font.size':10,'axes.spines.top':False,'axes.spines.right':False})
|
||||
def save(fig,name):
|
||||
fig.tight_layout()
|
||||
for ext in ('png','pdf','svg'):fig.savefig(figdir/(name+'.'+ext),dpi=180,bbox_inches='tight')
|
||||
plt.close(fig)
|
||||
fig,axs=plt.subplots(2,2,figsize=(11,7),sharex=True,sharey=True)
|
||||
for i,center in enumerate(('origin','mean')):
|
||||
for j,kind in enumerate(('matrix','head_rows')):
|
||||
rs=subset(center=center,kind=kind);ax=axs[i,j]
|
||||
for ref,label in [('ball','Uniform ball'),('range_box','Range-matched box'),('moment_box','Moment-matched box')]:
|
||||
v=np.sort([r[ref+'_interior90_gap'] for r in rs]);ax.plot(v,np.arange(1,len(v)+1)/len(v),label=label)
|
||||
ax.axvline(0,color='gray',lw=.8);ax.set_title(f'{center} center / {kind}, n={len(rs)}');ax.set_xlabel('F(data r/Rcal <= 0.9) - F(null)');ax.set_ylabel('Fraction of sampled groups')
|
||||
axs[0,0].legend(fontsize=8);fig.suptitle('Actual Qwen3.5-2B: interior mass depends on the reference');save(fig,'interior_reference_sensitivity')
|
||||
fig,axs=plt.subplots(1,2,figsize=(11,4.3))
|
||||
for ax,center in zip(axs,('origin','mean')):
|
||||
for kind,marker in [('matrix','o'),('head_rows','.')]:
|
||||
rs=subset(center=center,kind=kind);ax.scatter([r['moment_box_rms_cdf_positive_gap'] for r in rs],[r['moment_box_rms_cdf_negative_gap'] for r in rs],s=10,alpha=.25,label=kind,marker=marker)
|
||||
ax.axvline(.1,color='gray',lw=.7);ax.axhline(.1,color='gray',lw=.7);ax.set(xlabel='Largest positive CDF gap (more inner tail)',ylabel='Largest negative CDF gap (more outer tail)',title=f'{center} / moment-matched box',xlim=(-.02,1),ylim=(-.02,1));ax.legend()
|
||||
fig.suptitle('RMS-normalized radial shape: both tails can grow together');save(fig,'radial_shape_two_tails')
|
||||
fig,axs=plt.subplots(1,2,figsize=(11,4.3))
|
||||
for ax,center in zip(axs,('origin','mean')):
|
||||
b=subset(center=center,changed=True);t=subset(model='finetuned',center=center,changed=True)
|
||||
x=[r['median_max'] for r in b];y=[r['median_max'] for r in t];ax.scatter(x,y,s=10,alpha=.3);low=min(x+y);high=max(x+y);ax.plot([low,high],[low,high],color='gray',lw=1)
|
||||
ax.set(xlabel='Original median r/Rcal',ylabel='GRPO-LoRA median r/Rcal',title=f'{center}, changed primary groups n={len(b)}')
|
||||
fig.suptitle('Paired same-index comparison; unchanged groups excluded');save(fig,'finetune_paired_radii')
|
||||
fig,axs=plt.subplots(1,2,figsize=(11,4.3))
|
||||
for ax,center in zip(axs,('origin','mean')):
|
||||
for kind in ('matrix','head_rows'):
|
||||
rs=subset(center=center,kind=kind);v=np.sort([r['moment_box_participation_ratio'] for r in rs]);ax.plot(v,np.arange(1,len(v)+1)/len(v),label=kind)
|
||||
ax.set(xlabel='Empirical participation / equal-sample moment-box reference',ylabel='Fraction of groups',title=center);ax.axvline(1,color='gray',lw=.8);ax.legend()
|
||||
fig.suptitle('Finite-sample participation is not the true intrinsic dimension');save(fig,'participation_reference')
|
||||
examples=[('model.language_model.layers.0.linear_attn.in_proj_qkv.weight::Q_head0','L0 GDN Q head0'),
|
||||
('model.language_model.layers.3.self_attn.q_proj.weight::Q_head0','L3 full-attention Q head0'),
|
||||
('model.language_model.layers.0.mlp.up_proj.weight::rows','L0 MLP up rows')]
|
||||
fig,axs=plt.subplots(2,3,figsize=(13,7),sharey=True)
|
||||
byid={r['group']['id']:r for r in records}
|
||||
indices={g['id']:i for i,g in enumerate(groups)}
|
||||
base=load_tensors(Path('runs/cpu-qwen35-2b/weights'));example_radii={}
|
||||
for j,(gid,label) in enumerate(examples):
|
||||
p=byid[gid]['original']['origin']
|
||||
g=byid[gid]['group'];x,digest=sample(base[g['tensor']],g);assert digest==byid[gid]['sampling_sha256']
|
||||
ncal=len(x)//2;d=x.shape[1];cal=x[:ncal].astype(np.float64)
|
||||
rng=np.random.default_rng(seed(gid+'/uniform'));u=rng.random((ncal+512,d),dtype=np.float32)
|
||||
mu=cal.mean(0);sd=cal.std(0,ddof=1);lo=mu-np.sqrt(3)*sd;hi=mu+np.sqrt(3)*sd
|
||||
moment=(lo+(hi-lo)*u).astype(np.float32)
|
||||
rng=np.random.default_rng(seed(f'ball/{d}/{ncal}'));ball=rng.standard_normal((ncal+512,d),dtype=np.float32)
|
||||
ball*=((rng.random(len(ball))**(1/d))/np.linalg.norm(ball,axis=1))[:,None]
|
||||
arrays={'Actual weights':(radii[f'{indices[gid]}_original_origin_cal'],radii[f'{indices[gid]}_original_origin_eval'])}
|
||||
for name,ref,raw in [('Uniform ball','ball',ball),('Moment box','moment_box',moment)]:
|
||||
nr=np.linalg.norm(raw.astype(np.float64),axis=1);arrays[name]=(nr[:ncal],nr[ncal:])
|
||||
np.testing.assert_allclose([np.mean(nr[ncal:]/nr[:ncal].max()<=z) for z in MAX_GRID],p['references'][ref]['origin']['cdf_max'])
|
||||
np.testing.assert_allclose([np.mean(nr[ncal:]/np.sqrt(np.mean(nr[:ncal]**2))<=z) for z in RMS_GRID],p['references'][ref]['origin']['cdf_rms'])
|
||||
for name,(nc,ne) in arrays.items():example_radii[f'example{j}_{name.replace(" ","_")}_cal']=nc;example_radii[f'example{j}_{name.replace(" ","_")}_eval']=ne
|
||||
for i,(field,grid) in enumerate([('cdf_max',MAX_GRID),('cdf_rms',RMS_GRID)]):
|
||||
ax=axs[i,j]
|
||||
for name,(nc,ne) in arrays.items():
|
||||
scale=nc.max() if i==0 else np.sqrt(np.mean(nc**2));v=np.sort(ne/scale)
|
||||
ax.step(np.r_[0,v, max(1.5,float(v[-1]))],np.r_[0,np.arange(1,len(v)+1)/len(v),1],where='post',label=name,lw=1.8,ls='-' if name=='Actual weights' else '--')
|
||||
ax.set(title=label,xlabel='r / calibration '+('maximum' if i==0 else 'RMS'),ylabel='Held-out CDF',ylim=(-.03,1.03),xlim=(0,1.5));ax.legend(fontsize=8)
|
||||
np.savez_compressed(out/'example_radii.npz',**example_radii)
|
||||
fig.suptitle('Fixed role examples: maximum-radius and RMS normalization');save(fig,'fixed_role_radial_cdfs')
|
||||
lines=['# 两个实际2B检查点：参数云的球壳检验','',f'完整覆盖：{len(groups)}组/模型，{len(rows)}个模型×中心组合。632个参数张量，2,274,069,824个参数（包含语言、视觉和MTP）。',
|
||||
'','## 分布计数与范围','', '固定规则给出554个矩阵行／列视图、1,004个语言/MTP头视图；视图互相重叠，不能相加成参数节省量。1,522组有至少64个可用点，36组只有16个点；1组高维嵌入列视图使用8,192个抽样坐标。主表只汇总样本充足且没有投影的1,521组。319个一维张量和18个低维卷积核不构成此规则下的高维样本云。',
|
||||
'','## 原模型结果（逐组等权，非独立观测）','', '|中心|视图|组数|相对均匀球：90%半径内质量差中位数|相对矩匹配均匀盒：质量差中位数|矩匹配且RMS归一后，内外两尾差均>10个百分点的组占比|','|---|---|---:|---:|---:|---:|']
|
||||
for key,s in summary.items():
|
||||
c,k=key.split('/');lines.append(f'|{c}|{k}|{s["n"]}|{s["ball"]["median_interior90_gap"]:.3f}|{s["moment_box"]["median_interior90_gap"]:.3f}|{s["moment_box"]["fraction_two_sided_rms_gt10pp"]:.1%}|')
|
||||
lines+=['','这些是描述性质量差，不是显著性检验。正差表示所指定半径以内有更多样本；不能将其自动解释为整条径向分布向内平移。若RMS归一后内外尾均变多，更准确的解释是径向分布更宽。',
|
||||
'', '|中心/视图|逐组零值率中位数|点坐标能量参与率÷维度，中位数|坐标方差参与率÷维度，中位数|','|---|---:|---:|---:|']
|
||||
for key,s in summary.items():lines.append(f'|{key}|{s["median_zero_fraction"]:.6g}|{s["median_coordinate_energy_participation_fraction"]:.4f}|{s["median_coordinate_variance_participation_fraction"]:.4f}|')
|
||||
lines+=['','能量参与率衡量坐标能量集中程度，依赖坐标基；有限样本Gram参与率反映样本间几何，另见图。两者均不是精确非零坐标计数，也不是模型功能的有效维度。',
|
||||
'','## 微调前后','', '原模型是官方Qwen3.5-2B；第二个模型是在同一revision上进行GSM8K GRPO的已有纯LoRA检查点，不是另一个完整模型。72个投影矩阵使用只读W+2BA展开，其余参数共享。972个视图受影响；其中936个属于主分析。未改变视图复用完全相同的分析结果。','', '|中心|指标|配对变化中位数|P10|P90|最大绝对变化|','|---|---|---:|---:|---:|---:|']
|
||||
for c,p in paired.items():
|
||||
for k,v in p.items():
|
||||
if k!='n':lines.append(f'|{c}|{k}|{v["median"]:.6g}|{v["p10"]:.6g}|{v["p90"]:.6g}|{v["max_abs"]:.6g}|')
|
||||
lines+=['','## 单独描述的小样本与坐标抽样组','', '|模型|中心|类别|组数|中位r/Rcal的组间中位数|相对矩匹配盒的90%半径质量差中位数|','|---|---|---|---:|---:|---:|']
|
||||
for s in secondary:lines.append(f'|{s["model"]}|{s["center"]}|{s["stratum"]}|{s["n"]}|{s["median_max"]:.4f}|{s["moment_interior90_gap"]:.4f}|')
|
||||
lines+=['','## 如何读图','', '1. `interior_reference_sensitivity`：不同均匀参照下，90%校准半径内质量差的组间CDF。横轴向右表示更多内侧质量。','2. `radial_shape_two_tails`：匹配矩且按各自校准RMS归一后的最大正／负CDF差。右上角同时增加两尾，并非单纯向内。','3. `finetune_paired_radii`：同一视图的微调前后中位半径；对角线表示不变。','4. `participation_reference`：Gram矩阵参与率相对同样本量参照的比值。受样本秩上限影响，不能当作真实内在维度。',
|
||||
'5. `fixed_role_radial_cdfs`：预先按角色固定的GDN Q、全注意力Q和MLP上投影示例，上排按校准最大半径、下排按RMS归一化。不是按结果挑选最明显的组。',
|
||||
'','## 方法与限制','', '每组最多512个不同参数向量，一半校准、一半留出；模型固定，留出指未用于拟合中心/半径的参数向量，不是独立训练模型。对照每组512个验证向量，最坏单项二项标准误约2.21个百分点；模型留出样本的误差还取决于样本数，且参数向量不保证独立。10个百分点仅是描述性筛选尺度，没有多重检验或总体显著性结论。',
|
||||
'','均匀球、坐标范围均匀盒、匹配均值和逐坐标方差的均匀盒都按同一规则拟合自身中心、最大半径及RMS。均匀球原点中心的精确拟合半径CDF为a^d*ncal/(ncal+1)，保留其对数以避免把模拟零命中误报成零概率。盒对照不是协方差匹配。校准最大半径不是已知总体支撑边界；留出越界率已记录。',
|
||||
'','`origin`保留坐标均值贡献；`mean`使用校准向量均值。没有使用所有数据来选边界、训练或推理；没有删除模型。坐标稀疏性、有限样本参与率与径向内侧质量分别测量，不能相互代替。该实验不能证明更好的模型更靠里，也不能证明功能重复、可剪枝或可复用。',
|
||||
'','复现：`OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze`，完成后 `... python -m model_shell.report`。只在已授权CPU服务器运行。分组协议见 `model_shell/PROTOCOL.md`；全量指标在metrics.csv，完整CDF/参照在profiles.json.gz，实测范数在measured_radii.npz。']
|
||||
(out/'REPORT.md').write_text('\n'.join(lines)+'\n')
|
||||
sources={'base':Path('runs/cpu-qwen35-2b/weights/model.safetensors-00001-of-00001.safetensors'),'adapter':Path('runs/finetune-head-comparison/adapter.safetensors'),
|
||||
'config':Path('runs/cpu-qwen35-2b/model_config.json'),'adapter_meta':Path('runs/finetune-head-comparison/adapter_meta.json')}
|
||||
sourcehash={k:dict(path=str(p),bytes=p.stat().st_size,sha256=sha(p)) for k,p in sources.items()}
|
||||
(out/'environment.json').write_text(json.dumps(dict(python=sys.version,platform=platform.platform(),numpy=np.__version__,matplotlib=matplotlib.__version__,
|
||||
source_code_sha256={str(p):sha(p) for p in sorted(Path('model_shell').glob('*.py'))},protocol_sha256=sha(Path('model_shell/PROTOCOL.md')),seed='first 8 SHA256 bytes, little endian; group id plus purpose suffix',
|
||||
thread_limit=1,server='8.211.154.134',training=False,inference=False,gpu=False),indent=2))
|
||||
assert sourcehash['base']['sha256']=='aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1'
|
||||
assert sourcehash['adapter']['sha256']=='93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6'
|
||||
audit=dict(complete_groups=len(groups),profile_rows=len(rows),all_finite=True,cdf_recomputed=True,sample_rank_checked=True,sources=sourcehash,files={str(p.relative_to(out)):sha(p) for p in sorted(out.rglob('*')) if p.is_file() and p.name!='audit.json'})
|
||||
(out/'audit.json').write_text(json.dumps(audit,indent=2));print(json.dumps(dict(summary=summary,paired=paired),indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Run on the authorized CPU server: python -m unittest model_shell.test_analysis."""
|
||||
import unittest
|
||||
import numpy as np
|
||||
from model_shell.analyze import sample,geometry,analyze,seed
|
||||
from finetune_compare.lora import EffectiveWeight
|
||||
|
||||
class ArrayTensor:
|
||||
def __init__(self,x):self.x=x;self.shape=x.shape;self.numel=x.size;self.dtype='F32'
|
||||
def read(self,s=slice(None)):return self.x.reshape(-1)[s].copy()
|
||||
def evict(self):pass
|
||||
|
||||
class Checks(unittest.TestCase):
|
||||
def test_sampling_and_lora(self):
|
||||
rng=np.random.default_rng(17);w=rng.normal(size=(80,64)).astype('f');a=rng.normal(size=(2,64)).astype('f');b=rng.normal(size=(80,2)).astype('f')
|
||||
effective=EffectiveWeight(ArrayTensor(w),a,b,2);explicit=ArrayTensor(w+2*b@a)
|
||||
for axis,n,d,start,stop in [('rows',80,64,0,80),('columns',64,80,0,80),('rows',32,64,16,48)]:
|
||||
g=dict(id=axis+str(start),orientation=axis,population=n,dimension=d,start=start,stop=stop)
|
||||
x,h=sample(effective,g);y,k=sample(explicit,g);np.testing.assert_allclose(x,y,atol=2e-6,rtol=1e-5);self.assertEqual(h,k)
|
||||
self.assertEqual(x.shape,(n,d))
|
||||
rr=np.random.default_rng(seed(g['id']+'/indices'));indices=np.sort(rr.choice(n,n,replace=False));order=rr.permutation(n)
|
||||
cloud=explicit.x[start:stop] if axis=='rows' else explicit.x.T
|
||||
np.testing.assert_allclose(y,cloud[indices][order],atol=0,rtol=0)
|
||||
def test_center_scale_and_rank(self):
|
||||
rng=np.random.default_rng(5);x=rng.normal(size=(128,96)).astype('f')
|
||||
for center in ('origin','mean'):
|
||||
p=geometry(x,64,center);q=geometry(x*3,64,center)
|
||||
np.testing.assert_allclose(p['cdf_rms'],q['cdf_rms']);self.assertLessEqual(p['covariance_participation'],p['rank_ceiling']+1e-6)
|
||||
p=geometry(x,64,'mean');q=geometry(x+4,64,'mean');np.testing.assert_allclose(p['median_rms'],q['median_rms'],rtol=1e-6)
|
||||
self.assertFalse(geometry(np.zeros((32,64)),16,'mean')['valid'])
|
||||
def test_ball_fit_and_sparse_counterexample(self):
|
||||
rng=np.random.default_rng(33);d=32;ncal=256
|
||||
# Analytic expectation over independent calibration maxima.
|
||||
c=rng.random((20000,ncal))**(1/d);v=rng.random(20000)**(1/d)
|
||||
observed=np.mean(v<=.99*c.max(1));expected=.99**d*ncal/(ncal+1)
|
||||
self.assertLess(abs(observed-expected),.015)
|
||||
radii=rng.random(512)**(1/d);x=np.zeros((512,d),dtype='f');x[:,0]=radii
|
||||
p=geometry(x,256,'origin');self.assertGreater(p['median_max'],.9)
|
||||
# Sparse support alone retains the full-dimensional radial law.
|
||||
self.assertLess(p['cdf_max'][2],.08)
|
||||
x[:,0]=rng.random(512)**.5
|
||||
self.assertGreater(geometry(x,256,'origin')['cdf_max'][2],.65)
|
||||
sphere=rng.normal(size=(512,d));sphere/=np.linalg.norm(sphere,axis=1)[:,None]
|
||||
self.assertEqual(geometry(sphere,256,'origin')['cdf_max'][2],0.)
|
||||
def test_reproducible_controls(self):
|
||||
x=np.random.default_rng(7).normal(size=(32,40)).astype('f');a=analyze(x,'test');b=analyze(x,'test')
|
||||
self.assertEqual(a,b)
|
||||
self.assertTrue(a['mean']['valid'])
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,73 @@
|
||||
# 两个实际2B检查点：参数云的球壳检验
|
||||
|
||||
完整覆盖：1558组/模型，6232个模型×中心组合。632个参数张量，2,274,069,824个参数（包含语言、视觉和MTP）。
|
||||
|
||||
## 分布计数与范围
|
||||
|
||||
固定规则给出554个矩阵行／列视图、1,004个语言/MTP头视图；视图互相重叠，不能相加成参数节省量。1,522组有至少64个可用点，36组只有16个点；1组高维嵌入列视图使用8,192个抽样坐标。主表只汇总样本充足且没有投影的1,521组。319个一维张量和18个低维卷积核不构成此规则下的高维样本云。
|
||||
|
||||
## 原模型结果（逐组等权，非独立观测）
|
||||
|
||||
|中心|视图|组数|相对均匀球：90%半径内质量差中位数|相对矩匹配均匀盒：质量差中位数|矩匹配且RMS归一后，内外两尾差均>10个百分点的组占比|
|
||||
|---|---|---:|---:|---:|---:|
|
||||
|origin|matrix|517|0.980|0.977|93.8%|
|
||||
|origin|head_rows|1004|0.973|0.973|87.8%|
|
||||
|mean|matrix|517|0.977|0.977|94.4%|
|
||||
|mean|head_rows|1004|0.969|0.969|86.4%|
|
||||
|
||||
这些是描述性质量差，不是显著性检验。正差表示所指定半径以内有更多样本；不能将其自动解释为整条径向分布向内平移。若RMS归一后内外尾均变多，更准确的解释是径向分布更宽。
|
||||
|
||||
|中心/视图|逐组零值率中位数|点坐标能量参与率÷维度，中位数|坐标方差参与率÷维度，中位数|
|
||||
|---|---:|---:|---:|
|
||||
|origin/matrix|0|0.3067|0.9346|
|
||||
|origin/head_rows|0|0.3117|0.9055|
|
||||
|mean/matrix|0|0.3072|0.9346|
|
||||
|mean/head_rows|0|0.3118|0.9055|
|
||||
|
||||
能量参与率衡量坐标能量集中程度，依赖坐标基；有限样本Gram参与率反映样本间几何，另见图。两者均不是精确非零坐标计数，也不是模型功能的有效维度。
|
||||
|
||||
## 微调前后
|
||||
|
||||
原模型是官方Qwen3.5-2B；第二个模型是在同一revision上进行GSM8K GRPO的已有纯LoRA检查点，不是另一个完整模型。72个投影矩阵使用只读W+2BA展开，其余参数共享。972个视图受影响；其中936个属于主分析。未改变视图复用完全相同的分析结果。
|
||||
|
||||
|中心|指标|配对变化中位数|P10|P90|最大绝对变化|
|
||||
|---|---|---:|---:|---:|---:|
|
||||
|origin|median_max|6.30233e-07|-3.20939e-05|3.49557e-05|0.000402869|
|
||||
|origin|median_rms|-2.02235e-06|-4.00888e-05|3.91646e-05|0.000121968|
|
||||
|origin|covariance_participation|0.00011293|-0.0007975|0.00116302|0.0136976|
|
||||
|origin|moment_box_interior90_gap|0|0|0|0|
|
||||
|mean|median_max|3.08748e-07|-3.29473e-05|3.37828e-05|0.000395056|
|
||||
|mean|median_rms|-1.97695e-06|-4.08665e-05|3.90134e-05|0.000306282|
|
||||
|mean|covariance_participation|0.000126928|-0.000792364|0.00112596|0.0127131|
|
||||
|mean|moment_box_interior90_gap|0|0|0|0.00390625|
|
||||
|
||||
## 单独描述的小样本与坐标抽样组
|
||||
|
||||
|模型|中心|类别|组数|中位r/Rcal的组间中位数|相对矩匹配盒的90%半径质量差中位数|
|
||||
|---|---|---|---:|---:|---:|
|
||||
|original|origin|small_sample|36|0.7490|0.7500|
|
||||
|original|origin|projected|1|0.4199|0.9961|
|
||||
|original|mean|small_sample|36|0.8109|0.6875|
|
||||
|original|mean|projected|1|0.4245|0.9961|
|
||||
|finetuned|origin|small_sample|36|0.7490|0.7500|
|
||||
|finetuned|origin|projected|1|0.4199|0.9961|
|
||||
|finetuned|mean|small_sample|36|0.8110|0.6875|
|
||||
|finetuned|mean|projected|1|0.4245|0.9961|
|
||||
|
||||
## 如何读图
|
||||
|
||||
1. `interior_reference_sensitivity`：不同均匀参照下，90%校准半径内质量差的组间CDF。横轴向右表示更多内侧质量。
|
||||
2. `radial_shape_two_tails`：匹配矩且按各自校准RMS归一后的最大正／负CDF差。右上角同时增加两尾，并非单纯向内。
|
||||
3. `finetune_paired_radii`：同一视图的微调前后中位半径；对角线表示不变。
|
||||
4. `participation_reference`：Gram矩阵参与率相对同样本量参照的比值。受样本秩上限影响，不能当作真实内在维度。
|
||||
5. `fixed_role_radial_cdfs`：预先按角色固定的GDN Q、全注意力Q和MLP上投影示例，上排按校准最大半径、下排按RMS归一化。不是按结果挑选最明显的组。
|
||||
|
||||
## 方法与限制
|
||||
|
||||
每组最多512个不同参数向量，一半校准、一半留出；模型固定，留出指未用于拟合中心/半径的参数向量，不是独立训练模型。对照每组512个验证向量，最坏单项二项标准误约2.21个百分点；模型留出样本的误差还取决于样本数，且参数向量不保证独立。10个百分点仅是描述性筛选尺度，没有多重检验或总体显著性结论。
|
||||
|
||||
均匀球、坐标范围均匀盒、匹配均值和逐坐标方差的均匀盒都按同一规则拟合自身中心、最大半径及RMS。均匀球原点中心的精确拟合半径CDF为a^d*ncal/(ncal+1)，保留其对数以避免把模拟零命中误报成零概率。盒对照不是协方差匹配。校准最大半径不是已知总体支撑边界；留出越界率已记录。
|
||||
|
||||
`origin`保留坐标均值贡献；`mean`使用校准向量均值。没有使用所有数据来选边界、训练或推理；没有删除模型。坐标稀疏性、有限样本参与率与径向内侧质量分别测量，不能相互代替。该实验不能证明更好的模型更靠里，也不能证明功能重复、可剪枝或可复用。
|
||||
|
||||
复现：`OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze`，完成后 `... python -m model_shell.report`。只在已授权CPU服务器运行。分组协议见 `model_shell/PROTOCOL.md`；全量指标在metrics.csv，完整CDF/参照在profiles.json.gz，实测范数在measured_radii.npz。
|
||||
@@ -0,0 +1,157 @@
|
||||
1/1558 elapsed=25.4s
|
||||
11/1558 elapsed=65.2s
|
||||
21/1558 elapsed=75.1s
|
||||
31/1558 elapsed=84.5s
|
||||
41/1558 elapsed=93.3s
|
||||
51/1558 elapsed=102.0s
|
||||
61/1558 elapsed=115.8s
|
||||
71/1558 elapsed=137.8s
|
||||
81/1558 elapsed=149.6s
|
||||
91/1558 elapsed=159.8s
|
||||
101/1558 elapsed=169.0s
|
||||
111/1558 elapsed=178.5s
|
||||
121/1558 elapsed=193.0s
|
||||
131/1558 elapsed=215.0s
|
||||
141/1558 elapsed=225.0s
|
||||
151/1558 elapsed=234.5s
|
||||
161/1558 elapsed=243.4s
|
||||
171/1558 elapsed=252.9s
|
||||
181/1558 elapsed=262.7s
|
||||
191/1558 elapsed=283.8s
|
||||
201/1558 elapsed=297.3s
|
||||
211/1558 elapsed=305.0s
|
||||
221/1558 elapsed=311.2s
|
||||
231/1558 elapsed=328.1s
|
||||
241/1558 elapsed=337.7s
|
||||
251/1558 elapsed=348.0s
|
||||
261/1558 elapsed=359.5s
|
||||
271/1558 elapsed=368.0s
|
||||
281/1558 elapsed=382.3s
|
||||
291/1558 elapsed=405.5s
|
||||
301/1558 elapsed=414.4s
|
||||
311/1558 elapsed=424.6s
|
||||
321/1558 elapsed=433.5s
|
||||
331/1558 elapsed=442.5s
|
||||
341/1558 elapsed=455.7s
|
||||
351/1558 elapsed=476.6s
|
||||
361/1558 elapsed=485.6s
|
||||
371/1558 elapsed=494.2s
|
||||
381/1558 elapsed=503.4s
|
||||
391/1558 elapsed=512.8s
|
||||
401/1558 elapsed=524.4s
|
||||
411/1558 elapsed=544.7s
|
||||
421/1558 elapsed=557.9s
|
||||
431/1558 elapsed=565.7s
|
||||
441/1558 elapsed=571.8s
|
||||
451/1558 elapsed=586.9s
|
||||
461/1558 elapsed=596.2s
|
||||
471/1558 elapsed=605.1s
|
||||
481/1558 elapsed=614.1s
|
||||
491/1558 elapsed=623.4s
|
||||
501/1558 elapsed=637.7s
|
||||
511/1558 elapsed=660.1s
|
||||
521/1558 elapsed=668.4s
|
||||
531/1558 elapsed=677.4s
|
||||
541/1558 elapsed=686.7s
|
||||
551/1558 elapsed=696.0s
|
||||
561/1558 elapsed=709.0s
|
||||
571/1558 elapsed=730.0s
|
||||
581/1558 elapsed=738.9s
|
||||
591/1558 elapsed=747.4s
|
||||
601/1558 elapsed=757.0s
|
||||
611/1558 elapsed=766.0s
|
||||
621/1558 elapsed=775.5s
|
||||
631/1558 elapsed=797.3s
|
||||
641/1558 elapsed=810.4s
|
||||
651/1558 elapsed=818.2s
|
||||
661/1558 elapsed=823.8s
|
||||
671/1558 elapsed=840.0s
|
||||
681/1558 elapsed=850.1s
|
||||
691/1558 elapsed=859.5s
|
||||
701/1558 elapsed=868.9s
|
||||
711/1558 elapsed=878.3s
|
||||
721/1558 elapsed=891.7s
|
||||
731/1558 elapsed=913.7s
|
||||
741/1558 elapsed=922.2s
|
||||
751/1558 elapsed=931.2s
|
||||
761/1558 elapsed=941.2s
|
||||
771/1558 elapsed=951.8s
|
||||
781/1558 elapsed=968.3s
|
||||
791/1558 elapsed=989.6s
|
||||
801/1558 elapsed=999.1s
|
||||
811/1558 elapsed=1009.5s
|
||||
821/1558 elapsed=1018.5s
|
||||
831/1558 elapsed=1027.4s
|
||||
841/1558 elapsed=1037.3s
|
||||
851/1558 elapsed=1056.4s
|
||||
861/1558 elapsed=1071.4s
|
||||
871/1558 elapsed=1081.0s
|
||||
881/1558 elapsed=1089.9s
|
||||
891/1558 elapsed=1099.9s
|
||||
901/1558 elapsed=1109.1s
|
||||
911/1558 elapsed=1131.1s
|
||||
921/1558 elapsed=1143.5s
|
||||
931/1558 elapsed=1152.5s
|
||||
941/1558 elapsed=1159.0s
|
||||
951/1558 elapsed=1174.1s
|
||||
961/1558 elapsed=1182.3s
|
||||
971/1558 elapsed=1188.2s
|
||||
981/1558 elapsed=1196.0s
|
||||
991/1558 elapsed=1211.6s
|
||||
1001/1558 elapsed=1220.8s
|
||||
1011/1558 elapsed=1230.4s
|
||||
1021/1558 elapsed=1239.1s
|
||||
1031/1558 elapsed=1248.8s
|
||||
1041/1558 elapsed=1267.3s
|
||||
1051/1558 elapsed=1285.1s
|
||||
1061/1558 elapsed=1293.8s
|
||||
1071/1558 elapsed=1303.3s
|
||||
1081/1558 elapsed=1311.9s
|
||||
1091/1558 elapsed=1321.5s
|
||||
1101/1558 elapsed=1338.0s
|
||||
1111/1558 elapsed=1356.8s
|
||||
1121/1558 elapsed=1366.6s
|
||||
1131/1558 elapsed=1375.4s
|
||||
1141/1558 elapsed=1384.4s
|
||||
1151/1558 elapsed=1393.2s
|
||||
1161/1558 elapsed=1406.7s
|
||||
1171/1558 elapsed=1430.1s
|
||||
1181/1558 elapsed=1438.1s
|
||||
1191/1558 elapsed=1444.4s
|
||||
1201/1558 elapsed=1451.6s
|
||||
1211/1558 elapsed=1467.4s
|
||||
1221/1558 elapsed=1477.1s
|
||||
1231/1558 elapsed=1489.0s
|
||||
1241/1558 elapsed=1501.1s
|
||||
1251/1558 elapsed=1513.7s
|
||||
1261/1558 elapsed=1538.7s
|
||||
1271/1558 elapsed=1564.9s
|
||||
1281/1558 elapsed=1576.8s
|
||||
1291/1558 elapsed=1587.4s
|
||||
1301/1558 elapsed=1598.2s
|
||||
1311/1558 elapsed=1610.2s
|
||||
1321/1558 elapsed=1628.9s
|
||||
1331/1558 elapsed=1641.2s
|
||||
1341/1558 elapsed=1649.6s
|
||||
1351/1558 elapsed=1659.0s
|
||||
1361/1558 elapsed=1670.9s
|
||||
1371/1558 elapsed=1680.2s
|
||||
1381/1558 elapsed=1688.9s
|
||||
1391/1558 elapsed=1698.7s
|
||||
1401/1558 elapsed=1709.4s
|
||||
1411/1558 elapsed=1718.6s
|
||||
1421/1558 elapsed=1727.1s
|
||||
1431/1558 elapsed=1736.3s
|
||||
1441/1558 elapsed=1749.4s
|
||||
1451/1558 elapsed=1757.2s
|
||||
1461/1558 elapsed=1765.7s
|
||||
1471/1558 elapsed=1775.4s
|
||||
1481/1558 elapsed=1786.3s
|
||||
1491/1558 elapsed=1794.9s
|
||||
1501/1558 elapsed=1803.7s
|
||||
1511/1558 elapsed=1812.9s
|
||||
1521/1558 elapsed=1826.2s
|
||||
1531/1558 elapsed=1844.9s
|
||||
1541/1558 elapsed=1852.7s
|
||||
1551/1558 elapsed=1858.5s
|
||||
COMPLETE 1558
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"complete_groups": 1558,
|
||||
"profile_rows": 6232,
|
||||
"all_finite": true,
|
||||
"cdf_recomputed": true,
|
||||
"sample_rank_checked": true,
|
||||
"sources": {
|
||||
"base": {
|
||||
"path": "runs/cpu-qwen35-2b/weights/model.safetensors-00001-of-00001.safetensors",
|
||||
"bytes": 4548221488,
|
||||
"sha256": "aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1"
|
||||
},
|
||||
"adapter": {
|
||||
"path": "runs/finetune-head-comparison/adapter.safetensors",
|
||||
"bytes": 18932336,
|
||||
"sha256": "93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6"
|
||||
},
|
||||
"config": {
|
||||
"path": "runs/cpu-qwen35-2b/model_config.json",
|
||||
"bytes": 2908,
|
||||
"sha256": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4"
|
||||
},
|
||||
"adapter_meta": {
|
||||
"path": "runs/finetune-head-comparison/adapter_meta.json",
|
||||
"bytes": 573,
|
||||
"sha256": "b8d4737a5b940bcdfed72258600227a89f883dc863e9724bf84e457b46a53703"
|
||||
}
|
||||
},
|
||||
"files": {
|
||||
"REPORT.md": "43c644d2cf1e1100cb547377e5407db606e04e568bdaa1ba0deb0e9e70fe3ff6",
|
||||
"analysis-output.txt": "a6905d3ec23d68fa4c2ff1b1a4ea6b19e6b94b93aef65ca220dca3823cd87fd4",
|
||||
"environment.json": "7ccc05cb9ee75302982886d1e6746d5e97753ca91c10629a69f6acde68848a41",
|
||||
"example_radii.npz": "787fb0e986ed621604b64e704ce9f7cd6c37a04c70475d1611fcb8f986c27b51",
|
||||
"figures/finetune_paired_radii.pdf": "80a0b5b90b04104bf89ea90676b6694af6ebe4e41101ccf4b771414b870bfa56",
|
||||
"figures/finetune_paired_radii.png": "1ba80adfa87ac69537fa74cbe1c4b2237a5fca9d3a536319bb630fd84b59b534",
|
||||
"figures/finetune_paired_radii.svg": "9f65fa49e08cb93e4467f15c2beb2a499f29267392a531db86ec98e3fce9a689",
|
||||
"figures/fixed_role_radial_cdfs.pdf": "cbb200dcc5f0812181720ad62047a07b4310eea35d32dc0652d891d86eeecac5",
|
||||
"figures/fixed_role_radial_cdfs.png": "c7175229b3d9637774635db8a6fef313bba83351f87c6e6916b8d065b5200891",
|
||||
"figures/fixed_role_radial_cdfs.svg": "09194a9734f496b12f8e699083e494c35ff6f2d95f70fac683aa3e528e0c4684",
|
||||
"figures/interior_reference_sensitivity.pdf": "b6b77c5b02e0a133db884297cbb5f939df82f67db2bccfb0efc142627d032b8f",
|
||||
"figures/interior_reference_sensitivity.png": "f3f612974c19841aa960d14824dfbf713304a60b74af87bade35cfe70dd1f86f",
|
||||
"figures/interior_reference_sensitivity.svg": "b61ee318190c584c389f051023796edc3111cb288d5790059b94e54cdd01e2ac",
|
||||
"figures/participation_reference.pdf": "ea500a0838178b1747fe4d9325c927f8a19557b26de266cd0171ada2bc04da1c",
|
||||
"figures/participation_reference.png": "89782ec43f88874b6b822e91a90ff4b8f5cb40edeae5229286c80781c5336bab",
|
||||
"figures/participation_reference.svg": "f24de314649ea9d8b2b6ce66187515857f5b0eb10e68d8f8e0437fee2fcf6272",
|
||||
"figures/radial_shape_two_tails.pdf": "36d6e9df0db1c1986da949330cc22daf04dac7a3e524778447a68fbd9adac359",
|
||||
"figures/radial_shape_two_tails.png": "3b6149a4b0ef154f58c06f518e8a6a972f1ff6c913d0058c5698e64ae74213eb",
|
||||
"figures/radial_shape_two_tails.svg": "f665d434b5511ad480a32a565e0ce433a1dfaf6b9855b6caa524b0fab24a1a75",
|
||||
"inventory.json": "fe7c88d5d4ae58dcd45709b44041e3d216953fae64dfc4e22961b7051727f83a",
|
||||
"measured_radii.npz": "bc2a8298aa94e3c9481474f91400f16a119a5083f8112145727cbab080ebd075",
|
||||
"metrics.csv": "d4176fa88b9126c00ba7e56bb54a031828a7481e9049a03c5058c58d0e8c1c30",
|
||||
"profiles.json.gz": "7a97cf2dccd4993e2e5318ace39536bbd592d133bc3d393e7631f3911e500faf",
|
||||
"summary.json": "7320790885813d2afa9b688685efcf8c9ed72ef7f7e97557658602cf2008eef2",
|
||||
"test-output.txt": "3794ae58181c17343dc3660a54b70c53f4f75dbd25fa6637e5f185553fe094d4"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"python": "3.14.4 (main, Aug 20 2026, 10:41:58) [GCC 15.2.0]",
|
||||
"platform": "Linux-7.0.0-31-generic-x86_64-with-glibc2.43",
|
||||
"numpy": "2.5.4",
|
||||
"matplotlib": "3.11.2",
|
||||
"source_code_sha256": {
|
||||
"model_shell/analyze.py": "e791368b5ef9094ee6b3a1d4603fe11b0161b647353fa4d25c52aadfae914fe2",
|
||||
"model_shell/groups.py": "eafbce2f0eeac89cfd3c63c57e91d1b7a260f4de4a1dd38be0aaceda5f317b15",
|
||||
"model_shell/report.py": "af0617373a2b06cbe5728330157c1d2e60615eb7577cf3dd95be699a4b27ae19",
|
||||
"model_shell/test_analysis.py": "a837d787d00e5a6302ae46ca4079bbaabe3f548cc71705cce514c92acd0943e7"
|
||||
},
|
||||
"protocol_sha256": "dfece26a7f2779c511d7a8f9a6c62c1ed49fec2dc5a3aa8c112681a52189d366",
|
||||
"seed": "first 8 SHA256 bytes, little endian; group id plus purpose suffix",
|
||||
"thread_limit": 1,
|
||||
"server": "8.211.154.134",
|
||||
"training": false,
|
||||
"inference": false,
|
||||
"gpu": false
|
||||
}
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 314 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 240 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 162 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 167 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 111 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 114 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 89 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 293 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 500 KiB |
File diff suppressed because it is too large
Load Diff
Binary file not shown.
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@@ -0,0 +1,246 @@
|
||||
{
|
||||
"strata": {
|
||||
"origin/matrix": {
|
||||
"n": 517,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.30673555734079716,
|
||||
"median_coordinate_variance_participation_fraction": 0.934645955851128,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.98046875,
|
||||
"fraction_gap_gt10pp": 0.9806576402321083,
|
||||
"median_positive_rms_gap": 0.37109375,
|
||||
"median_negative_rms_gap": 0.3515625,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9516441005802708,
|
||||
"median_participation_ratio": 0.8297024600513981
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9806576402321083,
|
||||
"median_positive_rms_gap": 0.32421875,
|
||||
"median_negative_rms_gap": 0.310546875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9090909090909091,
|
||||
"median_participation_ratio": 1.3431227073360341
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.9765625,
|
||||
"fraction_gap_gt10pp": 0.9806576402321083,
|
||||
"median_positive_rms_gap": 0.34375,
|
||||
"median_negative_rms_gap": 0.330078125,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9381044487427466,
|
||||
"median_participation_ratio": 0.8524481165697501
|
||||
}
|
||||
},
|
||||
"origin/head_rows": {
|
||||
"n": 1004,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.31172156343430024,
|
||||
"median_coordinate_variance_participation_fraction": 0.9055279134912191,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.97265625,
|
||||
"fraction_gap_gt10pp": 0.99800796812749,
|
||||
"median_positive_rms_gap": 0.578125,
|
||||
"median_negative_rms_gap": 0.265625,
|
||||
"fraction_two_sided_rms_gt10pp": 0.896414342629482,
|
||||
"median_participation_ratio": 0.7070664507223768
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.99800796812749,
|
||||
"median_positive_rms_gap": 0.55078125,
|
||||
"median_negative_rms_gap": 0.244140625,
|
||||
"fraction_two_sided_rms_gt10pp": 0.852589641434263,
|
||||
"median_participation_ratio": 1.1235217859798587
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.97265625,
|
||||
"fraction_gap_gt10pp": 0.99800796812749,
|
||||
"median_positive_rms_gap": 0.56640625,
|
||||
"median_negative_rms_gap": 0.251953125,
|
||||
"fraction_two_sided_rms_gt10pp": 0.8784860557768924,
|
||||
"median_participation_ratio": 0.7369770671990965
|
||||
}
|
||||
},
|
||||
"mean/matrix": {
|
||||
"n": 517,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.30718321338258303,
|
||||
"median_coordinate_variance_participation_fraction": 0.934645955851128,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.9765625,
|
||||
"fraction_gap_gt10pp": 0.9787234042553191,
|
||||
"median_positive_rms_gap": 0.46484375,
|
||||
"median_negative_rms_gap": 0.37109375,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9922630560928434,
|
||||
"median_participation_ratio": 0.8366753780099343
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.97265625,
|
||||
"fraction_gap_gt10pp": 0.9787234042553191,
|
||||
"median_positive_rms_gap": 0.3359375,
|
||||
"median_negative_rms_gap": 0.32421875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9303675048355899,
|
||||
"median_participation_ratio": 0.868224555240548
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.9765625,
|
||||
"fraction_gap_gt10pp": 0.9787234042553191,
|
||||
"median_positive_rms_gap": 0.33984375,
|
||||
"median_negative_rms_gap": 0.3359375,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9439071566731141,
|
||||
"median_participation_ratio": 0.8465704560846329
|
||||
}
|
||||
},
|
||||
"mean/head_rows": {
|
||||
"n": 1004,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.31181881259530586,
|
||||
"median_coordinate_variance_participation_fraction": 0.9055279134912191,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9970119521912351,
|
||||
"median_positive_rms_gap": 0.62109375,
|
||||
"median_negative_rms_gap": 0.296875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9322709163346613,
|
||||
"median_participation_ratio": 0.7143476292983669
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9970119521912351,
|
||||
"median_positive_rms_gap": 0.560546875,
|
||||
"median_negative_rms_gap": 0.248046875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.8615537848605578,
|
||||
"median_participation_ratio": 0.7194449549938213
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9970119521912351,
|
||||
"median_positive_rms_gap": 0.5625,
|
||||
"median_negative_rms_gap": 0.25,
|
||||
"fraction_two_sided_rms_gt10pp": 0.8635458167330677,
|
||||
"median_participation_ratio": 0.7159821443435122
|
||||
}
|
||||
}
|
||||
},
|
||||
"paired_changed": {
|
||||
"origin": {
|
||||
"n": 936,
|
||||
"median_max": {
|
||||
"median": 6.302327853480172e-07,
|
||||
"p10": -3.2093888658157876e-05,
|
||||
"p90": 3.49557393942157e-05,
|
||||
"max_abs": 0.0004028694837374669
|
||||
},
|
||||
"median_rms": {
|
||||
"median": -2.0223542534503203e-06,
|
||||
"p10": -4.008879155420475e-05,
|
||||
"p90": 3.916460868902405e-05,
|
||||
"max_abs": 0.00012196820638044059
|
||||
},
|
||||
"covariance_participation": {
|
||||
"median": 0.00011293028622105794,
|
||||
"p10": -0.0007975000317692604,
|
||||
"p90": 0.0011630175829182576,
|
||||
"max_abs": 0.01369755006576412
|
||||
},
|
||||
"moment_box_interior90_gap": {
|
||||
"median": 0.0,
|
||||
"p10": 0.0,
|
||||
"p90": 0.0,
|
||||
"max_abs": 0.0
|
||||
}
|
||||
},
|
||||
"mean": {
|
||||
"n": 936,
|
||||
"median_max": {
|
||||
"median": 3.087481980079687e-07,
|
||||
"p10": -3.294732610958917e-05,
|
||||
"p90": 3.3782823366856896e-05,
|
||||
"max_abs": 0.0003950555852355597
|
||||
},
|
||||
"median_rms": {
|
||||
"median": -1.9769477522801893e-06,
|
||||
"p10": -4.086651887480475e-05,
|
||||
"p90": 3.901337185857612e-05,
|
||||
"max_abs": 0.00030628174134661457
|
||||
},
|
||||
"covariance_participation": {
|
||||
"median": 0.00012692754527421357,
|
||||
"p10": -0.000792364212092167,
|
||||
"p90": 0.0011259624874320195,
|
||||
"max_abs": 0.012713060927119102
|
||||
},
|
||||
"moment_box_interior90_gap": {
|
||||
"median": 0.0,
|
||||
"p10": 0.0,
|
||||
"p90": 0.0,
|
||||
"max_abs": 0.00390625
|
||||
}
|
||||
}
|
||||
},
|
||||
"secondary": [
|
||||
{
|
||||
"model": "original",
|
||||
"center": "origin",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.7489558868770938,
|
||||
"moment_interior90_gap": 0.75
|
||||
},
|
||||
{
|
||||
"model": "original",
|
||||
"center": "origin",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.41986786500966855,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
},
|
||||
{
|
||||
"model": "original",
|
||||
"center": "mean",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.8109468212379675,
|
||||
"moment_interior90_gap": 0.6875
|
||||
},
|
||||
{
|
||||
"model": "original",
|
||||
"center": "mean",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.4244519268106477,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "origin",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.7489607494253279,
|
||||
"moment_interior90_gap": 0.75
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "origin",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.41986786500966855,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "mean",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.8110356927186851,
|
||||
"moment_interior90_gap": 0.6875
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "mean",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.4244519268106477,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
test_ball_fit_and_sparse_counterexample (model_shell.test_analysis.Checks.test_ball_fit_and_sparse_counterexample) ... ok
|
||||
test_center_scale_and_rank (model_shell.test_analysis.Checks.test_center_scale_and_rank) ... ok
|
||||
test_reproducible_controls (model_shell.test_analysis.Checks.test_reproducible_controls) ... ok
|
||||
test_sampling_and_lora (model_shell.test_analysis.Checks.test_sampling_and_lora) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 4 tests in 0.394s
|
||||
|
||||
OK
|
||||
Reference in New Issue
Block a user