Measure parameter shell geometry in paired Qwen3.5-2B checkpoints
@@ -0,0 +1,73 @@
|
||||
# 两个实际2B检查点:参数云的球壳检验
|
||||
|
||||
完整覆盖:1558组/模型,6232个模型×中心组合。632个参数张量,2,274,069,824个参数(包含语言、视觉和MTP)。
|
||||
|
||||
## 分布计数与范围
|
||||
|
||||
固定规则给出554个矩阵行/列视图、1,004个语言/MTP头视图;视图互相重叠,不能相加成参数节省量。1,522组有至少64个可用点,36组只有16个点;1组高维嵌入列视图使用8,192个抽样坐标。主表只汇总样本充足且没有投影的1,521组。319个一维张量和18个低维卷积核不构成此规则下的高维样本云。
|
||||
|
||||
## 原模型结果(逐组等权,非独立观测)
|
||||
|
||||
|中心|视图|组数|相对均匀球:90%半径内质量差中位数|相对矩匹配均匀盒:质量差中位数|矩匹配且RMS归一后,内外两尾差均>10个百分点的组占比|
|
||||
|---|---|---:|---:|---:|---:|
|
||||
|origin|matrix|517|0.980|0.977|93.8%|
|
||||
|origin|head_rows|1004|0.973|0.973|87.8%|
|
||||
|mean|matrix|517|0.977|0.977|94.4%|
|
||||
|mean|head_rows|1004|0.969|0.969|86.4%|
|
||||
|
||||
这些是描述性质量差,不是显著性检验。正差表示所指定半径以内有更多样本;不能将其自动解释为整条径向分布向内平移。若RMS归一后内外尾均变多,更准确的解释是径向分布更宽。
|
||||
|
||||
|中心/视图|逐组零值率中位数|点坐标能量参与率÷维度,中位数|坐标方差参与率÷维度,中位数|
|
||||
|---|---:|---:|---:|
|
||||
|origin/matrix|0|0.3067|0.9346|
|
||||
|origin/head_rows|0|0.3117|0.9055|
|
||||
|mean/matrix|0|0.3072|0.9346|
|
||||
|mean/head_rows|0|0.3118|0.9055|
|
||||
|
||||
能量参与率衡量坐标能量集中程度,依赖坐标基;有限样本Gram参与率反映样本间几何,另见图。两者均不是精确非零坐标计数,也不是模型功能的有效维度。
|
||||
|
||||
## 微调前后
|
||||
|
||||
原模型是官方Qwen3.5-2B;第二个模型是在同一revision上进行GSM8K GRPO的已有纯LoRA检查点,不是另一个完整模型。72个投影矩阵使用只读W+2BA展开,其余参数共享。972个视图受影响;其中936个属于主分析。未改变视图复用完全相同的分析结果。
|
||||
|
||||
|中心|指标|配对变化中位数|P10|P90|最大绝对变化|
|
||||
|---|---|---:|---:|---:|---:|
|
||||
|origin|median_max|6.30233e-07|-3.20939e-05|3.49557e-05|0.000402869|
|
||||
|origin|median_rms|-2.02235e-06|-4.00888e-05|3.91646e-05|0.000121968|
|
||||
|origin|covariance_participation|0.00011293|-0.0007975|0.00116302|0.0136976|
|
||||
|origin|moment_box_interior90_gap|0|0|0|0|
|
||||
|mean|median_max|3.08748e-07|-3.29473e-05|3.37828e-05|0.000395056|
|
||||
|mean|median_rms|-1.97695e-06|-4.08665e-05|3.90134e-05|0.000306282|
|
||||
|mean|covariance_participation|0.000126928|-0.000792364|0.00112596|0.0127131|
|
||||
|mean|moment_box_interior90_gap|0|0|0|0.00390625|
|
||||
|
||||
## 单独描述的小样本与坐标抽样组
|
||||
|
||||
|模型|中心|类别|组数|中位r/Rcal的组间中位数|相对矩匹配盒的90%半径质量差中位数|
|
||||
|---|---|---|---:|---:|---:|
|
||||
|original|origin|small_sample|36|0.7490|0.7500|
|
||||
|original|origin|projected|1|0.4199|0.9961|
|
||||
|original|mean|small_sample|36|0.8109|0.6875|
|
||||
|original|mean|projected|1|0.4245|0.9961|
|
||||
|finetuned|origin|small_sample|36|0.7490|0.7500|
|
||||
|finetuned|origin|projected|1|0.4199|0.9961|
|
||||
|finetuned|mean|small_sample|36|0.8110|0.6875|
|
||||
|finetuned|mean|projected|1|0.4245|0.9961|
|
||||
|
||||
## 如何读图
|
||||
|
||||
1. `interior_reference_sensitivity`:不同均匀参照下,90%校准半径内质量差的组间CDF。横轴向右表示更多内侧质量。
|
||||
2. `radial_shape_two_tails`:匹配矩且按各自校准RMS归一后的最大正/负CDF差。右上角同时增加两尾,并非单纯向内。
|
||||
3. `finetune_paired_radii`:同一视图的微调前后中位半径;对角线表示不变。
|
||||
4. `participation_reference`:Gram矩阵参与率相对同样本量参照的比值。受样本秩上限影响,不能当作真实内在维度。
|
||||
5. `fixed_role_radial_cdfs`:预先按角色固定的GDN Q、全注意力Q和MLP上投影示例,上排按校准最大半径、下排按RMS归一化。不是按结果挑选最明显的组。
|
||||
|
||||
## 方法与限制
|
||||
|
||||
每组最多512个不同参数向量,一半校准、一半留出;模型固定,留出指未用于拟合中心/半径的参数向量,不是独立训练模型。对照每组512个验证向量,最坏单项二项标准误约2.21个百分点;模型留出样本的误差还取决于样本数,且参数向量不保证独立。10个百分点仅是描述性筛选尺度,没有多重检验或总体显著性结论。
|
||||
|
||||
均匀球、坐标范围均匀盒、匹配均值和逐坐标方差的均匀盒都按同一规则拟合自身中心、最大半径及RMS。均匀球原点中心的精确拟合半径CDF为a^d*ncal/(ncal+1),保留其对数以避免把模拟零命中误报成零概率。盒对照不是协方差匹配。校准最大半径不是已知总体支撑边界;留出越界率已记录。
|
||||
|
||||
`origin`保留坐标均值贡献;`mean`使用校准向量均值。没有使用所有数据来选边界、训练或推理;没有删除模型。坐标稀疏性、有限样本参与率与径向内侧质量分别测量,不能相互代替。该实验不能证明更好的模型更靠里,也不能证明功能重复、可剪枝或可复用。
|
||||
|
||||
复现:`OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze`,完成后 `... python -m model_shell.report`。只在已授权CPU服务器运行。分组协议见 `model_shell/PROTOCOL.md`;全量指标在metrics.csv,完整CDF/参照在profiles.json.gz,实测范数在measured_radii.npz。
|
||||
@@ -0,0 +1,157 @@
|
||||
1/1558 elapsed=25.4s
|
||||
11/1558 elapsed=65.2s
|
||||
21/1558 elapsed=75.1s
|
||||
31/1558 elapsed=84.5s
|
||||
41/1558 elapsed=93.3s
|
||||
51/1558 elapsed=102.0s
|
||||
61/1558 elapsed=115.8s
|
||||
71/1558 elapsed=137.8s
|
||||
81/1558 elapsed=149.6s
|
||||
91/1558 elapsed=159.8s
|
||||
101/1558 elapsed=169.0s
|
||||
111/1558 elapsed=178.5s
|
||||
121/1558 elapsed=193.0s
|
||||
131/1558 elapsed=215.0s
|
||||
141/1558 elapsed=225.0s
|
||||
151/1558 elapsed=234.5s
|
||||
161/1558 elapsed=243.4s
|
||||
171/1558 elapsed=252.9s
|
||||
181/1558 elapsed=262.7s
|
||||
191/1558 elapsed=283.8s
|
||||
201/1558 elapsed=297.3s
|
||||
211/1558 elapsed=305.0s
|
||||
221/1558 elapsed=311.2s
|
||||
231/1558 elapsed=328.1s
|
||||
241/1558 elapsed=337.7s
|
||||
251/1558 elapsed=348.0s
|
||||
261/1558 elapsed=359.5s
|
||||
271/1558 elapsed=368.0s
|
||||
281/1558 elapsed=382.3s
|
||||
291/1558 elapsed=405.5s
|
||||
301/1558 elapsed=414.4s
|
||||
311/1558 elapsed=424.6s
|
||||
321/1558 elapsed=433.5s
|
||||
331/1558 elapsed=442.5s
|
||||
341/1558 elapsed=455.7s
|
||||
351/1558 elapsed=476.6s
|
||||
361/1558 elapsed=485.6s
|
||||
371/1558 elapsed=494.2s
|
||||
381/1558 elapsed=503.4s
|
||||
391/1558 elapsed=512.8s
|
||||
401/1558 elapsed=524.4s
|
||||
411/1558 elapsed=544.7s
|
||||
421/1558 elapsed=557.9s
|
||||
431/1558 elapsed=565.7s
|
||||
441/1558 elapsed=571.8s
|
||||
451/1558 elapsed=586.9s
|
||||
461/1558 elapsed=596.2s
|
||||
471/1558 elapsed=605.1s
|
||||
481/1558 elapsed=614.1s
|
||||
491/1558 elapsed=623.4s
|
||||
501/1558 elapsed=637.7s
|
||||
511/1558 elapsed=660.1s
|
||||
521/1558 elapsed=668.4s
|
||||
531/1558 elapsed=677.4s
|
||||
541/1558 elapsed=686.7s
|
||||
551/1558 elapsed=696.0s
|
||||
561/1558 elapsed=709.0s
|
||||
571/1558 elapsed=730.0s
|
||||
581/1558 elapsed=738.9s
|
||||
591/1558 elapsed=747.4s
|
||||
601/1558 elapsed=757.0s
|
||||
611/1558 elapsed=766.0s
|
||||
621/1558 elapsed=775.5s
|
||||
631/1558 elapsed=797.3s
|
||||
641/1558 elapsed=810.4s
|
||||
651/1558 elapsed=818.2s
|
||||
661/1558 elapsed=823.8s
|
||||
671/1558 elapsed=840.0s
|
||||
681/1558 elapsed=850.1s
|
||||
691/1558 elapsed=859.5s
|
||||
701/1558 elapsed=868.9s
|
||||
711/1558 elapsed=878.3s
|
||||
721/1558 elapsed=891.7s
|
||||
731/1558 elapsed=913.7s
|
||||
741/1558 elapsed=922.2s
|
||||
751/1558 elapsed=931.2s
|
||||
761/1558 elapsed=941.2s
|
||||
771/1558 elapsed=951.8s
|
||||
781/1558 elapsed=968.3s
|
||||
791/1558 elapsed=989.6s
|
||||
801/1558 elapsed=999.1s
|
||||
811/1558 elapsed=1009.5s
|
||||
821/1558 elapsed=1018.5s
|
||||
831/1558 elapsed=1027.4s
|
||||
841/1558 elapsed=1037.3s
|
||||
851/1558 elapsed=1056.4s
|
||||
861/1558 elapsed=1071.4s
|
||||
871/1558 elapsed=1081.0s
|
||||
881/1558 elapsed=1089.9s
|
||||
891/1558 elapsed=1099.9s
|
||||
901/1558 elapsed=1109.1s
|
||||
911/1558 elapsed=1131.1s
|
||||
921/1558 elapsed=1143.5s
|
||||
931/1558 elapsed=1152.5s
|
||||
941/1558 elapsed=1159.0s
|
||||
951/1558 elapsed=1174.1s
|
||||
961/1558 elapsed=1182.3s
|
||||
971/1558 elapsed=1188.2s
|
||||
981/1558 elapsed=1196.0s
|
||||
991/1558 elapsed=1211.6s
|
||||
1001/1558 elapsed=1220.8s
|
||||
1011/1558 elapsed=1230.4s
|
||||
1021/1558 elapsed=1239.1s
|
||||
1031/1558 elapsed=1248.8s
|
||||
1041/1558 elapsed=1267.3s
|
||||
1051/1558 elapsed=1285.1s
|
||||
1061/1558 elapsed=1293.8s
|
||||
1071/1558 elapsed=1303.3s
|
||||
1081/1558 elapsed=1311.9s
|
||||
1091/1558 elapsed=1321.5s
|
||||
1101/1558 elapsed=1338.0s
|
||||
1111/1558 elapsed=1356.8s
|
||||
1121/1558 elapsed=1366.6s
|
||||
1131/1558 elapsed=1375.4s
|
||||
1141/1558 elapsed=1384.4s
|
||||
1151/1558 elapsed=1393.2s
|
||||
1161/1558 elapsed=1406.7s
|
||||
1171/1558 elapsed=1430.1s
|
||||
1181/1558 elapsed=1438.1s
|
||||
1191/1558 elapsed=1444.4s
|
||||
1201/1558 elapsed=1451.6s
|
||||
1211/1558 elapsed=1467.4s
|
||||
1221/1558 elapsed=1477.1s
|
||||
1231/1558 elapsed=1489.0s
|
||||
1241/1558 elapsed=1501.1s
|
||||
1251/1558 elapsed=1513.7s
|
||||
1261/1558 elapsed=1538.7s
|
||||
1271/1558 elapsed=1564.9s
|
||||
1281/1558 elapsed=1576.8s
|
||||
1291/1558 elapsed=1587.4s
|
||||
1301/1558 elapsed=1598.2s
|
||||
1311/1558 elapsed=1610.2s
|
||||
1321/1558 elapsed=1628.9s
|
||||
1331/1558 elapsed=1641.2s
|
||||
1341/1558 elapsed=1649.6s
|
||||
1351/1558 elapsed=1659.0s
|
||||
1361/1558 elapsed=1670.9s
|
||||
1371/1558 elapsed=1680.2s
|
||||
1381/1558 elapsed=1688.9s
|
||||
1391/1558 elapsed=1698.7s
|
||||
1401/1558 elapsed=1709.4s
|
||||
1411/1558 elapsed=1718.6s
|
||||
1421/1558 elapsed=1727.1s
|
||||
1431/1558 elapsed=1736.3s
|
||||
1441/1558 elapsed=1749.4s
|
||||
1451/1558 elapsed=1757.2s
|
||||
1461/1558 elapsed=1765.7s
|
||||
1471/1558 elapsed=1775.4s
|
||||
1481/1558 elapsed=1786.3s
|
||||
1491/1558 elapsed=1794.9s
|
||||
1501/1558 elapsed=1803.7s
|
||||
1511/1558 elapsed=1812.9s
|
||||
1521/1558 elapsed=1826.2s
|
||||
1531/1558 elapsed=1844.9s
|
||||
1541/1558 elapsed=1852.7s
|
||||
1551/1558 elapsed=1858.5s
|
||||
COMPLETE 1558
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"complete_groups": 1558,
|
||||
"profile_rows": 6232,
|
||||
"all_finite": true,
|
||||
"cdf_recomputed": true,
|
||||
"sample_rank_checked": true,
|
||||
"sources": {
|
||||
"base": {
|
||||
"path": "runs/cpu-qwen35-2b/weights/model.safetensors-00001-of-00001.safetensors",
|
||||
"bytes": 4548221488,
|
||||
"sha256": "aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1"
|
||||
},
|
||||
"adapter": {
|
||||
"path": "runs/finetune-head-comparison/adapter.safetensors",
|
||||
"bytes": 18932336,
|
||||
"sha256": "93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6"
|
||||
},
|
||||
"config": {
|
||||
"path": "runs/cpu-qwen35-2b/model_config.json",
|
||||
"bytes": 2908,
|
||||
"sha256": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4"
|
||||
},
|
||||
"adapter_meta": {
|
||||
"path": "runs/finetune-head-comparison/adapter_meta.json",
|
||||
"bytes": 573,
|
||||
"sha256": "b8d4737a5b940bcdfed72258600227a89f883dc863e9724bf84e457b46a53703"
|
||||
}
|
||||
},
|
||||
"files": {
|
||||
"REPORT.md": "43c644d2cf1e1100cb547377e5407db606e04e568bdaa1ba0deb0e9e70fe3ff6",
|
||||
"analysis-output.txt": "a6905d3ec23d68fa4c2ff1b1a4ea6b19e6b94b93aef65ca220dca3823cd87fd4",
|
||||
"environment.json": "7ccc05cb9ee75302982886d1e6746d5e97753ca91c10629a69f6acde68848a41",
|
||||
"example_radii.npz": "787fb0e986ed621604b64e704ce9f7cd6c37a04c70475d1611fcb8f986c27b51",
|
||||
"figures/finetune_paired_radii.pdf": "80a0b5b90b04104bf89ea90676b6694af6ebe4e41101ccf4b771414b870bfa56",
|
||||
"figures/finetune_paired_radii.png": "1ba80adfa87ac69537fa74cbe1c4b2237a5fca9d3a536319bb630fd84b59b534",
|
||||
"figures/finetune_paired_radii.svg": "9f65fa49e08cb93e4467f15c2beb2a499f29267392a531db86ec98e3fce9a689",
|
||||
"figures/fixed_role_radial_cdfs.pdf": "cbb200dcc5f0812181720ad62047a07b4310eea35d32dc0652d891d86eeecac5",
|
||||
"figures/fixed_role_radial_cdfs.png": "c7175229b3d9637774635db8a6fef313bba83351f87c6e6916b8d065b5200891",
|
||||
"figures/fixed_role_radial_cdfs.svg": "09194a9734f496b12f8e699083e494c35ff6f2d95f70fac683aa3e528e0c4684",
|
||||
"figures/interior_reference_sensitivity.pdf": "b6b77c5b02e0a133db884297cbb5f939df82f67db2bccfb0efc142627d032b8f",
|
||||
"figures/interior_reference_sensitivity.png": "f3f612974c19841aa960d14824dfbf713304a60b74af87bade35cfe70dd1f86f",
|
||||
"figures/interior_reference_sensitivity.svg": "b61ee318190c584c389f051023796edc3111cb288d5790059b94e54cdd01e2ac",
|
||||
"figures/participation_reference.pdf": "ea500a0838178b1747fe4d9325c927f8a19557b26de266cd0171ada2bc04da1c",
|
||||
"figures/participation_reference.png": "89782ec43f88874b6b822e91a90ff4b8f5cb40edeae5229286c80781c5336bab",
|
||||
"figures/participation_reference.svg": "f24de314649ea9d8b2b6ce66187515857f5b0eb10e68d8f8e0437fee2fcf6272",
|
||||
"figures/radial_shape_two_tails.pdf": "36d6e9df0db1c1986da949330cc22daf04dac7a3e524778447a68fbd9adac359",
|
||||
"figures/radial_shape_two_tails.png": "3b6149a4b0ef154f58c06f518e8a6a972f1ff6c913d0058c5698e64ae74213eb",
|
||||
"figures/radial_shape_two_tails.svg": "f665d434b5511ad480a32a565e0ce433a1dfaf6b9855b6caa524b0fab24a1a75",
|
||||
"inventory.json": "fe7c88d5d4ae58dcd45709b44041e3d216953fae64dfc4e22961b7051727f83a",
|
||||
"measured_radii.npz": "bc2a8298aa94e3c9481474f91400f16a119a5083f8112145727cbab080ebd075",
|
||||
"metrics.csv": "d4176fa88b9126c00ba7e56bb54a031828a7481e9049a03c5058c58d0e8c1c30",
|
||||
"profiles.json.gz": "7a97cf2dccd4993e2e5318ace39536bbd592d133bc3d393e7631f3911e500faf",
|
||||
"summary.json": "7320790885813d2afa9b688685efcf8c9ed72ef7f7e97557658602cf2008eef2",
|
||||
"test-output.txt": "3794ae58181c17343dc3660a54b70c53f4f75dbd25fa6637e5f185553fe094d4"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"python": "3.14.4 (main, Aug 20 2026, 10:41:58) [GCC 15.2.0]",
|
||||
"platform": "Linux-7.0.0-31-generic-x86_64-with-glibc2.43",
|
||||
"numpy": "2.5.4",
|
||||
"matplotlib": "3.11.2",
|
||||
"source_code_sha256": {
|
||||
"model_shell/analyze.py": "e791368b5ef9094ee6b3a1d4603fe11b0161b647353fa4d25c52aadfae914fe2",
|
||||
"model_shell/groups.py": "eafbce2f0eeac89cfd3c63c57e91d1b7a260f4de4a1dd38be0aaceda5f317b15",
|
||||
"model_shell/report.py": "af0617373a2b06cbe5728330157c1d2e60615eb7577cf3dd95be699a4b27ae19",
|
||||
"model_shell/test_analysis.py": "a837d787d00e5a6302ae46ca4079bbaabe3f548cc71705cce514c92acd0943e7"
|
||||
},
|
||||
"protocol_sha256": "dfece26a7f2779c511d7a8f9a6c62c1ed49fec2dc5a3aa8c112681a52189d366",
|
||||
"seed": "first 8 SHA256 bytes, little endian; group id plus purpose suffix",
|
||||
"thread_limit": 1,
|
||||
"server": "8.211.154.134",
|
||||
"training": false,
|
||||
"inference": false,
|
||||
"gpu": false
|
||||
}
|
||||
|
After Width: | Height: | Size: 136 KiB |
|
After Width: | Height: | Size: 314 KiB |
|
After Width: | Height: | Size: 240 KiB |
|
After Width: | Height: | Size: 162 KiB |
|
After Width: | Height: | Size: 167 KiB |
|
After Width: | Height: | Size: 111 KiB |
|
After Width: | Height: | Size: 114 KiB |
|
After Width: | Height: | Size: 89 KiB |
|
After Width: | Height: | Size: 293 KiB |
|
After Width: | Height: | Size: 500 KiB |
@@ -0,0 +1,246 @@
|
||||
{
|
||||
"strata": {
|
||||
"origin/matrix": {
|
||||
"n": 517,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.30673555734079716,
|
||||
"median_coordinate_variance_participation_fraction": 0.934645955851128,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.98046875,
|
||||
"fraction_gap_gt10pp": 0.9806576402321083,
|
||||
"median_positive_rms_gap": 0.37109375,
|
||||
"median_negative_rms_gap": 0.3515625,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9516441005802708,
|
||||
"median_participation_ratio": 0.8297024600513981
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9806576402321083,
|
||||
"median_positive_rms_gap": 0.32421875,
|
||||
"median_negative_rms_gap": 0.310546875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9090909090909091,
|
||||
"median_participation_ratio": 1.3431227073360341
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.9765625,
|
||||
"fraction_gap_gt10pp": 0.9806576402321083,
|
||||
"median_positive_rms_gap": 0.34375,
|
||||
"median_negative_rms_gap": 0.330078125,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9381044487427466,
|
||||
"median_participation_ratio": 0.8524481165697501
|
||||
}
|
||||
},
|
||||
"origin/head_rows": {
|
||||
"n": 1004,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.31172156343430024,
|
||||
"median_coordinate_variance_participation_fraction": 0.9055279134912191,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.97265625,
|
||||
"fraction_gap_gt10pp": 0.99800796812749,
|
||||
"median_positive_rms_gap": 0.578125,
|
||||
"median_negative_rms_gap": 0.265625,
|
||||
"fraction_two_sided_rms_gt10pp": 0.896414342629482,
|
||||
"median_participation_ratio": 0.7070664507223768
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.99800796812749,
|
||||
"median_positive_rms_gap": 0.55078125,
|
||||
"median_negative_rms_gap": 0.244140625,
|
||||
"fraction_two_sided_rms_gt10pp": 0.852589641434263,
|
||||
"median_participation_ratio": 1.1235217859798587
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.97265625,
|
||||
"fraction_gap_gt10pp": 0.99800796812749,
|
||||
"median_positive_rms_gap": 0.56640625,
|
||||
"median_negative_rms_gap": 0.251953125,
|
||||
"fraction_two_sided_rms_gt10pp": 0.8784860557768924,
|
||||
"median_participation_ratio": 0.7369770671990965
|
||||
}
|
||||
},
|
||||
"mean/matrix": {
|
||||
"n": 517,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.30718321338258303,
|
||||
"median_coordinate_variance_participation_fraction": 0.934645955851128,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.9765625,
|
||||
"fraction_gap_gt10pp": 0.9787234042553191,
|
||||
"median_positive_rms_gap": 0.46484375,
|
||||
"median_negative_rms_gap": 0.37109375,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9922630560928434,
|
||||
"median_participation_ratio": 0.8366753780099343
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.97265625,
|
||||
"fraction_gap_gt10pp": 0.9787234042553191,
|
||||
"median_positive_rms_gap": 0.3359375,
|
||||
"median_negative_rms_gap": 0.32421875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9303675048355899,
|
||||
"median_participation_ratio": 0.868224555240548
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.9765625,
|
||||
"fraction_gap_gt10pp": 0.9787234042553191,
|
||||
"median_positive_rms_gap": 0.33984375,
|
||||
"median_negative_rms_gap": 0.3359375,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9439071566731141,
|
||||
"median_participation_ratio": 0.8465704560846329
|
||||
}
|
||||
},
|
||||
"mean/head_rows": {
|
||||
"n": 1004,
|
||||
"median_zero_fraction": 0.0,
|
||||
"median_coordinate_energy_participation_fraction": 0.31181881259530586,
|
||||
"median_coordinate_variance_participation_fraction": 0.9055279134912191,
|
||||
"ball": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9970119521912351,
|
||||
"median_positive_rms_gap": 0.62109375,
|
||||
"median_negative_rms_gap": 0.296875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.9322709163346613,
|
||||
"median_participation_ratio": 0.7143476292983669
|
||||
},
|
||||
"range_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9970119521912351,
|
||||
"median_positive_rms_gap": 0.560546875,
|
||||
"median_negative_rms_gap": 0.248046875,
|
||||
"fraction_two_sided_rms_gt10pp": 0.8615537848605578,
|
||||
"median_participation_ratio": 0.7194449549938213
|
||||
},
|
||||
"moment_box": {
|
||||
"median_interior90_gap": 0.96875,
|
||||
"fraction_gap_gt10pp": 0.9970119521912351,
|
||||
"median_positive_rms_gap": 0.5625,
|
||||
"median_negative_rms_gap": 0.25,
|
||||
"fraction_two_sided_rms_gt10pp": 0.8635458167330677,
|
||||
"median_participation_ratio": 0.7159821443435122
|
||||
}
|
||||
}
|
||||
},
|
||||
"paired_changed": {
|
||||
"origin": {
|
||||
"n": 936,
|
||||
"median_max": {
|
||||
"median": 6.302327853480172e-07,
|
||||
"p10": -3.2093888658157876e-05,
|
||||
"p90": 3.49557393942157e-05,
|
||||
"max_abs": 0.0004028694837374669
|
||||
},
|
||||
"median_rms": {
|
||||
"median": -2.0223542534503203e-06,
|
||||
"p10": -4.008879155420475e-05,
|
||||
"p90": 3.916460868902405e-05,
|
||||
"max_abs": 0.00012196820638044059
|
||||
},
|
||||
"covariance_participation": {
|
||||
"median": 0.00011293028622105794,
|
||||
"p10": -0.0007975000317692604,
|
||||
"p90": 0.0011630175829182576,
|
||||
"max_abs": 0.01369755006576412
|
||||
},
|
||||
"moment_box_interior90_gap": {
|
||||
"median": 0.0,
|
||||
"p10": 0.0,
|
||||
"p90": 0.0,
|
||||
"max_abs": 0.0
|
||||
}
|
||||
},
|
||||
"mean": {
|
||||
"n": 936,
|
||||
"median_max": {
|
||||
"median": 3.087481980079687e-07,
|
||||
"p10": -3.294732610958917e-05,
|
||||
"p90": 3.3782823366856896e-05,
|
||||
"max_abs": 0.0003950555852355597
|
||||
},
|
||||
"median_rms": {
|
||||
"median": -1.9769477522801893e-06,
|
||||
"p10": -4.086651887480475e-05,
|
||||
"p90": 3.901337185857612e-05,
|
||||
"max_abs": 0.00030628174134661457
|
||||
},
|
||||
"covariance_participation": {
|
||||
"median": 0.00012692754527421357,
|
||||
"p10": -0.000792364212092167,
|
||||
"p90": 0.0011259624874320195,
|
||||
"max_abs": 0.012713060927119102
|
||||
},
|
||||
"moment_box_interior90_gap": {
|
||||
"median": 0.0,
|
||||
"p10": 0.0,
|
||||
"p90": 0.0,
|
||||
"max_abs": 0.00390625
|
||||
}
|
||||
}
|
||||
},
|
||||
"secondary": [
|
||||
{
|
||||
"model": "original",
|
||||
"center": "origin",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.7489558868770938,
|
||||
"moment_interior90_gap": 0.75
|
||||
},
|
||||
{
|
||||
"model": "original",
|
||||
"center": "origin",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.41986786500966855,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
},
|
||||
{
|
||||
"model": "original",
|
||||
"center": "mean",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.8109468212379675,
|
||||
"moment_interior90_gap": 0.6875
|
||||
},
|
||||
{
|
||||
"model": "original",
|
||||
"center": "mean",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.4244519268106477,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "origin",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.7489607494253279,
|
||||
"moment_interior90_gap": 0.75
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "origin",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.41986786500966855,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "mean",
|
||||
"stratum": "small_sample",
|
||||
"n": 36,
|
||||
"median_max": 0.8110356927186851,
|
||||
"moment_interior90_gap": 0.6875
|
||||
},
|
||||
{
|
||||
"model": "finetuned",
|
||||
"center": "mean",
|
||||
"stratum": "projected",
|
||||
"n": 1,
|
||||
"median_max": 0.4244519268106477,
|
||||
"moment_interior90_gap": 0.99609375
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
test_ball_fit_and_sparse_counterexample (model_shell.test_analysis.Checks.test_ball_fit_and_sparse_counterexample) ... ok
|
||||
test_center_scale_and_rank (model_shell.test_analysis.Checks.test_center_scale_and_rank) ... ok
|
||||
test_reproducible_controls (model_shell.test_analysis.Checks.test_reproducible_controls) ... ok
|
||||
test_sampling_and_lora (model_shell.test_analysis.Checks.test_sampling_and_lora) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 4 tests in 0.394s
|
||||
|
||||
OK
|
||||