Archive research charts and compare GSM8K LoRA head concentration

2026-11-11 00:06:15 +08:00
parent d6d985487c
commit 52e6d13fe5
142 changed files with 18505 additions and 0 deletions
@@ -0,0 +1,39 @@
+# 研究图册：Qwen3.5-2B 头贡献集中度
+
+这份图册与原始PNG/PDF/SVG一同保存在Git中。图片文件已随PR #6合并到main；以下固定提交链接不依赖聊天附件或本机路径。
+
+原始结果提交：`098ea8650c7d0e06339c66e3a0509e38b85b3d7d`；模型revision：`15852e8c16360a2fea060d615a32b45270f8a8fc`。
+
+| 基线指标（逐观测中位数） | QKV复合变化 | √JS检索距离 |
+|---|---:|---:|
+| 有效头数÷总头数 | 76.0% | 55.8% |
+| 最强25%头的影响份额 | 44.6% | 56.5% |
+| 最弱25%头的影响份额 | 10.6% | 6.8% |
+
+**√JS有效头数：8.931822/16；P10=5.784081，P90=11.672658。** 这是消融影响的集中程度，不是注意力概率质量。原基线为4条文本各32 token，普通注意力接收层的query7–31。
+
+## 连续分布与尺度敏感性
+
+![连续分布与尺度敏感性](../results/head-dominance/figures/retrieval_scale_sensitivity.png)
+
+[PNG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.png)
+[PDF固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.pdf)
+[SVG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.svg)
+
+## 联合强弱头消融
+
+![联合强弱头消融](../results/head-dominance/figures/joint_removal_effects.png)
+
+[PNG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.png)
+[PDF固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.pdf)
+[SVG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.svg)
+
+[完整基线报告](../results/head-dominance/REPORT.md) · [全部图表目录](../results/head-dominance/figures) · [图册SHA清单](figure-gallery-manifest.json)
+
+使用√JS距离进行微调前后配对比较；保留JS散度作为尺度敏感性参考，避免混淆平方尺度与距离尺度。
+
+## GSM8K微调前后配对图
+
+[配对报告](../results/finetune-comparison/REPORT.md) · [CDF](../results/finetune-comparison/figures/paired_effective_heads_cdf.png) · [逐层分布](../results/finetune-comparison/figures/paired_effective_heads_by_layer.png) · [配对变化](../results/finetune-comparison/figures/paired_effective_heads_delta.png)
+
+同目录保留PDF/SVG。图表随代码、原始数据和SHA清单一同提交，不依赖聊天附件。
@@ -0,0 +1,48 @@
+{
+  "source_commit": "098ea8650c7d0e06339c66e3a0509e38b85b3d7d",
+  "base_model_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "sqrt_js_effective_heads": {
+    "median": 8.93182211792415,
+    "p10": 5.7840811612328125,
+    "p90": 11.67265834734814,
+    "donor_heads": 16
+  },
+  "files": [
+    {
+      "path": "results/head-dominance/figures/retrieval_scale_sensitivity.png",
+      "sha256": "4f2dac5fdb5506ee5282d459dcb86cc6b58fac05d494d6409c29ea6924365add",
+      "immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.png",
+      "http_status": 200,
+      "remote_sha_verified": true,
+      "content_type": "image/png"
+    },
+    {
+      "path": "results/head-dominance/figures/retrieval_scale_sensitivity.pdf",
+      "sha256": "56a2f5655196018f4310dbccb29cb55c314111007598491496c9a36256958525",
+      "immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.pdf"
+    },
+    {
+      "path": "results/head-dominance/figures/retrieval_scale_sensitivity.svg",
+      "sha256": "f51803e68df40df6dda91bce7236c999f405ccf30be56e7819242147a5e9a5b8",
+      "immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.svg"
+    },
+    {
+      "path": "results/head-dominance/figures/joint_removal_effects.png",
+      "sha256": "7dc26da9fda80e62c6445c5dee29fe52c06146dea0cd65d444755cf0eb5bb729",
+      "immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.png",
+      "http_status": 200,
+      "remote_sha_verified": true,
+      "content_type": "image/png"
+    },
+    {
+      "path": "results/head-dominance/figures/joint_removal_effects.pdf",
+      "sha256": "b35132d3e1cf3d202bc6695e64f64c9e495196bc6c9a92659ccf65aceb566040",
+      "immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.pdf"
+    },
+    {
+      "path": "results/head-dominance/figures/joint_removal_effects.svg",
+      "sha256": "03e6b299c1dbfa703ac72cbc7e9749e3173b649a22d829316dd5b5dccbe8728d",
+      "immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.svg"
+    }
+  ]
+}