Compare commits
9 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Zayden-Jung
|
ea9bfd2a23 | ||
 codex-bot
|
2f1113323e | ||
|
Zayden-Jung
|
35090663e2 | ||
|
codex-bot
|
e1514f4952 | ||
|
Zayden-Jung
|
9e0e659287 | ||
|
codex-bot
|
52e6d13fe5 | ||
|
Zayden-Jung
|
d6d985487c | ||
|
codex-bot
|
098ea8650c | ||
|
Zayden-Jung
|
cd260a2483 |
@@ -4,3 +4,6 @@ results/** -text
|
||||
results/**/*.csv -whitespace
|
||||
results/**/*.svg -whitespace
|
||||
results/**/*.log -whitespace
|
||||
|
||||
# Preserve captured scientific stdout byte-for-byte, including blank final lines.
|
||||
results/head-dominance/*-output.txt -whitespace
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Model parameter similarity experiments
|
||||
|
||||
**[研究图册与固定在线副本](docs/FIGURE_GALLERY.md)**：连续分布、联合消融及原始PDF/SVG。
|
||||
|
||||
[两个实际2B检查点的参数球壳结论](docs/MODEL_SHELL_FINDINGS.md) · [完整数值报告](results/model-shell/REPORT.md) · [固定分组和均匀参照协议](model_shell/PROTOCOL.md)。
|
||||
|
||||
Research on repeated learned functions in **Qwen3.5-2B**: a completed static weight baseline
|
||||
and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm
|
||||
functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks.
|
||||
@@ -42,3 +46,16 @@ Full resolved versions and run provenance accompany each result. See the SOP for
|
||||
## CPU block activation measurements
|
||||
|
||||
See [measurement definitions](activation/README.md) and [observed activation report](results/block-activation/REPORT.md) for all 24 layers, contiguous output-channel blocks, exact/near-zero rates and output energy concentration. This is an observational SiLU-gated study, not a pruning or reuse quality claim.
|
||||
|
||||
## Adjacent-layer head influence concentration
|
||||
|
||||
The CPU study measures all 23 adjacent transitions, observation-wise effective
|
||||
head counts and Q/K/V influence shares, with equal-size weak/strong/random joint
|
||||
removal controls. See [definitions and reproduction](dominance/README.md),
|
||||
[fixed protocol](dominance/PROTOCOL.md), and [measured report](results/head-dominance/REPORT.md).
|
||||
These are descriptive intervention effects, not additive contribution probabilities
|
||||
or validated final-LM pruning gains.
|
||||
|
||||
## Existing GSM8K task-finetuning comparison
|
||||
|
||||
[Paired study and reproduction](finetune_compare/README.md) · [Fixed protocol](finetune_compare/PROTOCOL.md) · [Results](results/finetune-comparison/REPORT.md). This compares an existing pure-LoRA GRPO checkpoint to exactly the same original release; no training is performed here.
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
# 研究图册：Qwen3.5-2B 参数几何与头贡献
|
||||
|
||||
这份图册与原始PNG/PDF/SVG一同保存在Git中。头贡献基线图片已随PR #6合并到main；其固定提交链接不依赖聊天附件或本机路径。后续实验分别列于下方，可通过图册所在的Git提交查看对应版本。
|
||||
|
||||
原始结果提交：`098ea8650c7d0e06339c66e3a0509e38b85b3d7d`；模型revision：`15852e8c16360a2fea060d615a32b45270f8a8fc`。
|
||||
|
||||
| 基线指标（逐观测中位数） | QKV复合变化 | √JS检索距离 |
|
||||
|---|---:|---:|
|
||||
| 有效头数÷总头数 | 76.0% | 55.8% |
|
||||
| 最强25%头的影响份额 | 44.6% | 56.5% |
|
||||
| 最弱25%头的影响份额 | 10.6% | 6.8% |
|
||||
|
||||
**√JS有效头数：8.931822/16；P10=5.784081，P90=11.672658。** 这是消融影响的集中程度，不是注意力概率质量。原基线为4条文本各32 token，普通注意力接收层的query7–31。
|
||||
|
||||
## 连续分布与尺度敏感性
|
||||
|
||||
|
||||
|
||||
[PNG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.png)
|
||||
[PDF固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.pdf)
|
||||
[SVG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.svg)
|
||||
|
||||
## 联合强弱头消融
|
||||
|
||||
|
||||
|
||||
[PNG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.png)
|
||||
[PDF固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.pdf)
|
||||
[SVG固定提交副本](https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.svg)
|
||||
|
||||
[完整基线报告](../results/head-dominance/REPORT.md) · [全部图表目录](../results/head-dominance/figures) · [图册SHA清单](figure-gallery-manifest.json)
|
||||
|
||||
使用√JS距离进行微调前后配对比较；保留JS散度作为尺度敏感性参考，避免混淆平方尺度与距离尺度。
|
||||
|
||||
## GSM8K微调前后配对图
|
||||
|
||||
[配对报告](../results/finetune-comparison/REPORT.md) · [CDF](../results/finetune-comparison/figures/paired_effective_heads_cdf.png) · [逐层分布](../results/finetune-comparison/figures/paired_effective_heads_by_layer.png) · [配对变化](../results/finetune-comparison/figures/paired_effective_heads_delta.png)
|
||||
|
||||
同目录保留PDF/SVG。图表随代码、原始数据和SHA清单一同提交，不依赖聊天附件。
|
||||
|
||||
## 高维壳与稀疏性对照
|
||||
|
||||
|
||||
|
||||
[条件定理、反例与数据说明](../results/sparse-shell/REPORT.md) · [PDF](../results/sparse-shell/sparse_shell_controls.pdf) · [SVG](../results/sparse-shell/sparse_shell_controls.svg)
|
||||
|
||||
## 两个实际2B检查点的参数球壳检验
|
||||
|
||||
[结论与解释](MODEL_SHELL_FINDINGS.md) · [实测报告、分组数量与限制](../results/model-shell/REPORT.md) · [固定协议](../model_shell/PROTOCOL.md) · [指标源表](../results/model-shell/metrics.csv) · [校验清单](../results/model-shell/audit.json)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
[微调前后配对图](../results/model-shell/figures/finetune_paired_radii.png) · [有限样本参与率](../results/model-shell/figures/participation_reference.png)。每幅图同目录均保存PDF/SVG，完整CDF及参照数据保存在`profiles.json.gz`，实测范数保存在`measured_radii.npz`；均随本次成果进入Git。
|
||||
@@ -0,0 +1,50 @@
|
||||
# 实际2B参数：比均匀参照更靠里吗？
|
||||
|
||||
**在明确的“校准最大半径以内”口径下，是；但数据同时显示更宽的径向分布，不能概括为整团参数向内收缩。此次GSM8K微调几乎没有改变这些几何指标。**
|
||||
|
||||
这轮直接读取了保留的Qwen3.5-2B权重和同一底座的已有纯LoRA GRPO检查点，在8.211.154.134上使用CPU完成。没有训练、推理或删除权重。
|
||||
|
||||
## 一个模型有多少组高维分布？
|
||||
|
||||
按事先固定的参数矩阵行／列与语言注意力头视图规则，每个模型有 **1,558组**：554个矩阵视图、1,004个按头视图。它们有重叠，不是模型固有的唯一分布数量。
|
||||
|
||||
主要汇总使用1,521个样本充足、未抽取坐标的视图。另有36个小样本视图和1个坐标抽样视图，全部单列并保留数据。整个检查点包括语言、视觉与MTP，共632个参数张量；319个一维张量和18个低维卷积核不满足这次“高维向量云”的规则。
|
||||
|
||||
## 有更多内侧质量，但也有更长的外尾
|
||||
|
||||
每组最多512个参数向量，一半校准中心及半径，另一半测量。对照也使用同样的校准规则；没有用所有向量的最大范数事后包住全部数据。
|
||||
|
||||
以原点为中心，比较归一化半径 `r / R_cal <= 0.9`：
|
||||
|
||||
|视图|组数|相对均匀球的内侧质量差中位数|相对匹配均值、逐坐标方差的均匀盒的质量差中位数|
|
||||
|---|---:|---:|---:|
|
||||
|矩阵行／列|517|+98.0个百分点|+97.7个百分点|
|
||||
|按头|1,004|+97.3个百分点|+97.3个百分点|
|
||||
|
||||
改用校准均值作为中心，结果仍有约96.9–97.7个百分点的差异。因此，这个现象并非仅来自中心放在原点。
|
||||
|
||||
但最大半径很容易受到少数大范数向量影响。换成各自的校准RMS尺度，并比较完整CDF后，**93.8%的矩阵视图和87.8%的头视图同时存在超过10个百分点的内尾、外尾差异**。去均值后分别为94.4%和86.4%。这说明两尾同时增加是普遍现象，比“所有向量整体更靠内”更符合这些数据。
|
||||
|
||||
|
||||
|
||||
这三组示例按模块角色固定：GDN的Q头、全注意力的Q头、MLP上投影行。上排用校准最大半径，下排用RMS；它们只是帮助读图，不代替全量统计。
|
||||
|
||||
## 是否说明稀疏或低维？
|
||||
|
||||
还不能如此推断。采样视图的逐组精确零值率中位数为0；点坐标能量参与率除以维度的中位数约0.31。径向内侧质量、坐标能量集中和样本间的有效维数是不同的测量对象。
|
||||
|
||||
本轮同时记录了有限样本Gram参与率，并与同样本数的均匀参照比较。它受校准样本秩上限限制，不能作为真实内在维度的直接估计。盒参照也只匹配边际范围或逐坐标矩，并未匹配相关结构。因此，本轮没有单独识别“稀疏支持”导致的效应。
|
||||
|
||||
## 微调前后改变了吗？
|
||||
|
||||
LoRA实际改变72个投影矩阵，影响972个视图，其中936个属于主要汇总。只对这些受影响的主要视图配对，以免大量未改动参数稀释差异。
|
||||
|
||||
原点中心下，逐组中位 `r/R_cal` 的配对变化中位数为 **6.30×10⁻⁷**，最大绝对变化 **4.03×10⁻⁴**。去均值后的结论相近。90%半径质量差在原点中心口径下未跨越当前经验CDF的分辨率；这不表示两套权重完全相同。
|
||||
|
||||
因此，这两个检查点不能支持“微调后、更好的模型更靠内”的结论；本轮也没有测任务质量。参数几何中的宽径向结构可以继续挖掘，但它本身还不是功能重复或可互相替代的证据。
|
||||
|
||||
## 数据与核验
|
||||
|
||||
[完整数值报告](../results/model-shell/REPORT.md) · [冻结协议](../model_shell/PROTOCOL.md) · [复现方式](../model_shell/README.md) · [指标CSV](../results/model-shell/metrics.csv) · [摘要JSON](../results/model-shell/summary.json) · [审计SHA](../results/model-shell/audit.json)
|
||||
|
||||
6,232个模型×中心×视图组合全部核验；CDF从保留的校准／留出范数复算，检查有限样本秩上限，原模型和适配器SHA均与既有记录一致。远程4项测试通过，覆盖索引、LoRA读取、中心与尺度、均匀球校准和稀疏贴壳反例。图表、完整CDF及对照摘要、实测范数一同保存。结论为描述性统计：视图相关且重叠，没有将其当作独立重复实验进行显著性检验。
|
||||
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"source_commit": "098ea8650c7d0e06339c66e3a0509e38b85b3d7d",
|
||||
"base_model_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
|
||||
"sqrt_js_effective_heads": {
|
||||
"median": 8.93182211792415,
|
||||
"p10": 5.7840811612328125,
|
||||
"p90": 11.67265834734814,
|
||||
"donor_heads": 16
|
||||
},
|
||||
"files": [
|
||||
{
|
||||
"path": "results/head-dominance/figures/retrieval_scale_sensitivity.png",
|
||||
"sha256": "4f2dac5fdb5506ee5282d459dcb86cc6b58fac05d494d6409c29ea6924365add",
|
||||
"immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.png",
|
||||
"http_status": 200,
|
||||
"remote_sha_verified": true,
|
||||
"content_type": "image/png"
|
||||
},
|
||||
{
|
||||
"path": "results/head-dominance/figures/retrieval_scale_sensitivity.pdf",
|
||||
"sha256": "56a2f5655196018f4310dbccb29cb55c314111007598491496c9a36256958525",
|
||||
"immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.pdf"
|
||||
},
|
||||
{
|
||||
"path": "results/head-dominance/figures/retrieval_scale_sensitivity.svg",
|
||||
"sha256": "f51803e68df40df6dda91bce7236c999f405ccf30be56e7819242147a5e9a5b8",
|
||||
"immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/retrieval_scale_sensitivity.svg"
|
||||
},
|
||||
{
|
||||
"path": "results/head-dominance/figures/joint_removal_effects.png",
|
||||
"sha256": "7dc26da9fda80e62c6445c5dee29fe52c06146dea0cd65d444755cf0eb5bb729",
|
||||
"immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.png",
|
||||
"http_status": 200,
|
||||
"remote_sha_verified": true,
|
||||
"content_type": "image/png"
|
||||
},
|
||||
{
|
||||
"path": "results/head-dominance/figures/joint_removal_effects.pdf",
|
||||
"sha256": "b35132d3e1cf3d202bc6695e64f64c9e495196bc6c9a92659ccf65aceb566040",
|
||||
"immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.pdf"
|
||||
},
|
||||
{
|
||||
"path": "results/head-dominance/figures/joint_removal_effects.svg",
|
||||
"sha256": "03e6b299c1dbfa703ac72cbc7e9749e3173b649a22d829316dd5b5dccbe8728d",
|
||||
"immutable_url": "https://gitea.entroia.com/Zayden-Jung/exp-model-parameter-similarity/raw/commit/098ea8650c7d0e06339c66e3a0509e38b85b3d7d/results/head-dominance/figures/joint_removal_effects.svg"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
# Head influence concentration: measurement protocol
|
||||
|
||||
Target: quantify the statistical distribution of large and small upstream head
|
||||
influences on the next layer's Q/K/V; identity of dominant heads may vary at each
|
||||
sample, query token and recipient head. No training, GPU, deletion or model edits.
|
||||
|
||||
Scope: all 23 adjacent layer transitions of the retained Qwen3.5-2B text model,
|
||||
including 16-head linear attention and 8-head full attention. Measure effective
|
||||
Q/K/V, and retrieval probability changes where the recipient uses full attention.
|
||||
The intervening source-layer residual/MLP and recipient input normalization must
|
||||
be recomputed after interventions. Residual stream is preserved: removing an
|
||||
attention head is not removing all information reaching the recipient.
|
||||
|
||||
For each observation, remove one upstream attention head's projected contribution
|
||||
at a time. Record nonnegative influence magnitude d_i and its absolute scale.
|
||||
When sum(d_i)>0, p_i=d_i/sum(d_i) is normalized intervention influence, NOT an
|
||||
additive causal attribution or a native attention probability over upstream heads.
|
||||
All-zero influence has undefined concentration and must be flagged, not assigned
|
||||
uniform or concentrated values. Record small total influences explicitly.
|
||||
|
||||
Primary continuous measures:
|
||||
- Effective head count: N_eff=1/sum(p_i^2), range [1,H].
|
||||
- Effective head fraction: N_eff/H, supports different H.
|
||||
- Normalized concentration: C=(sum(p_i^2)-1/H)/(1-1/H), range [0,1].
|
||||
These are related summaries, not independent evidence. Report distributions over
|
||||
texts/tokens/recipient heads/layers, not only concentration of globally averaged
|
||||
influences. Primary query positions 7 onward avoid trivial short-prefix attention.
|
||||
|
||||
Secondary intuitive measures: top1/top25%/top50% influence share, bottom25%
|
||||
share, and number of heads reaching 90% influence. Preserve raw influence tables.
|
||||
No RMS activity threshold is used to define influence concentration.
|
||||
|
||||
Joint interventions are required: per observation's weak-head set and strong-head
|
||||
set must be removed together and measured, because individual ablations are not
|
||||
additive and may conceal cancellation or synergy. Rankings may be input-specific;
|
||||
these experiments cannot imply a universally removable fixed head set. Distinguish
|
||||
local next-layer QKV/probability effects from unmeasured final LM quality.
|
||||
|
||||
CPU measurements, scientific plots, tests and audits execute on the previously
|
||||
authorized server 8.211.154.134. All weights and existing caches remain retained.
|
||||
|
||||
## Fixed execution details (before real-model measurements)
|
||||
|
||||
Four texts are reused from crossqkv/fresh_samples.json (two Chinese/two English),
|
||||
32 tokens each; no independent generalization claim. All tokens are computed,
|
||||
queries7..31 are primary statistical observations. The observer caches source
|
||||
residual inputs and exact per-head projected output components; their sum is
|
||||
checked against the native branch output. First text's full forward is checked
|
||||
against the canonical streaming forward, and every transition has a null-mask
|
||||
control. No cached tensor is deleted.
|
||||
|
||||
For each query/recipient query-head, individual influence is the L2 change divided
|
||||
by that same baseline vector's L2 (floor1e-12). Q,K,V are analyzed separately.
|
||||
A declared balanced score sqrt((relative_Q^2+relative_K^2+relative_V^2)/3) is also
|
||||
analyzed; it does not treat their different physical scales as directly additive.
|
||||
Full attention: Q/K are measured after learned norm/partial RoPE, V before output
|
||||
mixing, and actual softmax retrieval JS is measured. Linear attention: Q/K/V are
|
||||
after causal convolution+SiLU; Q/K include the delta-rule L2 normalization and Q
|
||||
scale. It has no ordinary softmax retrieval distribution. Full-attention GQA K/V
|
||||
are repeated to query-head slots for aligned joint tests; these slots are not
|
||||
independent KV heads, and statistical observations are not independent samples.
|
||||
|
||||
Joint weak-half/strong-half/random-half/all-head removal uses a separate coherent
|
||||
sequence for each recipient query-head. At each source token, choose the donor
|
||||
half by that recipient query-head's QKV influence ranking at the same token.
|
||||
Recompute full recipient sequence; convolution and QK retrieval can therefore
|
||||
propagate changes between positions. Only the intended recipient head is scored
|
||||
for each scenario. This is an input-adaptive diagnostic, not a computationally
|
||||
free head-selection policy. Random uses fixed seed31000+sample_index*100+layer.
|
||||
Compare joint effects to sums of corresponding single effects but do not assume
|
||||
additivity. For JS use the same joint sets (selected by QKV, not by JS).
|
||||
|
||||
No success criterion assumes that dominance exists. Report effective-head and
|
||||
absolute-effect distributions even if contributions are diffuse. No final-LM
|
||||
NLL or deployable pruning quality claim. Scientific output includes all raw
|
||||
single/joint effects and deletion masks; large baseline caches remain external.
|
||||
@@ -0,0 +1,75 @@
|
||||
# Statistical head influence dominance (CPU)
|
||||
|
||||
This study answers how concentrated the **distribution of measured upstream-head
|
||||
influences** is, without assuming that dominant head identities are fixed across
|
||||
texts, positions or recipient heads. Read [registered protocol](PROTOCOL.md).
|
||||
|
||||
- `core.py`: normalized effective Q/K/V extraction for both attention types,
|
||||
source projected head decomposition, exact nonlinear downstream recomputation,
|
||||
coherent masks, continuous concentration and JS.
|
||||
- `run.py`: four fixed prior texts, all23 adjacent connections, single-head and
|
||||
weak/strong/random-half/all-head interventions, per-transition checkpointing,
|
||||
canonical-forward/null controls, preserved baseline caches.
|
||||
- `report.py`: full saved-data distributions, per-transition CSV, continuous
|
||||
effective head plots, influence rank curves, joint tests and Chinese report.
|
||||
- `audit.py`: independently recomputes concentration/masks from raw effects,
|
||||
validates scope/null/head sums, and hashes retained caches, source and results.
|
||||
- `tests/test_dominance.py`: continuous limiting cases, undefined zero-total,
|
||||
token-dependent masks, JS identity, both canonical mixed-layer paths, and joint
|
||||
interventions on a small checkpoint with known arrays.
|
||||
|
||||
Influence for Q/K/V is relative L2 change after deleting one whole donor attention
|
||||
head (at all sequence positions), preserving the prior residual and recomputing
|
||||
source MLP + recipient normalization. Source **output/value heads** are the units:
|
||||
16 for linear attention,8 for full attention. These are not arbitrary individual
|
||||
Q/K/V weight blocks. For a recipient query-head at one token, normalize donor
|
||||
influences to p. Primary continuous summary is N_eff=1/sum(p²), and N_eff/H.
|
||||
C=(sum(p²)-1/H)/(1-1/H) is a related0..1 concentration score. These are not
|
||||
independent confirmations. No RMS near-zero threshold or R fitting is used.
|
||||
|
||||
Q/K/V are separate primary scales; balanced QKV score is RMS of their three
|
||||
relative changes. It is a declared summary, not a norm in a single physical space.
|
||||
Zero total implies undefined concentration, not a uniform distribution. Single-head relative effects before head-share normalization and true joint
|
||||
effects are retained as well as shares. The saved metric named max_absolute is
|
||||
the largest unsigned relative-L2 effect, not an unnormalized hidden-space norm.
|
||||
The total field sums individual relative effects; it is not the measured joint
|
||||
effect of removing the whole attention branch.
|
||||
|
||||
Full-attention recipients additionally have real token-key probability JS changes;
|
||||
linear attention does not have an ordinary softmax attention probability matrix.
|
||||
Full-attention K/V repeat their GQA groups to query-head slots for aligned joint
|
||||
experiments, not as extra independent KV observations. Every headline statistic
|
||||
is descriptive: query/token/head observations are dependent, and four short prior
|
||||
texts do not establish universal dominance or independent generalization.
|
||||
|
||||
Joint masks vary per source token and recipient head. Recompute a coherent full
|
||||
sequence for each recipient head, scoring that intended head afterward. Rank by
|
||||
single QKV influence, remove weakest half/strongest half/a fixed-seed random half,
|
||||
or all attention heads. This investigates interactions and cross-token effects;
|
||||
it is not a validated free selector, fixed removable head subset or LM pruning.
|
||||
|
||||
Only run model work and scientific plotting on the authorized CPU server:
|
||||
```
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_dominance -v
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m dominance.run --source runs/cpu-qwen35-2b --out runs/NEW_DOMINANCE_DIR --seconds 7200
|
||||
venv/bin/python -B -m dominance.report runs/NEW_DOMINANCE_DIR
|
||||
venv/bin/python -B -m dominance.audit runs/NEW_DOMINANCE_DIR runs/cpu-qwen35-2b
|
||||
```
|
||||
|
||||
No training, GPU, new model download or automatic deletion. Raw intervention
|
||||
arrays and masks are delivered to Git; large baseline caches stay on the server
|
||||
with path/size/SHA metadata. Existing checkpoints/caches are never removed.
|
||||
|
||||
The saved sum of selected individual effects is only a query-wise mask-weighted
|
||||
reference: individual ablations remove a head at all positions, whereas joint
|
||||
masks can vary across positions. Their difference cannot isolate pure nonlinear
|
||||
interaction, since temporal intervention supports also differ. Use the measured
|
||||
weak/strong/random joint effects directly; do not call this reference an additive
|
||||
causal decomposition or a Shapley attribution.
|
||||
|
||||
Scale sensitivity: after reviewing the first complete summaries, add sqrt(JS)
|
||||
(Jensen–Shannon distance) alongside primary JS divergence. Divergence is roughly
|
||||
quadratic for small changes, so shares and effective head counts depend on this
|
||||
scale. All original primary statistics and raw interventions remain unchanged;
|
||||
primary-analysis-summary.json preserves the pre-sensitivity aggregate for exact
|
||||
audit comparison. Retrieval-scale plots and source tables show both definitions.
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Recompute saved concentration, masks, scale and scope checks; preserve caches."""
|
||||
import json,hashlib,csv,sys
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from .core import concentration
|
||||
|
||||
|
||||
def main(root,source):
|
||||
root=Path(root);source=Path(source);status=json.loads((root/'status.json').read_text());assert status['state']=='completed'
|
||||
samples=json.loads((root/'samples.json').read_text());assert len(samples)==4 and all(len(s['token_ids'])==32 for s in samples)
|
||||
expected={(s['id'],i) for s in samples for i in range(23)};actual={(x['sample'],x['source']) for x in status['completed_links']};assert actual==expected and len(status['completed_links'])==92
|
||||
cfg=json.loads((source/'model_config.json').read_text())['text_config'];obs=dict(np.load(root/'observations.npz'));lookup=sorted(s['id'] for s in samples);counts={};undefined={};full_js_links=0;joint_values={}
|
||||
for sample,layer in sorted(actual):
|
||||
f=root/'raw'/f'{sample}-link-{layer:02d}.npz';d=dict(np.load(f));hs=8 if cfg['layer_types'][layer]=='full_attention' else 16;hr=8 if cfg['layer_types'][layer+1]=='full_attention' else 16
|
||||
if hr==8:
|
||||
full_js_links+=1;d['single_sqrtJS']=np.sqrt(d['single_JS'])
|
||||
for label in ('weak_half','strong_half','random_half','all_heads'):
|
||||
d['joint_'+label+'_sqrtJS']=np.sqrt(d['joint_'+label+'_JS'])
|
||||
mask=d.get('mask_'+label,np.ones((hr,32,hs),dtype=np.float32))
|
||||
d['individual_sum_'+label+'_sqrtJS']=np.sum(d['single_sqrtJS'].transpose(2,1,0)*mask,-1).T
|
||||
d['null_sqrtJS']=np.sqrt(d['null_JS'])
|
||||
masks={label:d['mask_'+label] for label in ('weak_half','strong_half','random_half')}
|
||||
for label,m in masks.items():
|
||||
assert m.shape==(hr,32,hs);assert np.isin(m,[0,1]).all();np.testing.assert_array_equal(m.sum(-1),hs//2)
|
||||
np.testing.assert_array_equal(masks['weak_half']+masks['strong_half'],1)
|
||||
ranked=np.argsort(d['single_QKV'].transpose(2,1,0),axis=-1,kind='stable')
|
||||
np.testing.assert_array_equal(np.take_along_axis(masks['weak_half'],ranked[...,:hs//2],-1),1)
|
||||
for key in ('Q','K','V','QKV')+(('JS','sqrtJS') if hr==8 else ()):
|
||||
single=d['single_'+key];assert single.shape==(hs,32,hr) and np.isfinite(single).all() and np.min(single)>=0
|
||||
effect=single[:,7:,:].transpose(1,2,0);stats=concentration(effect);counts[key]=counts.get(key,0)+effect.shape[0]*hr;undefined[key]=undefined.get(key,0)+int((~stats['valid']).sum())
|
||||
choose=(obs[key+'_source']==layer)&(obs[key+'_sample_index']==lookup.index(sample))
|
||||
assert int(choose.sum())==25*hr
|
||||
for metric,values in stats.items():
|
||||
if metric=='valid':continue
|
||||
np.testing.assert_allclose(obs[key+'_'+metric][choose],values.ravel(),rtol=1e-6,atol=1e-7,equal_nan=True)
|
||||
for label in ('weak_half','strong_half','random_half','all_heads'):
|
||||
mask=masks.get(label,np.ones((hr,32,hs),dtype=np.float32));expected_sum=np.sum(single.transpose(2,1,0)*mask,-1).T
|
||||
np.testing.assert_allclose(d['individual_sum_'+label+'_'+key],expected_sum,rtol=1e-6,atol=1e-8)
|
||||
assert d['joint_'+label+'_'+key].shape==(32,hr) and np.isfinite(d['joint_'+label+'_'+key]).all()
|
||||
joint_values.setdefault((key,label),[]).extend(d['joint_'+label+'_'+key][7:].ravel())
|
||||
assert float(np.max(d['null_'+key]))<=2e-5
|
||||
assert full_js_links==24
|
||||
summary=json.loads((root/'analysis-summary.json').read_text())
|
||||
def check_stats(values,record):
|
||||
v=np.asarray(values,dtype=np.float64);v=v[np.isfinite(v)];assert len(v)==record['count']
|
||||
if len(v):
|
||||
for name,x in zip(['p10','median','p90','mean'],[*np.quantile(v,[.1,.5,.9]),v.mean()]):np.testing.assert_allclose(record[name],x,rtol=2e-6,atol=1e-8)
|
||||
for key in counts:
|
||||
for metric,record in summary['concentration'][key].items():check_stats(obs[key+'_'+metric],record)
|
||||
primary=json.loads((root/'primary-analysis-summary.json').read_text())
|
||||
for section in ('concentration','joint','paired_joint','by_recipient_type'):
|
||||
for key,value in primary[section].items():assert summary[section][key]==value,(section,key)
|
||||
full_sources=sorted({x['source'] for x in status['completed_links'] if x['full_attention_recipient']})
|
||||
assert summary['full_attention_source_indices']==full_sources
|
||||
for key,groups in summary['by_recipient_type'].items():
|
||||
src=obs[key+'_source']
|
||||
for label,metrics in groups.items():
|
||||
choose=np.isin(src,full_sources) if label=='full_attention' else ~np.isin(src,full_sources)
|
||||
for metric,record in metrics.items():check_stats(obs[key+'_'+metric][choose],record)
|
||||
for (key,label),values in joint_values.items():check_stats(values,summary['joint'][key][label])
|
||||
for key in counts:
|
||||
weak=np.asarray(joint_values[(key,'weak_half')]);strong=np.asarray(joint_values[(key,'strong_half')]);random=np.asarray(joint_values[(key,'random_half')]);valid=weak>0;record=summary['paired_joint'][key]
|
||||
assert record['zero_weak_effect_count']==int((~valid).sum())
|
||||
np.testing.assert_allclose(record['weak_less_than_strong_fraction'],np.mean(weak<strong))
|
||||
np.testing.assert_allclose(record['weak_less_than_random_fraction'],np.mean(weak<random))
|
||||
check_stats(strong[valid]/weak[valid],record['strong_over_weak']);check_stats(random[valid]/weak[valid],record['random_over_weak'])
|
||||
controls=json.loads((root/'head-sum-controls.json').read_text());assert len(controls)==96 and max(x['relative_l2'] for x in controls)<2e-5
|
||||
assert status['canonical_forward_relative_error']<2e-5
|
||||
caches=[]
|
||||
for f in sorted((root/'cache').rglob('*')):
|
||||
if f.is_file():caches.append({'path':str(f.resolve()),'bytes':f.stat().st_size,'sha256':hashlib.sha256(f.read_bytes()).hexdigest()})
|
||||
weights=list((source/'weights').rglob('*.safetensors'));assert weights and sum(f.stat().st_size for f in weights)==4548221488
|
||||
manifest=json.loads((source/'manifest.json').read_text());weight_records=[]
|
||||
for f in weights:
|
||||
with f.open('rb') as stream:sha=hashlib.file_digest(stream,'sha256').hexdigest()
|
||||
expected=next(x['upstream_sha256'] for x in manifest['files'] if x['name']==f.name);assert sha==expected
|
||||
weight_records.append({'path':str(f.resolve()),'bytes':f.stat().st_size,'sha256':sha,'matches_upstream':True})
|
||||
external={'host':'8.211.154.134','checkpoint_directory':str((source/'weights').resolve()),'checkpoint_bytes':sum(f.stat().st_size for f in weights),'checkpoint_files':weight_records,'revision':'15852e8c16360a2fea060d615a32b45270f8a8fc','baseline_caches':caches,'automatic_deletion':False,'prior_caches_untouched':True}
|
||||
(root/'external-caches.json').write_text(json.dumps(external,indent=2)+'\n')
|
||||
reference=json.loads((source/'reference-check.json').read_text())
|
||||
assert reference['passed'] and all(x['passed'] for x in reference['tests'])
|
||||
assert hashlib.sha256(Path('functional/model.py').read_bytes()).hexdigest()==reference['implementation_sha256']
|
||||
(root/'prior-official-reference-check.json').write_bytes((source/'reference-check.json').read_bytes())
|
||||
files=list(Path('dominance').glob('*.py'))+[Path('dominance/PROTOCOL.md'),Path('dominance/README.md'),Path('tests/test_dominance.py'),Path('functional/model.py'),Path('crossqkv/fresh_samples.json')]
|
||||
(root/'source-hashes.json').write_text(json.dumps({str(f):hashlib.sha256(f.read_bytes()).hexdigest() for f in files if not f.name.startswith('._')},indent=2)+'\n')
|
||||
audit={'passed':True,'transitions':23,'texts':4,'link_observations':92,'primary_observations_per_metric':counts,'undefined_zero_total_per_metric':undefined,'full_attention_js_links':full_js_links,'head_sum_controls':96,'canonical_forward_relative_error':status['canonical_forward_relative_error'],'all_raw_concentration_and_masks_recomputed':True,'pooled_concentration_and_joint_summary_recomputed':True,'original_primary_summaries_unchanged':True,'sqrt_JS_scale_sensitivity_recomputed':True,'retained_baseline_cache_files':len(caches),'checkpoint_sha_matches_upstream':True,'prior_official_reference_check_passed_same_implementation':True,'prior_official_reference_tests':len(reference['tests'])}
|
||||
(root/'audit.json').write_text(json.dumps(audit,indent=2)+'\n');print(json.dumps(audit,indent=2),flush=True)
|
||||
included=[f for f in root.rglob('*') if f.is_file() and 'cache' not in f.relative_to(root).parts and f.name!='checksums.json']
|
||||
(root/'checksums.json').write_text(json.dumps({str(f.relative_to(root)):hashlib.sha256(f.read_bytes()).hexdigest() for f in sorted(included)},indent=2)+'\n')
|
||||
if __name__=='__main__':main(sys.argv[1],sys.argv[2])
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Frozen-model head interventions and continuous influence concentration."""
|
||||
import numpy as np
|
||||
from functional.model import StreamingLM,PREFIX,rope,causal_conv,attention
|
||||
|
||||
|
||||
def concentration(d):
|
||||
"""Last axis donor head; zero total is undefined, represented by NaNs."""
|
||||
d=np.asarray(d,dtype=np.float64)
|
||||
if not np.isfinite(d).all() or np.any(d<0):raise ValueError('Finite nonnegative influences required')
|
||||
h=d.shape[-1];total=d.sum(-1);valid=total>0
|
||||
p=np.divide(d,total[...,None],out=np.zeros_like(d),where=valid[...,None]);square=np.sum(p*p,-1)
|
||||
neff=np.divide(1,square,out=np.full_like(total,np.nan),where=valid)
|
||||
ordered=np.sort(p,axis=-1)[...,::-1]
|
||||
return {'total':total,'max_absolute':d.max(-1),'valid':valid,'effective_heads':neff,'effective_fraction':neff/h,
|
||||
'concentration':np.where(valid,(square-1/h)/(1-1/h),np.nan),
|
||||
'top1':np.where(valid,ordered[...,0],np.nan),'top25':np.where(valid,ordered[...,:max(1,h//4)].sum(-1),np.nan),
|
||||
'top50':np.where(valid,ordered[...,:max(1,h//2)].sum(-1),np.nan),
|
||||
'bottom25':np.where(valid,ordered[...,-max(1,h//4):].sum(-1),np.nan),
|
||||
'heads90':np.where(valid,1+np.sum(np.cumsum(ordered,axis=-1)<.9,axis=-1),np.nan)}
|
||||
|
||||
|
||||
def masks_from_influences(d,weak=True):
|
||||
"""d [donor,T,recipient]; output masks [recipient,T,donor]."""
|
||||
ranked=np.argsort(d.transpose(2,1,0),axis=-1,kind='stable');n=d.shape[0]//2
|
||||
selected=ranked[...,:n] if weak else ranked[...,-n:]
|
||||
mask=np.zeros_like(ranked,dtype=np.float32);np.put_along_axis(mask,selected,1,axis=-1)
|
||||
return mask
|
||||
|
||||
|
||||
def js_rows(a,b):
|
||||
a=np.asarray(a,dtype=np.float64);b=np.asarray(b,dtype=np.float64);m=(a+b)/2
|
||||
def term(x):return np.where(x>0,x*np.log(np.maximum(x,1e-300)/np.maximum(m,1e-300)),0)
|
||||
return np.maximum(0,.5*np.sum(term(a)+term(b),axis=-1))
|
||||
|
||||
|
||||
class InfluenceLM(StreamingLM):
|
||||
capturing=False
|
||||
def linear(self,x,name,rows=256):
|
||||
y=super().linear(x,name,rows)
|
||||
if self.capturing and (name.endswith('self_attn.o_proj') or name.endswith('linear_attn.out_proj')):
|
||||
layer=int(name.split('.')[1]);nh=self.config['num_attention_heads'] if 'self_attn' in name else self.config['linear_num_value_heads']
|
||||
xx=x.reshape(*x.shape[:2],nh,-1);dim=xx.shape[-1];t=self.tensors[PREFIX+name+'.weight'];od,idim=t.shape
|
||||
out=np.empty((*x.shape[:2],nh,od),dtype=np.float32)
|
||||
for start in range(0,od,256):
|
||||
self.guard();stop=min(od,start+256);w=t.read(slice(start*idim,stop*idim)).reshape(stop-start,idim)
|
||||
for head in range(nh):out[:,:,head,start:stop]=xx[:,:,head]@w[:,head*dim:(head+1)*dim].T
|
||||
del w;t.evict()
|
||||
err=float(np.linalg.norm(out.sum(2).astype(np.float64)-y)/max(np.linalg.norm(y.astype(np.float64)),1e-30))
|
||||
if err>2e-5:raise AssertionError(('head sum mismatch',layer,err))
|
||||
self.head_outputs=out;self.head_sum_errors.append({'layer':layer,'relative_l2':err})
|
||||
return y
|
||||
|
||||
def baseline(self,ids,root):
|
||||
h=self.embedding(ids);self.head_sum_errors=[]
|
||||
for layer,kind in enumerate(self.config['layer_types']):
|
||||
raw=h.copy();n=self.normalization(h,f'layers.{layer}.input_layernorm');self.capturing=True
|
||||
out,_=self.full_attention(n,layer) if kind=='full_attention' else (self.linear_attention(n,layer),None)
|
||||
self.capturing=False;heads=self.head_outputs;self.head_outputs=None
|
||||
mid=raw+out;h=mid+self.mlp(self.normalization(mid,f'layers.{layer}.post_attention_layernorm'),layer)
|
||||
np.savez(root/f'layer-{layer}.npz',raw=raw,out=out,heads=heads,next_raw=h)
|
||||
print('baseline layer',layer,flush=True)
|
||||
return h
|
||||
|
||||
def recipient(self,raw,layer):
|
||||
h=self.normalization(raw,f'layers.{layer}.input_layernorm');c=self.config;b,t=h.shape[:2]
|
||||
if c['layer_types'][layer]=='full_attention':
|
||||
p=f'layers.{layer}.self_attn.';nh=c['num_attention_heads'];nk=c['num_key_value_heads'];d=c['head_dim']
|
||||
q=self.linear(h,p+'q_proj').reshape(b,t,nh,2*d)[...,:d];k=self.linear(h,p+'k_proj').reshape(b,t,nk,d);v=self.linear(h,p+'v_proj').reshape(b,t,nk,d)
|
||||
q=self.normalization(q,p+'q_norm');k=self.normalization(k,p+'k_norm');r=c['rope_parameters'];q,k=(rope(z,r['rope_theta'],r['partial_rotary_factor']) for z in (q,k))
|
||||
k=np.repeat(k,nh//nk,axis=2);v=np.repeat(v,nh//nk,axis=2)
|
||||
_,prob=attention(q,k,v)
|
||||
return {'Q':q,'K':k,'V':v,'P':prob}
|
||||
p=f'layers.{layer}.linear_attn.';kh=c['linear_num_key_heads'];vh=c['linear_num_value_heads'];kd=c['linear_key_head_dim'];vd=c['linear_value_head_dim']
|
||||
x=causal_conv(self.linear(h,p+'in_proj_qkv'),self.small(p+'conv1d.weight')[:,0,:]);q,k,v=np.split(x,[kh*kd,2*kh*kd],axis=-1)
|
||||
q=q.reshape(b,t,kh,kd);k=k.reshape(b,t,kh,kd);v=v.reshape(b,t,vh,vd)
|
||||
if vh!=kh:q,k=np.repeat(q,vh//kh,axis=2),np.repeat(k,vh//kh,axis=2)
|
||||
q=q/np.sqrt(np.sum(q*q,axis=-1,keepdims=True)+1e-6)/np.float32(np.sqrt(kd));k=k/np.sqrt(np.sum(k*k,axis=-1,keepdims=True)+1e-6)
|
||||
return {'Q':q,'K':k,'V':v}
|
||||
|
||||
def intervene(self,cache,layer,masks):
|
||||
# Each scenario has a full coherent token-dependent deletion mask.
|
||||
removed=np.einsum('bth,thd->btd',masks,cache['heads'][0],optimize=True)
|
||||
mid=cache['raw']+cache['out']-removed
|
||||
nxt=mid+self.mlp(self.normalization(mid,f'layers.{layer}.post_attention_layernorm'),layer)
|
||||
return self.recipient(nxt,layer+1)
|
||||
@@ -0,0 +1,140 @@
|
||||
"""Saved-data scientific analysis; no model invocation or fitting."""
|
||||
import csv,json
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
import matplotlib
|
||||
matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
from .core import concentration
|
||||
|
||||
|
||||
def stats(x):
|
||||
x=np.asarray(x,dtype=np.float64);valid=x[np.isfinite(x)]
|
||||
if not valid.size:return {'count':0,'p10':None,'median':None,'p90':None,'mean':None}
|
||||
q=np.quantile(valid,[.1,.5,.9]);return {'count':int(valid.size),'p10':float(q[0]),'median':float(q[1]),'p90':float(q[2]),'mean':float(valid.mean())}
|
||||
|
||||
|
||||
def csvwrite(p,rows):
|
||||
with p.open('w') as f:
|
||||
w=csv.DictWriter(f,fieldnames=list(rows[0]));w.writeheader();w.writerows(rows)
|
||||
|
||||
|
||||
def main(root):
|
||||
root=Path(root);figdir=root/'figures';figdir.mkdir(exist_ok=True);status=json.loads((root/'status.json').read_text());assert status['state']=='completed'
|
||||
rows=[];joint_rows=[];pool={};curves={};joint_pool={};obs={};winner=[];joint_by_layer={}
|
||||
for entry in status['completed_links']:
|
||||
sample=entry['sample'];source=entry['source'];data=dict(np.load(root/'raw'/f'{sample}-link-{source:02d}.npz'))
|
||||
if 'single_JS' in data:
|
||||
data['single_sqrtJS']=np.sqrt(data['single_JS'])
|
||||
for label in ('weak_half','strong_half','random_half','all_heads'):
|
||||
data['joint_'+label+'_sqrtJS']=np.sqrt(data['joint_'+label+'_JS'])
|
||||
masks=data.get('mask_'+label,np.ones_like(data['mask_weak_half']))
|
||||
data['individual_sum_'+label+'_sqrtJS']=np.sum(data['single_sqrtJS'].transpose(2,1,0)*masks,-1).T
|
||||
for key in ('Q','K','V','QKV','JS','sqrtJS'):
|
||||
if 'single_'+key not in data:continue
|
||||
d=data['single_'+key][:,7:,:].transpose(1,2,0);c=concentration(d);valid=c['valid'];h=d.shape[-1]
|
||||
metric={k:v.reshape(-1) for k,v in c.items() if k!='valid'}
|
||||
row={'sample':sample,'source':source,'recipient':source+1,'metric':key,'source_heads':h,'recipient_attention_type':'full_attention' if entry['full_attention_recipient'] else 'linear_attention','observations':d.shape[0]*d.shape[1],'undefined_zero_total':int((~valid).sum())}
|
||||
for k,v in metric.items():
|
||||
pool.setdefault(key,{}).setdefault(k,[]).extend(v)
|
||||
for s,z in stats(v).items():row[k+'_'+s]=z
|
||||
rows.append(row)
|
||||
# Raw per-observation statistics for independently reproducing figures.
|
||||
for k,v in metric.items():obs.setdefault(key+'_'+k,[]).append(v.astype(np.float32))
|
||||
n=d.shape[0]*d.shape[1]
|
||||
for k,v in [('source',np.full(n,source)),('sample_index',np.full(n,sorted({e['sample'] for e in status['completed_links']}).index(sample))),('token',np.repeat(np.arange(7,32),d.shape[1])),('recipient_head',np.tile(np.arange(d.shape[1]),25))]:obs.setdefault(key+'_'+k,[]).append(v.astype(np.int16))
|
||||
p=np.divide(d,d.sum(-1,keepdims=True),out=np.zeros_like(d),where=d.sum(-1,keepdims=True)>0);cum=np.concatenate([np.zeros((*d.shape[:2],1)),np.cumsum(np.sort(p,-1)[...,::-1],-1)],-1)
|
||||
grid=np.linspace(0,1,101);cc=np.array([np.interp(grid,np.linspace(0,1,h+1),v) for v in cum[valid]])
|
||||
curves.setdefault(key,[]).extend(cc)
|
||||
win=np.argmax(d,-1);counts=np.bincount(win[valid],minlength=h)
|
||||
winner.append({'sample':sample,'source':source,'metric':key,'most_frequent_winner_fraction':float(counts.max()/max(counts.sum(),1)),'distinct_winning_heads':int((counts>0).sum()),'head_counts':json.dumps(counts.tolist())})
|
||||
for label in ('weak_half','strong_half','random_half','all_heads'):
|
||||
values=data['joint_'+label+'_'+key][7:];summed=data['individual_sum_'+label+'_'+key][7:]
|
||||
jr={'sample':sample,'source':source,'recipient':source+1,'metric':key,'removed':label}
|
||||
for k,x in [('joint_effect',values),('sum_individual',summed)]:
|
||||
for s,z in stats(x).items():jr[k+'_'+s]=z
|
||||
joint_rows.append(jr);joint_pool.setdefault(key,{}).setdefault(label,[]).extend(values.ravel());joint_by_layer.setdefault((key,label,source),[]).extend(values.ravel())
|
||||
csvwrite(root/'concentration-by-link.csv',rows);csvwrite(root/'joint-by-link.csv',joint_rows);csvwrite(root/'winner-identities.csv',winner)
|
||||
np.savez_compressed(root/'observations.npz',**{k:np.concatenate(v) for k,v in obs.items()})
|
||||
pooled={k:{m:stats(v) for m,v in metrics.items()} for k,metrics in pool.items()};joint_stats={k:{m:stats(v) for m,v in metrics.items()} for k,metrics in joint_pool.items()}
|
||||
paired={}
|
||||
for key,labels in joint_pool.items():
|
||||
weak=np.asarray(labels['weak_half']);strong=np.asarray(labels['strong_half']);random=np.asarray(labels['random_half']);valid=weak>0
|
||||
paired[key]={'weak_less_than_strong_fraction':float(np.mean(weak<strong)),'weak_less_than_random_fraction':float(np.mean(weak<random)),'zero_weak_effect_count':int((~valid).sum()),'strong_over_weak':stats(strong[valid]/weak[valid]),'random_over_weak':stats(random[valid]/weak[valid])}
|
||||
full_sources=sorted({x['source'] for x in status['completed_links'] if x['full_attention_recipient']})
|
||||
by_type={}
|
||||
for key in pool:
|
||||
src=np.concatenate(obs[key+'_source']);by_type[key]={}
|
||||
for label,choose in [('full_attention',np.isin(src,full_sources)),('linear_attention',~np.isin(src,full_sources))]:
|
||||
if not choose.any():continue
|
||||
by_type[key][label]={m:stats(np.concatenate(obs[key+'_'+m])[choose]) for m in pool[key]}
|
||||
summary={'scale_sensitivity':'sqrt(JS) added after reviewing completed primary summaries; primary JS and all raw interventions unchanged; squared divergences cannot be equated to probability mass or linear-scale effects','by_recipient_type':by_type,'full_attention_source_indices':full_sources,'paired_joint':paired,'pooling':'query7..31, per text/token/recipient-query-head; descriptive dependent observations, not independent samples','concentration':pooled,'joint':joint_stats,'counts':{'links':len(status['completed_links']),'texts':len(json.loads((root/'samples.json').read_text())),'layer_transitions':23},'sample_index_key':sorted({e['sample'] for e in status['completed_links']}),'winner_identity_file':'winner-identities.csv'}
|
||||
(root/'analysis-summary.json').write_text(json.dumps(summary,indent=2,allow_nan=False)+'\n')
|
||||
def save(fig,name):
|
||||
for ext in ('png','pdf','svg'):fig.savefig(figdir/(name+'.'+ext),dpi=180)
|
||||
plt.close(fig)
|
||||
fig,axs=plt.subplots(1,2,figsize=(14,4.8),layout='constrained')
|
||||
colors={'Q':'C0','K':'C1','V':'C2','QKV':'C3'}
|
||||
for key in colors:
|
||||
for ax,metric in zip(axs,['effective_fraction','top1']):
|
||||
med=[];lo=[];hi=[]
|
||||
source= np.concatenate(obs[key+'_source']);vals=np.concatenate(obs[key+'_'+metric])
|
||||
for layer in range(23):
|
||||
z=stats(vals[source==layer]);lo.append(z['p10']);med.append(z['median']);hi.append(z['p90'])
|
||||
ax.plot(range(1,24),med,label=key,color=colors[key]);ax.fill_between(range(1,24),lo,hi,color=colors[key],alpha=.10)
|
||||
axs[0].set(title='Continuous effective-head fraction (median, p10–p90)',ylabel='N_eff / number of donor heads')
|
||||
axs[1].set(title='Largest donor-head influence share',ylabel='Top-1 normalized ablation effect')
|
||||
for ax in axs:
|
||||
for j,source in enumerate(full_sources):ax.axvspan(source+.6,source+1.4,color='gray',alpha=.08,label='Full-attention receiver' if j==0 else None)
|
||||
ax.set(xlabel='Recipient layer index',ylim=(0,1.02));ax.legend();ax.grid(alpha=.2)
|
||||
fig.suptitle('All 23 adjacent transitions; four texts, query positions 7–31; no training')
|
||||
save(fig,'continuous_dominance')
|
||||
fig,axs=plt.subplots(1,2,figsize=(13,4.8),layout='constrained');curve_rows=[]
|
||||
for key in colors:
|
||||
a=np.asarray(curves[key]);lo,med,hi=np.quantile(a,[.1,.5,.9],axis=0);x=np.linspace(0,1,101)
|
||||
axs[0].plot(x,med,label=key,color=colors[key]);axs[0].fill_between(x,lo,hi,color=colors[key],alpha=.1)
|
||||
for j,xx in enumerate(x):curve_rows.append({'metric':key,'fraction_ranked_heads':xx,'p10':lo[j],'median':med[j],'p90':hi[j]})
|
||||
v=np.sort(np.asarray(pool[key]['effective_fraction']));axs[1].plot(v,np.arange(1,len(v)+1)/len(v),label=key,color=colors[key])
|
||||
axs[0].plot([0,1],[0,1],':',color='gray',label='Uniform');axs[0].set(title='Influence concentration curve',xlabel='Fraction of donor heads, largest first',ylabel='Cumulative normalized single-head effects')
|
||||
axs[1].set(title='Continuous effective-head fraction: empirical CDF',xlabel='N_eff / donor heads',ylabel='Fraction of observed cases')
|
||||
for ax in axs:ax.set(xlim=(0,1),ylim=(0,1.02));ax.legend();ax.grid(alpha=.2)
|
||||
fig.suptitle('Observation-wise ranking; dominant identities can differ; shaded p10–p90')
|
||||
csvwrite(root/'concentration-curve.csv',curve_rows);save(fig,'ranked_influence_distribution')
|
||||
fig,axs=plt.subplots(1,2,figsize=(14,4.8),layout='constrained')
|
||||
for label,color in zip(['weak_half','strong_half','random_half','all_heads'],['C0','C3','C1','gray']):
|
||||
for ax,key in zip(axs,['QKV','JS']):
|
||||
selected=[r for r in joint_rows if r['metric']==key and r['removed']==label]
|
||||
xs=sorted(set(r['recipient'] for r in selected));ys=[np.median(joint_by_layer[(key,label,x-1)]) for x in xs]
|
||||
ax.plot(xs,np.maximum(ys,1e-12),'o-',label=label,color=color,markersize=4)
|
||||
axs[0].set(title='Joint removal: balanced QKV relative effect',ylabel='Median token/head relative L2 score')
|
||||
axs[1].set(title='Full-attention recipients: retrieval change',ylabel='Median token/head JS divergence (nats)')
|
||||
for ax in axs:ax.set(xlabel='Recipient layer index',yscale='log');ax.legend();ax.grid(alpha=.2)
|
||||
fig.suptitle('Equal-count weak/strong/random removal; token-wise masks; log display floor 1e-12')
|
||||
save(fig,'joint_removal_effects')
|
||||
fig,axs=plt.subplots(1,2,figsize=(13,4.8),layout='constrained')
|
||||
for key,label,color in [('JS','JS divergence (squared scale)','C4'),('sqrtJS','sqrt(JS) distance','C5')]:
|
||||
a=np.asarray(curves[key]);lo,med,hi=np.quantile(a,[.1,.5,.9],axis=0);x=np.linspace(0,1,101)
|
||||
axs[0].plot(x,med,label=label,color=color);axs[0].fill_between(x,lo,hi,color=color,alpha=.12)
|
||||
for j,xx in enumerate(x):curve_rows.append({'metric':key,'fraction_ranked_heads':xx,'p10':lo[j],'median':med[j],'p90':hi[j]})
|
||||
v=np.sort(np.asarray(pool[key]['effective_fraction']));axs[1].plot(v,np.arange(1,len(v)+1)/len(v),label=label,color=color)
|
||||
axs[0].plot([0,1],[0,1],':',color='gray',label='Uniform');axs[0].set(title='Retrieval influence rank curve',xlabel='Fraction of donor heads, largest first',ylabel='Cumulative normalized intervention effects')
|
||||
axs[1].set(title='Sensitivity to divergence versus distance scale',xlabel='Effective donor-head fraction',ylabel='Fraction of observations')
|
||||
for ax in axs:ax.set(xlim=(0,1),ylim=(0,1.02));ax.legend();ax.grid(alpha=.2)
|
||||
fig.suptitle('Full-attention recipients; sqrt(JS) is post-summary scale sensitivity, not a new model run')
|
||||
csvwrite(root/'concentration-curve.csv',curve_rows);save(fig,'retrieval_scale_sensitivity')
|
||||
lines=['# 上游注意力头对下一层 Q/K/V 的影响集中度','', '实测全部23个相邻层连接、4条既有中英文文本各32 token。主统计取query7–31，按文本/token/接收query头分别计算，再汇总。不是独立样本的显著性检验。','', '影响是移除上游一个头后，下一层Q/K/V向量的相对L2变化；重新计算中间MLP/残差/归一化。各头影响归一化仅构成消融影响分布，不是天然可相加的贡献概率。','', '| 对象 | 有效头比例中位数 [P10,P90] | 最大头份额中位数 | 前25%头份额中位数 | 最小25%头份额中位数 | 90%影响所需头数中位数 |','|---|---:|---:|---:|---:|---:|']
|
||||
for k in ('Q','K','V','QKV','JS','sqrtJS'):
|
||||
if k not in pooled:continue
|
||||
z=pooled[k];a=z['effective_fraction'];lines.append(f"| {k} | {a['median']:.3f} [{a['p10']:.3f},{a['p90']:.3f}] | {z['top1']['median']:.1%} | {z['top25']['median']:.1%} | {z['bottom25']['median']:.1%} | {z['heads90']['median']:.1f} |")
|
||||
lines+=['','sqrtJS为JS距离尺度敏感性分析：在初次完整汇总后增加，原始消融和原JS主结果不变。JS散度近似平方尺度，归一化后会放大强弱差距；JS影响份额不是注意力概率质量份额。','', '按接收层类型分层（避免总体分布混合不同机制）：','', '| 接收层类型 | Q有效头比例中位数 | K | V | QKV |','|---|---:|---:|---:|---:|']
|
||||
for label in ('linear_attention','full_attention'):
|
||||
lines.append('| '+label+' | '+' | '.join(format(by_type[k][label]['effective_fraction']['median'],'.3f') for k in ('Q','K','V','QKV'))+' |')
|
||||
lines+=['','QKV是三个相对变化平方的均值再开根号，避免混加不同Q/K/V尺度；JS仅普通注意力接收层有。普通注意力GQA的K/V复制到query头索引用于配对，不是额外独立KV头。','', '| 联合移除 | QKV相对影响中位数 [P10,P90] | 检索JS中位数 [P10,P90] |','|---|---:|---:|']
|
||||
for label in ('weak_half','strong_half','random_half','all_heads'):
|
||||
a=joint_stats['QKV'][label];b=joint_stats['JS'][label];lines.append(f"| {label} | {a['median']:.5f} [{a['p10']:.5f},{a['p90']:.5f}] | {b['median']:.6f} [{b['p10']:.6f},{b['p90']:.6f}] |")
|
||||
lines+=['', '配对观测：QKV弱半数的实际联合影响小于强半数的比例为'+format(paired['QKV']['weak_less_than_strong_fraction'],'.1%')+'；强/弱影响比中位数为'+format(paired['QKV']['strong_over_weak']['median'],'.3f')+'。这是逐观测比值的中位数，不是两个总体中位数之比。弱效应为零的观测另计，不做除法。', '', '联合实验的弱/强/随机半数头，在各token、各接收头上分别选择，形成完整序列再重算。排名依据单头QKV影响，同样用于JS；它是输入自适应诊断，不是免费可用的剪枝策略。线性注意力的卷积和普通注意力的QK可能传播跨token影响。所有上游头移除仍保留残差背景，不能把它称为删除全部上游信息。', '', '连续指标：N_eff=1/Σp²；有效头比例=N_eff/H；C=(Σp²−1/H)/(1−1/H)。全零总影响时集中度未定义，原始数组保留NaN并在统计时排除、另计数量。单头相对L2影响在做头间占比归一化之前的总和及最大值均保留，避免把微小效应中的集中误认作重要。字段max_absolute指最大无符号相对效应，并非隐藏坐标中的未归一化L2；total是单头效应之和，真正全头联合效果另测。', '', '联合实验中保存的单头影响之和仅为按query掩码加权的参考。单头消融移除整段序列的一个头，联合掩码则可跨位置变化；二者的差不能单独解释成纯非线性交互，因为时间支持也不同。应直接比较实际弱/强/随机联合效果。', '', '## 文件与限制','', '`raw/*.npz`保留每次单头影响、联合影响、相应单头影响之和、删除掩码及空操作对照；`observations.npz`保留逐观测连续指标及索引；CSV和JSON为全部图表源表。原模型权重和大基线缓存继续保留在服务器。','', '只测短文本上的局部下一层作用；不代表全LM质量、可直接删头、所有输入普遍集中，或相似头可互相替代。所有头贡献可能是广泛分布的；报告不以发现稀疏为成功条件。','',f"CPU测量耗时{status['elapsed_seconds']:.1f}秒，峰值RSS{status['peak_rss_kib']/1024:.1f} MiB；无训练、GPU或清理。"]
|
||||
(root/'REPORT.md').write_text('\n'.join(lines)+'\n')
|
||||
print((root/'REPORT.md').read_text())
|
||||
if __name__=='__main__':
|
||||
import sys
|
||||
main(sys.argv[1])
|
||||
@@ -0,0 +1,84 @@
|
||||
"""All adjacent layers, input-dependent influence distributions; CPU only."""
|
||||
import argparse,json,time,resource,hashlib,traceback
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from tokenizers import Tokenizer
|
||||
from functional.model import StreamingLM
|
||||
from .core import InfluenceLM,concentration,masks_from_influences,js_rows
|
||||
|
||||
|
||||
def dump(path,data):path.write_text(json.dumps(data,indent=2,allow_nan=False)+'\n')
|
||||
|
||||
|
||||
def distances(base,changed):
|
||||
out={}
|
||||
for key in ('Q','K','V'):
|
||||
a=base[key].astype(np.float64);b=changed[key].astype(np.float64)
|
||||
out[key]=np.linalg.norm(b-a,axis=-1)/np.maximum(np.linalg.norm(a,axis=-1),1e-12)
|
||||
out['QKV']=np.sqrt(sum(out[k]**2 for k in ('Q','K','V'))/3)
|
||||
if 'P' in base:out['JS']=js_rows(base['P'],changed['P']).transpose(0,2,1)
|
||||
return out
|
||||
|
||||
|
||||
def link(lm,cache,layer,path,seed):
|
||||
started=time.monotonic();base=lm.recipient(cache['next_raw'],layer+1);hs=cache['heads'].shape[2];t=cache['raw'].shape[1];hr=base['Q'].shape[2]
|
||||
# Whole donor head across the full sequence is the single-ablation intervention.
|
||||
singles={k:[] for k in ('Q','K','V','QKV')+ (('JS',) if 'P' in base else ())}
|
||||
for start in range(0,hs,4):
|
||||
index=list(range(start,min(start+4,hs)));masks=np.zeros((len(index),t,hs),dtype=np.float32)
|
||||
for n,h in enumerate(index):masks[n,:,h]=1
|
||||
scores=distances(base,lm.intervene(cache,layer,masks))
|
||||
for k,v in scores.items():singles[k].extend(v)
|
||||
singles={k:np.asarray(v,dtype=np.float32) for k,v in singles.items()}
|
||||
# Identical cardinality weak/strong/random removals, ranking by balanced QKV influence.
|
||||
# One coherent full-sequence mask per recipient head; only that head's output scored.
|
||||
rank=singles['QKV'];rng=np.random.default_rng(seed)
|
||||
weak=masks_from_influences(rank,True);strong=masks_from_influences(rank,False)
|
||||
random=np.zeros_like(weak)
|
||||
for r in range(hr):
|
||||
for q in range(t):random[r,q,rng.choice(hs,hs//2,replace=False)]=1
|
||||
joint={};sum_individual={}
|
||||
for label,masks in [('weak_half',weak),('strong_half',strong),('random_half',random),('all_heads',np.ones_like(weak))]:
|
||||
values={k:[] for k in singles}
|
||||
for start in range(0,hr,4):
|
||||
stop=min(start+4,hr);scores=distances(base,lm.intervene(cache,layer,masks[start:stop]))
|
||||
for k,v in scores.items():
|
||||
for n,r in enumerate(range(start,stop)):values[k].append(v[n,:,r])
|
||||
for k,v in values.items():
|
||||
joint[label+'_'+k]=np.asarray(v,dtype=np.float32).T
|
||||
sum_individual[label+'_'+k]=np.sum(singles[k].transpose(2,1,0)*masks,axis=-1).T
|
||||
null=distances(base,lm.intervene(cache,layer,np.zeros((1,t,hs),dtype=np.float32)))
|
||||
if max(float(np.max(v)) for v in null.values())>2e-5:raise AssertionError('Null intervention changed recipient')
|
||||
payload={'single_'+k:v for k,v in singles.items()};payload.update({'joint_'+k:v for k,v in joint.items()});payload.update({'individual_sum_'+k:v for k,v in sum_individual.items()})
|
||||
payload.update({'mask_weak_half':weak,'mask_strong_half':strong,'mask_random_half':random})
|
||||
payload.update({'null_'+k:v for k,v in null.items()})
|
||||
np.savez_compressed(path,**payload)
|
||||
return {'source':layer,'recipient':layer+1,'source_heads':hs,'recipient_query_heads':hr,'full_attention_recipient':'P' in base,'elapsed_seconds':time.monotonic()-started,'null_max':max(float(np.max(v)) for v in null.values())}
|
||||
|
||||
|
||||
def main():
|
||||
p=argparse.ArgumentParser();p.add_argument('--source',type=Path,required=True);p.add_argument('--out',type=Path,required=True);p.add_argument('--seconds',type=int,default=7200);a=p.parse_args()
|
||||
a.out.mkdir(parents=True,exist_ok=False);(a.out/'cache').mkdir();(a.out/'raw').mkdir();started=time.monotonic()
|
||||
protocol={'scope':'all 23 adjacent layer transitions; source attention head contribution to next effective Q/K/V','samples':'4 previously authored zh/en texts, first32 tokens, descriptive','primary_queries':list(range(7,32)), 'influence':'L2 change / baseline token recipient-head L2; QKV score sqrt(mean of three squared relative changes))','linear_QK':'after causal convolution/SiLU and delta-rule L2 normalization; Q additionally divided by sqrt(key dim)','full_QK':'QK norm and partial RoPE applied; K/V repeated to query heads for indexing (not independent KV observations)','single_ablation':'remove complete source head projected output at all tokens; recompute source postattention normalization, MLP, residual and next input normalization','joint_ablation':'weak/strong/random half and all attention heads; one coherent token-varying removal mask for each recipient query head; ranking uses QKV single effects; no fitting','continuous':'effective heads=1/sum(p^2); fraction=effective/H; concentration=(sum(p^2)-1/H)/(1-1/H)','zero_total':'undefined concentration (NaN arrays), count explicitly','cost_budget_seconds':a.seconds,'no_training':True,'keep_all_weights_and_caches':True,'limitations':['not final LM quality or pruning proof','individual effects not additive; normalized scores not true contribution probabilities','token-varying masks are input-specific diagnostic and not a deployable fixed head subset','linear attention does not have an ordinary softmax probability matrix','Q/K/V scales handled separately; combined QKV is a declared balanced relative-effect score']}
|
||||
dump(a.out/'protocol.json',protocol);status={'state':'running','completed_links':[]};dump(a.out/'status.json',status)
|
||||
try:
|
||||
lm=InfluenceLM(a.source/'weights',json.loads((a.source/'model_config.json').read_text()),seconds=a.seconds)
|
||||
samples=json.loads(Path('crossqkv/fresh_samples.json').read_text())['samples'];tok=Tokenizer.from_file(str(a.source/'tokenizer.json'));controls=[]
|
||||
for si,s in enumerate(samples):
|
||||
lm.guard();s['token_ids']=tok.encode(s['text']).ids[:32];ids=np.array([s['token_ids']],dtype=np.int64);cache=a.out/'cache'/s['id'];cache.mkdir()
|
||||
final=lm.baseline(ids,cache);np.save(cache/'final.npy',final)
|
||||
for x in lm.head_sum_errors:controls.append({'sample':s['id'],**x})
|
||||
if si==0:
|
||||
reference,_=StreamingLM.forward(lm,ids);normalized=lm.normalization(final,'norm');err=float(np.linalg.norm(normalized-reference)/max(np.linalg.norm(reference),1e-30));assert err<2e-5;status['canonical_forward_relative_error']=err
|
||||
dump(a.out/'head-sum-controls.json',controls);dump(a.out/'samples.json',samples)
|
||||
for layer in range(23):
|
||||
target=a.out/'raw'/f'{s["id"]}-link-{layer:02d}.npz'
|
||||
with np.load(cache/f'layer-{layer}.npz') as data:loaded={k:data[k] for k in data.files}
|
||||
info=link(lm,loaded,layer,target,31000+si*100+layer)
|
||||
del loaded
|
||||
status['completed_links'].append({'sample':s['id'],**info});status['elapsed_seconds']=time.monotonic()-started;dump(a.out/'status.json',status)
|
||||
print('completed',s['id'],layer,'seconds',round(info['elapsed_seconds'],2),flush=True)
|
||||
status.update(state='completed',elapsed_seconds=time.monotonic()-started,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss);dump(a.out/'status.json',status)
|
||||
except BaseException as e:
|
||||
status.update(state='failed',error=repr(e),elapsed_seconds=time.monotonic()-started);dump(a.out/'status.json',status);raise
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,42 @@
|
||||
# Paired task-finetuning comparison (registered before reading finetuned effects)
|
||||
|
||||
Primary checkpoint: Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo,
|
||||
HF revision038d8be9bd0edd3883fa019651b02bdf65609468, pure lora/seed0,
|
||||
step150 after shared40-step warmup. GSM8K GRPO, not SFT. Selected by smallest
|
||||
seed index before observing concentration, not by benchmark score or our metric.
|
||||
No gate01/gate02 arm or architecture changes. Exact base revision matches retained
|
||||
Qwen/Qwen3.5-2B15852e8c16360a2fea060d615a32b45270f8a8fc.
|
||||
|
||||
Download existing18.9MB F32 LoRA tensors, verify official file SHA. All144 tensors
|
||||
must form72 A/B pairs on the4 declared GDN projections of18 linear-attention
|
||||
layers, rank16 alpha32. Apply FP32 effective W +2BA in streamed row chunks, also
|
||||
when decomposing source output heads. Do not quantize merged weights, mutate or
|
||||
save over base weights, execute foreign .pt files, train or use GPU. Validate
|
||||
against separate base-output +2B(Ax) forward and check head sums/null masks.
|
||||
|
||||
Primary comparison exactly matches the reported8.931822/16 statistic: all six
|
||||
ordinary attention receivers3/7/11/15/19/23, previous-layer16 donor output heads,
|
||||
all32 tokens computed, query7..31, original4 plain-text inputs and same token IDs.
|
||||
Reuse corresponding immutable base single-ablation arrays, recompute FT baselines
|
||||
and all16 single donor removals, source MLP/residual/recipient normalization exact.
|
||||
Use the same sqrt(JS) distance influence, normalized shares and N_eff=1/sum(p²).
|
||||
Preserve JS divergence as a sensitivity row; no change of ranking or threshold.
|
||||
All4800 per-token/query-head observations are paired. No new joint-mask search:
|
||||
the requested statistic depends on single removals, not the previous joint tests.
|
||||
|
||||
Secondary task-conditioned comparison: first4 original GSM8K test questions with
|
||||
at least32 base-tokenizer tokens (length selection only, before model effects),
|
||||
question-only plain text, first32 tokens. Freeze original author repository commit,
|
||||
JSONL SHA, item indices, question hashes and exact token IDs. No answers, generation,
|
||||
chat-template change or accuracy claim. Compute both base and FT on these same
|
||||
questions; never compare old general-text base to new GSM8K FT as if paired.
|
||||
|
||||
Report overall median/P10/P90, paired differences, per receiver layer and per text,
|
||||
strong/weak-quartile shares, and dominant donor identity agreement. No iid-token
|
||||
p-values, no claim over training seeds (one fixed trained seed), no proof of
|
||||
GSM8K accuracy improvement or long-context behavior. Positive, negative and near
|
||||
zero changes all count as results. Source cohort remains4+4 short texts.
|
||||
|
||||
All model work/tests/plots on authorized CPU server8.211.154.134, bounded7200s.
|
||||
Preserve all base/FT weights, adapters and caches; no automatic cleanup. Save raw
|
||||
arrays, source provenance, SHA manifests, fixed chart gallery and reviewable PR.
|
||||
@@ -0,0 +1,61 @@
|
||||
# Existing GSM8K fine-tune: paired head-concentration comparison
|
||||
|
||||
First restore the original charts through [the fixed figure gallery](../docs/FIGURE_GALLERY.md).
|
||||
The original 8.931822/16, P10=5.784081 and P90=11.672658 refer to sqrt(JS)
|
||||
influences of all16 prior-layer heads on the six full-attention receivers.
|
||||
|
||||
Selected release: [Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo](https://huggingface.co/Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo),
|
||||
pinned038d8be9bd0edd3883fa019651b02bdf65609468, pure lora/seed0. The publisher
|
||||
records GSM8K GRPO from a shared warmup, rank16 alpha32, exact same original
|
||||
Qwen3.5-2B revision. This is extra RL fine-tuning of the post-trained release,
|
||||
not pretraining-to-SFT, and not a beta-gate architecture modification. Our seed
|
||||
choice is fixed before seeing effects; no new training is performed.
|
||||
|
||||
`lora.py` validates all144 tensors/72 pairs, maps only declared GDN projections,
|
||||
and provides read-only FP32 effective row chunks W+2BA. Source head decomposition
|
||||
uses the same effective weights. `ExplicitLoRA` adds2B(Ax) independently to the
|
||||
base projection as a numerical reference. No BF16 re-rounding is introduced:
|
||||
the author's training code also warns that rounding merged increments can erase
|
||||
small LoRA changes. Author configuration commit/SHA and public HF metadata are
|
||||
saved alongside the protocol; foreign training code and pickle checkpoints are
|
||||
not executed.
|
||||
|
||||
`run.py` reuses the exact original single-ablation arrays for the original cohort,
|
||||
then computes matched finetuned states; it computes both models on4 question-only
|
||||
GSM8K test prefixes as a separate cohort. Every input has32 tokens. All16 donor
|
||||
heads, six receiver layers andquery7..31 remain identical to the original metric.
|
||||
The task corpus is the canonical author's test JSONL at a fixed Git commit, and
|
||||
selection uses only the first4 items with at least32 tokenizer tokens. All item
|
||||
indices, question hashes and token IDs are recorded; answers are not model inputs.
|
||||
|
||||
`report.py` generates paired CDFs, per-layer medians/quantiles, paired change
|
||||
histograms, raw observations and tables. `audit.py` independently reconstructs
|
||||
effective-head counts via(sum d)^2/sum(d^2), checks pairing, exact old baselines,
|
||||
head sums, null masks, adapter/dataset/base SHA and retained external caches.
|
||||
`tests/test_finetune_compare.py` checks streamed row arithmetic, an independent
|
||||
low-rank forward and head decomposition, plus exact zero-scale restoration.
|
||||
|
||||
No iid-token p-values, training-seed generalization, GSM8K accuracy result,
|
||||
long-context claim, pruning recommendation or functional-duplicate claim.
|
||||
The paired delta median is distinguished from the difference of pooled medians.
|
||||
All weights, adapters and caches remain on the authorized CPU server until user
|
||||
explicit deletion. Git stores all raw influence arrays, not the model/adapter.
|
||||
|
||||
Remote workflow on8.211.154.134:
|
||||
```
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_finetune_compare -v
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m finetune_compare.run --source runs/cpu-qwen35-2b --parent runs/head-dominance --assets runs/finetune-head-comparison --out runs/finetune-head-comparison/NEW_EXPERIMENT
|
||||
venv/bin/python -B -m finetune_compare.report runs/finetune-head-comparison/NEW_EXPERIMENT
|
||||
venv/bin/python -B -m finetune_compare.audit runs/finetune-head-comparison/NEW_EXPERIMENT runs/head-dominance runs/finetune-head-comparison runs/cpu-qwen35-2b
|
||||
```
|
||||
|
||||
The supplemental QKV field is measured only on the same six full-attention
|
||||
receivers, so its baseline must not be confused with the older76.0% pooled
|
||||
QKV effective-head fraction across all23 adjacent transitions. The gallery
|
||||
preserves that original table separately; the requested8.931822 metric uses
|
||||
exactly the matching six receivers in both studies.
|
||||
|
||||
The audit also records all72 low-rank update Frobenius norms (via small Gram
|
||||
matrices) and final pre-normalization residual changes per text. These separate
|
||||
a stable concentration statistic from an accidentally inactive adapter. Residual
|
||||
changes are not logit distributions, accuracy differences or capability scores.
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Independent paired effective-head, source, preservation and delivery checks."""
|
||||
import json,hashlib,sys
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
|
||||
def sha(path):
|
||||
with Path(path).open('rb') as f:return hashlib.file_digest(f,'sha256').hexdigest()
|
||||
def main(root,parent,assets,source):
|
||||
root,parent,assets,source=map(Path,[root,parent,assets,source]);status=json.loads((root/'status.json').read_text());assert status['state']=='completed' and len(status['links'])==96
|
||||
samples=json.loads((root/'samples.json').read_text());assert set(samples)=={'legacy','gsm8k'} and all(len(v)==4 for v in samples.values());assert all(len(s['token_ids'])==32 for ss in samples.values() for s in ss)
|
||||
assert status['explicit_lora_forward_relative_error']<1e-4;rawstats={};expected=set();obs=dict(np.load(root/'observations.npz'));summary=json.loads((root/'summary.json').read_text())
|
||||
def check(x,record):
|
||||
x=np.asarray(x,dtype=np.float64);x=x[np.isfinite(x)];assert record['count']==len(x)
|
||||
for name,value in zip(('p10','median','p90','mean'),[*np.quantile(x,[.1,.5,.9]),x.mean()]):np.testing.assert_allclose(record[name],value,atol=1e-8,rtol=1e-6)
|
||||
for cohort,ss in samples.items():
|
||||
for role in ('base','finetuned'):
|
||||
for sample in ss:
|
||||
for layer in (2,6,10,14,18,22):
|
||||
name=f'{cohort}-{role}-{sample["id"]}-{layer:02d}.npz';expected.add(name);d=dict(np.load(root/'raw'/name))
|
||||
for key in ('Q','K','V','QKV','JS'):
|
||||
assert d['single_'+key].shape==(16,32,8) and np.isfinite(d['single_'+key]).all() and np.min(d['single_'+key])>=0
|
||||
assert np.max(d['null_'+key])<2e-5
|
||||
if cohort=='legacy' and role=='base':
|
||||
with np.load(parent/'raw'/f'{sample["id"]}-link-{layer:02d}.npz') as original:
|
||||
for key in d:np.testing.assert_array_equal(d[key],original[key])
|
||||
for metric,impacts in [('distance',np.sqrt(d['single_JS'])),('divergence',d['single_JS']),('QKV',d['single_QKV'])]:
|
||||
x=impacts[:,7:,:].transpose(1,2,0).astype(np.float64);total=x.sum(-1);den=np.sum(x*x,-1);assert np.all(den>0)
|
||||
neff=total*total/den
|
||||
rawstats.setdefault((cohort,role,metric),[]).append(neff.ravel())
|
||||
assert expected=={f.name for f in (root/'raw').glob('*.npz')}
|
||||
for cohort in samples:
|
||||
for metric in ('distance','divergence','QKV'):
|
||||
for role in ('base','finetuned'):
|
||||
x=np.concatenate(rawstats[(cohort,role,metric)]);assert len(x)==4800
|
||||
np.testing.assert_allclose(obs['__'.join((cohort,role,metric,'effective_heads'))],x,rtol=1e-8,atol=1e-8)
|
||||
check(x,summary[cohort][metric][role]['effective_heads'])
|
||||
before=np.concatenate(rawstats[(cohort,'base',metric)]);after=np.concatenate(rawstats[(cohort,'finetuned',metric)]);check(after-before,summary[cohort][metric]['paired']['effective_heads_delta'])
|
||||
base=summary['legacy']['distance']['base']['effective_heads'];np.testing.assert_allclose([base[x] for x in ('median','p10','p90')],[8.93182211792415,5.7840811612328125,11.67265834734814],rtol=1e-10)
|
||||
controls=json.loads((root/'head-sum-controls.json').read_text());assert len(controls)==288 and all(c['relative_l2']<2e-5 for c in controls)
|
||||
refs=json.loads((root/'baseline-references.json').read_text());assert len(refs)==24 and all(sha(v['source'])==v['sha256'] for v in refs)
|
||||
assert sha(assets/'adapter.safetensors')=='93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6'
|
||||
dataset=json.loads((root/'gsm8k-source.json').read_text());assert sha(assets/'gsm8k-test.jsonl')==dataset['sha256']
|
||||
oldweights=list((source/'weights').glob('*.safetensors'));assert len(oldweights)==1
|
||||
base_sha=sha(oldweights[0]);assert base_sha=='aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1'
|
||||
from pilot.tensors import load_tensors
|
||||
import re
|
||||
tensors=load_tensors(assets);pairs={}
|
||||
for name,t in tensors.items():
|
||||
stem,which=name.split('.lora_');pairs.setdefault(stem,{})[which[0]]=t.read().reshape(t.shape).copy();t.evict()
|
||||
updates=[]
|
||||
for name,pair in pairs.items():
|
||||
a=pair['A'].astype(np.float64);b=pair['B'].astype(np.float64)
|
||||
norm2=4*float(np.sum((b.T@b)*(a@a.T).T))
|
||||
updates.append({'module':name,'delta_frobenius':float(np.sqrt(max(norm2,0))),'A_nonzero':int(np.count_nonzero(a)),'B_nonzero':int(np.count_nonzero(b))})
|
||||
assert len(updates)==72 and any(x['delta_frobenius']>0 for x in updates)
|
||||
(root/'weight-update-summary.json').write_text(json.dumps({'scale':2,'modules':updates,'nonzero_update_modules':sum(x['delta_frobenius']>0 for x in updates),'norm_identity':'||2BA||_F^2 = 4 trace((B^T B)(A A^T))'},indent=2)+'\n')
|
||||
changes=[]
|
||||
for cohort,ss in samples.items():
|
||||
for sample in ss:
|
||||
basefile=parent/'cache'/sample['id']/'final.npy' if cohort=='legacy' else root/'cache'/f"{cohort}-base-{sample['id']}"/'final.npy'
|
||||
ftfile=root/'cache'/f"{cohort}-finetuned-{sample['id']}"/'final.npy'
|
||||
before=np.load(basefile).astype(np.float64);after=np.load(ftfile).astype(np.float64)
|
||||
changes.append({'cohort':cohort,'sample':sample['id'],'final_pre_norm_residual_relative_l2':float(np.linalg.norm(after-before)/np.linalg.norm(before))})
|
||||
assert any(x['final_pre_norm_residual_relative_l2']>0 for x in changes)
|
||||
(root/'hidden-state-changes.json').write_text(json.dumps({'definition':'final residual before final RMSNorm, not logits, accuracy or LM quality','changes':changes},indent=2)+'\n')
|
||||
caches=[{'path':str(p.resolve()),'bytes':p.stat().st_size,'sha256':sha(p)} for p in sorted((root/'cache').rglob('*')) if p.is_file()]
|
||||
assert len(caches)==300
|
||||
(root/'external-caches.json').write_text(json.dumps({'host':'8.211.154.134','base_weights':str(oldweights[0].resolve()),'base_sha256':base_sha,'adapter_path':str((assets/'adapter.safetensors').resolve()),'adapter_sha256':sha(assets/'adapter.safetensors'),'caches':caches,'automatic_deletion':False},indent=2)+'\n')
|
||||
files=list(Path('finetune_compare').glob('*.py'))+[Path('finetune_compare/PROTOCOL.md'),Path('tests/test_finetune_compare.py'),Path('dominance/core.py'),Path('dominance/run.py'),Path('functional/model.py')]
|
||||
(root/'source-hashes.json').write_text(json.dumps({str(f):sha(f) for f in files},indent=2)+'\n')
|
||||
proof={'passed':True,'links':96,'paired_observations_per_cohort_metric':4800,'legacy_baseline_exact':True,'effective_head_formula_independently_recomputed':True,'head_sum_checks':288,'explicit_lora_forward_relative_error':status['explicit_lora_forward_relative_error'],'adapter_sha_verified':True,'base_sha_verified':True,'dataset_sha_verified':True,'retained_cache_files':len(caches),'nonzero_update_modules':sum(x['delta_frobenius']>0 for x in updates),'nonzero_hidden_state_changes_verified':True}
|
||||
(root/'audit.json').write_text(json.dumps(proof,indent=2)+'\n');print(json.dumps(proof,indent=2),flush=True)
|
||||
included=[p for p in root.rglob('*') if p.is_file() and 'cache' not in p.relative_to(root).parts and p.name!='checksums.json'];(root/'checksums.json').write_text(json.dumps({str(p.relative_to(root)):sha(p) for p in sorted(included)},indent=2)+'\n')
|
||||
if __name__=='__main__':main(*sys.argv[1:])
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"source_repository": "https://github.com/avifenesh/gdn-beta-gate",
|
||||
"source_revision": "f12f61f24ebe5997aac58667f413dcd1dcd96f54",
|
||||
"lora_configuration_path": "src/gdn_gate/broad/arms.py",
|
||||
"lora_configuration_sha256": "fc5b8de096eaa259ab166b8083bb774f9e14859d566ce82ea15986cc8d6eb5b5",
|
||||
"verified_fields": [
|
||||
"LoraConfig r=rank",
|
||||
"lora_alpha=2*rank",
|
||||
"use_rslora not enabled"
|
||||
],
|
||||
"effective_scale": 2
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6 lora/seed0/adapter.safetensors
|
||||
b8d4737a5b940bcdfed72258600227a89f883dc863e9724bf84e457b46a53703 lora/seed0/adapter_meta.json
|
||||
63d54e1c2f6bf7d07ca0f6e83a3299bd4b6d00740919bc7cae0b422af05a828e lora/seed1/adapter.safetensors
|
||||
f8be6c4218876b7b12a47945cafe18fcc27d59ff06447db6d3e5b21f0bb9fc29 lora/seed1/adapter_meta.json
|
||||
702b764e2c39b3529914dac1547148ef2f462d9ee04e256201e79f00b20ccc62 lora/seed2/adapter.safetensors
|
||||
9acee331b9383f49a81cfc057163e034f06ea1ba4b8167fea441f0fbfd490f94 lora/seed2/adapter_meta.json
|
||||
7dbf35899fe1186453e599ad8c7a85c56b7fad3d630e371ca026a1e95da2c5a7 lora/seed3/adapter.safetensors
|
||||
b5bfbefa8ce6b428feea761c39a148f003bf02396ca6e01b68abfba2ab3f71af lora/seed3/adapter_meta.json
|
||||
a5cb1e7fa74dff467bd7f163e1db23d30ef8b1a1c935cfa55dcc8cd1b39082a7 lora/seed4/adapter.safetensors
|
||||
23a6c3f198978e79014361a326af20ec364ea266cf9b5e9ced4c9da06feab30e lora/seed4/adapter_meta.json
|
||||
d513d499dfd409da88b3db7da025ba9872fb48ec92d6b29bc2ce7a092a69745d lora/seed5/adapter.safetensors
|
||||
9384d0b4ce4cec6b3dc15756fb4d46c5625a4dacd3805af728fc37b89627d0ed lora/seed5/adapter_meta.json
|
||||
fda9b4c15d283ef8ee778c4416d69b12e1922527ea330bfc67bab5cde1b6741a gate01/seed0/adapter.safetensors
|
||||
131e60bc768412f4d344e5b17cbc811f35d149aad0c328bc6503646513bbd8fc gate01/seed0/adapter_meta.json
|
||||
6ada79d07414641bed58d74fd23090937abcf52337e75e8f06483d502fa64c57 gate01/seed1/adapter.safetensors
|
||||
0350240ed838a93a26f406ef0d63f29ced21ea2284c9430d07aefccd30f8d924 gate01/seed1/adapter_meta.json
|
||||
a9359e3f3be68468604f9bf93f20f4b65114084f3fc6a3c13363e7062c4cf547 gate01/seed2/adapter.safetensors
|
||||
17fa2e876770ec636f8a0a0eb0b3f7b58c91a5f6f9910c09c38c593d856b035f gate01/seed2/adapter_meta.json
|
||||
a2d0c27fee0263c9a8152e0849975edefedc9d10061bbf789a3ead223db8703c gate01/seed3/adapter.safetensors
|
||||
4da32ea52fe6e7c2a0a8858e4d3255540fcc7adf18a7ab2ce9fd3164e4bb0a05 gate01/seed3/adapter_meta.json
|
||||
810b8f8d6147af3620cf08d2cf575c0eb2b0134ae0253a93ceeae5be9dca8695 gate01/seed4/adapter.safetensors
|
||||
f943fc6874af4e4b24763ee6e0cf6e737bc62720045190441d0b33dd561a439f gate01/seed4/adapter_meta.json
|
||||
b34e8d999612415c7395f341633ff3d2365a020b9e27b7e371c5f80f0105a486 gate01/seed5/adapter.safetensors
|
||||
98983ecbf303bf1b10e11c816f850db13af8a0ca5ff2e4efd121f7fd70262918 gate01/seed5/adapter_meta.json
|
||||
1fc5a4a4392c35820c26e24d0075824b3426c4a77f3119713b4a1e577d31df5b gate02/seed0/adapter.safetensors
|
||||
0a17ea0542fb52751d0e628985f8d07240a3ccc4b44a7e2c970e7029c00f6c3f gate02/seed0/adapter_meta.json
|
||||
b742f4ffed0e293f27e4646c364a85ae359a3eae1690d3840e531dca7fba9cd8 gate02/seed1/adapter.safetensors
|
||||
4b6c7bf77edf3049ca904863d59d4d3bfdbe1d0b1b3f563b7fd3bea7f179f97c gate02/seed1/adapter_meta.json
|
||||
dc36c27a98379535007f2df798953624b6a67f0677bd41dec42349f8ce868afa gate02/seed2/adapter.safetensors
|
||||
20009502b1bd6aa9c67b509b77eac0aa0d367af7f347caf4d05c7a51367b8f61 gate02/seed2/adapter_meta.json
|
||||
1dcb6132afeec8101daf9a3fd73d42141fb871bff11fff28069f238dcff6afd0 gate02/seed3/adapter.safetensors
|
||||
518289370e0eb9fc064c3226c2b3811f94cec180a9ae366a026f30cf1d3a3131 gate02/seed3/adapter_meta.json
|
||||
5b97b0601585925c19bf849ecb11ad68597d1b0d55551b917ec0f5091f85c2ff gate02/seed4/adapter.safetensors
|
||||
8419e8e22aa583f97e570950baf37e7c6370856b7cacefaf9c5ebeb0ea4f731c gate02/seed4/adapter_meta.json
|
||||
3f944a6d210958de5840e5b76e263f2105b9f49d15b9ab505950c2b29a5343af gate02/seed5/adapter.safetensors
|
||||
94d9c1e202f7962da61fa060d27941bbef7ff3d23ca5548f5d73fb6f5ac39aac gate02/seed5/adapter_meta.json
|
||||
9108bce2acfcc3ccd38d0dfae67ac33253b95bdc94133bd25deb4daa45d41700 warmup/seed0/adapter.safetensors
|
||||
d2d4346179104830410be9d2776d79983481503daeb5c8383e2a062de43714ef warmup/seed0/adapter_meta.json
|
||||
@@ -0,0 +1,379 @@
|
||||
{
|
||||
"_id": "6accb5a881b0c228f413b5e5",
|
||||
"id": "Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo",
|
||||
"private": false,
|
||||
"pipeline_tag": "text-generation",
|
||||
"library_name": "pytorch",
|
||||
"tags": [
|
||||
"pytorch",
|
||||
"lora",
|
||||
"gated-deltanet",
|
||||
"qwen3.5",
|
||||
"beta-gate",
|
||||
"grpo",
|
||||
"reasoning",
|
||||
"math",
|
||||
"safetensors",
|
||||
"research",
|
||||
"text-generation",
|
||||
"en",
|
||||
"dataset:openai/gsm8k",
|
||||
"base_model:Qwen/Qwen3.5-2B",
|
||||
"base_model:adapter:Qwen/Qwen3.5-2B",
|
||||
"license:apache-2.0",
|
||||
"region:us"
|
||||
],
|
||||
"downloads": 0,
|
||||
"likes": 2,
|
||||
"modelId": "Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo",
|
||||
"author": "Avifenesh",
|
||||
"sha": "038d8be9bd0edd3883fa019651b02bdf65609468",
|
||||
"lastModified": "2026-11-09T19:45:55.000Z",
|
||||
"gated": false,
|
||||
"disabled": false,
|
||||
"widgetData": [
|
||||
{
|
||||
"text": "Hi, what can you help me with?"
|
||||
},
|
||||
{
|
||||
"text": "What is 84 * 3 / 2?"
|
||||
},
|
||||
{
|
||||
"text": "Tell me an interesting fact about the universe!"
|
||||
},
|
||||
{
|
||||
"text": "Explain quantum computing in simple terms."
|
||||
}
|
||||
],
|
||||
"model-index": null,
|
||||
"config": {},
|
||||
"cardData": {
|
||||
"license": "apache-2.0",
|
||||
"base_model": "Qwen/Qwen3.5-2B",
|
||||
"base_model_relation": "adapter",
|
||||
"library_name": "pytorch",
|
||||
"pipeline_tag": "text-generation",
|
||||
"language": [
|
||||
"en"
|
||||
],
|
||||
"datasets": [
|
||||
"openai/gsm8k"
|
||||
],
|
||||
"tags": [
|
||||
"lora",
|
||||
"gated-deltanet",
|
||||
"qwen3.5",
|
||||
"beta-gate",
|
||||
"grpo",
|
||||
"reasoning",
|
||||
"math",
|
||||
"safetensors",
|
||||
"research"
|
||||
]
|
||||
},
|
||||
"siblings": [
|
||||
{
|
||||
"rfilename": ".gitattributes",
|
||||
"blobId": "a6344aac8c09253b3b630fb776ae94478aa0275b",
|
||||
"size": 1519
|
||||
},
|
||||
{
|
||||
"rfilename": "README.md",
|
||||
"blobId": "916419ce9530318d51d71325023fb2a5024475f0",
|
||||
"size": 6527
|
||||
},
|
||||
{
|
||||
"rfilename": "SHA256SUMS",
|
||||
"blobId": "cc8927482e300cb79fed740be333a8ac5639386c",
|
||||
"size": 3700
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed0/adapter.safetensors",
|
||||
"blobId": "86a05f3f118a88940a9122fab59a8e7db73d4b4f",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "fda9b4c15d283ef8ee778c4416d69b12e1922527ea330bfc67bab5cde1b6741a",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed0/adapter_meta.json",
|
||||
"blobId": "0652fd0ab9647ed184e2c65a670da74de1a24a4e",
|
||||
"size": 661
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed1/adapter.safetensors",
|
||||
"blobId": "596ed265e665aea6f674b81eecfdaaada202be45",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "6ada79d07414641bed58d74fd23090937abcf52337e75e8f06483d502fa64c57",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed1/adapter_meta.json",
|
||||
"blobId": "934875448487af7efe63e3eae775b24d2d388964",
|
||||
"size": 661
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed2/adapter.safetensors",
|
||||
"blobId": "6413c056f06232ad2ca3d513fa44cd01a90d251b",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "a9359e3f3be68468604f9bf93f20f4b65114084f3fc6a3c13363e7062c4cf547",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed2/adapter_meta.json",
|
||||
"blobId": "abc653d6a65aa8b3b1f2a18aaf12dc1260409573",
|
||||
"size": 661
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed3/adapter.safetensors",
|
||||
"blobId": "f17c2715463cadb293f36a0fc2c15fb0542990a6",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "a2d0c27fee0263c9a8152e0849975edefedc9d10061bbf789a3ead223db8703c",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed3/adapter_meta.json",
|
||||
"blobId": "f511293207bf322ce11a4c26dc83cc57b20edf52",
|
||||
"size": 661
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed4/adapter.safetensors",
|
||||
"blobId": "32503f87fd062bf0631c8eac7d8dbba09ef0e6cb",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "810b8f8d6147af3620cf08d2cf575c0eb2b0134ae0253a93ceeae5be9dca8695",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed4/adapter_meta.json",
|
||||
"blobId": "3fef81591cd7a41c636c39e72e22d5b3394dc6a5",
|
||||
"size": 661
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed5/adapter.safetensors",
|
||||
"blobId": "63b9a6dd509a288e6095ed9531fe76a055e2e5b4",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "b34e8d999612415c7395f341633ff3d2365a020b9e27b7e371c5f80f0105a486",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate01/seed5/adapter_meta.json",
|
||||
"blobId": "d0daa05e6348923237cdac83935118e4f020e9f7",
|
||||
"size": 661
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed0/adapter.safetensors",
|
||||
"blobId": "d566d86611edc89edb7cac62dc73d72619df55bd",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "1fc5a4a4392c35820c26e24d0075824b3426c4a77f3119713b4a1e577d31df5b",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed0/adapter_meta.json",
|
||||
"blobId": "36173814d1d838077da330e92fa5f0b1b222ba17",
|
||||
"size": 660
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed1/adapter.safetensors",
|
||||
"blobId": "cba04a7ef1bcb48952a723b6aa80d1b6847ef8dc",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "b742f4ffed0e293f27e4646c364a85ae359a3eae1690d3840e531dca7fba9cd8",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed1/adapter_meta.json",
|
||||
"blobId": "3c7751cefbe375871d0b99bfda0b3ba1e6ca896c",
|
||||
"size": 660
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed2/adapter.safetensors",
|
||||
"blobId": "af547c0fcaa9e69024e14818eb82c46f155dfa59",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "dc36c27a98379535007f2df798953624b6a67f0677bd41dec42349f8ce868afa",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed2/adapter_meta.json",
|
||||
"blobId": "baaefc1b64b0dce0c87bbc2a7c4342f87111aba6",
|
||||
"size": 660
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed3/adapter.safetensors",
|
||||
"blobId": "21066e00dfc55fc619ca74202ff8ad8dd6017d4e",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "1dcb6132afeec8101daf9a3fd73d42141fb871bff11fff28069f238dcff6afd0",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed3/adapter_meta.json",
|
||||
"blobId": "f675496569bbec647fcb31784c4a6d9d83872ee0",
|
||||
"size": 660
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed4/adapter.safetensors",
|
||||
"blobId": "bec0713c08368303a25e2ccef690d37a8cf2d54b",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "5b97b0601585925c19bf849ecb11ad68597d1b0d55551b917ec0f5091f85c2ff",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed4/adapter_meta.json",
|
||||
"blobId": "8535e4793f089b37fa8bf5724cf49ac24fd2750c",
|
||||
"size": 660
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed5/adapter.safetensors",
|
||||
"blobId": "128821f8b3b0591990a3b18c70ca66a16d69aff3",
|
||||
"size": 21293968,
|
||||
"lfs": {
|
||||
"sha256": "3f944a6d210958de5840e5b76e263f2105b9f49d15b9ab505950c2b29a5343af",
|
||||
"size": 21293968,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "gate02/seed5/adapter_meta.json",
|
||||
"blobId": "ae2076c513dc6898a8e30f2cf0fafc4486d8f7c3",
|
||||
"size": 660
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed0/adapter.safetensors",
|
||||
"blobId": "c3d80937be6f8262769b08a53f5e19548c516420",
|
||||
"size": 18932336,
|
||||
"lfs": {
|
||||
"sha256": "93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6",
|
||||
"size": 18932336,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed0/adapter_meta.json",
|
||||
"blobId": "c67a9c7f3db4db8f73cc71d27d4a8a664967b624",
|
||||
"size": 573
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed1/adapter.safetensors",
|
||||
"blobId": "5f0b88be4fa8d1a7fd586ecc640b7ac73fb8a68d",
|
||||
"size": 18932336,
|
||||
"lfs": {
|
||||
"sha256": "63d54e1c2f6bf7d07ca0f6e83a3299bd4b6d00740919bc7cae0b422af05a828e",
|
||||
"size": 18932336,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed1/adapter_meta.json",
|
||||
"blobId": "2f1ebca329c8579362d80f0e9e731b832a139349",
|
||||
"size": 573
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed2/adapter.safetensors",
|
||||
"blobId": "d66df32e3e1e67c4c6b3eccf6229f223fa61a5f8",
|
||||
"size": 18932336,
|
||||
"lfs": {
|
||||
"sha256": "702b764e2c39b3529914dac1547148ef2f462d9ee04e256201e79f00b20ccc62",
|
||||
"size": 18932336,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed2/adapter_meta.json",
|
||||
"blobId": "7e4737b99685c9f9542f7d15ad4efebc45fc26f3",
|
||||
"size": 573
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed3/adapter.safetensors",
|
||||
"blobId": "abb761e518f2bcd7285c7c2cb11579943b942d55",
|
||||
"size": 18932336,
|
||||
"lfs": {
|
||||
"sha256": "7dbf35899fe1186453e599ad8c7a85c56b7fad3d630e371ca026a1e95da2c5a7",
|
||||
"size": 18932336,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed3/adapter_meta.json",
|
||||
"blobId": "655fd5600b67568bc9da02e4480a14483e0a1067",
|
||||
"size": 573
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed4/adapter.safetensors",
|
||||
"blobId": "8bd506ae2220c7d0b025d69a4fbeb9ddaf37198d",
|
||||
"size": 18932336,
|
||||
"lfs": {
|
||||
"sha256": "a5cb1e7fa74dff467bd7f163e1db23d30ef8b1a1c935cfa55dcc8cd1b39082a7",
|
||||
"size": 18932336,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed4/adapter_meta.json",
|
||||
"blobId": "d8d7b0df8c3d7d1e2f5ef2f2f6e02e22478e2055",
|
||||
"size": 573
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed5/adapter.safetensors",
|
||||
"blobId": "487c771e11d8e7ca1fa5bd2670d060e8f77fbaee",
|
||||
"size": 18932336,
|
||||
"lfs": {
|
||||
"sha256": "d513d499dfd409da88b3db7da025ba9872fb48ec92d6b29bc2ce7a092a69745d",
|
||||
"size": 18932336,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "lora/seed5/adapter_meta.json",
|
||||
"blobId": "26fc54919eeaffc92a453ddd31cec660c2ab7f0b",
|
||||
"size": 573
|
||||
},
|
||||
{
|
||||
"rfilename": "warmup/seed0/adapter.safetensors",
|
||||
"blobId": "378bc7e4f70afb2e947fc6f53ae822dc4bc422d2",
|
||||
"size": 18932344,
|
||||
"lfs": {
|
||||
"sha256": "9108bce2acfcc3ccd38d0dfae67ac33253b95bdc94133bd25deb4daa45d41700",
|
||||
"size": 18932344,
|
||||
"pointerSize": 133
|
||||
}
|
||||
},
|
||||
{
|
||||
"rfilename": "warmup/seed0/adapter_meta.json",
|
||||
"blobId": "20989af8c7f2ff5959625878167aca5e0fc7203a",
|
||||
"size": 576
|
||||
}
|
||||
],
|
||||
"spaces": [],
|
||||
"createdAt": "2026-10-12T10:25:44.000Z",
|
||||
"usedStorage": 388053976
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"record": "qwen3.5-2b-grpo",
|
||||
"arm": "lora",
|
||||
"receipt_arm": "ctrl",
|
||||
"seed": 0,
|
||||
"step": 150,
|
||||
"base_model": "Qwen/Qwen3.5-2B",
|
||||
"base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
|
||||
"recipe": "GRPO, correctness-gated length penalty alpha 0.2, 150 steps from the shared warm-up (paper Section 4)",
|
||||
"lora_rank": 16,
|
||||
"lora_alpha": 32,
|
||||
"lora_targets": [
|
||||
"in_proj_qkv",
|
||||
"in_proj_a",
|
||||
"in_proj_b",
|
||||
"out_proj"
|
||||
],
|
||||
"gate": null,
|
||||
"n_tensors": 144,
|
||||
"n_params": 4727808,
|
||||
"source_sha256": "de2794c085f5096fea53374ba1cea31075071bb6de3f227bb3350b3035970447"
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Read-only FP32 effective weights for an existing pure-LoRA checkpoint."""
|
||||
import re
|
||||
import numpy as np
|
||||
from functional.model import PREFIX,StreamingLM
|
||||
from pilot.tensors import load_tensors
|
||||
BASE_REVISION='15852e8c16360a2fea060d615a32b45270f8a8fc'
|
||||
TARGETS={'in_proj_qkv','in_proj_a','in_proj_b','out_proj'}
|
||||
|
||||
class EffectiveWeight:
|
||||
def __init__(self,base,a,b,scale):
|
||||
assert a.ndim==b.ndim==2 and (b.shape[0],a.shape[1])==base.shape and a.shape[0]==b.shape[1]
|
||||
self.base,self.a,self.b,self.scale=base,a,b,scale
|
||||
self.shape,self.numel,self.dtype=base.shape,base.numel,'F32'
|
||||
def read(self,selector=slice(None)):
|
||||
if not isinstance(selector,slice):raise ValueError('Only contiguous row chunks supported')
|
||||
start,stop,step=selector.indices(self.numel);dim=self.shape[1]
|
||||
if step!=1 or start%dim or stop%dim:raise ValueError('Row aligned chunks required')
|
||||
w=self.base.read(slice(start,stop)).reshape(-1,dim)
|
||||
return (w+self.scale*(self.b[start//dim:stop//dim]@self.a)).astype(np.float32).reshape(-1)
|
||||
def evict(self):self.base.evict()
|
||||
|
||||
|
||||
def install(lm,directory,meta,multiplier=1.):
|
||||
assert meta['base_model']=='Qwen/Qwen3.5-2B' and meta['base_revision']==BASE_REVISION and meta['gate'] is None and meta['arm']=='lora'
|
||||
tensors=load_tensors(directory);pairs={};pattern=re.compile(r'^base_model\.model\.model\.(layers\.(\d+)\.linear_attn\.(in_proj_qkv|in_proj_a|in_proj_b|out_proj))\.lora_([AB])\.default\.weight$')
|
||||
for name,tensor in tensors.items():
|
||||
match=pattern.fullmatch(name)
|
||||
if not match:raise ValueError('Unexpected trainable tensor: '+name)
|
||||
target=PREFIX+match[1]+'.weight';assert lm.config['layer_types'][int(match[2])]=='linear_attention'
|
||||
assert target in lm.tensors
|
||||
value=tensor.read().reshape(tensor.shape).copy();tensor.evict();assert np.isfinite(value).all()
|
||||
pairs.setdefault(target,{})[match[4]]=value
|
||||
expected={PREFIX+f'layers.{layer}.linear_attn.{role}.weight' for layer,kind in enumerate(lm.config['layer_types']) if kind=='linear_attention' for role in TARGETS}
|
||||
assert set(pairs)==expected and len(tensors)==meta['n_tensors']
|
||||
scale=meta['lora_alpha']/meta['lora_rank']*multiplier
|
||||
for target,pair in pairs.items():
|
||||
assert set(pair)=={'A','B'} and pair['A'].shape[0]==meta['lora_rank']
|
||||
lm.tensors[target]=EffectiveWeight(lm.tensors[target],pair['A'],pair['B'],scale)
|
||||
return pairs,scale
|
||||
|
||||
|
||||
class ExplicitLoRA(StreamingLM):
|
||||
"""Independent reference: base projection + scaled low-rank output, no W merge."""
|
||||
def linear(self,x,name,rows=256):
|
||||
y=super().linear(x,name,rows);pair=self.pairs.get(PREFIX+name+'.weight')
|
||||
if pair is not None:y=y+self.scale*((x@pair['A'].T)@pair['B'].T)
|
||||
return y.astype(np.float32)
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Paired concentration changes from saved immutable intervention arrays."""
|
||||
import csv,json,sys
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
import matplotlib
|
||||
matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
from dominance.core import concentration
|
||||
|
||||
def stats(x):
|
||||
x=np.asarray(x,dtype=np.float64);v=x[np.isfinite(x)];q=np.quantile(v,[.1,.5,.9])
|
||||
return {'count':int(v.size),'p10':float(q[0]),'median':float(q[1]),'p90':float(q[2]),'mean':float(v.mean())}
|
||||
def csvwrite(path,rows):
|
||||
with path.open('w') as f:w=csv.DictWriter(f,fieldnames=list(rows[0]));w.writeheader();w.writerows(rows)
|
||||
def main(root):
|
||||
root=Path(root);status=json.loads((root/'status.json').read_text());assert status['state']=='completed';samples=json.loads((root/'samples.json').read_text());pool={};observations={};summary={};rows=[];layer_rows=[]
|
||||
for cohort in ('legacy','gsm8k'):
|
||||
for role in ('base','finetuned'):
|
||||
for sample in samples[cohort]:
|
||||
for source in (2,6,10,14,18,22):
|
||||
data=np.load(root/'raw'/f'{cohort}-{role}-{sample["id"]}-{source:02d}.npz')
|
||||
for metric,key,transform in [('distance','JS',np.sqrt),('divergence','JS',lambda x:x),('QKV','QKV',lambda x:x)]:
|
||||
impacts=transform(data['single_'+key])[:,7:,:].transpose(1,2,0);c=concentration(impacts)
|
||||
record={'cohort':cohort,'role':role,'sample':sample['id'],'recipient':source+1,'metric':metric}
|
||||
for name,x in c.items():
|
||||
observations.setdefault((cohort,role,metric,name),[]).append(x.reshape(-1))
|
||||
if name!='valid':
|
||||
for stat,value in stats(x).items():record[name+'_'+stat]=value
|
||||
pool.setdefault((cohort,role,metric,'recipient'),[]).append(np.full(200,source+1))
|
||||
pool.setdefault((cohort,role,metric,'sample'),[]).append(np.full(200,sample['id']))
|
||||
observations.setdefault((cohort,role,metric,'winner'),[]).append(np.argmax(impacts,-1).reshape(-1))
|
||||
rows.append(record)
|
||||
arrays={k:np.concatenate(v) for k,v in observations.items()};np.savez_compressed(root/'observations.npz',**{'__'.join(k):v for k,v in arrays.items()})
|
||||
for cohort in ('legacy','gsm8k'):
|
||||
summary[cohort]={}
|
||||
for metric in ('distance','divergence','QKV'):
|
||||
item={role:{name:stats(arrays[(cohort,role,metric,name)]) for name in ('effective_heads','effective_fraction','concentration','top1','top25','bottom25','heads90','total')} for role in ('base','finetuned')}
|
||||
before=arrays[(cohort,'base',metric,'effective_heads')];after=arrays[(cohort,'finetuned',metric,'effective_heads')];delta=after-before
|
||||
item['paired']={'effective_heads_delta':stats(delta),'increased_fraction':float(np.mean(delta>0)),'decreased_fraction':float(np.mean(delta<0)),'winner_agreement':float(np.mean(arrays[(cohort,'base',metric,'winner')]==arrays[(cohort,'finetuned',metric,'winner')]))}
|
||||
summary[cohort][metric]=item
|
||||
recipients=np.concatenate(pool[(cohort,'base',metric,'recipient')]);names=np.concatenate(pool[(cohort,'base',metric,'sample')])
|
||||
for layer in (3,7,11,15,19,23):
|
||||
which=recipients==layer;a=stats(before[which]);b=stats(after[which]);d=stats(delta[which]);layer_rows.append({'cohort':cohort,'metric':metric,'recipient':layer,'base_median':a['median'],'base_p10':a['p10'],'base_p90':a['p90'],'finetuned_median':b['median'],'finetuned_p10':b['p10'],'finetuned_p90':b['p90'],'paired_delta_median':d['median']})
|
||||
for sample in samples[cohort]:
|
||||
which=names==sample['id'];item.setdefault('per_text',{})[sample['id']]={'base':stats(before[which]),'finetuned':stats(after[which]),'paired_delta':stats(delta[which])}
|
||||
expected=(8.93182211792415,5.7840811612328125,11.67265834734814);b=summary['legacy']['distance']['base']['effective_heads'];np.testing.assert_allclose([b[k] for k in ['median','p10','p90']],expected,rtol=1e-10)
|
||||
(root/'summary.json').write_text(json.dumps(summary,indent=2)+'\n');csvwrite(root/'per-text-layer.csv',rows);csvwrite(root/'per-layer.csv',layer_rows)
|
||||
figs=root/'figures';figs.mkdir(exist_ok=True)
|
||||
def save(fig,name):
|
||||
for ext in ('png','pdf','svg'):fig.savefig(figs/(name+'.'+ext),dpi=180)
|
||||
plt.close(fig)
|
||||
fig,axs=plt.subplots(1,2,figsize=(13,4.8),layout='constrained')
|
||||
for ax,cohort in zip(axs,['legacy','gsm8k']):
|
||||
for role,label,color in [('base','Original release','C0'),('finetuned','GSM8K GRPO LoRA seed0','C1')]:
|
||||
x=np.sort(arrays[(cohort,role,'distance','effective_heads')]);ax.plot(x,np.arange(1,len(x)+1)/len(x),label=label,color=color)
|
||||
ax.set(title='Original four texts' if cohort=='legacy' else 'Four GSM8K held-out question prefixes',xlabel='Effective donor heads: sqrt(JS) influence',ylabel='Empirical CDF',xlim=(1,16),ylim=(0,1));ax.grid(alpha=.2);ax.legend()
|
||||
fig.suptitle('Same inputs before/after existing task fine-tuning; 32 tokens; one training seed; no training here');save(fig,'paired_effective_heads_cdf')
|
||||
fig,axs=plt.subplots(1,2,figsize=(13,4.8),layout='constrained')
|
||||
for ax,cohort in zip(axs,['legacy','gsm8k']):
|
||||
for role,prefix,color in [('base','base','C0'),('finetuned','finetuned','C1')]:
|
||||
rr=[r for r in layer_rows if r['cohort']==cohort and r['metric']=='distance'];x=[r['recipient'] for r in rr];med=[r[prefix+'_median'] for r in rr];lo=[r[prefix+'_p10'] for r in rr];hi=[r[prefix+'_p90'] for r in rr];ax.plot(x,med,'o-',color=color,label=role);ax.fill_between(x,lo,hi,color=color,alpha=.12)
|
||||
ax.set(title=cohort,xlabel='Full-attention recipient layer index',ylabel='Effective donor heads: median / P10–P90',ylim=(1,16));ax.legend();ax.grid(alpha=.2)
|
||||
fig.suptitle('All six full-attention receivers, sixteen donor heads each; descriptive dependent observations');save(fig,'paired_effective_heads_by_layer')
|
||||
fig,axs=plt.subplots(1,2,figsize=(13,4.8),layout='constrained')
|
||||
for ax,cohort in zip(axs,['legacy','gsm8k']):
|
||||
delta=arrays[(cohort,'finetuned','distance','effective_heads')]-arrays[(cohort,'base','distance','effective_heads')];ax.hist(delta,bins=60,density=True,alpha=.7);ax.axvline(0,color='black',linestyle='--');ax.axvline(np.median(delta),color='C3',label='Paired median');ax.set(title=cohort,xlabel='Finetuned minus original effective head count',ylabel='Density');ax.legend()
|
||||
fig.suptitle('Paired changes; positive = more distributed, negative = more concentrated');save(fig,'paired_effective_heads_delta')
|
||||
lines=['# GSM8K任务微调前后的有效头数','', '比较原Qwen3.5-2B发布权重与同一revision上的纯LoRA seed0（额外GSM8K GRPO，非SFT）。没有训练新模型。主指标为√JS距离消融影响的N_eff；正变化表示更分散，负变化表示更集中。','', '| 输入组 | 原权重中位数 [P10,P90] | 微调后中位数 [P10,P90] | 配对变化中位数 | 增加的观测比例 |','|---|---:|---:|---:|---:|']
|
||||
for cohort,label in [('legacy','原四条文本'),('gsm8k','GSM8K测试题前缀')]:
|
||||
item=summary[cohort]['distance'];a=item['base']['effective_heads'];b=item['finetuned']['effective_heads'];d=item['paired']['effective_heads_delta'];lines.append(f"| {label} | {a['median']:.6f} [{a['p10']:.6f},{a['p90']:.6f}] | {b['median']:.6f} [{b['p10']:.6f},{b['p90']:.6f}] | {d['median']:+.6f} | {item['paired']['increased_fraction']:.1%} |")
|
||||
lines+=['', '每组4条输入×6个接收层×25个query位置×8个接收query头=4800个配对观测；它们不是4800个独立文本样本。前后使用同一原模型tokenizer、完全相同token IDs。原四条输入的基线8.931822/16已精确复现。','', 'GSM8K输入取原作者固定版本test split中前4道至少32token的题目，仅用题目前32token，不含答案，不套不同聊天模板。另算了同组GSM8K输入的原权重基线；没有把旧通用文本与新任务文本混作配对。','', '## 方法与限定','', '所有6个普通注意力接收层和16个上游输出头均保留；每次移除一个上游头在全序列的投影输出，重算中间MLP/残差/归一化和检索分布。微调只改变18个GDN层的指定投影，普通注意力权重不变；隐藏状态和上游贡献仍会变化。','', '采用read-only FP32有效权重W+2BA，未把增量舍入为BF16、未修改底座；另用base projection+2B(Ax)独立前向校验。头输出求和、空消融和源数组复用校验均记录。JS散度及QKV复合相对变化也保留，避免只展示有利度量。这里QKV补充指标仅覆盖相同六个普通注意力接收层，其原基线不能与旧研究全部23连接的76.0%混作同一统计量。','', '单个发布者训练seed、4条短文本/组，仅支持这些输入上的描述性变化；不推广为所有微调都会如此，不报告GSM8K准确率，不据此推断删头可用或最终LM能力。配对变化中位数与两个总体中位数之差不是同一个统计量。','', '[公开微调权重与训练说明](https://huggingface.co/Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo)；固定版本、参数SHA、dataset版本和题目索引见protocol.json/gsm8k-source.json/samples.json。所有权重、适配器和缓存继续保留。']
|
||||
(root/'REPORT.md').write_text('\n'.join(lines)+'\n');print((root/'REPORT.md').read_text(),end='')
|
||||
if __name__=='__main__':main(sys.argv[1])
|
||||
@@ -0,0 +1,72 @@
|
||||
"""Paired CPU-only existing-adapter comparison for the published sqrt(JS) metric."""
|
||||
import argparse,json,hashlib,time,resource,gc
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
from tokenizers import Tokenizer
|
||||
from dominance.core import InfluenceLM,concentration
|
||||
from dominance.run import distances
|
||||
from .lora import install,ExplicitLoRA,BASE_REVISION
|
||||
SOURCES=(2,6,10,14,18,22)
|
||||
|
||||
def dump(p,d):p.write_text(json.dumps(d,indent=2,allow_nan=False)+'\n')
|
||||
def single(lm,cache,layer,path):
|
||||
base=lm.recipient(cache['next_raw'],layer+1);hs=cache['heads'].shape[2];t=cache['raw'].shape[1];assert hs==16
|
||||
scores={k:[] for k in ('Q','K','V','QKV','JS')}
|
||||
for start in range(0,hs,4):
|
||||
masks=np.zeros((4,t,hs),dtype=np.float32)
|
||||
for n in range(4):masks[n,:,start+n]=1
|
||||
effects=distances(base,lm.intervene(cache,layer,masks))
|
||||
for k,v in effects.items():scores[k].extend(v)
|
||||
null=distances(base,lm.intervene(cache,layer,np.zeros((1,t,hs),dtype=np.float32)))
|
||||
assert max(float(np.max(v)) for v in null.values())<2e-5
|
||||
payload={'single_'+k:np.asarray(v,dtype=np.float32) for k,v in scores.items()};payload.update({'null_'+k:v for k,v in null.items()});np.savez_compressed(path,**payload)
|
||||
|
||||
def main():
|
||||
p=argparse.ArgumentParser();p.add_argument('--source',type=Path,required=True);p.add_argument('--parent',type=Path,required=True);p.add_argument('--assets',type=Path,required=True);p.add_argument('--out',type=Path,required=True);a=p.parse_args();a.out.mkdir(parents=True,exist_ok=False);(a.out/'cache').mkdir();(a.out/'raw').mkdir()
|
||||
started=time.monotonic();meta=json.loads((a.assets/'adapter_meta.json').read_text());assert meta['base_revision']==BASE_REVISION
|
||||
adapter_sha=hashlib.sha256((a.assets/'adapter.safetensors').read_bytes()).hexdigest();assert adapter_sha=='93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6'
|
||||
tok=Tokenizer.from_file(str(a.source/'tokenizer.json'));legacy=json.loads((a.parent/'samples.json').read_text());gsm=[]
|
||||
for s in legacy:assert s['token_ids']==tok.encode(s['text']).ids[:32]
|
||||
for index,line in enumerate((a.assets/'gsm8k-test.jsonl').read_text().splitlines()):
|
||||
q=json.loads(line)['question'];ids=tok.encode(q).ids
|
||||
if len(ids)<32:continue
|
||||
gsm.append({'id':f'gsm8k-test-{index:04d}','row_index':index,'text':q,'question_sha256':hashlib.sha256(q.encode()).hexdigest(),'token_ids':ids[:32]})
|
||||
if len(gsm)==4:break
|
||||
assert len(gsm)==4
|
||||
cohorts={'legacy':legacy,'gsm8k':gsm};dump(a.out/'samples.json',cohorts);dump(a.out/'adapter_meta.json',meta);dump(a.out/'gsm8k-source.json',json.loads((a.assets/'gsm8k-source.json').read_text()))
|
||||
protocol={'primary':'same six full-attention receivers and original4x32 token cohort as published8.931822/16','adapter_repo':'Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo','adapter_revision':'038d8be9bd0edd3883fa019651b02bdf65609468','adapter_file':'lora/seed0/adapter.safetensors','adapter_sha256':adapter_sha,'base_revision':BASE_REVISION,'scale':2,'rank':16,'arm':'pure LoRA','training':'publisher-reported GSM8K GRPO; we do no training','sample_choice':'four original texts; first4 canonical GSM8K test questions with >=32tokens, length selection only, first32tokens; question only','source_layers':SOURCES,'primary_queries':list(range(7,32)),'main_metric':'concentration(sqrt(single_JS)); same FP32 tensors and sequence-wide donor-head deletion as prior run','cohort_pairing':'each comparison uses same question/tokenIDs before and after; base legacy arrays reused with elementwise checks','runtime':'read-only FP32 effective W+2BA; no BF16 re-rounding or quantization','controls':'head sum, null mask, explicit low-rank versus effective-weight full forward relL2<1e-4','budget_seconds':7200,'keep_all_weights_and_caches':True,'limitations':['one existing training seed','4 texts per cohort, 32 tokens each','dependent query/head observations, not iid textual samples','no generation or accuracy measurement','GDN targets only; ordinary attention weights stay unchanged','no pruning or functional interchangeability claim']}
|
||||
dump(a.out/'protocol.json',protocol);status={'state':'running','links':[]};dump(a.out/'status.json',status);cfg=json.loads((a.source/'model_config.json').read_text());controls=[];references=[]
|
||||
try:
|
||||
for cohort,samples in cohorts.items():
|
||||
for role in ('base','finetuned'):
|
||||
reuse=cohort=='legacy' and role=='base';lm=None
|
||||
if not reuse:
|
||||
lm=InfluenceLM(a.source/'weights',cfg,seconds=max(1,7200-(time.monotonic()-started)))
|
||||
if role=='finetuned':pairs,scale=install(lm,a.assets,meta)
|
||||
for si,s in enumerate(samples):
|
||||
if time.monotonic()-started>7200:raise TimeoutError('Paired CPU budget')
|
||||
if not reuse:
|
||||
cache=a.out/'cache'/f'{cohort}-{role}-{s["id"]}';cache.mkdir();ids=np.array([s['token_ids']],dtype=np.int64);final=lm.baseline(ids,cache);np.save(cache/'final.npy',final)
|
||||
controls.extend({'cohort':cohort,'role':role,'sample':s['id'],**v} for v in lm.head_sum_errors)
|
||||
if cohort=='legacy' and role=='finetuned' and si==0:
|
||||
ref=ExplicitLoRA(a.source/'weights',cfg,seconds=max(1,7200-(time.monotonic()-started)));ref.pairs,ref.scale=pairs,scale;expected,_=ref.forward(ids);actual=lm.normalization(final,'norm');error=float(np.linalg.norm(actual.astype(np.float64)-expected)/np.linalg.norm(expected.astype(np.float64)));assert error<1e-4
|
||||
status['explicit_lora_forward_relative_error']=error;del ref;gc.collect()
|
||||
for layer in SOURCES:
|
||||
out=a.out/'raw'/f'{cohort}-{role}-{s["id"]}-{layer:02d}.npz'
|
||||
if reuse:
|
||||
old=a.parent/'raw'/f'{s["id"]}-link-{layer:02d}.npz'
|
||||
with np.load(old) as d:payload={prefix+k:d[prefix+k] for prefix in ('single_','null_') for k in ('Q','K','V','QKV','JS')}
|
||||
np.savez_compressed(out,**payload)
|
||||
with np.load(out) as d:
|
||||
for k,v in payload.items():np.testing.assert_array_equal(d[k],v)
|
||||
references.append({'source':str(old.resolve()),'sha256':hashlib.sha256(old.read_bytes()).hexdigest(),'derived':out.name})
|
||||
else:
|
||||
with np.load(cache/f'layer-{layer}.npz') as d:loaded={k:d[k] for k in d.files}
|
||||
single(lm,loaded,layer,out);del loaded
|
||||
status['links'].append({'cohort':cohort,'role':role,'sample':s['id'],'source':layer,'path':str(out.relative_to(a.out))});status['elapsed_seconds']=time.monotonic()-started;dump(a.out/'status.json',status);print('completed',cohort,role,s['id'],layer,flush=True)
|
||||
dump(a.out/'head-sum-controls.json',controls);dump(a.out/'baseline-references.json',references)
|
||||
if lm is not None:del lm;gc.collect()
|
||||
status.update(state='completed',elapsed_seconds=time.monotonic()-started,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss);dump(a.out/'status.json',status)
|
||||
except BaseException as e:
|
||||
status.update(state='failed',error=repr(e),elapsed_seconds=time.monotonic()-started);dump(a.out/'status.json',status);raise
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,83 @@
|
||||
# Actual-model parameter clouds: frozen grouping and null protocol
|
||||
|
||||
Assets: retained Qwen3.5-2B original15852e8c16360a2fea060d615a32b45270f8a8fc,
|
||||
and pure GSM8K GRPO LoRA seed0 at HF038d8be9bd0edd3883fa019651b02bdf65609468.
|
||||
Use the established read-only FP32 W+2BA wrapper. No training, GPU, new models,
|
||||
weight mutation, inference requirement or deletion. Every retained asset stays.
|
||||
|
||||
## What constitutes a distribution
|
||||
|
||||
A model has no unique canonical number of parameter distributions. Enumerate a
|
||||
fixed family, not "all possible distributions":
|
||||
1. Each ndim>=2 tensor flattened as [axis0, product(other axes)]: row vectors
|
||||
and column vectors, when point count>=16 and feature dimension>=32.
|
||||
2. Decoder language/MTP full-attention Q and interleaved gate rows per query
|
||||
head; K/V rows per KV head. GDN concatenated Q/K/V rows per corresponding head.
|
||||
No claim that these groups are disjoint or iid; whole-matrix views overlap.
|
||||
3. Primary sample-size stratum has >=64 available points;16..63 is small-sample
|
||||
descriptive only. One-dimensional norm/bias vectors have no cloud of vectors;
|
||||
GDN kernel row dimension4 is low dimensional and excluded with explicit reason.
|
||||
Vision parameters are covered by matrix/flattened views, not vision-head splits.
|
||||
|
||||
Include all checkpoint components (language, vision and MTP). Per view choose
|
||||
up to512 point indices without replacement, split deterministically into calibration
|
||||
and held-out halves. Use full vector coordinates up to dimension8192; only larger
|
||||
vectors use a fixed random coordinate subset, labelled projected rather than full.
|
||||
Use identical indices, coordinates and null random draws for original/FT.
|
||||
No overlap-group parameter savings or total distribution counts are additive.
|
||||
|
||||
## Centers and radius/scaling
|
||||
|
||||
Analyze origin and calibration-mean centers separately. Fit R_max using calibration
|
||||
norms only; do not silently clip held-out points outside R_max. Also fit calibration
|
||||
RMS radius. Store held-out maximum, outside-radius mass and radial spread.
|
||||
|
||||
Null A: uniform d-dimensional ball. Simulate the SAME center-fitting and calibration
|
||||
maximum/RMS fitting before held-out evaluation. For origin, also retain exact
|
||||
expected max-fitted null CDF a^d*n_cal/(n_cal+1), a<=1; log probabilities remain
|
||||
available and zero Monte Carlo hits are not exact zeros.
|
||||
Null B: independent coordinate-uniform box bounded by calibration min/max,
|
||||
including asymmetric coordinates. Fit its center/radii using its own calibration
|
||||
sample by exactly the same procedure. Record energy/range mismatch and held-out
|
||||
coordinate violations; this null is conditional on fitted bounds, not a statement
|
||||
that all real vectors lie inside that box.
|
||||
Null C: independent coordinate-uniform box with calibration mean mu and standard
|
||||
deviation sigma, bounds mu +/- sqrt(3)*sigma (sample variance ddof1). This matches
|
||||
first and diagonal second moments in its generating law. Use its own calibration
|
||||
and held-out center/max/RMS fits too; RMS-normalized comparisons target shape
|
||||
rather than a change of global scale. Matching diagonal moments is not covariance
|
||||
matching and does not preserve correlations.
|
||||
|
||||
Use512 validation null draws (not iid model observations), same calibration count
|
||||
as each data cloud. Reuse isotropic-ball nulls only for identical dimension/count
|
||||
and center-fitting rules. Monte Carlo resolution/error must be stated. Fit both
|
||||
centers from the same raw draws where applicable; paired FT uses common randomness.
|
||||
|
||||
## Metrics and interpretation
|
||||
|
||||
Retain measured norm arrays, empirical CDF grids, max-normalized interior fractions
|
||||
at0.5/0.75/0.9/0.95/0.99 and outside1, RMS-normalized CDF from0.5 to1.5,
|
||||
median radial ratios and radial spread. Prefer probability-point differences over
|
||||
astronomical ratios against tiny a^d. Inspect positive AND negative CDF gaps:
|
||||
more inner tail plus more outer tail is not global inward stochastic dominance.
|
||||
|
||||
Record exact zeros, coordinate energy participation (sum squares)^2/sum fourths,
|
||||
coordinate-variance participation, empirical covariance/second-moment participation
|
||||
via calibration Gram matrices. Label sample rank ceiling n_cal-1 (mean centered)
|
||||
or n_cal (origin); compare to equally sampled nulls, do not interpret the raw
|
||||
finite-sample participation as true intrinsic dimension.
|
||||
|
||||
Report both full-radius and moment-controlled results for each center, whole-view
|
||||
and head subgroups, primary versus small sample/projection strata, original and
|
||||
changed72-LoRA-target groups. Unchanged parameter views can reuse identical results
|
||||
with explicit references rather than re-read/recompute. Results describe parameters,
|
||||
not activations, model quality, functional duplicates, or pruning gains.
|
||||
|
||||
Execution budget: one CPU, 512 MiB RSS ceiling, 7200 seconds per invocation;
|
||||
checkpoint every group, resume without replacing completed groups. Any incomplete
|
||||
coverage is reported before interpretation. No numerical analysis runs locally.
|
||||
|
||||
Illustrative radial CDFs use these fixed original/origin views, selected by module
|
||||
role before the census completes, not by the resulting similarity/interior score:
|
||||
language layer0 linear-attention Q head0, layer3 full-attention Q head0, and layer0
|
||||
MLP up-projection rows. Their plots are illustrations, not independent replications.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Actual-model shell census
|
||||
|
||||
Read [PROTOCOL.md](PROTOCOL.md) before interpreting the results. This analyzes
|
||||
retained parameter arrays, not model activations or task accuracy.
|
||||
|
||||
Run only on the authorized CPU server from the repository root:
|
||||
|
||||
```sh
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m unittest model_shell.test_analysis -v
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze
|
||||
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.report
|
||||
```
|
||||
|
||||
Inputs are the already retained `runs/cpu-qwen35-2b/weights` and
|
||||
`runs/finetune-head-comparison/adapter.safetensors` with their existing JSON
|
||||
configuration and metadata. The adapter is evaluated as FP32 W+2BA. No weights
|
||||
are modified, no training or inference is run, and no files are deleted.
|
||||
|
||||
`runs/model-shell/checkpoints` is a resumable group ledger. Re-running skips
|
||||
completed groups; do not reuse a ledger after changing its protocol/code/inputs.
|
||||
One invocation stops at 7,200 seconds, RSS above 512 MiB or available memory below
|
||||
128 MiB. Resume after addressing a resource stop; do not relax guards silently.
|
||||
|
||||
`results/model-shell` contains the report, group inventory, per-profile metric
|
||||
CSV, compressed complete CDF/control summaries, and measured calibration/held-out
|
||||
radii. The NPZ keys are `GROUPINDEX_MODEL_CENTER_cal` / `_eval`, with zero-based
|
||||
group indices into `inventory.json`. Unchanged fine-tune profiles explicitly
|
||||
reference the original profiles in `profiles.json.gz`; the CSV expands both.
|
||||
Raw null draws are reproducible from the recorded deterministic seeds; null CDFs,
|
||||
scales and participation summaries are stored. `audit.json` verifies coverage,
|
||||
recomputes empirical CDFs from the saved norms, checks finite-sample rank limits,
|
||||
verifies the original asset SHA256 values, and hashes the output artifacts.
|
||||
`example_radii.npz` also saves calibration/held-out radii for the three fixed role
|
||||
examples and their regenerated nulls; their grid CDFs are checked against the
|
||||
original ledger before plotting full empirical step curves from radius zero.
|
||||
|
||||
The count of views is defined by the protocol and is not a unique number of
|
||||
distributions inherent in a neural network. These overlapping views are equally
|
||||
weighted for description; they are not independent replicates. The box nulls
|
||||
match fitted marginal ranges or moments, not correlations. The maximum fitted
|
||||
radius is not the unknown support boundary. Quantities near numerical or Monte
|
||||
Carlo resolution cannot support a definitive population statement.
|
||||
@@ -0,0 +1,124 @@
|
||||
"""CPU-only, read-only parameter cloud census with paired uniform references."""
|
||||
import argparse, hashlib, json, math, os, time
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
import numpy as np
|
||||
from pilot.tensors import load_tensors
|
||||
from finetune_compare.lora import install
|
||||
from model_shell.groups import enumerate_groups
|
||||
|
||||
MAX_GRID=np.array([.5,.75,.9,.95,.99,1.,1.05,1.1,1.25])
|
||||
RMS_GRID=np.linspace(.5,1.5,41)
|
||||
|
||||
def seed(key):return int.from_bytes(hashlib.sha256(key.encode()).digest()[:8],'little')
|
||||
|
||||
def guard(start):
|
||||
mem=dict((s.split(':')[0],int(s.split()[1])) for s in Path('/proc/meminfo').read_text().splitlines() if s.startswith('MemAvailable:'))
|
||||
rss=int(Path('/proc/self/statm').read_text().split()[1])*os.sysconf('SC_PAGE_SIZE')
|
||||
if rss>512*1024**2 or mem['MemAvailable']<128*1024 or time.monotonic()-start>7200:
|
||||
raise RuntimeError(f'Resource guard: RSS={rss}, available_kB={mem}, elapsed={time.monotonic()-start:.0f}')
|
||||
|
||||
def sample(t,g):
|
||||
rng=np.random.default_rng(seed(g['id']+'/indices'))
|
||||
n=min(512,g['population']);d=min(8192,g['dimension'])
|
||||
points=np.sort(rng.choice(g['population'],n,replace=False))
|
||||
features=np.sort(rng.choice(g['dimension'],d,replace=False)) if d<g['dimension'] else np.arange(d)
|
||||
width=math.prod(t.shape[1:]);out=np.empty((n,d),np.float32)
|
||||
if g['orientation']=='rows':
|
||||
for j,i in enumerate(points):
|
||||
row=i+g['start'];out[j]=t.read(slice(row*width,(row+1)*width))[features]
|
||||
if j%32==0:t.evict()
|
||||
else:
|
||||
for j,i in enumerate(features):
|
||||
out[:,j]=t.read(slice(i*width,(i+1)*width))[points]
|
||||
if j%32==0:t.evict()
|
||||
t.evict();out=out[rng.permutation(n)]
|
||||
digest=hashlib.sha256(points.tobytes()+features.tobytes()).hexdigest()
|
||||
return out,digest
|
||||
|
||||
def participation(y):
|
||||
gram=y@y.T;energy=float(np.trace(gram));den=float(np.square(gram).sum())
|
||||
return energy**2/den if den else 0.
|
||||
|
||||
def geometry(x,ncal,center,raw=False):
|
||||
# Float64 arithmetic; bounded sample matrix, no model-wide conversion.
|
||||
z=x.astype(np.float64);mu=z[:ncal].mean(0) if center=='mean' else np.zeros(z.shape[1])
|
||||
y=z-mu;norm=np.linalg.norm(y,axis=1);cal=norm[:ncal];r=norm[ncal:]
|
||||
maximum=float(cal.max());rms=float(np.sqrt(np.mean(cal**2)))
|
||||
valid=maximum>0 and rms>0
|
||||
a=r/maximum if maximum else np.zeros_like(r);b=r/rms if rms else np.zeros_like(r)
|
||||
coordvar=z[:ncal].var(0,ddof=1);cv=float(coordvar.sum()**2/np.square(coordvar).sum()) if coordvar.any() else 0.
|
||||
e=np.square(y[ncal:]);den=np.square(e).sum(1)
|
||||
pe=np.divide(e.sum(1)**2,den,out=np.zeros(len(e)),where=den>0)
|
||||
out=dict(valid=bool(valid),ncal=ncal,neval=len(r),d=z.shape[1],rmax=maximum,rms=rms,
|
||||
median_max=float(np.median(a)),median_rms=float(np.median(b)),
|
||||
p10_rms=float(np.quantile(b,.1)),p90_rms=float(np.quantile(b,.9)),
|
||||
outside=float(np.mean(a>1)),cdf_max=[float(np.mean(a<=v)) for v in MAX_GRID],
|
||||
cdf_rms=[float(np.mean(b<=v)) for v in RMS_GRID],
|
||||
covariance_participation=participation(y[:ncal]),rank_ceiling=ncal-(center=='mean'),
|
||||
coordinate_variance_participation=cv,point_energy_participation_median=float(np.median(pe)),
|
||||
zero_fraction=float(np.mean(z==0)))
|
||||
if raw:out.update(cal_norm=cal.tolist(),eval_norm=r.tolist())
|
||||
return out
|
||||
|
||||
BALL_CACHE={}
|
||||
def references(x,ncal,key):
|
||||
d=x.shape[1];cachekey=(d,ncal)
|
||||
if cachekey not in BALL_CACHE:
|
||||
rng=np.random.default_rng(seed(f'ball/{d}/{ncal}'))
|
||||
y=rng.standard_normal((ncal+512,d),dtype=np.float32)
|
||||
y*=((rng.random(len(y))**(1/d))/np.linalg.norm(y,axis=1))[:,None]
|
||||
BALL_CACHE[cachekey]={c:geometry(y,ncal,c) for c in ('origin','mean')}
|
||||
cal=x[:ncal].astype(np.float64);lo=cal.min(0);hi=cal.max(0);mu=cal.mean(0);sd=cal.std(0,ddof=1)
|
||||
rng=np.random.default_rng(seed(key+'/uniform'))
|
||||
u=rng.random((ncal+512,d),dtype=np.float32)
|
||||
refs={'ball':BALL_CACHE[cachekey]}
|
||||
for name,left,right in [('range_box',lo,hi),('moment_box',mu-np.sqrt(3)*sd,mu+np.sqrt(3)*sd)]:
|
||||
y=(left+(right-left)*u).astype(np.float32)
|
||||
refs[name]={c:geometry(y,ncal,c) for c in ('origin','mean')}
|
||||
return refs,dict(heldout_outside_coordinate_range=float(np.mean((x[ncal:]<lo)|(x[ncal:]>hi))),
|
||||
range_box_expected_energy=float(np.sum((lo**2+lo*hi+hi**2)/3)),
|
||||
observed_cal_energy=float(np.mean(np.sum(cal**2,axis=1))))
|
||||
|
||||
def analyze(x,key):
|
||||
ncal=len(x)//2;refs,extra=references(x,ncal,key);out={}
|
||||
for center in ('origin','mean'):
|
||||
p=geometry(x,ncal,center,True);p['references']=refs; p.update(extra)
|
||||
p['comparisons']={}
|
||||
for name,bycenter in refs.items():
|
||||
ref=bycenter[center];diff=np.array(p['cdf_rms'])-ref['cdf_rms']
|
||||
p['comparisons'][name]=dict(interior90_gap=p['cdf_max'][2]-ref['cdf_max'][2],
|
||||
interior95_gap=p['cdf_max'][3]-ref['cdf_max'][3],
|
||||
rms_cdf_positive_gap=float(max(0,diff.max())),rms_cdf_negative_gap=float(max(0,-diff.min())),
|
||||
median_rms_delta=p['median_rms']-ref['median_rms'],
|
||||
participation_ratio=p['covariance_participation']/ref['covariance_participation'] if ref['covariance_participation'] else None)
|
||||
p['ball_origin_log_expected_cdf90']=float(x.shape[1]*np.log(.9)+np.log(ncal/(ncal+1)))
|
||||
out[center]=p
|
||||
return out
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser();ap.add_argument('--root',default='runs');ap.add_argument('--out',default='runs/model-shell');args=ap.parse_args()
|
||||
root=Path(args.root);out=Path(args.out);out.mkdir(exist_ok=True,parents=True);(out/'checkpoints').mkdir(exist_ok=True)
|
||||
cfg=json.loads((root/'cpu-qwen35-2b/model_config.json').read_text())['text_config']
|
||||
base=load_tensors(root/'cpu-qwen35-2b/weights');ft=SimpleNamespace(tensors=dict(base),config=cfg)
|
||||
meta=json.loads((root/'finetune-head-comparison/adapter_meta.json').read_text())
|
||||
pairs,scale=install(ft,root/'finetune-head-comparison',meta)
|
||||
groups,skips=enumerate_groups(base,cfg)
|
||||
manifest=dict(groups=groups,skips=skips,total_tensors=len(base),parameters=sum(t.numel for t in base.values()),changed_tensors=sorted(pairs),adapter_meta=meta,
|
||||
max_grid=MAX_GRID.tolist(),rms_grid=RMS_GRID.tolist(),null_eval=512,point_cap=512,feature_cap=8192)
|
||||
(out/'inventory.json').write_text(json.dumps(manifest,indent=2));start=time.monotonic()
|
||||
for i,g in enumerate(groups):
|
||||
target=out/'checkpoints'/f'{i:04d}.json'
|
||||
if target.exists():continue
|
||||
guard(start);x,digest=sample(base[g['tensor']],g);p=analyze(x,g['id']);del x
|
||||
changed=g['tensor'] in pairs
|
||||
if changed:
|
||||
guard(start);x,digest2=sample(ft.tensors[g['tensor']],g);assert digest==digest2
|
||||
q=analyze(x,g['id']);del x
|
||||
else:q=None
|
||||
record=dict(group=g,sampling_sha256=digest,changed=changed,original=p,finetuned=q,finetuned_reuses_original=not changed)
|
||||
temp=target.with_suffix('.tmp');temp.write_text(json.dumps(record,allow_nan=False));temp.replace(target)
|
||||
if i%10==0:print(f'{i+1}/{len(groups)} elapsed={time.monotonic()-start:.1f}s',flush=True)
|
||||
print('COMPLETE',len(groups),flush=True)
|
||||
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,29 @@
|
||||
"""Deterministic finite family of matrix/vector clouds; no data-dependent choice."""
|
||||
import math,re
|
||||
|
||||
def enumerate_groups(tensors,cfg):
|
||||
groups=[];skips=[]
|
||||
for name,t in sorted(tensors.items()):
|
||||
if len(t.shape)<2:skips.append({'name':name,'reason':'one vector, no vector cloud','shape':t.shape});continue
|
||||
rows=t.shape[0];cols=math.prod(t.shape[1:]);count=0
|
||||
for axis,n,d in [('rows',rows,cols),('columns',cols,rows)]:
|
||||
if n>=16 and d>=32:
|
||||
groups.append({'id':name+'::'+axis,'tensor':name,'kind':'matrix','orientation':axis,'start':0,'stop':rows,'population':n,'dimension':d,'stratum':'primary' if n>=64 else 'small_sample'});count+=1
|
||||
if not count:skips.append({'name':name,'reason':'insufficient points or dimension <32','shape':t.shape})
|
||||
if not (name.startswith('model.language_model.layers.') or name.startswith('mtp.layers.')):continue
|
||||
segments=[]
|
||||
if name.endswith('.linear_attn.in_proj_qkv.weight'):
|
||||
kh,kd,vh,vd=[cfg[k] for k in ('linear_num_key_heads','linear_key_head_dim','linear_num_value_heads','linear_value_head_dim')]
|
||||
for role,num,dim,offset in [('Q',kh,kd,0),('K',kh,kd,kh*kd),('V',vh,vd,2*kh*kd)]:
|
||||
for h in range(num):segments.append((role,h,offset+h*dim,offset+(h+1)*dim))
|
||||
elif name.endswith('.self_attn.q_proj.weight'):
|
||||
dim=cfg['head_dim'];assert rows==cfg['num_attention_heads']*2*dim
|
||||
for h in range(cfg['num_attention_heads']):
|
||||
for role,offset in [('Q',0),('gate',dim)]:segments.append((role,h,h*2*dim+offset,h*2*dim+offset+dim))
|
||||
elif name.endswith(('.self_attn.k_proj.weight','.self_attn.v_proj.weight')):
|
||||
role='K' if '.k_proj.' in name else 'V';dim=cfg['head_dim'];assert rows==cfg['num_key_value_heads']*dim
|
||||
for h in range(cfg['num_key_value_heads']):segments.append((role,h,h*dim,(h+1)*dim))
|
||||
for role,h,start,stop in segments:
|
||||
assert stop<=rows
|
||||
groups.append({'id':name+f'::{role}_head{h}','tensor':name,'kind':'head_rows','orientation':'rows','start':start,'stop':stop,'population':stop-start,'dimension':cols,'stratum':'primary' if stop-start>=64 else 'small_sample'})
|
||||
return groups,skips
|
||||
@@ -0,0 +1,176 @@
|
||||
"""Validate complete checkpoints; write compact scientific data, figures and report."""
|
||||
import argparse,csv,gzip,hashlib,json,platform,sys
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
import matplotlib
|
||||
matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
from model_shell.analyze import MAX_GRID,RMS_GRID,sample,seed
|
||||
from pilot.tensors import load_tensors
|
||||
|
||||
def sha(p):
|
||||
h=hashlib.sha256()
|
||||
with p.open('rb') as f:
|
||||
for b in iter(lambda:f.read(1024*1024),b''):h.update(b)
|
||||
return h.hexdigest()
|
||||
|
||||
def main():
|
||||
ap=argparse.ArgumentParser();ap.add_argument('--input',default='runs/model-shell');ap.add_argument('--out',default='results/model-shell');a=ap.parse_args()
|
||||
source=Path(a.input);out=Path(a.out);out.mkdir(parents=True,exist_ok=True);figdir=out/'figures';figdir.mkdir(exist_ok=True)
|
||||
inventory=json.loads((source/'inventory.json').read_text());groups=inventory['groups'];records=[];rows=[];radii={}
|
||||
for i,g in enumerate(groups):
|
||||
r=json.loads((source/'checkpoints'/f'{i:04d}.json').read_text());assert r['group']==g
|
||||
assert r['changed']==(g['tensor'] in inventory['changed_tensors'])
|
||||
for model in ('original','finetuned'):
|
||||
profiles=r[model] if r[model] is not None else r['original']
|
||||
for center,p in profiles.items():
|
||||
if center=='origin':
|
||||
p['ball_log_expected_max_cdf']=[float(p['d']*np.log(z)+np.log(p['ncal']/(p['ncal']+1))) for z in MAX_GRID if z<=1]
|
||||
assert p['valid'] and p['d']==min(8192,g['dimension'])
|
||||
assert p['covariance_participation']<=p['rank_ceiling']+1e-6
|
||||
v=np.asarray(p['eval_norm']);cal=np.asarray(p['cal_norm'])
|
||||
assert abs(p['rmax']-cal.max())<1e-12
|
||||
assert abs(p['outside']-np.mean(v>p['rmax']))<1e-12
|
||||
np.testing.assert_allclose(p['cdf_max'],[np.mean(v<=z*p['rmax']) for z in MAX_GRID])
|
||||
np.testing.assert_allclose(p['cdf_rms'],[np.mean(v<=z*p['rms']) for z in RMS_GRID])
|
||||
key=f'{i}_{model}_{center}';radii[key+'_cal']=cal;radii[key+'_eval']=v
|
||||
row=dict(group_index=i,group=g['id'],model=model,center=center,kind=g['kind'],stratum=g['stratum'],changed=r['changed'],projected=g['dimension']>8192,
|
||||
dimension=g['dimension'],analyzed_dimension=p['d'],population=g['population'],ncal=p['ncal'],neval=p['neval'],sampling_sha256=r['sampling_sha256'])
|
||||
for k in ('median_max','median_rms','p10_rms','p90_rms','outside','rmax','rms','covariance_participation','rank_ceiling','coordinate_variance_participation','point_energy_participation_median','zero_fraction','heldout_outside_coordinate_range'):
|
||||
row[k]=p[k]
|
||||
for name,c in p['comparisons'].items():
|
||||
for k,vv in c.items():row[name+'_'+k]=vv
|
||||
rows.append(row)
|
||||
# Compact profiles keep CDF and controls; raw measured radii live in NPZ.
|
||||
for model in ('original','finetuned'):
|
||||
if r[model] is not None:
|
||||
for p in r[model].values():p.pop('cal_norm');p.pop('eval_norm')
|
||||
records.append(r)
|
||||
with gzip.open(out/'profiles.json.gz','wt',encoding='utf-8') as f:json.dump(records,f,allow_nan=False)
|
||||
np.savez_compressed(out/'measured_radii.npz',**radii)
|
||||
with (out/'metrics.csv').open('w',newline='') as f:
|
||||
w=csv.DictWriter(f,fieldnames=list(rows[0]));w.writeheader();w.writerows(rows)
|
||||
(out/'inventory.json').write_text(json.dumps(inventory,indent=2))
|
||||
def subset(model='original',center='origin',kind=None,changed=None):
|
||||
return [r for r in rows if r['model']==model and r['center']==center and r['stratum']=='primary' and not r['projected'] and (kind is None or r['kind']==kind) and (changed is None or r['changed']==changed)]
|
||||
summary={}
|
||||
for center in ('origin','mean'):
|
||||
for kind in ('matrix','head_rows'):
|
||||
rs=subset(center=center,kind=kind);s={'n':len(rs),
|
||||
'median_zero_fraction':float(np.median([r['zero_fraction'] for r in rs])),
|
||||
'median_coordinate_energy_participation_fraction':float(np.median([r['point_energy_participation_median']/r['analyzed_dimension'] for r in rs])),
|
||||
'median_coordinate_variance_participation_fraction':float(np.median([r['coordinate_variance_participation']/r['analyzed_dimension'] for r in rs]))}
|
||||
for ref in ('ball','range_box','moment_box'):
|
||||
gap=np.array([r[ref+'_interior90_gap'] for r in rs]);pos=np.array([r[ref+'_rms_cdf_positive_gap'] for r in rs]);neg=np.array([r[ref+'_rms_cdf_negative_gap'] for r in rs])
|
||||
s[ref]=dict(median_interior90_gap=float(np.median(gap)),fraction_gap_gt10pp=float(np.mean(gap>.1)),median_positive_rms_gap=float(np.median(pos)),median_negative_rms_gap=float(np.median(neg)),
|
||||
fraction_two_sided_rms_gt10pp=float(np.mean((pos>.1)&(neg>.1))),median_participation_ratio=float(np.median([r[ref+'_participation_ratio'] for r in rs])))
|
||||
summary[center+'/'+kind]=s
|
||||
paired={}
|
||||
for center in ('origin','mean'):
|
||||
b=subset(center=center,changed=True);t=subset(model='finetuned',center=center,changed=True);assert [r['group'] for r in b]==[r['group'] for r in t]
|
||||
paired[center]={'n':len(b)}
|
||||
for field in ('median_max','median_rms','covariance_participation','moment_box_interior90_gap'):
|
||||
delta=np.array([q[field]-p[field] for p,q in zip(b,t)])
|
||||
paired[center][field]=dict(median=float(np.median(delta)),p10=float(np.quantile(delta,.1)),p90=float(np.quantile(delta,.9)),max_abs=float(np.max(abs(delta))))
|
||||
secondary=[]
|
||||
for model in ('original','finetuned'):
|
||||
for center in ('origin','mean'):
|
||||
for label in ('small_sample','projected'):
|
||||
ss=[r for r in rows if r['model']==model and r['center']==center and (r['stratum']=='small_sample' if label=='small_sample' else r['projected'])]
|
||||
secondary.append(dict(model=model,center=center,stratum=label,n=len(ss),median_max=float(np.median([r['median_max'] for r in ss])),moment_interior90_gap=float(np.median([r['moment_box_interior90_gap'] for r in ss]))))
|
||||
(out/'summary.json').write_text(json.dumps(dict(strata=summary,paired_changed=paired,secondary=secondary),indent=2))
|
||||
plt.rcParams.update({'font.size':10,'axes.spines.top':False,'axes.spines.right':False})
|
||||
def save(fig,name):
|
||||
fig.tight_layout()
|
||||
for ext in ('png','pdf','svg'):fig.savefig(figdir/(name+'.'+ext),dpi=180,bbox_inches='tight')
|
||||
plt.close(fig)
|
||||
fig,axs=plt.subplots(2,2,figsize=(11,7),sharex=True,sharey=True)
|
||||
for i,center in enumerate(('origin','mean')):
|
||||
for j,kind in enumerate(('matrix','head_rows')):
|
||||
rs=subset(center=center,kind=kind);ax=axs[i,j]
|
||||
for ref,label in [('ball','Uniform ball'),('range_box','Range-matched box'),('moment_box','Moment-matched box')]:
|
||||
v=np.sort([r[ref+'_interior90_gap'] for r in rs]);ax.plot(v,np.arange(1,len(v)+1)/len(v),label=label)
|
||||
ax.axvline(0,color='gray',lw=.8);ax.set_title(f'{center} center / {kind}, n={len(rs)}');ax.set_xlabel('F(data r/Rcal <= 0.9) - F(null)');ax.set_ylabel('Fraction of sampled groups')
|
||||
axs[0,0].legend(fontsize=8);fig.suptitle('Actual Qwen3.5-2B: interior mass depends on the reference');save(fig,'interior_reference_sensitivity')
|
||||
fig,axs=plt.subplots(1,2,figsize=(11,4.3))
|
||||
for ax,center in zip(axs,('origin','mean')):
|
||||
for kind,marker in [('matrix','o'),('head_rows','.')]:
|
||||
rs=subset(center=center,kind=kind);ax.scatter([r['moment_box_rms_cdf_positive_gap'] for r in rs],[r['moment_box_rms_cdf_negative_gap'] for r in rs],s=10,alpha=.25,label=kind,marker=marker)
|
||||
ax.axvline(.1,color='gray',lw=.7);ax.axhline(.1,color='gray',lw=.7);ax.set(xlabel='Largest positive CDF gap (more inner tail)',ylabel='Largest negative CDF gap (more outer tail)',title=f'{center} / moment-matched box',xlim=(-.02,1),ylim=(-.02,1));ax.legend()
|
||||
fig.suptitle('RMS-normalized radial shape: both tails can grow together');save(fig,'radial_shape_two_tails')
|
||||
fig,axs=plt.subplots(1,2,figsize=(11,4.3))
|
||||
for ax,center in zip(axs,('origin','mean')):
|
||||
b=subset(center=center,changed=True);t=subset(model='finetuned',center=center,changed=True)
|
||||
x=[r['median_max'] for r in b];y=[r['median_max'] for r in t];ax.scatter(x,y,s=10,alpha=.3);low=min(x+y);high=max(x+y);ax.plot([low,high],[low,high],color='gray',lw=1)
|
||||
ax.set(xlabel='Original median r/Rcal',ylabel='GRPO-LoRA median r/Rcal',title=f'{center}, changed primary groups n={len(b)}')
|
||||
fig.suptitle('Paired same-index comparison; unchanged groups excluded');save(fig,'finetune_paired_radii')
|
||||
fig,axs=plt.subplots(1,2,figsize=(11,4.3))
|
||||
for ax,center in zip(axs,('origin','mean')):
|
||||
for kind in ('matrix','head_rows'):
|
||||
rs=subset(center=center,kind=kind);v=np.sort([r['moment_box_participation_ratio'] for r in rs]);ax.plot(v,np.arange(1,len(v)+1)/len(v),label=kind)
|
||||
ax.set(xlabel='Empirical participation / equal-sample moment-box reference',ylabel='Fraction of groups',title=center);ax.axvline(1,color='gray',lw=.8);ax.legend()
|
||||
fig.suptitle('Finite-sample participation is not the true intrinsic dimension');save(fig,'participation_reference')
|
||||
examples=[('model.language_model.layers.0.linear_attn.in_proj_qkv.weight::Q_head0','L0 GDN Q head0'),
|
||||
('model.language_model.layers.3.self_attn.q_proj.weight::Q_head0','L3 full-attention Q head0'),
|
||||
('model.language_model.layers.0.mlp.up_proj.weight::rows','L0 MLP up rows')]
|
||||
fig,axs=plt.subplots(2,3,figsize=(13,7),sharey=True)
|
||||
byid={r['group']['id']:r for r in records}
|
||||
indices={g['id']:i for i,g in enumerate(groups)}
|
||||
base=load_tensors(Path('runs/cpu-qwen35-2b/weights'));example_radii={}
|
||||
for j,(gid,label) in enumerate(examples):
|
||||
p=byid[gid]['original']['origin']
|
||||
g=byid[gid]['group'];x,digest=sample(base[g['tensor']],g);assert digest==byid[gid]['sampling_sha256']
|
||||
ncal=len(x)//2;d=x.shape[1];cal=x[:ncal].astype(np.float64)
|
||||
rng=np.random.default_rng(seed(gid+'/uniform'));u=rng.random((ncal+512,d),dtype=np.float32)
|
||||
mu=cal.mean(0);sd=cal.std(0,ddof=1);lo=mu-np.sqrt(3)*sd;hi=mu+np.sqrt(3)*sd
|
||||
moment=(lo+(hi-lo)*u).astype(np.float32)
|
||||
rng=np.random.default_rng(seed(f'ball/{d}/{ncal}'));ball=rng.standard_normal((ncal+512,d),dtype=np.float32)
|
||||
ball*=((rng.random(len(ball))**(1/d))/np.linalg.norm(ball,axis=1))[:,None]
|
||||
arrays={'Actual weights':(radii[f'{indices[gid]}_original_origin_cal'],radii[f'{indices[gid]}_original_origin_eval'])}
|
||||
for name,ref,raw in [('Uniform ball','ball',ball),('Moment box','moment_box',moment)]:
|
||||
nr=np.linalg.norm(raw.astype(np.float64),axis=1);arrays[name]=(nr[:ncal],nr[ncal:])
|
||||
np.testing.assert_allclose([np.mean(nr[ncal:]/nr[:ncal].max()<=z) for z in MAX_GRID],p['references'][ref]['origin']['cdf_max'])
|
||||
np.testing.assert_allclose([np.mean(nr[ncal:]/np.sqrt(np.mean(nr[:ncal]**2))<=z) for z in RMS_GRID],p['references'][ref]['origin']['cdf_rms'])
|
||||
for name,(nc,ne) in arrays.items():example_radii[f'example{j}_{name.replace(" ","_")}_cal']=nc;example_radii[f'example{j}_{name.replace(" ","_")}_eval']=ne
|
||||
for i,(field,grid) in enumerate([('cdf_max',MAX_GRID),('cdf_rms',RMS_GRID)]):
|
||||
ax=axs[i,j]
|
||||
for name,(nc,ne) in arrays.items():
|
||||
scale=nc.max() if i==0 else np.sqrt(np.mean(nc**2));v=np.sort(ne/scale)
|
||||
ax.step(np.r_[0,v, max(1.5,float(v[-1]))],np.r_[0,np.arange(1,len(v)+1)/len(v),1],where='post',label=name,lw=1.8,ls='-' if name=='Actual weights' else '--')
|
||||
ax.set(title=label,xlabel='r / calibration '+('maximum' if i==0 else 'RMS'),ylabel='Held-out CDF',ylim=(-.03,1.03),xlim=(0,1.5));ax.legend(fontsize=8)
|
||||
np.savez_compressed(out/'example_radii.npz',**example_radii)
|
||||
fig.suptitle('Fixed role examples: maximum-radius and RMS normalization');save(fig,'fixed_role_radial_cdfs')
|
||||
lines=['# 两个实际2B检查点：参数云的球壳检验','',f'完整覆盖：{len(groups)}组/模型，{len(rows)}个模型×中心组合。632个参数张量，2,274,069,824个参数（包含语言、视觉和MTP）。',
|
||||
'','## 分布计数与范围','', '固定规则给出554个矩阵行／列视图、1,004个语言/MTP头视图；视图互相重叠，不能相加成参数节省量。1,522组有至少64个可用点，36组只有16个点；1组高维嵌入列视图使用8,192个抽样坐标。主表只汇总样本充足且没有投影的1,521组。319个一维张量和18个低维卷积核不构成此规则下的高维样本云。',
|
||||
'','## 原模型结果（逐组等权，非独立观测）','', '|中心|视图|组数|相对均匀球：90%半径内质量差中位数|相对矩匹配均匀盒：质量差中位数|矩匹配且RMS归一后，内外两尾差均>10个百分点的组占比|','|---|---|---:|---:|---:|---:|']
|
||||
for key,s in summary.items():
|
||||
c,k=key.split('/');lines.append(f'|{c}|{k}|{s["n"]}|{s["ball"]["median_interior90_gap"]:.3f}|{s["moment_box"]["median_interior90_gap"]:.3f}|{s["moment_box"]["fraction_two_sided_rms_gt10pp"]:.1%}|')
|
||||
lines+=['','这些是描述性质量差，不是显著性检验。正差表示所指定半径以内有更多样本；不能将其自动解释为整条径向分布向内平移。若RMS归一后内外尾均变多，更准确的解释是径向分布更宽。',
|
||||
'', '|中心/视图|逐组零值率中位数|点坐标能量参与率÷维度，中位数|坐标方差参与率÷维度，中位数|','|---|---:|---:|---:|']
|
||||
for key,s in summary.items():lines.append(f'|{key}|{s["median_zero_fraction"]:.6g}|{s["median_coordinate_energy_participation_fraction"]:.4f}|{s["median_coordinate_variance_participation_fraction"]:.4f}|')
|
||||
lines+=['','能量参与率衡量坐标能量集中程度，依赖坐标基；有限样本Gram参与率反映样本间几何，另见图。两者均不是精确非零坐标计数，也不是模型功能的有效维度。',
|
||||
'','## 微调前后','', '原模型是官方Qwen3.5-2B；第二个模型是在同一revision上进行GSM8K GRPO的已有纯LoRA检查点，不是另一个完整模型。72个投影矩阵使用只读W+2BA展开，其余参数共享。972个视图受影响；其中936个属于主分析。未改变视图复用完全相同的分析结果。','', '|中心|指标|配对变化中位数|P10|P90|最大绝对变化|','|---|---|---:|---:|---:|---:|']
|
||||
for c,p in paired.items():
|
||||
for k,v in p.items():
|
||||
if k!='n':lines.append(f'|{c}|{k}|{v["median"]:.6g}|{v["p10"]:.6g}|{v["p90"]:.6g}|{v["max_abs"]:.6g}|')
|
||||
lines+=['','## 单独描述的小样本与坐标抽样组','', '|模型|中心|类别|组数|中位r/Rcal的组间中位数|相对矩匹配盒的90%半径质量差中位数|','|---|---|---|---:|---:|---:|']
|
||||
for s in secondary:lines.append(f'|{s["model"]}|{s["center"]}|{s["stratum"]}|{s["n"]}|{s["median_max"]:.4f}|{s["moment_interior90_gap"]:.4f}|')
|
||||
lines+=['','## 如何读图','', '1. `interior_reference_sensitivity`：不同均匀参照下，90%校准半径内质量差的组间CDF。横轴向右表示更多内侧质量。','2. `radial_shape_two_tails`：匹配矩且按各自校准RMS归一后的最大正／负CDF差。右上角同时增加两尾，并非单纯向内。','3. `finetune_paired_radii`：同一视图的微调前后中位半径；对角线表示不变。','4. `participation_reference`：Gram矩阵参与率相对同样本量参照的比值。受样本秩上限影响，不能当作真实内在维度。',
|
||||
'5. `fixed_role_radial_cdfs`：预先按角色固定的GDN Q、全注意力Q和MLP上投影示例，上排按校准最大半径、下排按RMS归一化。不是按结果挑选最明显的组。',
|
||||
'','## 方法与限制','', '每组最多512个不同参数向量，一半校准、一半留出；模型固定，留出指未用于拟合中心/半径的参数向量，不是独立训练模型。对照每组512个验证向量，最坏单项二项标准误约2.21个百分点；模型留出样本的误差还取决于样本数，且参数向量不保证独立。10个百分点仅是描述性筛选尺度，没有多重检验或总体显著性结论。',
|
||||
'','均匀球、坐标范围均匀盒、匹配均值和逐坐标方差的均匀盒都按同一规则拟合自身中心、最大半径及RMS。均匀球原点中心的精确拟合半径CDF为a^d*ncal/(ncal+1)，保留其对数以避免把模拟零命中误报成零概率。盒对照不是协方差匹配。校准最大半径不是已知总体支撑边界；留出越界率已记录。',
|
||||
'','`origin`保留坐标均值贡献；`mean`使用校准向量均值。没有使用所有数据来选边界、训练或推理；没有删除模型。坐标稀疏性、有限样本参与率与径向内侧质量分别测量，不能相互代替。该实验不能证明更好的模型更靠里，也不能证明功能重复、可剪枝或可复用。',
|
||||
'','复现：`OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -m model_shell.analyze`，完成后 `... python -m model_shell.report`。只在已授权CPU服务器运行。分组协议见 `model_shell/PROTOCOL.md`；全量指标在metrics.csv，完整CDF/参照在profiles.json.gz，实测范数在measured_radii.npz。']
|
||||
(out/'REPORT.md').write_text('\n'.join(lines)+'\n')
|
||||
sources={'base':Path('runs/cpu-qwen35-2b/weights/model.safetensors-00001-of-00001.safetensors'),'adapter':Path('runs/finetune-head-comparison/adapter.safetensors'),
|
||||
'config':Path('runs/cpu-qwen35-2b/model_config.json'),'adapter_meta':Path('runs/finetune-head-comparison/adapter_meta.json')}
|
||||
sourcehash={k:dict(path=str(p),bytes=p.stat().st_size,sha256=sha(p)) for k,p in sources.items()}
|
||||
(out/'environment.json').write_text(json.dumps(dict(python=sys.version,platform=platform.platform(),numpy=np.__version__,matplotlib=matplotlib.__version__,
|
||||
source_code_sha256={str(p):sha(p) for p in sorted(Path('model_shell').glob('*.py'))},protocol_sha256=sha(Path('model_shell/PROTOCOL.md')),seed='first 8 SHA256 bytes, little endian; group id plus purpose suffix',
|
||||
thread_limit=1,server='8.211.154.134',training=False,inference=False,gpu=False),indent=2))
|
||||
assert sourcehash['base']['sha256']=='aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1'
|
||||
assert sourcehash['adapter']['sha256']=='93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6'
|
||||
audit=dict(complete_groups=len(groups),profile_rows=len(rows),all_finite=True,cdf_recomputed=True,sample_rank_checked=True,sources=sourcehash,files={str(p.relative_to(out)):sha(p) for p in sorted(out.rglob('*')) if p.is_file() and p.name!='audit.json'})
|
||||
(out/'audit.json').write_text(json.dumps(audit,indent=2));print(json.dumps(dict(summary=summary,paired=paired),indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
@@ -0,0 +1,49 @@
|
||||
"""Run on the authorized CPU server: python -m unittest model_shell.test_analysis."""
|
||||
import unittest
|
||||
import numpy as np
|
||||
from model_shell.analyze import sample,geometry,analyze,seed
|
||||
from finetune_compare.lora import EffectiveWeight
|
||||
|
||||
class ArrayTensor:
|
||||
def __init__(self,x):self.x=x;self.shape=x.shape;self.numel=x.size;self.dtype='F32'
|
||||
def read(self,s=slice(None)):return self.x.reshape(-1)[s].copy()
|
||||
def evict(self):pass
|
||||
|
||||
class Checks(unittest.TestCase):
|
||||
def test_sampling_and_lora(self):
|
||||
rng=np.random.default_rng(17);w=rng.normal(size=(80,64)).astype('f');a=rng.normal(size=(2,64)).astype('f');b=rng.normal(size=(80,2)).astype('f')
|
||||
effective=EffectiveWeight(ArrayTensor(w),a,b,2);explicit=ArrayTensor(w+2*b@a)
|
||||
for axis,n,d,start,stop in [('rows',80,64,0,80),('columns',64,80,0,80),('rows',32,64,16,48)]:
|
||||
g=dict(id=axis+str(start),orientation=axis,population=n,dimension=d,start=start,stop=stop)
|
||||
x,h=sample(effective,g);y,k=sample(explicit,g);np.testing.assert_allclose(x,y,atol=2e-6,rtol=1e-5);self.assertEqual(h,k)
|
||||
self.assertEqual(x.shape,(n,d))
|
||||
rr=np.random.default_rng(seed(g['id']+'/indices'));indices=np.sort(rr.choice(n,n,replace=False));order=rr.permutation(n)
|
||||
cloud=explicit.x[start:stop] if axis=='rows' else explicit.x.T
|
||||
np.testing.assert_allclose(y,cloud[indices][order],atol=0,rtol=0)
|
||||
def test_center_scale_and_rank(self):
|
||||
rng=np.random.default_rng(5);x=rng.normal(size=(128,96)).astype('f')
|
||||
for center in ('origin','mean'):
|
||||
p=geometry(x,64,center);q=geometry(x*3,64,center)
|
||||
np.testing.assert_allclose(p['cdf_rms'],q['cdf_rms']);self.assertLessEqual(p['covariance_participation'],p['rank_ceiling']+1e-6)
|
||||
p=geometry(x,64,'mean');q=geometry(x+4,64,'mean');np.testing.assert_allclose(p['median_rms'],q['median_rms'],rtol=1e-6)
|
||||
self.assertFalse(geometry(np.zeros((32,64)),16,'mean')['valid'])
|
||||
def test_ball_fit_and_sparse_counterexample(self):
|
||||
rng=np.random.default_rng(33);d=32;ncal=256
|
||||
# Analytic expectation over independent calibration maxima.
|
||||
c=rng.random((20000,ncal))**(1/d);v=rng.random(20000)**(1/d)
|
||||
observed=np.mean(v<=.99*c.max(1));expected=.99**d*ncal/(ncal+1)
|
||||
self.assertLess(abs(observed-expected),.015)
|
||||
radii=rng.random(512)**(1/d);x=np.zeros((512,d),dtype='f');x[:,0]=radii
|
||||
p=geometry(x,256,'origin');self.assertGreater(p['median_max'],.9)
|
||||
# Sparse support alone retains the full-dimensional radial law.
|
||||
self.assertLess(p['cdf_max'][2],.08)
|
||||
x[:,0]=rng.random(512)**.5
|
||||
self.assertGreater(geometry(x,256,'origin')['cdf_max'][2],.65)
|
||||
sphere=rng.normal(size=(512,d));sphere/=np.linalg.norm(sphere,axis=1)[:,None]
|
||||
self.assertEqual(geometry(sphere,256,'origin')['cdf_max'][2],0.)
|
||||
def test_reproducible_controls(self):
|
||||
x=np.random.default_rng(7).normal(size=(32,40)).astype('f');a=analyze(x,'test');b=analyze(x,'test')
|
||||
self.assertEqual(a,b)
|
||||
self.assertTrue(a['mean']['valid'])
|
||||
|
||||
if __name__=='__main__':unittest.main()
|
||||
@@ -0,0 +1,22 @@
|
||||
# GSM8K任务微调前后的有效头数
|
||||
|
||||
比较原Qwen3.5-2B发布权重与同一revision上的纯LoRA seed0（额外GSM8K GRPO，非SFT）。没有训练新模型。主指标为√JS距离消融影响的N_eff；正变化表示更分散，负变化表示更集中。
|
||||
|
||||
| 输入组 | 原权重中位数 [P10,P90] | 微调后中位数 [P10,P90] | 配对变化中位数 | 增加的观测比例 |
|
||||
|---|---:|---:|---:|---:|
|
||||
| 原四条文本 | 8.931822 [5.784081,11.672658] | 8.918301 [5.776576,11.683940] | -0.007827 | 45.6% |
|
||||
| GSM8K测试题前缀 | 8.906046 [6.013504,11.801882] | 8.904117 [5.982593,11.818115] | -0.006747 | 47.2% |
|
||||
|
||||
每组4条输入×6个接收层×25个query位置×8个接收query头=4800个配对观测；它们不是4800个独立文本样本。前后使用同一原模型tokenizer、完全相同token IDs。原四条输入的基线8.931822/16已精确复现。
|
||||
|
||||
GSM8K输入取原作者固定版本test split中前4道至少32token的题目，仅用题目前32token，不含答案，不套不同聊天模板。另算了同组GSM8K输入的原权重基线；没有把旧通用文本与新任务文本混作配对。
|
||||
|
||||
## 方法与限定
|
||||
|
||||
所有6个普通注意力接收层和16个上游输出头均保留；每次移除一个上游头在全序列的投影输出，重算中间MLP/残差/归一化和检索分布。微调只改变18个GDN层的指定投影，普通注意力权重不变；隐藏状态和上游贡献仍会变化。
|
||||
|
||||
采用read-only FP32有效权重W+2BA，未把增量舍入为BF16、未修改底座；另用base projection+2B(Ax)独立前向校验。头输出求和、空消融和源数组复用校验均记录。JS散度及QKV复合相对变化也保留，避免只展示有利度量。这里QKV补充指标仅覆盖相同六个普通注意力接收层，其原基线不能与旧研究全部23连接的76.0%混作同一统计量。
|
||||
|
||||
单个发布者训练seed、4条短文本/组，仅支持这些输入上的描述性变化；不推广为所有微调都会如此，不报告GSM8K准确率，不据此推断删头可用或最终LM能力。配对变化中位数与两个总体中位数之差不是同一个统计量。
|
||||
|
||||
[公开微调权重与训练说明](https://huggingface.co/Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo)；固定版本、参数SHA、dataset版本和题目索引见protocol.json/gsm8k-source.json/samples.json。所有权重、适配器和缓存继续保留。
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"record": "qwen3.5-2b-grpo",
|
||||
"arm": "lora",
|
||||
"receipt_arm": "ctrl",
|
||||
"seed": 0,
|
||||
"step": 150,
|
||||
"base_model": "Qwen/Qwen3.5-2B",
|
||||
"base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
|
||||
"recipe": "GRPO, correctness-gated length penalty alpha 0.2, 150 steps from the shared warm-up (paper Section 4)",
|
||||
"lora_rank": 16,
|
||||
"lora_alpha": 32,
|
||||
"lora_targets": [
|
||||
"in_proj_qkv",
|
||||
"in_proj_a",
|
||||
"in_proj_b",
|
||||
"out_proj"
|
||||
],
|
||||
"gate": null,
|
||||
"n_tensors": 144,
|
||||
"n_params": 4727808,
|
||||
"source_sha256": "de2794c085f5096fea53374ba1cea31075071bb6de3f227bb3350b3035970447"
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"passed": true,
|
||||
"links": 96,
|
||||
"paired_observations_per_cohort_metric": 4800,
|
||||
"legacy_baseline_exact": true,
|
||||
"effective_head_formula_independently_recomputed": true,
|
||||
"head_sum_checks": 288,
|
||||
"explicit_lora_forward_relative_error": 8.33777860933606e-07,
|
||||
"adapter_sha_verified": true,
|
||||
"base_sha_verified": true,
|
||||
"dataset_sha_verified": true,
|
||||
"retained_cache_files": 300,
|
||||
"nonzero_update_modules": 72,
|
||||
"nonzero_hidden_state_changes_verified": true
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"passed": true,
|
||||
"links": 96,
|
||||
"paired_observations_per_cohort_metric": 4800,
|
||||
"legacy_baseline_exact": true,
|
||||
"effective_head_formula_independently_recomputed": true,
|
||||
"head_sum_checks": 288,
|
||||
"explicit_lora_forward_relative_error": 8.33777860933606e-07,
|
||||
"adapter_sha_verified": true,
|
||||
"base_sha_verified": true,
|
||||
"dataset_sha_verified": true,
|
||||
"retained_cache_files": 300,
|
||||
"nonzero_update_modules": 72,
|
||||
"nonzero_hidden_state_changes_verified": true
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
[
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-1-link-02.npz",
|
||||
"sha256": "7f3662c3f9badc8d7cd7d7bb312afbc8b7e0e3a3890c64b503040d0d9c11025a",
|
||||
"derived": "legacy-base-fresh-zh-1-02.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-1-link-06.npz",
|
||||
"sha256": "c196a05605c73ad28a1eb18299743e5c2ab9b5edacb31d8bb1afbb5addd90b5a",
|
||||
"derived": "legacy-base-fresh-zh-1-06.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-1-link-10.npz",
|
||||
"sha256": "5f8e151ce7656350b6e1505356e22fe7fc91a543e19a4315b1be0730e9504c74",
|
||||
"derived": "legacy-base-fresh-zh-1-10.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-1-link-14.npz",
|
||||
"sha256": "bedeb732de606f62e6af6bbc85d90146b0ae0bd0b97889d67e026f7374e89212",
|
||||
"derived": "legacy-base-fresh-zh-1-14.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-1-link-18.npz",
|
||||
"sha256": "1cffd0b08074027d75997f602e3f42440bc5d2126618da7e74c28a13a34f3bd6",
|
||||
"derived": "legacy-base-fresh-zh-1-18.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-1-link-22.npz",
|
||||
"sha256": "62326fccae8a66a39ba5cd076464ff4e29d0c0f13492fa588a692d2e8dd8f313",
|
||||
"derived": "legacy-base-fresh-zh-1-22.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-2-link-02.npz",
|
||||
"sha256": "f479c9681dea7c9a20937964eddda0f44fb6c7d896258e8d0be0da897fe4b0f1",
|
||||
"derived": "legacy-base-fresh-zh-2-02.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-2-link-06.npz",
|
||||
"sha256": "9bc654f5eb7c8d9ee55afcbc0d2afad1fc2f0472101abadc3c2859d14532c72a",
|
||||
"derived": "legacy-base-fresh-zh-2-06.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-2-link-10.npz",
|
||||
"sha256": "0253a2fe6dc2a429e79bbc42ee8ae36605577df417e84154c6accd2ea674ea37",
|
||||
"derived": "legacy-base-fresh-zh-2-10.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-2-link-14.npz",
|
||||
"sha256": "be9496bbe6fc6628e89e712d3196faff89560769fd42956aa147f78f265019b9",
|
||||
"derived": "legacy-base-fresh-zh-2-14.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-2-link-18.npz",
|
||||
"sha256": "59027a05d715123275ce00e5a999df58cda31807e4b329d91a3f5698c7f2cee6",
|
||||
"derived": "legacy-base-fresh-zh-2-18.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-zh-2-link-22.npz",
|
||||
"sha256": "d8ad49c825823916d7208672ab86e002e32b0f2b49e2ecbf6238b8244a8487fb",
|
||||
"derived": "legacy-base-fresh-zh-2-22.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-1-link-02.npz",
|
||||
"sha256": "ffb16b2e53428a6cd7dd60c08424f519cae1ce9a1ef0e0de8314d62217cbdda7",
|
||||
"derived": "legacy-base-fresh-en-1-02.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-1-link-06.npz",
|
||||
"sha256": "a936f7fd99905cd1a93a646e267d7204581ff2ddfd88da700261dbcc6af1e6e7",
|
||||
"derived": "legacy-base-fresh-en-1-06.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-1-link-10.npz",
|
||||
"sha256": "33b76293fe9cea3c198408b77981f88c866271ae6d4396c1269de402ff02f124",
|
||||
"derived": "legacy-base-fresh-en-1-10.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-1-link-14.npz",
|
||||
"sha256": "e2a7fcafc95e6ee1caab2cce5b089a73d9b51aad3fb3ad6dcb898d1932ca874c",
|
||||
"derived": "legacy-base-fresh-en-1-14.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-1-link-18.npz",
|
||||
"sha256": "87577b06589afcbc0c51663ef82d3be2a53730b91834b748e1e08c9e60093153",
|
||||
"derived": "legacy-base-fresh-en-1-18.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-1-link-22.npz",
|
||||
"sha256": "5decb8e48cc8399c671e39f98418f8920a89259ada3b339bb85d165b6736c747",
|
||||
"derived": "legacy-base-fresh-en-1-22.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-2-link-02.npz",
|
||||
"sha256": "d21f11c5ab5d554ddbb06d6b937281bc9474f0f45e27ddc83f9c3447b2242e10",
|
||||
"derived": "legacy-base-fresh-en-2-02.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-2-link-06.npz",
|
||||
"sha256": "003fb8c74875cff6def86871280218dd6166b8d248da0f4a2f68beacee2ba2fe",
|
||||
"derived": "legacy-base-fresh-en-2-06.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-2-link-10.npz",
|
||||
"sha256": "7d69b262b259562f02517f3c57b978d0cf8f85d93e257f6cb8514847b01924c5",
|
||||
"derived": "legacy-base-fresh-en-2-10.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-2-link-14.npz",
|
||||
"sha256": "c2fe9b11d500af37dbf2dd86c2cf0fb4a30dc4e9d45a1f3c2db7ad898f4c51f2",
|
||||
"derived": "legacy-base-fresh-en-2-14.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-2-link-18.npz",
|
||||
"sha256": "d3b5ce2c4dbadfb8864c698b67fbf8561e01536d03befbd9a80bb3bf9d0341bf",
|
||||
"derived": "legacy-base-fresh-en-2-18.npz"
|
||||
},
|
||||
{
|
||||
"source": "/opt/exp-model-parameter-similarity/runs/head-dominance/raw/fresh-en-2-link-22.npz",
|
||||
"sha256": "bb71e2d13fbff283728037ecff93d87e2475e182fde385216c6d59ab68113f0f",
|
||||
"derived": "legacy-base-fresh-en-2-22.npz"
|
||||
}
|
||||
]
|
||||
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"REPORT.md": "5d9a955c2e47539aa4d6b3946d2695307793a4ac908781c18378aecbdbb96a64",
|
||||
"adapter_meta.json": "c1262193a71bdca14ed9b762842adb34521c0a4abcdd4944606d28fbf37491c4",
|
||||
"audit-output.txt": "43eadee842fd6aa6830b91f8490442f83f8127c30bff6bdbadc012800e7adc7e",
|
||||
"audit.json": "43eadee842fd6aa6830b91f8490442f83f8127c30bff6bdbadc012800e7adc7e",
|
||||
"baseline-references.json": "08d55f493049fe590039250d187977f7af88c32f9342f4631bc360b2b5a33cf6",
|
||||
"external-caches.json": "3fcc5f222bddaea60df8241c104c092d2e807a1899f9aaa2889db28186737a05",
|
||||
"figures/paired_effective_heads_by_layer.pdf": "1606898509bcd023e8dbb925cff35b798efb00046179e2182f291cd590cd6b44",
|
||||
"figures/paired_effective_heads_by_layer.png": "5dc16dacbab27f2d96c95f744a72124e8eba25d398f434b666639fc3765ef82c",
|
||||
"figures/paired_effective_heads_by_layer.svg": "29557150e6b9da53f3d4b792c709cbc9d929248c1e8cc2f4a7174282b091c7cd",
|
||||
"figures/paired_effective_heads_cdf.pdf": "1c7e0203218dcf1dc04461c2df17a8d771a0c5f142155f0ecc2d9912eac569c2",
|
||||
"figures/paired_effective_heads_cdf.png": "31ecf93a05585d2729a3426a5ceaaedb9f3d43a89fc114f12edc7d6764a898c5",
|
||||
"figures/paired_effective_heads_cdf.svg": "fcfe932c82423ba4fea6f8488a5031d008b2f5484383f40e883c2e566361b107",
|
||||
"figures/paired_effective_heads_delta.pdf": "35c574848fbbabc5d5952f11e73164a5be1417fbc7c670e0560e8d33451010e1",
|
||||
"figures/paired_effective_heads_delta.png": "ed9232e86e945737389d5b0b00a3f32f62f49a92499fc8748dd1b0bbcd97df16",
|
||||
"figures/paired_effective_heads_delta.svg": "ed5e254afa515d11935b32586ca8f530e59321ab4eb2b449a3996a7a7985a1d6",
|
||||
"final-tests.txt": "d162aad1f7a8b8ad78e4e465a5618f0f201cb41701ec21acf29ba44774e77185",
|
||||
"gsm8k-source.json": "1aa995a8e41c299d61749f95a6cda0478e6947c2d104d88ee2c88ac5b9cf2ae1",
|
||||
"head-sum-controls.json": "13ccab8e890ae6c9898355ea93b9bfdc131974b124152050628ab5a10a61bfeb",
|
||||
"hidden-state-changes.json": "9c3132ab5d71a03962d056bd11c36930fcd5dcc66fa12f747f539bbfff9e65ce",
|
||||
"observations.npz": "48c33b8e1eaf3eb722f83b34507c1d4138d78b706b4cff673945c8c2f8f11a67",
|
||||
"per-layer.csv": "b98afe569ca48b310f8b54af05556e06c6d9a08de2f6d6318f519d6cc1b059fd",
|
||||
"per-text-layer.csv": "52b13443f17662167476fe7657c543397a549c49f767192faf8f59470a9c41d5",
|
||||
"protocol.json": "70f107499364a1c43d85d4f8a0014a96fdf2eec533613cbfc9cb940887474439",
|
||||
"raw/gsm8k-base-gsm8k-test-0000-02.npz": "ecfc2fe3fe70281feca0667e1f26d8a69a6106a36073ef584cbbdda66010b373",
|
||||
"raw/gsm8k-base-gsm8k-test-0000-06.npz": "e7e4a7cffd8916aefb03e8aa25729e012b6f67d6bf6fc96e18885adb3e1fe566",
|
||||
"raw/gsm8k-base-gsm8k-test-0000-10.npz": "135aa65770cc8d561d134051ce995758fd326a4e077369f567c2328701e5f3fa",
|
||||
"raw/gsm8k-base-gsm8k-test-0000-14.npz": "6af0f8858693eca8ccaf409e62de26b1df00185dc2416b019090b267377150fc",
|
||||
"raw/gsm8k-base-gsm8k-test-0000-18.npz": "33ea81625978e5e616bc51ebef4115a2c6e1cde70010448bde7810bb58200a7f",
|
||||
"raw/gsm8k-base-gsm8k-test-0000-22.npz": "029854a53076fd7583a854ce74b23ab9a68779e3876b77b2a7c54093a2e30686",
|
||||
"raw/gsm8k-base-gsm8k-test-0002-02.npz": "9d435e4a3d4096799727ebaf9df544c7b31cd044836bc60711f085d3f574aa07",
|
||||
"raw/gsm8k-base-gsm8k-test-0002-06.npz": "47c165bedb771582c4718c5d25cec3c790b0c9e26b420f0f391e04d2c6c90220",
|
||||
"raw/gsm8k-base-gsm8k-test-0002-10.npz": "bcaf0686fc04515330ad234776df491d59acaa2568ff84dcbf3c0c61522d83fb",
|
||||
"raw/gsm8k-base-gsm8k-test-0002-14.npz": "d1f17789ef5e7d63390da5ff9baeaf81fa1647b54ea700794b38315486636b3e",
|
||||
"raw/gsm8k-base-gsm8k-test-0002-18.npz": "458fc2ce51fc2c4bd52fbbc2490654ad08ffe708fd30e40cb3af27141e0a787c",
|
||||
"raw/gsm8k-base-gsm8k-test-0002-22.npz": "040004d851fc6ea916054307e9cb1be6e98a8393865be1a55d0fe69f1fe2a843",
|
||||
"raw/gsm8k-base-gsm8k-test-0003-02.npz": "7bb4247e7a0b1db976bbeb3e67f54ab3dec94ae7e399a32a34dd3f81c5b7001f",
|
||||
"raw/gsm8k-base-gsm8k-test-0003-06.npz": "962976972eb64108068262a349aba53b7c81c6f7753ce5f9cf2c16c50e3f7df3",
|
||||
"raw/gsm8k-base-gsm8k-test-0003-10.npz": "e6a554f3477c373a0d662c8e7469099fcd0f3b0647b203ceaa5cb2c7fdcdc39b",
|
||||
"raw/gsm8k-base-gsm8k-test-0003-14.npz": "898f490b0989e5967d56e760fa9f8f6fff2b39449d5528ae461710e8e1fefa84",
|
||||
"raw/gsm8k-base-gsm8k-test-0003-18.npz": "91197613bf169d3f45381e8660811f4eedf4883d7e744c11be8f18035efaa606",
|
||||
"raw/gsm8k-base-gsm8k-test-0003-22.npz": "b9f0c1a7f9a3c0ecb77443c6ffa2d8a4f8addc75b5acf5a65a48d7e493bbaa62",
|
||||
"raw/gsm8k-base-gsm8k-test-0004-02.npz": "4bd803b726afe24f486b8f6795c576b82cbf66216f081c4377f3148cc945f064",
|
||||
"raw/gsm8k-base-gsm8k-test-0004-06.npz": "b2480beb27a7097e1ffc9ffccf36607bdb466a693bf681743d6dc27bd6b8ef25",
|
||||
"raw/gsm8k-base-gsm8k-test-0004-10.npz": "d97ec969621c07cf415c32894bd1d58d0ceaaafa09d52e794ab64fc4f77743a9",
|
||||
"raw/gsm8k-base-gsm8k-test-0004-14.npz": "5202e7e67545d28c098b8a69c64df7df9ce0cee14159b3c63b242ec413962845",
|
||||
"raw/gsm8k-base-gsm8k-test-0004-18.npz": "3967869d031d0dfc369843b17598a8ed974eef29a92a67ca3039e5f8d54756fc",
|
||||
"raw/gsm8k-base-gsm8k-test-0004-22.npz": "65fc463f1a0c2ab6f8ee43a1b5b7550c237ce68439139bff68f8fcb6929774bd",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0000-02.npz": "70ca3d17ab016639e7b82e0a86ff7ac8010e010568a3d557314af81c8ae56f02",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0000-06.npz": "c3dd0bfa9c2aadcad83406740a7cb26103f36869e86e8dde54bc444f20486f88",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0000-10.npz": "dd4dc2adf0f138748734269e5c567c1ecd1409f613227de4fce4c6b0e496959c",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0000-14.npz": "444db92129fcf1361563bc694413772eeae5cd4a8468bbd5c8f20a5c8f84c581",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0000-18.npz": "d329c4e1f66ac7faba575af3c79159b661ecb9a0a0b8774fe261b29c46d596b3",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0000-22.npz": "8822354b5cb72d519d1999f4572d6dd01e73c3b8ab9d9c42c149d8a3c67ded9f",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0002-02.npz": "6382a19dbb82da1550b276fca614a49aa4eb2ff9432b661ee24a70583f10444c",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0002-06.npz": "7b3eabeed7793fc1c30ef04be2f5e94b73b6aafe635601cc543b2713b986b50a",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0002-10.npz": "1296bd5ef6bfc564cfe3d641fae2bcfc605ab1c8965e7f983e0e6332208ac956",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0002-14.npz": "31e344565c5e3935693ae72f488d7bad994f7a45b750b31b645642367dbdf945",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0002-18.npz": "b1b8c224355373caccaf3fa0d5542e9a39b726904ab154c67e36d46be2f5f6c4",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0002-22.npz": "d8b00f473c025ed6f55e72385b2e6e34ab4e3557af5e743d799834e2e3abf7f6",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0003-02.npz": "bbaa5a3f1a6978a8f1b0f2ae136986b2e6aa19cc37179c4490f273d8a1bd742b",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0003-06.npz": "48a1417b11e1188fd9b2262d283d366ea158ef6152e3ed38bacda61b44b88230",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0003-10.npz": "d57ac5c308fd2d7c8c47c9d9f8306c36e8502592adc535f48306a69f406b2e83",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0003-14.npz": "acee7b009e3ae76760a382c37c7bcde01aec93f5f18eec5df3bd63e5d40f7a46",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0003-18.npz": "21507efdf6cd88164a39a010a1d7892273ea30844af876ee1d0a8e458fce1b3a",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0003-22.npz": "9bf79f32a875142ccaff29f1a7f158281e4577c367899eaeea4dc3963d54dbcc",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0004-02.npz": "85e6678bf76d36936601b59d9974642eef6c56991dc73f81d3e50631d945b84a",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0004-06.npz": "c38802c693d40e9974e63b85625e7e4976859514e5c44b56fa037cde49fd6fcb",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0004-10.npz": "217d547f807bd056f5fa407d192aa4ccaae3c5e1ff11fbe77d8cf8a16a971021",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0004-14.npz": "7b93542d836bcc1c5e6053a4b447997e2f5afafc2dbee1834b0bf31f9da246a4",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0004-18.npz": "457a03603055b4ad7da5c4b347370e070904b4996a1fe508651e56043a323c20",
|
||||
"raw/gsm8k-finetuned-gsm8k-test-0004-22.npz": "580db43631651cb501f9a34051e0bbf4a875cd3f8e25b1ed1c79026dc1f2ce7c",
|
||||
"raw/legacy-base-fresh-en-1-02.npz": "02d39a0c9063d9b49026098e1026d9e8ed6658a52eea00a117cf8868fbe5a9c5",
|
||||
"raw/legacy-base-fresh-en-1-06.npz": "d9d37a188840807d1e8e3c68826e49054b2ac90c445f73022a5290f4160a7716",
|
||||
"raw/legacy-base-fresh-en-1-10.npz": "39d50060988427a999b9b7fee1a340cffa3d1b27e55e96b38f980ba4b6ca182c",
|
||||
"raw/legacy-base-fresh-en-1-14.npz": "985c54ad7e4d96fcd53ff1f1b290c5a5018ec7e4218b16b1104866745ba83719",
|
||||
"raw/legacy-base-fresh-en-1-18.npz": "a8ed3dd62428dca4b3fdad9bdb159609e0d2fe38657681d7acca42d79c2bb876",
|
||||
"raw/legacy-base-fresh-en-1-22.npz": "da1a5799d789e8561935448e2eb49a2427865ce26602b26f3df2c19fa8f061b9",
|
||||
"raw/legacy-base-fresh-en-2-02.npz": "918fb1ed19925ec01f60d02ac2edd813de6638c1b6dba8e2ccc16b2921d06a41",
|
||||
"raw/legacy-base-fresh-en-2-06.npz": "84447aec23901d8430b38a69a2601c48f7e57aa7224ab740362b7c36dc0300e7",
|
||||
"raw/legacy-base-fresh-en-2-10.npz": "6c89f1f7b87fd28ca9d3292b043280bfd1841ee38d8f077ece52d17d5aa9428b",
|
||||
"raw/legacy-base-fresh-en-2-14.npz": "20140c00b5df0750cb5929fa20ddd39f935ff536d6b0ad019abe5b75dfaa626f",
|
||||
"raw/legacy-base-fresh-en-2-18.npz": "eac15b1f5c19baecff37ae792195c69e978cd1d592090e28e580e8114a491c62",
|
||||
"raw/legacy-base-fresh-en-2-22.npz": "3e089d27887946450b5961f8bfe99d4725ff59872de86ef547a5512fcfc5f622",
|
||||
"raw/legacy-base-fresh-zh-1-02.npz": "3459f24661cdd9b58fa8aaed0081db19400ebd758d4ace7112df926f5c11baad",
|
||||
"raw/legacy-base-fresh-zh-1-06.npz": "502aa9cd493c863abddaa44c1b530d94703b2fbe0b04489d7638457576770291",
|
||||
"raw/legacy-base-fresh-zh-1-10.npz": "6ebc0d130320a1bc9037dbb17c258e2f8501af4f1dc7dbc750caac6ef38a4f39",
|
||||
"raw/legacy-base-fresh-zh-1-14.npz": "45f10901f3f1bfd9d069bf7c92ae9c6807b3ae1220bbbfe0a1d61a3cb3a1b5c5",
|
||||
"raw/legacy-base-fresh-zh-1-18.npz": "199ccf41bc26a03fafd69d3890b1f389a2152f1afaefcd972eff15f36f36b780",
|
||||
"raw/legacy-base-fresh-zh-1-22.npz": "2551355956dfdf20ff381e150eb1f618d134913403a30eda6e02270742b2e31a",
|
||||
"raw/legacy-base-fresh-zh-2-02.npz": "db0e721058e7149766328c2a2b17b55f02daff750daf732a061fa54248186a8f",
|
||||
"raw/legacy-base-fresh-zh-2-06.npz": "acc584949dea7baaefdf8f8a7e95ea0be812bbb364335df97274ea316181c309",
|
||||
"raw/legacy-base-fresh-zh-2-10.npz": "a54b0133e538d96c469e63ef5b33590a9c0895837bd53aef758377b7584d51e6",
|
||||
"raw/legacy-base-fresh-zh-2-14.npz": "8d58f0672075f97009009508978216bc131d8a69754046cfc619f7d5ed83aec0",
|
||||
"raw/legacy-base-fresh-zh-2-18.npz": "5503272cf1787c1c87f1502621bea2f18801a14d8424554736ea9640c778a2a8",
|
||||
"raw/legacy-base-fresh-zh-2-22.npz": "7daa469f1c68756cf3078ad691a35c2eb26cc78d420d79cd16cc03512df907d3",
|
||||
"raw/legacy-finetuned-fresh-en-1-02.npz": "dacb43fbde5b7cd0ae011141818832545a29d0e0ece532ad7f076ac1770d576e",
|
||||
"raw/legacy-finetuned-fresh-en-1-06.npz": "1adb8642c4c2767c99301a7f92107d98426636456d70bd3e69868df3332997f0",
|
||||
"raw/legacy-finetuned-fresh-en-1-10.npz": "4540b80c2c630c293841278066a4a17f36c9fe3fa2a1fae9ed0c7ba4d7b70c55",
|
||||
"raw/legacy-finetuned-fresh-en-1-14.npz": "1928fb89a08af6b843ffcb6ff58ab4d0ac5b0848d9e35a04a91884c1a929b893",
|
||||
"raw/legacy-finetuned-fresh-en-1-18.npz": "7348f577974ce49f0d618b78ea99f32c6e7ea66265ba048d688e1faccb1c4743",
|
||||
"raw/legacy-finetuned-fresh-en-1-22.npz": "aced3a86e6dec59d49a42f24d142b2bf62c93fbc31441395a42bcc46d786f749",
|
||||
"raw/legacy-finetuned-fresh-en-2-02.npz": "e3689374812d9fb4bc00827fd9e6b082ba73d8546e9ec9f7c5ef692c6946dafb",
|
||||
"raw/legacy-finetuned-fresh-en-2-06.npz": "ac9fcc744476e47494d3d60922573e90af8de4a45a5b426931386b8f0adb16cb",
|
||||
"raw/legacy-finetuned-fresh-en-2-10.npz": "d5ca98484ca83735319218b58c9a11146a9d884976034a839adb7036fec295e0",
|
||||
"raw/legacy-finetuned-fresh-en-2-14.npz": "b89dd50f99ebe5eccb99e153585566fab5f40178394e3992e1f7d12ff11089fe",
|
||||
"raw/legacy-finetuned-fresh-en-2-18.npz": "0f0ddd848aa51d98ba8c4a288ea93d6304c578bbf3a563e88a3028dce52ad9d9",
|
||||
"raw/legacy-finetuned-fresh-en-2-22.npz": "670883b063d993ac96ceb7e258d4de2a03bd7fbb8639aca24fc49602a6ba6d6a",
|
||||
"raw/legacy-finetuned-fresh-zh-1-02.npz": "f2bef255988bd0d947c65c06808c7d0caceab30d832015b55bcd8342d9bf28b9",
|
||||
"raw/legacy-finetuned-fresh-zh-1-06.npz": "c2d886c833ca4e51d53d57a2b076ea915c0d517ef26ed5b1d143b9224829922b",
|
||||
"raw/legacy-finetuned-fresh-zh-1-10.npz": "c20f2d166d175fbec7e236d6b95da6dd61865c7ab4cc17433a46647a0d511a11",
|
||||
"raw/legacy-finetuned-fresh-zh-1-14.npz": "c005ec04e8c439543cc5b3a9643f267d414c2757c43d1706f493f937d4f6ffc4",
|
||||
"raw/legacy-finetuned-fresh-zh-1-18.npz": "d01520126987ace2f73f3eaef5e560326158c16cf66c659626ab933e17622b1d",
|
||||
"raw/legacy-finetuned-fresh-zh-1-22.npz": "40933610f61bbb4d9c29c278b7fa1a188d4f8813fb827515a01dce90aa827feb",
|
||||
"raw/legacy-finetuned-fresh-zh-2-02.npz": "7d2828387978b21801a87d924e8e254c39c599c5781b82a7a5d05ecb9d239f27",
|
||||
"raw/legacy-finetuned-fresh-zh-2-06.npz": "a82b7ad2e05cd6019a05ae4bc2184171c5493cc26577bd347e5cc9176d8e28d1",
|
||||
"raw/legacy-finetuned-fresh-zh-2-10.npz": "49d0311d99960838036083e8d866ba418146538bae6b098f84d6a45824dc5dda",
|
||||
"raw/legacy-finetuned-fresh-zh-2-14.npz": "729cea1ba6f3f4d694552002900085256f4ffdf2dc06c0d5ae6d6829be3f1749",
|
||||
"raw/legacy-finetuned-fresh-zh-2-18.npz": "0a13c6bd541c3afe521de2fcc51bbcaf2ec180ffb8adfd98eb179171e2e80b20",
|
||||
"raw/legacy-finetuned-fresh-zh-2-22.npz": "2dbcd8ea5d4dc09bf160462ba0fc1f7cfd52ca1b1f9e4914227aa2b746ed3c35",
|
||||
"report-output.txt": "5d9a955c2e47539aa4d6b3946d2695307793a4ac908781c18378aecbdbb96a64",
|
||||
"run-log.txt": "afe8f52f7bf8694b088c15f610915fd91b77236d6b90282fd11d6164d229ae9e",
|
||||
"samples.json": "d5d5d707ba9365a0ad43e2a03d4c4659ddc4e6b1fc665ed14f9fceb1757f4d28",
|
||||
"source-hashes.json": "410aa0685f26e65311589abafc9661eb1054b5ab8e6beac5e85adbbd4c7396d5",
|
||||
"status.json": "41afcb194af4ed23fb1f80bee803e135059e06c403d9ab9b936d39be0ad3f4f9",
|
||||
"summary.json": "fa0df1bb7f7b7174885e7c9b37da0e05e46acc3932830c8f4f2476f23fa54b38",
|
||||
"weight-update-summary.json": "cb22c0e21a60c9d36ea2e7827b8b4b54b127e10e6e7aa47869d936f7e78e2013"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 185 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 75 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 144 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 142 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 78 KiB |
File diff suppressed because it is too large
Load Diff
|
After Width: | Height: | Size: 72 KiB |
@@ -0,0 +1,49 @@
|
||||
test_3d_reader_and_exact_block (tests.test_pilot.PilotTests.test_3d_reader_and_exact_block) ... ok
|
||||
test_bfloat16 (tests.test_pilot.PilotTests.test_bfloat16) ... ok
|
||||
test_directional_replacement_error_and_scaling (tests.test_pilot.PilotTests.test_directional_replacement_error_and_scaling) ... ok
|
||||
test_end_to_end_synthetic_groups_and_savings (tests.test_pilot.PilotTests.test_end_to_end_synthetic_groups_and_savings) ... Inventory: 4 tensors, 33792 bytes
|
||||
tensor: sampling 4 objects
|
||||
block: sampling 22 objects
|
||||
block: verified 100/109
|
||||
{"model": "synthetic", "revision": "test", "tensor_count": 4, "parameter_count": 8448, "model_weight_bytes": 33792, "ndim_counts": {"2": 2, "3": 2}, "elapsed_seconds": 0.09112083166837692, "analysis_budget_seconds": 60, "peak_rss_kib": 55772, "python": "3.14.4", "numpy": "2.5.4", "seed": 20261109, "limitations": ["No inference or functional equivalence measured", "No exhaustive pair search", "Block search uses per-role/shape/scale reservoirs of 256", "Different block scales overlap; savings must not be added", "Block savings count only verified disjoint sampled blocks, no extrapolation", "Spectral probes use aligned bounded subvolumes, not whole tensors", "Shared encoding is theoretical, not a runnable compressed checkpoint", "No cross-role grouping, neuron permutation alignment, or training"], "completed": true}
|
||||
ok
|
||||
test_no_transitive_chaining (tests.test_pilot.PilotTests.test_no_transitive_chaining) ... ok
|
||||
test_ranged_download_preserves_prefix (tests.test_pilot.PilotTests.test_ranged_download_preserves_prefix) ... Weight download 36/36 bytes (100.0%)
|
||||
ok
|
||||
test_causal_convolution_and_norm_boundary (tests.test_functional.FunctionalTests.test_causal_convolution_and_norm_boundary) ... ok
|
||||
test_causality_and_identical_function_metrics (tests.test_functional.FunctionalTests.test_causality_and_identical_function_metrics) ... ok
|
||||
test_neuron_function_metrics_and_targeted_replacement (tests.test_functional.FunctionalTests.test_neuron_function_metrics_and_targeted_replacement) ... ok
|
||||
test_nonrotary_gain_identity_and_rotary_negative_control (tests.test_functional.FunctionalTests.test_nonrotary_gain_identity_and_rotary_negative_control) ... ok
|
||||
test_streamed_projection_embedding_and_output_head (tests.test_functional.FunctionalTests.test_streamed_projection_embedding_and_output_head) ... ok
|
||||
test_uniform_rmsnorm_function_bound_is_tight (tests.test_functional.FunctionalTests.test_uniform_rmsnorm_function_bound_is_tight) ... ok
|
||||
test_assignment_against_exhaustive_and_inverse (tests.test_crossqkv.CrossQKVTests.test_assignment_against_exhaustive_and_inverse) ... ok
|
||||
test_cached_mode_rejects_future_state_reuse (tests.test_crossqkv.CrossQKVTests.test_cached_mode_rejects_future_state_reuse) ... ok
|
||||
test_controls_cover_multiple_projection_layers (tests.test_crossqkv.CrossQKVTests.test_controls_cover_multiple_projection_layers) ... ok
|
||||
test_head_matrix_metrics_and_gram_are_diagnostic (tests.test_crossqkv.CrossQKVTests.test_head_matrix_metrics_and_gram_are_diagnostic) ... ok
|
||||
test_probability_identity_and_nontrivial_queries (tests.test_crossqkv.CrossQKVTests.test_probability_identity_and_nontrivial_queries) ... ok
|
||||
test_profile_matches_canonical_and_head_sum (tests.test_crossqkv.CrossQKVTests.test_profile_matches_canonical_and_head_sum) ... ok
|
||||
test_recipient_value_gate_output_and_gqa_group_stay_fixed (tests.test_crossqkv.CrossQKVTests.test_recipient_value_gate_output_and_gqa_group_stay_fixed) ... ok
|
||||
test_topk_tail_rank_can_change_with_tiny_probability_difference (tests.test_crossqkv.CrossQKVTests.test_topk_tail_rank_can_change_with_tiny_probability_difference) ... ok
|
||||
test_known_activity (tests.test_activation.ActivationTests.test_known_activity) ... ok
|
||||
test_zero_and_scale (tests.test_activation.ActivationTests.test_zero_and_scale) ... ok
|
||||
test_continuous_limits_and_undefined_zero (tests.test_dominance.DominanceTests.test_continuous_limits_and_undefined_zero) ... ok
|
||||
test_joint_recomputation_null_and_saved_raw (tests.test_dominance.DominanceTests.test_joint_recomputation_null_and_saved_raw) ... baseline layer 0
|
||||
baseline layer 1
|
||||
ok
|
||||
test_js_identity_and_disjoint (tests.test_dominance.DominanceTests.test_js_identity_and_disjoint) ... ok
|
||||
test_mask_ranks_vary_by_token_and_recipient (tests.test_dominance.DominanceTests.test_mask_ranks_vary_by_token_and_recipient) ... ok
|
||||
test_mixed_layer_forward_and_effective_qkv (tests.test_dominance.DominanceTests.test_mixed_layer_forward_and_effective_qkv) ... baseline layer 0
|
||||
baseline layer 1
|
||||
baseline layer 0
|
||||
baseline layer 1
|
||||
ok
|
||||
test_effective_rows_equal_explicit_low_rank_output (tests.test_finetune_compare.FineTuneTests.test_effective_rows_equal_explicit_low_rank_output) ... ok
|
||||
test_forward_and_head_decomposition_match_independent_overlay (tests.test_finetune_compare.FineTuneTests.test_forward_and_head_decomposition_match_independent_overlay) ... baseline layer 0
|
||||
baseline layer 1
|
||||
ok
|
||||
test_zero_scale_preserves_original_and_bad_metadata_rejected (tests.test_finetune_compare.FineTuneTests.test_zero_scale_preserves_original_and_bad_metadata_rejected) ... ok
|
||||
|
||||
----------------------------------------------------------------------
|
||||
Ran 30 tests in 23.124s
|
||||
|
||||
OK
|
||||
@@ -0,0 +1,9 @@
|
||||
{
|
||||
"repository": "https://github.com/openai/grade-school-math",
|
||||
"revision": "3101c7d5072418e28b9008a6636bde82a006892c",
|
||||
"url": "https://raw.githubusercontent.com/openai/grade-school-math/3101c7d5072418e28b9008a6636bde82a006892c/grade_school_math/data/test.jsonl",
|
||||
"sha256": "3730d312f6e3440559ace48831e51066acaca737f6eabec99bccb9e4b3c39d14",
|
||||
"split": "test",
|
||||
"format": "original author JSONL",
|
||||
"rows": 1319
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,45 @@
|
||||
{
|
||||
"definition": "final residual before final RMSNorm, not logits, accuracy or LM quality",
|
||||
"changes": [
|
||||
{
|
||||
"cohort": "legacy",
|
||||
"sample": "fresh-zh-1",
|
||||
"final_pre_norm_residual_relative_l2": 0.021898728857078685
|
||||
},
|
||||
{
|
||||
"cohort": "legacy",
|
||||
"sample": "fresh-zh-2",
|
||||
"final_pre_norm_residual_relative_l2": 0.03829561458389978
|
||||
},
|
||||
{
|
||||
"cohort": "legacy",
|
||||
"sample": "fresh-en-1",
|
||||
"final_pre_norm_residual_relative_l2": 0.02432907925877068
|
||||
},
|
||||
{
|
||||
"cohort": "legacy",
|
||||
"sample": "fresh-en-2",
|
||||
"final_pre_norm_residual_relative_l2": 0.034934933219615986
|
||||
},
|
||||
{
|
||||
"cohort": "gsm8k",
|
||||
"sample": "gsm8k-test-0000",
|
||||
"final_pre_norm_residual_relative_l2": 0.03901556767460501
|
||||
},
|
||||
{
|
||||
"cohort": "gsm8k",
|
||||
"sample": "gsm8k-test-0002",
|
||||
"final_pre_norm_residual_relative_l2": 0.04043223109300957
|
||||
},
|
||||
{
|
||||
"cohort": "gsm8k",
|
||||
"sample": "gsm8k-test-0003",
|
||||
"final_pre_norm_residual_relative_l2": 0.045740534396548564
|
||||
},
|
||||
{
|
||||
"cohort": "gsm8k",
|
||||
"sample": "gsm8k-test-0004",
|
||||
"final_pre_norm_residual_relative_l2": 0.04190792993325029
|
||||
}
|
||||
]
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,37 @@
|
||||
cohort,metric,recipient,base_median,base_p10,base_p90,finetuned_median,finetuned_p10,finetuned_p90,paired_delta_median
|
||||
legacy,distance,3,8.739316033368453,4.950963035160957,10.863499972573624,8.731708727318438,4.9232497064004646,10.84212771798773,-0.02037733856762003
|
||||
legacy,distance,7,9.4363090529414,6.25085771836616,11.556565692274562,9.423484801453295,6.21315757199869,11.527370171007142,-0.019801758117782065
|
||||
legacy,distance,11,11.22490926849138,7.737020182887724,12.908045076028829,11.250679957928547,7.73525088572658,12.928017935515697,0.0038455741574283664
|
||||
legacy,distance,15,9.759881279287757,7.162904991854923,11.716165708007967,9.800072330188218,7.1630134470288285,11.731124776731967,0.03740503224688574
|
||||
legacy,distance,19,7.582995802914317,5.478640166334395,10.198341943349355,7.610650757626568,5.493471329919807,10.144988324005332,0.016808984407926886
|
||||
legacy,distance,23,7.388739661710135,5.651966220638869,9.249179535567297,7.251844559425095,5.554179144408622,9.126738186063735,-0.09932936375917611
|
||||
legacy,divergence,3,3.147912031314844,1.3810818357797214,5.537694943911305,3.1206879617749346,1.3758373946197866,5.50519735871187,-0.013426481470755869
|
||||
legacy,divergence,7,4.713474898063865,1.9437311034396794,7.289381192575258,4.697509688023275,1.9391684741219346,7.240142719104977,-0.02240482362572893
|
||||
legacy,divergence,11,5.747313395153364,2.349226790675758,8.479543942481612,5.7555205045992,2.342471242036245,8.45559258627536,0.008220859323202268
|
||||
legacy,divergence,15,4.927015096951064,2.8084538003789024,7.330506308274049,4.927768269102536,2.833041243881992,7.336922408013947,0.021762632364951795
|
||||
legacy,divergence,19,2.8494489425735994,1.7575837057447943,5.18654187756192,2.861665011032441,1.7442072067733334,5.143915153772987,0.006601984727423238
|
||||
legacy,divergence,23,3.0214471514630055,2.047270924772257,4.518937219975146,2.9701865807044205,2.0010686132133917,4.411307997114287,-0.05707591033976844
|
||||
legacy,QKV,3,8.777726631430976,7.189103615155911,10.713615974191201,8.76668332968768,7.190684779039996,10.719732488790717,-0.009309064163098846
|
||||
legacy,QKV,7,9.844072680466791,8.0413059359728,11.619560783151188,9.82825724015493,8.014544082220969,11.581538677554722,-0.016941599110913863
|
||||
legacy,QKV,11,12.089247576362546,10.201057391277503,13.571935182553943,12.09729379910231,10.196828217568173,13.57795553074303,0.01309251483925511
|
||||
legacy,QKV,15,10.345856861414816,8.507270281674957,11.85158988753179,10.38900205657722,8.564274444281775,11.951509121264612,0.037888922465287855
|
||||
legacy,QKV,19,12.044235479417448,10.732008460392013,13.1588691005697,12.061634540063746,10.722140503244573,13.161318190003938,-0.0025726092883973095
|
||||
legacy,QKV,23,8.09156575360802,5.833100089115012,10.736031379205444,7.9963938578096405,5.72057650519899,10.668444002058392,-0.09506145974499303
|
||||
gsm8k,distance,3,8.913673809873648,5.464073924885019,11.307334958094344,8.94684685981133,5.466046067182029,11.285099623450353,-0.0065539359628692395
|
||||
gsm8k,distance,7,9.691950173931822,6.449358553913276,11.717921799729858,9.682670113610445,6.420969364897694,11.686361680373526,-0.020282664056864075
|
||||
gsm8k,distance,11,11.13393122020824,6.4616586939758776,12.895309269291927,11.107427837202076,6.533455114023123,12.900256831750358,0.010065218794381714
|
||||
gsm8k,distance,15,9.974087630581401,7.069686056657516,11.884414604809061,9.997745229352889,7.038241611739143,11.927975457275181,0.04648967580843344
|
||||
gsm8k,distance,19,7.77346164330409,6.131638555699892,9.455856359148038,7.789480896051456,6.136266410549428,9.498390268941135,0.020110690923323293
|
||||
gsm8k,distance,23,7.442742494743904,5.505415997443188,9.042021588456507,7.346119301169663,5.469589684395749,8.936377940244807,-0.10260948221392674
|
||||
gsm8k,divergence,3,3.256158018271,1.5468419231568364,6.168010316005508,3.3172355634071002,1.5617019347980559,6.185732763079662,-0.0011366525249161263
|
||||
gsm8k,divergence,7,4.91009951654837,2.045185653260748,7.270690734337546,4.865016424592383,2.026755035640929,7.1787176847663465,-0.02816947521415858
|
||||
gsm8k,divergence,11,5.931935024726259,1.9898847070739774,8.459456344451384,5.89597820952082,1.960013675329343,8.45922799810914,0.0012143867431798627
|
||||
gsm8k,divergence,15,5.189684726756504,2.680351367454612,7.55120190386808,5.188558523298806,2.6608354187785666,7.605217486545163,0.025876923734101775
|
||||
gsm8k,divergence,19,2.7570989361980978,1.7919480814325728,4.193726547659212,2.775993252952797,1.806611536240062,4.181110519373861,0.015515300540850818
|
||||
gsm8k,divergence,23,3.137382710336168,1.9856284660015169,4.515373518991637,3.0794746938791713,1.946750149457116,4.410736135976342,-0.052616624917172894
|
||||
gsm8k,QKV,3,8.818242117491618,7.409517860590123,11.177278554033984,8.849729752069354,7.434097045185119,11.160648347342136,0.017932250236647462
|
||||
gsm8k,QKV,7,10.336483028296383,8.60447573440056,11.851083219477552,10.319640090875208,8.598477516226366,11.845101512915903,-0.011013131675529308
|
||||
gsm8k,QKV,11,11.951775082183595,9.533230055647254,13.350917876999524,11.918208649591278,9.580451951366072,13.340366194176807,0.00937949462949117
|
||||
gsm8k,QKV,15,10.23944961335793,8.248327352722713,12.111292674550386,10.31412657696469,8.27734572624579,12.14257864987415,0.06657806670976196
|
||||
gsm8k,QKV,19,12.3316091968019,10.589038220184888,13.241376894214595,12.367902101381986,10.614451371174253,13.264109489521545,0.015430541534902531
|
||||
gsm8k,QKV,23,8.242238784536774,6.1055890731614015,10.372955699499842,8.112160180891495,5.944619005378508,10.25239880721615,-0.12305989236341386
|
||||
|
@@ -0,0 +1,289 @@
|
||||
cohort,role,sample,recipient,metric,total_count,total_p10,total_median,total_p90,total_mean,max_absolute_count,max_absolute_p10,max_absolute_median,max_absolute_p90,max_absolute_mean,effective_heads_count,effective_heads_p10,effective_heads_median,effective_heads_p90,effective_heads_mean,effective_fraction_count,effective_fraction_p10,effective_fraction_median,effective_fraction_p90,effective_fraction_mean,concentration_count,concentration_p10,concentration_median,concentration_p90,concentration_mean,top1_count,top1_p10,top1_median,top1_p90,top1_mean,top25_count,top25_p10,top25_median,top25_p90,top25_mean,top50_count,top50_p10,top50_median,top50_p90,top50_mean,bottom25_count,bottom25_p10,bottom25_median,bottom25_p90,bottom25_mean,heads90_count,heads90_p10,heads90_median,heads90_p90,heads90_mean
|
||||
legacy,base,fresh-zh-1,3,distance,200,0.5068664275109768,0.66347974864766,0.8529259402770549,0.66853969598189,200,0.10391714274883271,0.1569887399673462,0.2834477216005325,0.17618735685944556,200,4.400645222422045,8.823949964387808,10.779013738541964,8.351667359484624,200,0.2750403264013778,0.551496872774238,0.6736883586588728,0.521979209967789,200,0.03229113834557456,0.054216715127082156,0.17572335128539068,0.07490029534993962,200,0.16524507391431523,0.23259758939678682,0.4443199621821767,0.2653715311832368,200,0.4897872200818199,0.5532730641446492,0.6650363202455166,0.5660363140298347,200,0.7454979994578955,0.7984299359235015,0.8522671606190597,0.798027595599657,200,0.04255478964937405,0.06211678658355077,0.08472368152523543,0.06290225602690902,200,10.0,11.0,12.0,11.12
|
||||
legacy,base,fresh-zh-1,3,divergence,200,0.028982629118581828,0.05374104665088453,0.09855992669154144,0.06108389672119415,200,0.010798964649438858,0.024645465426146984,0.08034284934401512,0.03651118309469894,200,1.3175070315523083,3.5480553550417184,6.017363110228346,3.5979467985382336,200,0.08234418947201927,0.2217534596901074,0.3760851943892716,0.2248716749086396,200,0.11061324421376217,0.23398621779410705,0.7429507366244389,0.3257845391446977,200,0.2976556613016461,0.488028777970165,0.8696035028760013,0.5342668192607622,200,0.7064194054950236,0.8101997774989895,0.9495847511402526,0.8153004192781174,200,0.8984980468457894,0.9461568854525526,0.9837933181787005,0.9424611423371019,200,0.0029290946529022268,0.009887020058842282,0.021721080315529443,0.01144397224682556,200,2.0,7.0,9.0,6.06
|
||||
legacy,base,fresh-zh-1,3,QKV,200,1.0199183085933328,1.1750049997936003,1.9069260513409971,1.3038114167831372,200,0.250296925008297,0.2935818135738373,0.3817494958639145,0.3097901244461536,200,7.196501267820574,8.441507481465084,9.751193704228024,8.44616190715833,200,0.4497813292387859,0.5275942175915678,0.6094496065142515,0.5278851191973957,200,0.04272226697397949,0.059693070665254866,0.08155381570809245,0.06124391513532672,200,0.19220363243778327,0.2452880404617358,0.28456213418232335,0.24238526627742943,200,0.5093232051398124,0.5697030938067207,0.6206605729117751,0.5686563434922721,200,0.7771778324174323,0.8286977027123357,0.8689578331982056,0.823016277118345,200,0.015305978073171437,0.04065693749556967,0.07249220059719527,0.041722741269430416,200,9.0,10.0,12.0,10.465
|
||||
legacy,base,fresh-zh-1,7,distance,200,0.43271681810729207,0.6038041174178943,0.7724204423837363,0.6074852672725684,200,0.08553836718201638,0.12564750388264656,0.22395427972078322,0.14068200470879674,200,5.5313715726687915,9.05250186330765,11.02686062187503,8.642337837729093,200,0.34571072329179947,0.5657813664567282,0.6891787888671894,0.5401461148580683,200,0.030066917695381033,0.05116493326388823,0.12617347149173305,0.06521773414851664,200,0.15726624767737105,0.20925916539896955,0.3767178850854693,0.23498832827314786,200,0.4891220651431828,0.5739843609213888,0.6664845470243277,0.5781163231024855,200,0.7490316363110263,0.809991048466753,0.8609176690120646,0.8073749254513416,200,0.04298112131965184,0.06218602194312592,0.08717511464006347,0.06344463270935115,200,10.0,11.0,12.0,10.985
|
||||
legacy,base,fresh-zh-1,7,divergence,200,0.0218641472303716,0.04420311772855712,0.07254370295777335,0.046871714137829486,200,0.007316942419856787,0.015788674354553223,0.05015565417706966,0.022694225640734658,200,1.6588479180249318,4.120520295512728,6.2380292228952285,4.145432773729307,200,0.10367799487655824,0.2575325184695455,0.3898768264309518,0.2590895483580817,200,0.10432844124517657,0.19221927366868563,0.5763515831250512,0.2563242193465016,200,0.2611877284354106,0.3885966051481249,0.77089108764741,0.45008556331872107,200,0.7068351247273077,0.8194089704456659,0.9291147357965708,0.8191980249013184,200,0.9010202346431437,0.953330766846342,0.981680517137843,0.9466400398116648,200,0.003264056999811985,0.009318286972849775,0.022254085990137965,0.011427831522990244,200,3.0,6.0,8.0,6.045
|
||||
legacy,base,fresh-zh-1,7,QKV,200,0.8571053378749639,1.0059685087762773,1.183649718435481,1.0213233802010655,200,0.1884602651000023,0.22348716855049133,0.2585281401872635,0.22458526909351348,200,7.7944570256649115,9.303971982080625,10.748833530405351,9.3710458372141,200,0.48715356410405697,0.5814982488800391,0.6718020956503344,0.5856903648258812,200,0.03256903217548453,0.047979708973495835,0.07018273007133141,0.048917588117041826,200,0.1801807823689268,0.22031704948065356,0.2803393183923416,0.22345551685200246,200,0.48813704243826284,0.5326324197050982,0.5923833864598406,0.5361561869350651,200,0.7333737956269141,0.7852361151915166,0.8346699733112655,0.7834009035789988,200,0.046062572291322504,0.07073412362494022,0.09617805857215993,0.07015698184671398,200,10.0,12.0,12.0,11.47
|
||||
legacy,base,fresh-zh-1,11,distance,200,0.40895284092985096,0.5534483557567,0.7309623930603266,0.5665644111053553,200,0.0644407406449318,0.09180759266018867,0.15148354917764661,0.10463369278237224,200,7.860647455468036,11.217304580067477,12.886992169252563,10.796342939084912,200,0.49129046596675224,0.7010815362542173,0.8054370105782852,0.674771433692807,200,0.016104150759088904,0.02842451321013062,0.06903769231945556,0.03846723173886477,200,0.1267601433826018,0.16769519761061916,0.27860473981048695,0.18530787381293057,200,0.4231478142339164,0.4907384095410912,0.5951599695983893,0.49822297329287757,200,0.6852193650415406,0.7369754038291347,0.8022838070372944,0.7395950195094779,200,0.07282213999186168,0.1024971813288944,0.13109117593541883,0.10247885237114591,200,11.0,13.0,13.0,12.45
|
||||
legacy,base,fresh-zh-1,11,divergence,200,0.016045292872877325,0.028061560849891976,0.0557658998943225,0.03360439002613021,200,0.004152655368670821,0.008428643923252821,0.022947734035551545,0.013680369750363752,200,2.6450400622520918,5.813073237536349,8.279876067241618,5.747200074222649,200,0.16531500389075574,0.3633170773460218,0.5174922542026011,0.35920000463891555,200,0.06216054871369186,0.11682778177670528,0.33709448395412145,0.16609993910311552,200,0.20591628311538554,0.3152610044344926,0.5464516179835875,0.352571257796797,200,0.6040428255496345,0.7240077921739443,0.8732768242837106,0.7241230388127226,200,0.8327742613094211,0.8957278587660789,0.9573599228471149,0.8942875428845476,200,0.011515956437173307,0.030337179298580402,0.0555964302539676,0.032318215880978524,200,5.900000000000002,9.0,11.0,8.335
|
||||
legacy,base,fresh-zh-1,11,QKV,200,0.7646997133269906,0.9415694242343307,1.1005946589633822,0.9381568700482603,200,0.10172526240348816,0.14520113170146942,0.17797999531030656,0.14074353359639644,200,10.556494727505484,12.132287690814861,13.526319939468888,12.102763513736924,200,0.6597809204690928,0.7582679806759288,0.8453949962168055,0.7564227196085578,200,0.012191935409086778,0.021252998106209633,0.034377010706376,0.022337764697260877,200,0.11199920705088734,0.14764846719062247,0.19376761916533158,0.1512855342947202,200,0.3976379062115118,0.4519844618754032,0.5082503153261523,0.4519074543448034,200,0.6643751919368126,0.7095712428180596,0.7495152168943192,0.7102956417116113,200,0.08967173405138439,0.11288960000503676,0.13725722023218334,0.1131325445636444,200,12.0,13.0,13.0,12.845
|
||||
legacy,base,fresh-zh-1,15,distance,200,0.43292285297065974,0.5795048493891954,0.8028445993550122,0.598374522572849,200,0.08124532625079155,0.11660745367407799,0.17312516421079635,0.1251748430915177,200,7.27187131575747,9.280908514420442,11.168043529324263,9.19956206262213,200,0.45449195723484187,0.5800567821512777,0.6980027205827665,0.5749726289138831,200,0.028844266270079803,0.04826461737077646,0.08001845356029015,0.052541086242545665,200,0.1521058685250012,0.20071546433315057,0.2787634172105522,0.20895935456315512,200,0.48676783085182757,0.5606942171838359,0.6427062218665469,0.5640766258320425,200,0.7445924303680195,0.799933686349982,0.850776678718176,0.7998207023229159,200,0.04972880916576016,0.07123257799778379,0.09336855652866997,0.07216344075869018,200,10.0,11.0,12.0,11.335
|
||||
legacy,base,fresh-zh-1,15,divergence,200,0.019945300565086654,0.03742143244858198,0.06954158965454553,0.04249801874332888,200,0.006600805884227156,0.013597384095191956,0.0299729011952877,0.01775519482791424,200,2.723727922639179,4.419071740960671,6.453804065895095,4.506546875024986,200,0.1702329951649487,0.27619198381004195,0.4033627541184434,0.2816591796890616,200,0.09861066336700716,0.17471540683448422,0.3249596363003885,0.19923313577665447,200,0.25891546649056973,0.36224561549091905,0.5691559297756097,0.3955265302614648,200,0.6979655399278633,0.8148032333471716,0.9034887493058987,0.802466854880239,200,0.9009231053331983,0.948978783177368,0.9757591574499868,0.9435997776413383,200,0.005237995813748987,0.012665228570550776,0.02465008873120709,0.014240310710541213,200,4.0,6.0,8.0,6.415
|
||||
legacy,base,fresh-zh-1,15,QKV,200,0.6458288012538105,0.7748749549500644,0.9652682330459356,0.7980924565205351,200,0.11018678247928619,0.13081123679876328,0.18139647394418718,0.14203495394438506,200,8.73400421523041,10.246712388513933,11.44923822507546,10.193826057069092,200,0.5458752634519006,0.6404195242821208,0.7155773890672162,0.6371141285668183,200,0.02649819570006886,0.03743176517140752,0.05546138875430876,0.03928103644010278,200,0.1449581241047069,0.17280844858768796,0.22752581143256917,0.179581028414308,200,0.47710706891360144,0.5362226771385954,0.5856630329020351,0.531838099732763,200,0.7295533715428937,0.7722563931895174,0.8144699553256052,0.7730578378245819,200,0.05475231118994709,0.07820279139223463,0.09810128173046476,0.0765137326843442,200,11.0,12.0,12.0,11.695
|
||||
legacy,base,fresh-zh-1,19,distance,200,0.24335797766689213,0.29331015225034207,0.3536794791696593,0.29521937190263997,200,0.05748218968510628,0.08412716910243034,0.10968977808952332,0.08364754555746913,200,5.711247110272558,7.024246621721719,8.823026151557187,7.196968633935012,200,0.3569529443920349,0.43901541385760745,0.5514391344723242,0.44981053962093825,200,0.054229127537616,0.08518829188617832,0.12009932598436433,0.08591510904237093,200,0.2150525931638637,0.2828655296956779,0.3538350878800364,0.28249004018624446,200,0.5713467217876544,0.6286265816374231,0.6790948191973873,0.6282328187848598,200,0.788729863215255,0.8190756238169092,0.8500392600289038,0.8184562214453764,200,0.0516866276508774,0.06341780080102401,0.07863042920113657,0.06433900282961363,200,10.0,11.0,12.0,11.0
|
||||
legacy,base,fresh-zh-1,19,divergence,200,0.007393096410646649,0.012181064067817715,0.018965315184050268,0.012800878722067068,200,0.003304202063009143,0.007077381014823914,0.012031846866011619,0.007414750217576511,200,1.8131037119282571,2.5861945507183703,3.8056885226410704,2.773240141124483,200,0.11331898199551607,0.16163715941989815,0.2378555326650669,0.1733275088202802,200,0.21362110907349677,0.3457801242352132,0.5216823690293247,0.35375158226090586,200,0.39054159599782057,0.5554795386289151,0.7298192826476227,0.5607862203853119,200,0.8382733198357065,0.9014582827651687,0.9417105610582329,0.8940616955368718,200,0.9446463243627984,0.9674112908715309,0.9823178519511492,0.964551388545188,200,0.004339754926425057,0.007983271427843467,0.013694931597522919,0.008508104633810631,200,3.0,4.0,6.0,4.64
|
||||
legacy,base,fresh-zh-1,19,QKV,200,0.3160225347615778,0.37049319688230753,0.5122994367033243,0.3931979568884708,200,0.05125492662191391,0.06661264598369598,0.09244697988033293,0.06970386667177081,200,10.207788622643587,11.45153724767245,12.62791443689083,11.39239019619744,200,0.6379867889152242,0.7157210779795281,0.7892446523056769,0.71202438726234,200,0.017802356626204725,0.02647952857663271,0.037828739337075895,0.027571846487599604,200,0.14179378283656605,0.1789277489871398,0.20475874522368429,0.17752368670248805,200,0.42932173162525245,0.468638335677255,0.5210576323533085,0.4737948927943687,200,0.6715027907704788,0.7155576126236289,0.7461306792649463,0.7130189220379008,200,0.09833837692914749,0.11557345949284488,0.13936113192895164,0.11688660634552829,200,12.0,13.0,13.099999999999994,12.98
|
||||
legacy,base,fresh-zh-1,23,distance,200,0.0633149473782396,0.19330154557246715,0.24875540377106517,0.17162828117536263,200,0.01779393944889307,0.049417492002248764,0.07126184999942779,0.04604077075608075,200,5.629146379436792,6.977966572711463,8.279055098310208,6.967945352487096,200,0.3518216487147995,0.4361229107944664,0.517440943644388,0.4354965845304435,200,0.0621725285869526,0.08619551993155491,0.12282329129553866,0.09071510895108638,200,0.21480057422104684,0.266990922186803,0.33301653823564925,0.2719936755675187,200,0.594021704609691,0.6606513508815572,0.7369617105159236,0.6616751262489386,200,0.8178922005537227,0.8545975415872406,0.8995344632204977,0.8573324619253084,200,0.028710226356380875,0.045251362467490786,0.06104690403816136,0.04556362815948265,200,8.900000000000002,10.0,11.0,9.84
|
||||
legacy,base,fresh-zh-1,23,divergence,200,0.0006433107142544969,0.005029468058637576,0.009329614385455896,0.004928803411408012,200,0.0003166275477269665,0.0024420906556770205,0.005078282160684466,0.002524947199417511,200,2.1931030652823855,2.9386259925524554,4.010336386724875,3.0725923688604064,200,0.1370689415801491,0.18366412453452846,0.2506460241703047,0.1920370230537754,200,0.19932094657634944,0.2963147732831179,0.41971051938784687,0.3026397028759481,200,0.36362632817018486,0.49651409730384194,0.6510295296725981,0.5048326123672355,200,0.842685359961803,0.9076509547744371,0.9508851429048261,0.9008439566152725,200,0.9593154414306101,0.978183274274075,0.9915364427157294,0.976535529056014,200,0.0014586561152327788,0.004169070780395262,0.008220400394240034,0.00460339436229952,200,3.0,4.0,6.0,4.405
|
||||
legacy,base,fresh-zh-1,23,QKV,200,0.29037469587055964,0.36007841280661523,0.42876356788910924,0.3618773313559359,200,0.06149344183504582,0.10106398910284042,0.14472270011901855,0.10212308172136546,200,5.839062505612701,7.612101863939762,9.64438690124938,7.665156123130973,200,0.3649414066007938,0.47575636649623515,0.6027741813280862,0.4790722576956858,200,0.043933354259961456,0.07346269657826368,0.11601131881578822,0.0775862528069393,200,0.18957159455392047,0.2814065255030375,0.36074185492119154,0.2785400452429898,200,0.5423959525219761,0.5951639662416559,0.6429841849355784,0.5942475007152278,200,0.775967739021575,0.8089392917502702,0.8332560576910509,0.8069472692661399,200,0.05220282240400577,0.06465804028412984,0.07959129731591676,0.0650774446869007,200,10.0,11.0,12.0,11.125
|
||||
legacy,base,fresh-zh-2,3,distance,200,0.5147837075870484,0.6829850850626826,0.8971190208103508,0.6926104764753837,200,0.11468573287129402,0.16545837372541428,0.24532595425844192,0.17407447088509798,200,5.304095096025002,9.12824132284305,10.916963993062089,8.593923252709432,200,0.33150594350156265,0.5705150826776906,0.6823102495663805,0.5371202032943395,200,0.031040645558570792,0.05018688622426508,0.134436893872321,0.06729200681748726,200,0.1707759904195831,0.233337792425879,0.3886315688481297,0.25794270892352494,200,0.4727869244530316,0.5464853478854644,0.6427440411339127,0.5502366009148489,200,0.7302245911601384,0.7823344771146692,0.8386868279110486,0.7832864792369361,200,0.04568047562806946,0.0678677686395084,0.08999645419244942,0.06769614052735536,200,10.0,11.0,12.0,11.305
|
||||
legacy,base,fresh-zh-2,3,divergence,200,0.033448167066580936,0.05564773845230775,0.09787187126348726,0.06004451266739807,200,0.013152833096683025,0.027376949787139893,0.06018529832363128,0.03327927593141794,200,1.4727544829126265,3.3988920187533527,5.725476907183045,3.5449320107920017,200,0.09204715518203915,0.21243075117208454,0.3578423066989403,0.2215582506745001,200,0.11963532755144395,0.24716110088194726,0.6576526947945271,0.3167158478088197,200,0.31645763911178215,0.4970100259065895,0.8216160846783133,0.5330661500627615,200,0.6906147329649192,0.8098432117780925,0.9257492856690333,0.8062379211162343,200,0.8846961358686755,0.9365625382050167,0.9766278558195673,0.9333179735345538,200,0.003490389515842565,0.011699572151314348,0.023815301075335624,0.012853122026607502,200,3.0,7.0,9.0,6.6
|
||||
legacy,base,fresh-zh-2,3,QKV,200,1.0465959789813497,1.326384887623135,1.8111236160621047,1.3640863217425068,200,0.23445787727832795,0.28515270352363586,0.3634243905544281,0.2942305117100477,200,7.3752083316840364,9.263852056851638,10.973150670801747,9.116421928105702,200,0.4609505207302523,0.5789907535532274,0.6858219169251092,0.5697763705066063,200,0.030540504034247008,0.04847636544377003,0.07796202004584099,0.05271926403711071,200,0.16824336792511158,0.220655243402105,0.2776208168794974,0.2207363346845586,200,0.48328261869321454,0.5376425134375862,0.6122147155048353,0.5465372692511494,200,0.7475225662214877,0.8073752435269923,0.8596723189911747,0.8073101963329276,200,0.01504699354534652,0.04803234550828406,0.0742059194321406,0.04670363722708493,200,10.0,11.0,12.0,10.705
|
||||
legacy,base,fresh-zh-2,7,distance,200,0.3461533321416937,0.5831202674889937,0.7570074148476124,0.5766709119178995,200,0.07040502727031708,0.11080574616789818,0.16701333373785018,0.11430789001286029,200,6.8480609902259015,9.477582448189732,11.462535995996191,9.2879402417683,200,0.42800381188911885,0.5923489030118583,0.716408499749762,0.5804962651105188,200,0.0263901150613913,0.04587963140386321,0.08909532664702488,0.05404535160656519,200,0.13910662840989046,0.18719585664591068,0.30449170860394,0.20526591532690439,200,0.4654513551144829,0.550799964667037,0.6596268322314756,0.5568742351932628,200,0.7499024924051205,0.8036992126753697,0.8627447404576886,0.8025792324184949,200,0.03819984821398796,0.061363312641139395,0.08371504206415634,0.06124599579158597,200,10.0,11.0,12.0,10.98
|
||||
legacy,base,fresh-zh-2,7,divergence,200,0.015560420188137416,0.03576696577965777,0.06376001235039438,0.03857223819759504,200,0.004956961330026388,0.012278192676603794,0.027893566712737083,0.014450385831296444,200,2.3441436284271195,4.85312814600349,7.294545866668523,4.932494843897084,200,0.14650897677669497,0.30332050912521813,0.4559091166667827,0.30828092774356775,200,0.07956148857028608,0.1531252197288483,0.38839506587827133,0.19481818163872655,200,0.20943451843543534,0.33797916279509155,0.6335231511834918,0.3776489454393901,200,0.6533904097570001,0.7909383258385765,0.904002350934321,0.7811691843380018,200,0.8894584007339179,0.9456311054989818,0.9804330583997438,0.9389277564448875,200,0.003463397057282541,0.009868600105001825,0.02267380658795696,0.011757646771249202,200,4.0,7.0,9.0,6.665
|
||||
legacy,base,fresh-zh-2,7,QKV,200,0.8218812965322286,1.0770739214494824,1.2469542897306383,1.0517673873191233,200,0.1626240238547325,0.2037961408495903,0.24288544803857803,0.20300193943083286,200,7.655344856465389,9.707328259401525,11.612704653324975,9.730204981630752,200,0.4784590535290868,0.6067080162125953,0.725794040832811,0.608137811351922,200,0.025186759288658715,0.04321621995332453,0.07267003375080719,0.04537094552500097,200,0.14522274418396963,0.19556793573579595,0.2502941221608022,0.1967809943997404,200,0.4590558288460326,0.5390097153036804,0.6216716665349686,0.5399644871645356,200,0.7293553921357455,0.786360543772143,0.8406465958743105,0.7882964969244451,200,0.04172663492609001,0.06605676506088651,0.09306152261354463,0.06697438966416348,200,10.0,11.0,12.0,11.3
|
||||
legacy,base,fresh-zh-2,11,distance,200,0.3440644766204059,0.5116504959296435,0.702592519018799,0.5206322756764712,200,0.057624995708465576,0.08500373363494873,0.17153594493865965,0.10536864038556815,200,7.678969183860344,11.249454889883909,12.929734635907007,10.561917864766247,200,0.4799355739912715,0.7030909306177443,0.8081084147441879,0.6601198665478905,200,0.015830636730338177,0.028152752798739116,0.07237079286306858,0.04651089632551939,200,0.12516382305205778,0.1733862229785058,0.29591365308423623,0.19928878972405076,200,0.4119472567522575,0.479859237667988,0.609922620484741,0.49941370888429515,200,0.6884653752354623,0.7337170812536673,0.806508031618447,0.743431284538109,200,0.06520874060840567,0.09888416782934102,0.12499918941467338,0.09649088118711639,200,11.0,12.0,13.0,12.26
|
||||
legacy,base,fresh-zh-2,11,divergence,200,0.011142155992820335,0.024161406952771358,0.05232254761458533,0.032801581233288746,200,0.003320680861361325,0.007225634763017297,0.02942522913217544,0.01661940539430361,200,2.0824668656602046,5.565923289458514,8.888684606686917,5.610989218469195,200,0.13015417910376278,0.3478702055911571,0.5555427879179323,0.35068682615432467,200,0.05333640346721704,0.12498647316508793,0.4455497111445807,0.1908647128637297,200,0.20188493853436654,0.32884190496612775,0.6768494902535758,0.38083407850613216,200,0.5701547550098055,0.7081905078252608,0.8814072250084485,0.720534753865286,200,0.8270653845106197,0.892816767226302,0.9605770554679033,0.8942457305006778,200,0.008964640819787316,0.028237855713225096,0.051504934569925824,0.029574690313179733,200,5.0,9.0,10.0,8.125
|
||||
legacy,base,fresh-zh-2,11,QKV,200,0.6919123398605734,0.9175626114010811,1.079559326544404,0.8972398960427381,200,0.0953451782464981,0.14140773564577103,0.17902919054031372,0.14258599277585746,200,9.51595316669769,11.85988651912448,13.944805415968524,11.769389525385082,200,0.5947470729186056,0.74124290744528,0.8715503384980328,0.7355868453365676,200,0.009825522948296406,0.023272394708995434,0.04542622555126366,0.02562800807654317,200,0.11031296338462056,0.15235756008040896,0.226956520186547,0.16121381079474326,200,0.3895726605760364,0.45783505177297035,0.5417679219453154,0.4573380716566514,200,0.6557039264870891,0.7174907094368088,0.7712403004479108,0.7180762613104094,200,0.08018298408239406,0.10563455649338205,0.14052965922963975,0.10714902018395173,200,12.0,13.0,13.0,12.645
|
||||
legacy,base,fresh-zh-2,15,distance,200,0.40327895022928717,0.5365054130088538,0.7292719057761132,0.5568849647592288,200,0.06461491361260414,0.10110463947057724,0.17801222801208494,0.11628661843016744,200,6.907880354399288,9.898420046463862,11.712565489288654,9.519281348602632,200,0.4317425221499555,0.6186512529039914,0.7320353430805409,0.5949550842876645,200,0.024403667524334528,0.041095185414681816,0.08774647100017062,0.0545292239919708,200,0.14136737762319898,0.1893759313449372,0.28623818724530736,0.20715795888301936,200,0.4635272175302825,0.532743655826585,0.6539505442689656,0.5486530923641578,200,0.7286047244913315,0.7788100167488672,0.8538148259340307,0.7870064222228049,200,0.04632639967097977,0.07189789721739584,0.10056367831479326,0.07256751745330087,200,10.0,12.0,13.0,11.375
|
||||
legacy,base,fresh-zh-2,15,divergence,200,0.01512076747858373,0.03115562235143443,0.06729762715403921,0.03770122548443737,200,0.004175087762996554,0.010222150478512049,0.03169021606445312,0.01672638167743571,200,2.5900783850252123,4.94621318500099,7.172146776592252,4.879790595054058,200,0.16187989906407577,0.30913832406256186,0.4482591735370158,0.30498691219087865,200,0.08205806728514975,0.14899000249238348,0.3451619100824118,0.1963934973272972,200,0.23590396148789686,0.35123879999721785,0.5772594587058741,0.3826099775251893,200,0.6510318304649428,0.7716097737657366,0.9082652232970603,0.7783450016384065,200,0.8792237458343642,0.9305443298912011,0.9751178867676571,0.929992078179312,200,0.004606125769961684,0.01396851584265563,0.030066682752376825,0.01600288792364242,200,4.0,7.0,9.0,6.895
|
||||
legacy,base,fresh-zh-2,15,QKV,200,0.6588485824642702,0.7985384785570204,0.9420318270102144,0.8033623660623561,200,0.11081496700644493,0.15258733183145523,0.21193956583738327,0.15762553729116915,200,7.865406633211771,9.933724253112251,12.03145499226041,9.954708294043309,200,0.4915879145757357,0.6208577658195157,0.7519659370162757,0.6221692683777068,200,0.021989867739256368,0.04071173031049086,0.06894840633322505,0.04300444815259835,200,0.14215656997081155,0.18820013325602533,0.26432110425775074,0.1973612923287556,200,0.467392659436809,0.5320164441979365,0.5976185966363394,0.5322297243982639,200,0.7199498207634355,0.7742148498107582,0.8182871099556205,0.7707348312245728,200,0.05028904003853427,0.0762599505403896,0.10466600239162002,0.07766264171257939,200,11.0,12.0,13.0,11.72
|
||||
legacy,base,fresh-zh-2,19,distance,200,0.1801991507643834,0.27761521888896823,0.3554695088416338,0.27363203835324384,200,0.031025007553398612,0.05724814906716347,0.08640452027320862,0.05799578634090721,200,7.334868986779317,9.412911961385927,11.077758879926002,9.36214126871173,200,0.4584293116737073,0.5883069975866204,0.6923599299953751,0.5851338292944831,200,0.029622701960780155,0.046653341153934144,0.0787574263658076,0.050307644538884484,200,0.15641748139966435,0.1969667030021279,0.28189193173812366,0.20965446579842933,200,0.4930179417021164,0.5578293720615657,0.6167904546741356,0.5545430430302752,200,0.7463542328960098,0.7865632191004863,0.8257781520688219,0.7867799748411661,200,0.06078319793718209,0.07615962590512901,0.09195109644556287,0.07667649122156311,200,11.0,12.0,12.0,11.6
|
||||
legacy,base,fresh-zh-2,19,divergence,200,0.0032219047937360303,0.008727351967536379,0.014426813827594741,0.008820296328706831,200,0.000962602801155299,0.003277353127487004,0.0074657529126852746,0.0038093298792955467,200,2.507341334910479,4.477504812221616,6.281203520584329,4.464567491200771,200,0.15670883343190495,0.279844050763851,0.39257522003652057,0.2790354682000482,200,0.10315228033370576,0.17156138441682822,0.35876087690123126,0.20615982106380534,200,0.2601172774806182,0.3719046340678134,0.6014802552390575,0.4050797685287161,200,0.6989415753564249,0.8121332768210971,0.8872430146730127,0.800883430708973,200,0.9016709446595074,0.9414718953816446,0.9677019058950461,0.936942245580977,200,0.0074748144061829395,0.01428673427781333,0.02403330531498279,0.01530508423302874,200,5.0,7.0,8.0,6.69
|
||||
legacy,base,fresh-zh-2,19,QKV,200,0.3294968142639846,0.3965512770228088,0.4856201363261789,0.4079517453722656,200,0.04950256980955601,0.061345502734184265,0.07889391258358952,0.06428984832018614,200,10.639726795051333,12.040430001644602,13.003814968129394,11.896175732781762,200,0.6649829246907083,0.7525268751027876,0.8127384355080871,0.7435109832988601,200,0.015360558173858537,0.021923786352624507,0.03358655348373444,0.02352446999362818,200,0.13557158666803165,0.15121115195854967,0.19280496233177397,0.1576780838303767,200,0.4227330702463845,0.4588547213598372,0.5105609907031137,0.46196181567596517,200,0.666401987984752,0.7140073510868454,0.7397872355751911,0.7096373145971054,200,0.09936859000570922,0.1181786828259131,0.14117991197913995,0.12074725231935556,200,12.0,13.0,14.0,13.01
|
||||
legacy,base,fresh-zh-2,23,distance,200,0.07260303560906324,0.23694017919478938,0.30805367498251146,0.21687023622929702,200,0.016900292597711086,0.060725798830389977,0.09685527831315995,0.06006529964040965,200,5.325534666811308,7.062343284874409,8.729592759987023,7.017189744398395,200,0.33284591667570673,0.44139645530465055,0.5455995474991889,0.43857435902489966,200,0.05552307967244228,0.08436913727368543,0.13362625792397423,0.09300369715738753,200,0.20715641115806013,0.2658266403035652,0.3515270913709329,0.27525113136498064,200,0.5768886759222794,0.6532001293957612,0.7380530980760734,0.6556035301474417,200,0.8010733283871241,0.8517030914115188,0.8938012135357822,0.8501187743965459,200,0.0280943046160524,0.046067984663694855,0.06711216303038915,0.04665465452250071,200,9.0,10.0,11.0,10.015
|
||||
legacy,base,fresh-zh-2,23,divergence,200,0.0007367950744956887,0.00772236482562505,0.01609503587471863,0.00822785697112763,200,0.00028562566149048507,0.0036876266822218895,0.009380965214222669,0.004447453631291865,200,1.9799851413106009,2.9869010799867652,4.352054934982288,3.102468910118243,200,0.12374907133191256,0.18668131749917283,0.272003433436393,0.1939043068823902,200,0.1784295163937018,0.29044817210863744,0.47205792347631015,0.3141883179825297,200,0.34939824391502955,0.4899743034386207,0.6879486427401185,0.5101724534354644,200,0.8297345313004074,0.9041209886600927,0.9574739533921884,0.8961743464279366,200,0.9497207380272148,0.9759621593939646,0.9900062278066352,0.9725891211013746,200,0.0013486610062881663,0.0042507848975992,0.011009112195801963,0.005332989943601728,200,3.0,4.0,6.0,4.42
|
||||
legacy,base,fresh-zh-2,23,QKV,200,0.33836357791442423,0.43763377936556935,0.5444308941019699,0.444455892132537,200,0.07637957483530045,0.1244831420481205,0.21034854501485822,0.13109229957684876,200,5.704836845565464,7.681773575478758,9.900431518716248,7.692501569460188,200,0.3565523028478415,0.48011084846742236,0.6187769699197655,0.4807813480912618,200,0.0410727895256019,0.07219018256859389,0.12030961605504087,0.07904438930688229,200,0.20460037665902386,0.28543954657863846,0.3743948821036761,0.2877652193445346,200,0.5205230142876641,0.5828079036475049,0.6447792654653233,0.5822370229148761,200,0.7659429372615634,0.7960861698590243,0.8323762151950365,0.7969292281607114,200,0.045599628145957256,0.07020308719780469,0.08575859071507999,0.06773132437934061,200,10.0,11.0,12.0,11.29
|
||||
legacy,base,fresh-en-1,3,distance,200,0.5252659021643922,0.7200451572425663,0.9775034859310835,0.7384794788464205,200,0.1308414176106453,0.19005780667066574,0.29805870950222013,0.20634597267955543,200,4.794488709749097,8.205655685427681,10.553169124508095,8.009031013569468,200,0.29965554435931857,0.5128534803392301,0.6595730702817559,0.5005644383480917,200,0.03440937443169004,0.06332576025209877,0.15581127173801881,0.0780638007280054,200,0.1883847248399948,0.2682357262649282,0.42470076515085625,0.28363085772702634,200,0.49196037242739715,0.5682752056098923,0.6521272322138194,0.5669872715332271,200,0.7415683566855604,0.7903184435067119,0.8402944752219776,0.7894673620889282,200,0.05013157079692451,0.0726078709155922,0.095399863961596,0.0727427586290444,200,10.0,12.0,12.0,11.435
|
||||
legacy,base,fresh-en-1,3,divergence,200,0.03657998311018673,0.06375922867209738,0.13071855896851048,0.07590858518343452,200,0.017119598388671876,0.03612196817994118,0.088840002566576,0.04750879812519997,200,1.3517448154556186,2.7036518264869738,4.922167708676823,2.9735965507494266,200,0.08448405096597617,0.16897823915543586,0.30763548179230144,0.18584978442183916,200,0.1500401072024709,0.3278649442112621,0.7224388057118413,0.3764720703865471,200,0.37722934025425736,0.584493422319244,0.8574665387411939,0.5955056245592193,200,0.720753322212144,0.8402769636062397,0.9366637648614378,0.8338450409046326,200,0.8986704348182338,0.9500650508651199,0.9800354476704709,0.943634389750679,200,0.004016308725895021,0.01166079986047167,0.023390373518317843,0.013164300579311177,200,3.0,6.0,9.0,5.92
|
||||
legacy,base,fresh-en-1,3,QKV,200,1.1010740990925114,1.3543728347722208,1.8310944105382077,1.4172562011762784,200,0.2824027270078659,0.3214397579431534,0.3999712496995926,0.33167507566511634,200,7.051466896914471,8.452564179047128,10.582944184630527,8.717685363378575,200,0.44071668105715445,0.5282852611904455,0.661434011539408,0.5448553352111609,200,0.034124609663255125,0.059527793142735264,0.08460267987083449,0.059096589201155805,200,0.19049060067661536,0.24169465643832275,0.28761437780167387,0.23929467458421333,200,0.4733027615244746,0.5589565563114933,0.6517027008846887,0.5584453388754818,200,0.7445807477412757,0.8094723748407489,0.8941137978363501,0.8111042453298671,200,0.014149877714614578,0.04733354365090493,0.08122643701197324,0.04911841556807503,200,9.0,11.0,12.0,10.68
|
||||
legacy,base,fresh-en-1,7,distance,200,0.42382513666525484,0.6078069985378534,0.7762976725120098,0.6047175329021411,200,0.08186280280351639,0.11928264796733856,0.20279795229434966,0.12993449360132217,200,5.2684249272688914,9.284846390697087,11.584071745937548,9.020189238112819,200,0.3292765579543057,0.5803028994185679,0.7240044841210967,0.5637618273820512,200,0.025414074667080077,0.04821633030923124,0.13579863895911298,0.06206218609821719,200,0.13628849605637222,0.19926919792139358,0.3573190311116594,0.22364927416547772,200,0.46442194635302225,0.5545787239127964,0.6535459545865228,0.5606615276495901,200,0.7472960366904517,0.8005331227891845,0.8561625252391857,0.8010191938059725,200,0.04311398972753782,0.06126412435345474,0.08558115694072826,0.06341542019751516,200,10.0,11.0,12.0,11.1
|
||||
legacy,base,fresh-en-1,7,divergence,200,0.0210299493551247,0.041246538941777544,0.06721432701724551,0.043585980836786435,200,0.006701526325196028,0.01422842824831605,0.04112701267004013,0.01909851892036386,200,1.7860569878673398,4.5360171233995725,7.3734922631087505,4.67952068477081,200,0.11162856174170874,0.2835010702124733,0.4608432664442969,0.2924700427981756,200,0.07799573239916363,0.1684882364505519,0.5305697372661342,0.23399521490718442,200,0.2163755568674938,0.3725131729390068,0.7405501321902432,0.41946526146364504,200,0.6392109590204115,0.7961574337460801,0.9319402144416754,0.7893348187273534,200,0.8923583286138922,0.9466221129758281,0.9795742236133427,0.9393528461758904,200,0.00338059084708163,0.009482982066217775,0.022206033332710727,0.01184926370899521,200,3.0,6.5,9.0,6.34
|
||||
legacy,base,fresh-en-1,7,QKV,200,0.8712185150827281,1.0402824106859043,1.223043517023325,1.0408493226702558,200,0.1742448091506958,0.2156040444970131,0.2560519903898239,0.21798965618014335,200,8.02160313162796,9.688187902457031,11.41896365440976,9.741112446786914,200,0.5013501957267475,0.6055117439035644,0.71368522840061,0.6088195279241821,200,0.0267452252926647,0.043433516698653155,0.06630807930044977,0.04465950294188996,200,0.1670947263614369,0.20846820360177704,0.2756778051675545,0.21290386664722902,200,0.46160249039682216,0.5224348477986831,0.5621384075676695,0.5168756794290319,200,0.7373330764017009,0.7815248457473716,0.8182293748928509,0.7767400682036921,200,0.04082117817627321,0.06183648480512356,0.08624314900080425,0.06445718695172137,200,11.0,11.0,12.0,11.425
|
||||
legacy,base,fresh-en-1,11,distance,200,0.3480386174749583,0.543599259108305,0.7279269386082887,0.5426706708426354,200,0.05968044437468052,0.09049655869603157,0.1500500649213791,0.10147279778495431,200,7.786795436381748,11.167258724565684,12.856941973850715,10.717228882353261,200,0.48667471477385926,0.6979536702853553,0.8035588733656697,0.6698268051470788,200,0.016297805213144903,0.028850671209787696,0.07032379809211387,0.03941906345986685,200,0.126183576611211,0.17118246445915597,0.27297499398166714,0.1883892132250232,200,0.4253922406250987,0.4836082610551373,0.5842930396744872,0.49745094528228123,200,0.6902138798479083,0.7384927740399874,0.8014818866030737,0.7431939922447773,200,0.07471577023129482,0.09971293189387676,0.128884513148028,0.09995610012552845,200,12.0,12.0,13.0,12.375
|
||||
legacy,base,fresh-en-1,11,divergence,200,0.012129754142551973,0.02756568178301677,0.053156213027978086,0.03111563401675994,200,0.0035617715446278455,0.008189627435058355,0.022515043430030347,0.012655710716499015,200,2.772891556438018,5.620292373824549,8.392975529674137,5.65763210199563,200,0.17330572227737612,0.3512682733640343,0.5245609706046336,0.3536020063747269,200,0.06042468716479553,0.12312224875165927,0.3180355820918079,0.17066951470487632,200,0.1990284378536723,0.3222974385676557,0.577510841922257,0.36134668632905187,200,0.5958839053122645,0.7174585738399295,0.8489595309124881,0.7218225477606007,200,0.8345675211164408,0.8958787219369339,0.953208964678737,0.8971598878108424,200,0.01245711316804196,0.028983649447609384,0.05255686712975372,0.03081421530304316,200,6.0,9.0,10.099999999999994,8.225
|
||||
legacy,base,fresh-en-1,11,QKV,200,0.7310712728649378,0.9247950157150626,1.100934425368905,0.9188851007819175,200,0.09555794224143029,0.14651821553707123,0.1867323637008667,0.14397828802466392,200,10.181497428447223,12.162679164102345,13.51513785670134,11.95559962381206,200,0.6363435892779514,0.7601674477563966,0.8446961160438338,0.7472249764882537,200,0.012257180879865775,0.021033367430259478,0.03809874098679807,0.023718019033761684,200,0.11219114064328034,0.1534239971476366,0.21174015876909552,0.15776689704574276,200,0.40151250960251883,0.44164458518768407,0.5117346580720709,0.45302702288091495,200,0.6732404801270179,0.7152158421781707,0.7554212286570804,0.7151006381798125,200,0.08130715972510741,0.1125337841224287,0.13469992492262145,0.10986820736768976,200,12.0,13.0,13.0,12.735
|
||||
legacy,base,fresh-en-1,15,distance,200,0.4415515260305256,0.5724745735060424,0.7562368151731789,0.5807246377062983,200,0.06811262890696526,0.10248782485723495,0.15816269665956495,0.10990720879286528,200,7.529556492098475,10.043100982924784,11.830367405808222,9.857777920876396,200,0.4705972807561547,0.627693811432799,0.7393979628630138,0.6161111200547748,200,0.02349704647211202,0.03954228448838887,0.07499737416111749,0.04573978544263805,200,0.13440912438352798,0.1823663584313931,0.2529708800712911,0.19014998749873235,200,0.4534672937478886,0.5284547764820187,0.6352246885760922,0.5384655484043196,200,0.737841351853865,0.7883864326779022,0.8530867643748598,0.789249829856754,200,0.046555005837892384,0.07025507592106536,0.09329942047575829,0.07066103067246497,200,10.0,11.0,12.0,11.305
|
||||
legacy,base,fresh-en-1,15,divergence,200,0.018004731212204205,0.03348266697184954,0.059227829484098035,0.03692166017076658,200,0.004639329807832837,0.010504491161555052,0.025016705133020872,0.013612156253657304,200,3.274025923895891,5.125033934823258,7.78695510128457,5.334980079610843,200,0.20462662024349318,0.32031462092645363,0.4866846938302856,0.3334362549756777,200,0.07031469472045071,0.14146500516297272,0.25914426730502294,0.16129882478698312,200,0.2029433759326427,0.32494952021570706,0.5013753590550546,0.34711214847798033,200,0.6287480978828671,0.7532603672391712,0.8900012906240546,0.7571391532104915,200,0.8834822446646576,0.9343652837231533,0.9743636498315802,0.9299282811985253,200,0.004901058654277581,0.013966580533476772,0.028204854633257292,0.015356057073776968,200,5.0,7.0,9.0,7.035
|
||||
legacy,base,fresh-en-1,15,QKV,200,0.6223420369904489,0.778354502748698,0.9231582049280405,0.7789445933763637,200,0.10168107002973557,0.1296810358762741,0.18023506850004192,0.1366561456397176,200,8.41350180833179,10.523084858081432,11.822431940106984,10.315722761810889,200,0.5258438630207368,0.6576928036300895,0.7389019962566865,0.6447326726131806,200,0.02355731133370658,0.034697786419297105,0.060114480614131806,0.03902258723578814,200,0.1334137715490158,0.16518611109273196,0.2417386911606085,0.1778114040581989,200,0.4630055955557551,0.5157883506966777,0.589183515042034,0.5209355972429051,200,0.7326387455106063,0.7717549500481755,0.8128654685207433,0.7724085128803674,200,0.05658227244761688,0.07334248693672096,0.0939632807903904,0.07454463698491803,200,11.0,12.0,12.0,11.63
|
||||
legacy,base,fresh-en-1,19,distance,200,0.30193768690805883,0.3825230540242046,0.45287913030479104,0.3826136138779111,200,0.06877774968743325,0.10705734044313431,0.1584443613886833,0.11202324468642473,200,5.280599074492594,7.197478374739278,8.886627395716664,7.092955534662397,200,0.3300374421557871,0.4498423984212049,0.5554142122322915,0.4433097209163998,200,0.05336540868347305,0.0815335800050039,0.13533071359772827,0.09134531188350514,200,0.2066737892973911,0.2802597576393344,0.3744575598529207,0.287352390095275,200,0.5709947479340537,0.636453290661577,0.7013767933232948,0.6394547653851447,200,0.7891912367314837,0.8343033990581601,0.8732837537132458,0.8324792919488169,200,0.045984321058947716,0.05991051438894515,0.07563192302723536,0.06042171254999941,200,10.0,11.0,12.0,10.665
|
||||
legacy,base,fresh-en-1,19,divergence,200,0.011181799773294188,0.020532311007855242,0.036456327223640984,0.023194204544329865,200,0.004730595322325826,0.011461302638053894,0.025105519965291017,0.014161832590471022,200,1.7412255463230686,2.659619212849171,4.217517466409308,2.8566335462318433,200,0.10882659664519179,0.16622620080307318,0.2635948416505817,0.1785395966394902,200,0.186248787168078,0.3343939097149974,0.5459485804755462,0.3479876147285212,200,0.37021580694925976,0.5612792278356824,0.7436607439631449,0.5598981971998174,200,0.8284835735130546,0.8959170292033419,0.952508250082405,0.8940651980407996,200,0.9457998846249187,0.9721946631458249,0.9865427685131878,0.9696579546258809,200,0.0032790084639732243,0.006570122373840076,0.01298995768820614,0.00743597117095286,200,3.0,5.0,6.0,4.545
|
||||
legacy,base,fresh-en-1,19,QKV,200,0.31665201601572335,0.39338733768090606,0.5794233224354685,0.41706886326428505,200,0.04845853969454765,0.06443605571985245,0.0888820044696331,0.06650541638955473,200,11.007434277440948,12.080045669666244,13.410993294223736,12.089160451072276,200,0.6879646423400593,0.7550028543541403,0.8381870808889835,0.7555725281920173,200,0.01287010285103627,0.02163322757509914,0.030237538393672932,0.02213954175105454,200,0.1312497628212408,0.15828317223401697,0.1958847498207491,0.1603144957098266,200,0.3972756301685534,0.4453810209838822,0.48398942204827167,0.44485647950146784,200,0.6651939623777732,0.6972101794953363,0.7309527729056692,0.6971647468186949,200,0.09977808790465417,0.11870345155728651,0.1370675737754651,0.11832447995976308,200,12.0,13.0,13.0,12.965
|
||||
legacy,base,fresh-en-1,23,distance,200,0.06319582018768415,0.21820172836305574,0.2689065179089084,0.19664797570407244,200,0.018564413674175738,0.046818315982818604,0.07088856622576714,0.048208457650616765,200,6.231787440009218,8.087427799617846,9.606946661964134,7.942121688052594,200,0.38948671500057613,0.5054642374761154,0.6004341663727584,0.4963826055032871,200,0.04436429250269472,0.06522627446553322,0.10450701511879815,0.07312091215983446,200,0.18335785834009402,0.23800731424651156,0.32609806366392924,0.24993717399838947,200,0.5431624918329219,0.6045795630472731,0.6805249844030334,0.6099194619996151,200,0.773016331414066,0.8156516430145131,0.8662197142008561,0.8177819383249778,200,0.042215155369451175,0.06066362336116296,0.08112226200805865,0.06092651660864405,200,10.0,11.0,12.0,10.86
|
||||
legacy,base,fresh-en-1,23,divergence,200,0.0006548373678775478,0.005710974774956412,0.010046254622852756,0.005727734705190684,200,0.0003446378308581188,0.002191954874433577,0.005025189882144332,0.0028395897474547383,200,2.0805880696920287,3.456884827849072,4.795167555360664,3.4264856378474575,200,0.1300367543557518,0.216055301740567,0.2996979722100415,0.2141553523654661,200,0.15577992324836368,0.24189662990558702,0.44600929059296407,0.2787465944918049,200,0.31801049757627503,0.4574181097015638,0.6713400086388607,0.4804506375313599,200,0.7890561926977303,0.8653031491092444,0.93487827550346,0.8635734485331668,200,0.9305897464303312,0.9601534309674675,0.9821965403596922,0.9579355198906288,200,0.003565159552665757,0.008419251055247715,0.01576835887178605,0.009210318846961947,200,4.0,5.0,7.0,5.425
|
||||
legacy,base,fresh-en-1,23,QKV,200,0.3165625500259921,0.3903192594880238,0.4747440696693957,0.39382400568167214,200,0.05849056728184223,0.08628058061003685,0.1325710192322731,0.0935366622172296,200,6.609176746232382,9.043481104332157,11.62641145284913,9.02073667887968,200,0.4130735466395239,0.5652175690207598,0.7266507158030706,0.56379604242998,200,0.025078482539711726,0.05128207313746439,0.09472850067847781,0.05787633461985751,200,0.1533327697107659,0.22689142102930943,0.33277540974781106,0.2369125603220013,200,0.46867103693328954,0.538736420168326,0.605079627042817,0.5383087529349373,200,0.7142758519297637,0.7724137837943109,0.8124701520823968,0.7661928601375356,200,0.05926498734040882,0.0789352885292616,0.09938865574208369,0.08012805172165825,200,11.0,12.0,12.099999999999994,11.81
|
||||
legacy,base,fresh-en-2,3,distance,200,0.619015127280727,0.8163227243348956,1.0459965398535132,0.8165744480350986,200,0.13608818650245666,0.1932932659983635,0.32764911055564877,0.21182226568460463,200,5.032549758296777,9.057259810463414,10.919899757335635,8.51093830816483,200,0.31453435989354855,0.5660787381539634,0.6824937348334772,0.5319336442603019,200,0.03101657788279797,0.05110258394874739,0.1452888043355792,0.06963123683933645,200,0.1743225468570771,0.2408677450329705,0.40932954352313083,0.2626818735898847,200,0.47965442934278435,0.5493611268914893,0.6477410004917799,0.5567935643343007,200,0.7380118569484899,0.7810466112897226,0.8285687368273159,0.7808712610072976,200,0.05596665307328854,0.07573437118325474,0.09884046993322032,0.07700475198656906,200,11.0,12.0,12.0,11.61
|
||||
legacy,base,fresh-en-2,3,divergence,200,0.04602197141357465,0.07628454504492765,0.1473817919920748,0.08694661521343278,200,0.018520138412714004,0.037362802773714066,0.10735419690608977,0.05066753458697349,200,1.4161641426749698,3.258386667467023,5.506307537049726,3.432265370737158,200,0.08851025891718561,0.20364916671668895,0.3441442210656079,0.21451658567107237,200,0.12705090325975432,0.2606942211190351,0.6865422169867628,0.32840330493091907,200,0.32065657972711803,0.5216855022234688,0.8383881975030106,0.5430453325998795,200,0.7088595794927229,0.8122219251794255,0.9368725031561941,0.8176237296849093,200,0.8909261078428391,0.9403066200847392,0.97681748180342,0.9366664684168806,200,0.0049321402912174805,0.013636212932307625,0.02948606380393602,0.01560636028815923,200,3.0,7.0,9.0,6.28
|
||||
legacy,base,fresh-en-2,3,QKV,200,1.1746613873168825,1.510028968565166,2.0858569553587585,1.555826693035633,200,0.28438410758972166,0.3382027745246887,0.4241101145744324,0.3472541347891092,200,7.2575944999530355,9.508067081538398,11.0402235143104,9.32555196008267,200,0.4535996562470647,0.5942541925961499,0.6900139696444,0.5828469975051669,200,0.02994973474423352,0.04551877659702507,0.08030585122282001,0.0507103968599783,200,0.18697113209949143,0.223359743917515,0.2870124131716494,0.22866373544002264,200,0.46995368469248416,0.5211731689492343,0.5982711448874869,0.5313307769316011,200,0.7317361846740532,0.7699960939205248,0.8574193990555813,0.7850918204550286,200,0.023050843473959227,0.07008059645349982,0.09458089018701459,0.0627653209542192,200,10.0,12.0,12.0,11.15
|
||||
legacy,base,fresh-en-2,7,distance,200,0.4461918576504104,0.6388791813515127,0.8278976137051358,0.6368944962022942,200,0.08254119008779526,0.11129002645611763,0.18417678624391556,0.12279642166569829,200,6.93346311189749,10.118440127963417,11.935910340242762,9.699909515080263,200,0.43334144449359313,0.6324025079977136,0.7459943962651726,0.6062443446925164,200,0.022699839718099902,0.03875150949571206,0.08717669813312172,0.049379957173508314,200,0.133885100412968,0.17813627303394444,0.3036569467751457,0.19936827184516573,200,0.45232076974069135,0.5352846335625829,0.639442797795787,0.5414969846474517,200,0.7297303629015461,0.7925243460125426,0.8421283591605325,0.7870725151767829,200,0.04920873150533187,0.06861594240979486,0.09464068989405404,0.07117687340126778,200,10.0,11.0,12.0,11.355
|
||||
legacy,base,fresh-en-2,7,divergence,200,0.02392248858304811,0.04078545750178364,0.07289505623994046,0.04433911242503797,200,0.006813149619847536,0.012385479174554348,0.033921096846461295,0.0166567829658743,200,2.2212617656104263,5.472744621966769,7.795142491361117,5.2085862491113835,200,0.13882886035065164,0.3420465388729231,0.48719640571006984,0.32553664056946147,200,0.07017116877961757,0.12823873508735031,0.41354198376766177,0.18576687870653763,200,0.21498545207837663,0.3169572328970117,0.6440030615827387,0.36982071258550403,200,0.6315613984189503,0.7645023129881782,0.9087314605696031,0.7661572128480736,200,0.8797246356687405,0.9362302054984842,0.9748901003878205,0.9285459982454779,200,0.005088591712897109,0.013080652225355354,0.029537841067918073,0.016327182162954226,200,4.0,7.0,9.0,6.975
|
||||
legacy,base,fresh-en-2,7,QKV,200,0.8439604148734361,1.0644298213301226,1.261562438122928,1.0658534347102977,200,0.1528593808412552,0.19050481170415878,0.22889656126499175,0.18935274064540863,200,8.836788695124728,10.754828477231866,12.225580782317998,10.629566412326012,200,0.5522992934452955,0.6721767798269916,0.7640987988948749,0.6643479007703758,200,0.02058208541327704,0.03251359570599585,0.05404294824057522,0.035148194787918224,200,0.13284157694416462,0.1788917007838831,0.24630757684507618,0.18245421864570616,200,0.44224690712953507,0.4957507159009583,0.5472988236519651,0.49561188525174443,200,0.7187324301286275,0.7608959973387576,0.7927744143058353,0.7594865973331963,200,0.05245626007919535,0.07794464082226368,0.09899585052478131,0.07606100718288497,200,11.0,12.0,12.0,11.755
|
||||
legacy,base,fresh-en-2,11,distance,200,0.35895347251789644,0.4935784398112446,0.6759456943720579,0.5069780787237687,200,0.05690678879618645,0.08286124840378761,0.14674851298332212,0.09638299074023962,200,7.737020182887723,11.242860968377068,12.887845618082206,10.77721058656071,200,0.4835637614304827,0.7026788105235667,0.8054903511301379,0.6735756616600443,200,0.016098762050105,0.028208400768005237,0.07119894107235233,0.03943889962605179,200,0.12761073696529185,0.16533935603053623,0.27651501951729623,0.1893869334868225,200,0.42097097663387156,0.4825174697427024,0.5681556709248554,0.49465358437463036,200,0.6907857172104901,0.7337084473227219,0.7925056622144059,0.7367745111306826,200,0.07294075788625516,0.10327805304595494,0.12654921058093285,0.10235385037661761,200,11.900000000000002,13.0,13.0,12.465
|
||||
legacy,base,fresh-en-2,11,divergence,200,0.011709015141423152,0.023373060074845853,0.043107747244721384,0.027677225003011473,200,0.0032383885001763704,0.006865991745144129,0.021535840816795822,0.012058183691115118,200,2.617147312838424,5.815193020227113,8.254860065664781,5.656953591645987,200,0.1635717070524015,0.36344956376419457,0.5159287541040488,0.3535595994778742,200,0.06255034166302798,0.11676127799835936,0.3412924810466721,0.17408890896488177,200,0.20915395264079262,0.31149713702621673,0.5872563399457557,0.3647079617739476,200,0.5989962260400519,0.7132297614635023,0.847698837654311,0.7207813611467898,200,0.8365399310151895,0.8915830096437295,0.9519819496264191,0.8918119046965123,200,0.012226791247638183,0.03101575282505921,0.05147123162550377,0.03220087854244019,200,6.0,9.0,10.0,8.37
|
||||
legacy,base,fresh-en-2,11,QKV,200,0.7359494174830615,0.9326356733217835,1.145884907245636,0.9298488935478963,200,0.09351496696472168,0.14523176848888397,0.18735939413309097,0.14044940054416657,200,10.717571898623689,12.14954155138565,13.478252681728977,12.129532125072826,200,0.6698482436639805,0.7593463469616031,0.842390792608061,0.7580957578170516,200,0.012473213794830885,0.02112817180101991,0.03285864636076488,0.021954165683340247,200,0.11469601524654695,0.14721384419545386,0.19606222929122147,0.1517285537708352,200,0.39547905106435316,0.4473112548724165,0.48655219965937613,0.4446178318221039,200,0.6678676742594226,0.7140064867943439,0.7477165556982082,0.7120801330591198,200,0.07804972536981533,0.10655233864527767,0.13865592130473667,0.1069933877044463,200,12.0,13.0,13.0,12.665
|
||||
legacy,base,fresh-en-2,15,distance,200,0.38702396722510457,0.506677265977487,0.707418407453224,0.5327011090918677,200,0.06411192044615746,0.09348417446017265,0.13972039222717286,0.10226278582587839,200,7.170381058306093,10.101540135856517,12.000093265242583,9.853055460709545,200,0.4481488161441308,0.6313462584910323,0.7500058290776614,0.6158159662943465,200,0.022221571063954997,0.038928103948196784,0.08209690865050921,0.046867267852027876,200,0.13829966249706946,0.1801738810065037,0.2613564666782947,0.19297507424713756,200,0.45189410439643896,0.5258846711689636,0.6351730220499668,0.5368699061987069,200,0.727661096728591,0.783550899996448,0.8467842429308771,0.7843328184845805,200,0.04680053111145946,0.07342024934251046,0.09930788474187263,0.07250668411934788,200,10.0,12.0,12.099999999999994,11.415
|
||||
legacy,base,fresh-en-2,15,divergence,200,0.014904379848849203,0.026413629350372503,0.0540445632568435,0.03173870663458047,200,0.004110403498634696,0.0087392907589674,0.019521798379719256,0.011973647387931123,200,2.861618878531048,5.263653175745896,7.449828850302678,5.275311005872961,200,0.1788511799081905,0.3289783234841185,0.4656143031439174,0.32970693786706007,200,0.07651338448512326,0.13598132604987107,0.3060867089903157,0.16751286446273184,200,0.2222518855844168,0.3230173395912448,0.528403858435191,0.3538174925693752,200,0.63932631152739,0.7556958862192491,0.8842163670081871,0.7586556977394497,200,0.8747839717769333,0.9299194141226115,0.9748954353768851,0.9263114769021038,200,0.0043998137459033675,0.013607161124437913,0.029801979308480706,0.016334177442489106,200,5.0,7.0,9.0,7.1
|
||||
legacy,base,fresh-en-2,15,QKV,200,0.6537230805493891,0.7794557483866811,0.911126397177577,0.7828541885322192,200,0.11116750985383987,0.12902627885341644,0.15805570930242538,0.13244120195508002,200,9.07384844548802,10.53057160834012,12.056999577608554,10.55645541397451,200,0.5671155278430012,0.6581607255212575,0.7535624736005346,0.6597784633734068,200,0.02180200952197641,0.03462583220526033,0.050887279333119116,0.03563068161830936,200,0.1369891791787797,0.16929981055153218,0.21023479356583566,0.17131985011908718,200,0.45246805026559067,0.511033573909291,0.5692540338094116,0.5111686588743951,200,0.7185958869888703,0.7621207160887895,0.8114361646969457,0.7644355925077724,200,0.053444759849850344,0.07224500930324185,0.10663638382072271,0.0755794682677014,200,11.0,12.0,13.0,11.7
|
||||
legacy,base,fresh-en-2,19,distance,200,0.2726464945706539,0.3351541784359142,0.4111551860230975,0.33943474419385894,200,0.06109372414648533,0.09853297099471092,0.16328971683979032,0.10558945152908564,200,4.581190917138312,6.8349791647576525,9.555185215208226,7.015182438322473,200,0.2863244323211445,0.4271861977973533,0.5971990759505141,0.43844890239515455,200,0.04496617828490613,0.08939369954827804,0.16617676051723185,0.09804455836945618,200,0.2054372444484531,0.3007280504621388,0.42127846114557815,0.30452780939168195,200,0.5441094234248772,0.6236289718739731,0.7210506092576822,0.624288171046703,200,0.7732819012082369,0.8172121234100993,0.8676815183336485,0.8195356628470304,200,0.04113805877573698,0.05931367226440325,0.0796330337137415,0.060107127351147156,200,10.0,11.0,12.0,10.84
|
||||
legacy,base,fresh-en-2,19,divergence,200,0.008706798572291063,0.015453939310873466,0.03442741068347459,0.01911709053846181,200,0.0037327888188883667,0.009708748199045658,0.0266639232635498,0.012770538186305202,200,1.5059478175314827,2.3283594286352995,4.116339641849155,2.6889541231606904,200,0.09412173859571767,0.14552246428970622,0.2572712276155722,0.16805963269754315,200,0.19252173614791507,0.3914527550231741,0.6416584119354828,0.3951190433640895,200,0.3956409484070341,0.6260428077448426,0.8054351139186814,0.6035878985229525,200,0.7962501512945249,0.8953137420203126,0.9632413753752798,0.8845474427626331,200,0.9318388328149693,0.9669840990752945,0.9886394396485646,0.9622941005012189,200,0.0023902814363606756,0.0070997641679930515,0.015701510673880717,0.00834999646838886,200,2.0,5.0,7.0,4.69
|
||||
legacy,base,fresh-en-2,19,QKV,200,0.3187912561465055,0.3973900239216164,0.5418821140192449,0.4125086717179511,200,0.04147509410977364,0.05560419522225857,0.07609003633260726,0.05739068113267422,200,11.783751877479908,12.551006713767334,13.390163316749993,12.594410992552143,200,0.7364844923424942,0.7844379196104584,0.8368852072968745,0.787150687034509,200,0.012993802096629491,0.018319966636286975,0.023853459206771628,0.018279396113407467,200,0.1188703471968148,0.1370972023157161,0.16924785426310193,0.1398349652228136,200,0.3940755289456063,0.429976479939846,0.4579784135669662,0.425522191419923,200,0.6661560316585446,0.6937954723285793,0.7228878057773872,0.6928534788094453,200,0.08730105550587129,0.10398373882169466,0.1271444609130742,0.10548742279737996,200,12.0,13.0,13.0,12.61
|
||||
legacy,base,fresh-en-2,23,distance,200,0.09078306090086699,0.23895533673930913,0.30426530304830524,0.22054257429139398,200,0.023469170182943346,0.055075300857424736,0.09086301401257516,0.058427924080751835,200,6.046355040516195,7.619268179768053,9.585119576904154,7.71124691949597,200,0.3778971900322622,0.4762042612355033,0.5990699735565096,0.48195293246849813,200,0.0446169532993613,0.0733306737639372,0.10978017114031512,0.0772446894596137,200,0.19773511533094978,0.2665851835263795,0.32795801759184384,0.26523541761548086,200,0.5384536096770526,0.6047399313588374,0.6823232703752686,0.6100649683213264,200,0.7721297210241375,0.8152846236318051,0.8642578252561324,0.8177328237062715,200,0.041858638850155665,0.06342044784640975,0.08085008476584839,0.062306743584434034,200,10.0,11.0,12.0,10.865
|
||||
legacy,base,fresh-en-2,23,divergence,200,0.0012135659423137214,0.007214870226903258,0.012908981638338446,0.007276498641497913,200,0.0005508021858986467,0.003033301909454167,0.00825609192252159,0.004046378126477066,200,1.9597116099469432,2.893045380434243,4.568850724774657,3.13245915280203,200,0.12248197562168395,0.1808153362771402,0.28555317029841604,0.19577869705012688,200,0.16679896723177956,0.30204067509107413,0.4776355939223207,0.31428818102730444,200,0.3570597728925854,0.5292406933011643,0.7051504978165276,0.5258299567516221,200,0.7918961961439432,0.8734844680449818,0.9327463827750408,0.8671854212585104,200,0.9305979376284169,0.9640200868536802,0.9839926204582717,0.9595663811844602,200,0.0030538325890119454,0.008476923657072453,0.016210899278690762,0.009102293092427282,200,3.900000000000002,5.0,7.0,5.245
|
||||
legacy,base,fresh-en-2,23,QKV,200,0.36500207604840396,0.463483129395172,0.591547960229218,0.46953482207958586,200,0.0683653011918068,0.1145361103117466,0.19546185880899422,0.12639000492170452,200,5.68450203936997,8.349205648359964,11.13561512926826,8.462177238303555,200,0.3552813774606231,0.5218253530224978,0.6959759455792662,0.5288860773939722,200,0.02912209851682172,0.06109168875431771,0.12097804262640702,0.06723230822763684,200,0.16227343022583332,0.26477958399582235,0.3677970639682868,0.2626599213938303,200,0.48099272016296724,0.554040017768743,0.6180777985127296,0.550899471825888,200,0.735756746659926,0.7769178279885305,0.8103034565834726,0.7748067273259538,200,0.062487366454123124,0.07966180443947153,0.09900878833797638,0.0796716360105363,200,11.0,12.0,12.0,11.785
|
||||
legacy,finetuned,fresh-zh-1,3,distance,200,0.5058500332059339,0.6628626624587923,0.852037683292292,0.6699390435486566,200,0.10404090806841851,0.15646206587553024,0.28703975677490234,0.1777297541871667,200,4.395692567192741,8.803375466564411,10.76394343539448,8.316407660385602,200,0.2747307854495463,0.5502109666602757,0.672746464712155,0.5197754787741001,200,0.03243044228352473,0.05449928034720569,0.17600115873574035,0.07563509671621003,200,0.16715145884996077,0.23030724041674924,0.44420778915854175,0.2668685851617101,200,0.4867893035323031,0.5541630771567907,0.6654395973650928,0.5668200905815562,200,0.7472577999380504,0.7986899798249977,0.8518329804777794,0.7984525336094458,200,0.04268226398387162,0.06147920457465449,0.08372153519057257,0.06265020682404816,200,10.0,11.0,12.0,11.11
|
||||
legacy,finetuned,fresh-zh-1,3,divergence,200,0.02931847478259897,0.053589895131494814,0.09934526282850127,0.06179363254720442,200,0.010824603587388992,0.024480387568473816,0.08239184990525246,0.037233291992452,200,1.3108847568908117,3.5384518717516316,5.9617133285787105,3.5647751137146053,200,0.08193029730567573,0.22115324198447697,0.3726070830361694,0.22279844460716283,200,0.1122530542686608,0.23486221640583888,0.7470385164559139,0.3291069414140813,200,0.302494426589565,0.49099334785973014,0.8717602636021168,0.5375326291518931,200,0.7065932373822549,0.8104842101748082,0.950059030700837,0.8164899741109575,200,0.9006257561093752,0.9464169837200549,0.984028572864482,0.9428581230811506,200,0.0028971611898841237,0.009529118767553487,0.02237759178521854,0.011324089351372078,200,2.0,6.5,8.0,6.035
|
||||
legacy,finetuned,fresh-zh-1,3,QKV,200,1.019851364917122,1.1762591541046277,1.9026943183969707,1.3046665861534712,200,0.2511228770017624,0.2945101410150528,0.38503326475620264,0.311181485876441,200,7.166181643825139,8.440300198315242,9.736210764117871,8.4267526992139,200,0.4478863527390712,0.5275187623947026,0.6085131727573669,0.5266720437008687,200,0.04289062356713357,0.05971114484761812,0.08218061774393147,0.06154035047705248,200,0.19341212993471657,0.245245074118074,0.2865691460455167,0.24329162074517072,200,0.5099524924918601,0.5698094965870104,0.6221295485306336,0.5689552795392061,200,0.7766692776643427,0.828698195294602,0.8689581932850793,0.8230958737500952,200,0.015407480887623422,0.04078588665844558,0.0725488179892378,0.041687923620119666,200,9.0,10.0,12.0,10.46
|
||||
legacy,finetuned,fresh-zh-1,7,distance,200,0.4324484004406258,0.6057263414841145,0.7774853199720383,0.6091870698693674,200,0.0862711451947689,0.12815707176923752,0.22534952908754347,0.14150357464328409,200,5.504741543950139,9.067028123428674,10.978584817096781,8.632231532900727,200,0.3440463464968837,0.5666892577142921,0.6861615510685488,0.5395144708062954,200,0.030492309750291057,0.05097598484681559,0.12710620490491414,0.06543259321724686,200,0.15857985673378547,0.21039925721692304,0.37720966123564753,0.23558569482654032,200,0.4879878951662466,0.5725008026270932,0.666341531007123,0.5782947736583549,200,0.7507237037484191,0.8090279780604719,0.859765051623952,0.8068755188242386,200,0.042981292156939965,0.06206726243792761,0.08735770666947336,0.06368928379273293,200,10.0,11.0,12.0,11.005
|
||||
legacy,finetuned,fresh-zh-1,7,divergence,200,0.022090163968096023,0.04455909232547128,0.07339166822748666,0.04723967095044543,200,0.007442784868180752,0.016424255445599556,0.0507831510156393,0.02298852427280508,200,1.646456676220792,4.145596089772347,6.183995372903456,4.1270754461426575,200,0.1029035422637995,0.2590997556107717,0.386499710806466,0.2579422153839161,200,0.1058216199784738,0.19064952545356226,0.58119823026703,0.2577928097738181,200,0.2600904034344505,0.391990666767591,0.7736004272574302,0.4515602608216342,200,0.7085287001020996,0.8183120739421845,0.9292011342368476,0.8198704926396558,200,0.9026721978532883,0.9541563198524243,0.9816107496644997,0.946476126387369,200,0.0032400265069020004,0.009360928390881483,0.02254261657878358,0.01150808945515212,200,3.0,6.0,8.0,6.03
|
||||
legacy,finetuned,fresh-zh-1,7,QKV,200,0.8556868876330554,1.0096935704350471,1.190343896584818,1.0226397854470997,200,0.18944542557001115,0.22531970590353012,0.2604521334171295,0.22566337943077086,200,7.755768062393754,9.271274994756078,10.823013466539697,9.348351549697465,200,0.4847355038996096,0.5794546871722549,0.6764383416587311,0.5842719718560916,200,0.03188875876381534,0.04838407248293264,0.07086538409971506,0.04923659205143114,200,0.1815934554255544,0.22122193456775457,0.28198694574414573,0.22432124028359227,200,0.48724648741714555,0.5326446722253313,0.5952965952754499,0.5369786486457215,200,0.7328409932963513,0.7843032347308431,0.8355567951205937,0.7832716455658647,200,0.04606816722602272,0.0711806251095087,0.09648412434793747,0.07021609973167924,200,10.0,12.0,12.0,11.475
|
||||
legacy,finetuned,fresh-zh-1,11,distance,200,0.41084286058321595,0.5572939380072057,0.7344947831705212,0.5687112854316365,200,0.06433839052915573,0.09116770327091217,0.15383956879377364,0.10480795765295625,200,7.976462629106428,11.248654543110526,12.946027597589563,10.81089596176704,200,0.49852891431915175,0.7030409089444078,0.8091267248493477,0.67568099761044,200,0.015726698173309538,0.02815949211011465,0.06706216961451329,0.03836192990070031,200,0.1265621889906236,0.167345437215523,0.2787154480023499,0.18487751303578126,200,0.42374438067520565,0.48989643643679454,0.5951493600618984,0.49780689101073816,200,0.6841314904283856,0.7361636762380916,0.7997339801457822,0.7392065198321948,200,0.07219343297254223,0.10248839326592957,0.13108602999373328,0.10266163878073385,200,11.900000000000002,13.0,13.0,12.475
|
||||
legacy,finetuned,fresh-zh-1,11,divergence,200,0.015997752759358264,0.028740824600390624,0.05659749056285363,0.03386089729315245,200,0.004139428865164519,0.008311574812978506,0.023667275160551067,0.013761867680004798,200,2.5997735775956965,5.85324786299834,8.36487709655275,5.767196091455653,200,0.16248584859973103,0.36582799143739625,0.5228048185345469,0.3604497557159783,200,0.060850679030224224,0.11556983699989326,0.34380929097023205,0.16535082689416336,200,0.20421507933774302,0.3138543596406624,0.5496067304813796,0.35127379828795313,200,0.604865857555347,0.7231844109204622,0.8733510240293834,0.7234125258254116,200,0.8325581707221371,0.8955780044783157,0.9569610937092922,0.8938331599140352,200,0.011333721221577976,0.0301347074936543,0.05579390999228313,0.03246257697360777,200,5.0,9.0,11.0,8.335
|
||||
legacy,finetuned,fresh-zh-1,11,QKV,200,0.7657808202318848,0.943809452932328,1.0997600063681603,0.9398530184756965,200,0.10149468258023261,0.1449493169784546,0.17893981337547302,0.14111157048493625,200,10.549364283814738,12.095780221603281,13.540838671307672,12.108655849733182,200,0.6593352677384211,0.7559862638502051,0.8463024169567295,0.7567909906083239,200,0.01210741140358697,0.0215183587770312,0.034445271378187044,0.02230761184316656,200,0.11169783590553935,0.14797619001378765,0.19435506363943136,0.15139872654470618,200,0.3967132696538513,0.45164191972764134,0.507530021753695,0.45150897847081845,200,0.6638159557495305,0.708599739052342,0.7486721861497224,0.7098045395650439,200,0.08967663714208574,0.11216090127978093,0.1376104110040412,0.1132716598940651,200,12.0,13.0,13.0,12.86
|
||||
legacy,finetuned,fresh-zh-1,15,distance,200,0.4344516339246184,0.5818915931158699,0.810145240323618,0.6003341202205047,200,0.08145828619599342,0.11726810410618782,0.17221415489912031,0.12483294671401382,200,7.3390123463788175,9.313575922510744,11.279186037870009,9.244137500322186,200,0.4586882716486761,0.5820984951569215,0.7049491273668755,0.5777585937701366,200,0.027902878509687094,0.04786287949342402,0.07867625800256621,0.051971020613570965,200,0.1526583542398343,0.19928519724345484,0.2779532232073168,0.2079116937288793,200,0.4856918880621381,0.5582643544076817,0.64115670803694,0.5622225103512138,200,0.7455133970199624,0.7991636860484783,0.8504190071265194,0.7986628395940001,200,0.0512908510502909,0.0716077773427323,0.09386556312100733,0.07280152930215614,200,10.0,11.0,12.0,11.34
|
||||
legacy,finetuned,fresh-zh-1,15,divergence,200,0.020187086798705425,0.037816209461198014,0.06988014294256573,0.042500427273574816,200,0.006635648198425771,0.013751808553934097,0.029658192768692967,0.017593142387340777,200,2.7148767221476544,4.426040258693139,6.525039339810002,4.540860574538271,200,0.1696797951342284,0.2766275161683212,0.4078149587381251,0.2838037859086419,200,0.09680625615474965,0.17433179741709215,0.32623026483974704,0.19713619417937633,200,0.2529115169847112,0.360298012431744,0.569102159401318,0.3934523128028139,200,0.6909230884292634,0.8123387145687058,0.8983965618152343,0.8000647222113804,200,0.8989825016392547,0.9484265228418335,0.976264430893593,0.942748737202337,200,0.005561033071680699,0.013081302774620293,0.025241745308402033,0.014544001209530664,200,4.900000000000002,6.0,9.0,6.465
|
||||
legacy,finetuned,fresh-zh-1,15,QKV,200,0.6473793068667874,0.7750650024972856,0.9644346908666194,0.797752159871161,200,0.1097166545689106,0.1304437592625618,0.18059764951467513,0.14170988105237484,200,8.75563440100556,10.271942119142665,11.463015335019222,10.218909360505252,200,0.5472271500628475,0.6419963824464165,0.7164384584387014,0.6386818350315783,200,0.026386220003833137,0.037176222729881025,0.05515964930823638,0.03901047908882769,200,0.14413774253328193,0.1726994724995819,0.22620954486968425,0.17916361069360875,200,0.4766160880732244,0.5342560140676518,0.5858173740595268,0.5309701951671477,200,0.7284822843610517,0.7705373284691537,0.8120899628980603,0.7717808253389769,200,0.05501135706554682,0.07869084953395192,0.09860408766047744,0.07729015506927146,200,11.0,12.0,12.0,11.71
|
||||
legacy,finetuned,fresh-zh-1,19,distance,200,0.24394142020028084,0.2938681370578706,0.3541193129029125,0.2955749484390253,200,0.05815389193594456,0.08397391811013222,0.10911938995122909,0.0834014512039721,200,5.745877763925563,7.082413011762741,8.848188346438823,7.222747869413911,200,0.3591173602453477,0.4426508132351713,0.5530117716524264,0.4514217418383694,200,0.053885319467945704,0.08394182041607726,0.11897378924403401,0.08534274154621295,200,0.21329289655433034,0.2811703788366924,0.35357117325420356,0.2813272788985441,200,0.5700806387168691,0.6278402587654703,0.6786012620686396,0.6275343818415676,200,0.7885468507980791,0.8194710216799583,0.8498878927360143,0.8183670903883007,200,0.051309397290370054,0.06321581962060643,0.07866743415141092,0.06415655806914607,200,10.0,11.0,12.0,10.98
|
||||
legacy,finetuned,fresh-zh-1,19,divergence,200,0.0074008882197631465,0.012117031229536224,0.019021286701354254,0.012783098907902968,200,0.0033818921307101845,0.007051622727885842,0.011907061468809842,0.007372068026452325,200,1.814506734826708,2.6135008377383597,3.8221419193430464,2.791253204101235,200,0.11340667092666926,0.16334380235864748,0.2388838699589404,0.17445332525632717,200,0.2124117354619713,0.3414720257258348,0.5212137841074999,0.3511943761010098,200,0.3874535283952043,0.5553387106458454,0.7311001872439624,0.5583334203933736,200,0.8363921225980061,0.8996627872761279,0.9405772414614808,0.8931127625882351,200,0.944578102369686,0.9670506523449405,0.9821499223988346,0.9643307658731601,200,0.00423177401168672,0.007982521478774315,0.013699100536728907,0.008491247488903557,200,3.0,5.0,6.0,4.67
|
||||
legacy,finetuned,fresh-zh-1,19,QKV,200,0.31691776202060284,0.3718169096391648,0.5124558109790086,0.3939738808805123,200,0.0516067948192358,0.0665995366871357,0.09195303618907928,0.06984324920922517,200,10.243038600260366,11.435176321745292,12.600614472474808,11.381453932479753,200,0.6401899125162729,0.7146985201090807,0.7875384045296755,0.7113408707799845,200,0.017985298153220277,0.02661275788053501,0.03746909926354166,0.027640124053748855,200,0.14230412210228746,0.1787906181018254,0.2038366337746456,0.17758923313910874,200,0.4315806202786547,0.4688090790785815,0.5210297732629997,0.4742419200357918,200,0.672300123802585,0.7153884725501973,0.7466853405836136,0.7134748558252226,200,0.09862851623955549,0.1157877239059813,0.1388357198465542,0.11651556850327144,200,12.0,13.0,13.099999999999994,12.98
|
||||
legacy,finetuned,fresh-zh-1,23,distance,200,0.06388159943016945,0.19623323471751064,0.2531446535198484,0.1740048539786949,200,0.01832182388752699,0.050499480217695236,0.07378452494740487,0.04731547483243048,200,5.56848939941006,6.900231694987033,8.184766874079095,6.890018820407261,200,0.34803058746312876,0.43126448093668956,0.5115479296299434,0.4306261762754538,200,0.06365673871294886,0.08791751530876998,0.12488739713170251,0.09237430622760362,200,0.21844588080577335,0.2715531670077999,0.33529793642926264,0.275012564583363,200,0.5989743090444954,0.6646067459694016,0.7387126828666677,0.6649732098087013,200,0.8201613689523232,0.8566206218490717,0.901596074484819,0.858774557727192,200,0.028365553406272084,0.04487124964615697,0.060253481300644514,0.04513804856082138,200,8.0,10.0,11.0,9.79
|
||||
legacy,finetuned,fresh-zh-1,23,divergence,200,0.0006901651987082148,0.005219324859638164,0.00973783218344124,0.005141098331360925,200,0.00033571708481758836,0.0025502695934846997,0.0054441556800156835,0.002671822046322632,200,2.149468958765974,2.9163593379163086,3.9388013626663843,3.0238367143365603,200,0.13434180992287337,0.1822724586197693,0.24617508516664902,0.18898979464603502,200,0.20414413749375154,0.29908651692363913,0.42958005311757463,0.30825469061619226,200,0.3745791902629142,0.5013590363868263,0.6536927174338794,0.5107459306686102,200,0.8482022043711428,0.9099793923768091,0.9515086242438802,0.9038239661286167,200,0.9607334170048832,0.9788869697810352,0.9917617313376438,0.9773135781435305,200,0.0014597811960382274,0.003999080855242934,0.008040422470733454,0.004463362970367698,200,3.0,4.0,6.0,4.315
|
||||
legacy,finetuned,fresh-zh-1,23,QKV,200,0.29459859055932613,0.3643820123979822,0.4325845808489248,0.36556585077778436,200,0.06299016773700715,0.10373371094465256,0.14728040844202042,0.10448838975280524,200,5.750191519955028,7.524958239422787,9.567252655927101,7.577272301936753,200,0.3593869699971893,0.4703098899639242,0.5979532909954438,0.47357951887104704,200,0.0448255459168806,0.0750848894282625,0.11883442821479043,0.07937492275046037,200,0.1915508888435106,0.2847504803844537,0.3648609321377722,0.28213759960920604,200,0.5440477841321977,0.5986055303098882,0.6448852314217055,0.5967787742622381,200,0.7783788635260818,0.8103846041193312,0.8344878558611886,0.8081491229081149,200,0.05159775969518733,0.06442523208388024,0.0796889942598373,0.06486026557838954,200,10.0,11.0,12.0,11.105
|
||||
legacy,finetuned,fresh-zh-2,3,distance,200,0.5125219866866246,0.6836832482367754,0.899123005149886,0.6916345105721848,200,0.11319038346409797,0.1644924432039261,0.2453740507364273,0.17445670306682587,200,5.235020619887475,9.135484506864762,11.01682485951343,8.583540573426399,200,0.3271887887429672,0.5709677816790476,0.6885515537195894,0.5364712858391499,200,0.030154954757239668,0.05009424408497622,0.13708930276597447,0.06767556949207643,200,0.17200205480832123,0.2339990958157555,0.39130544018641317,0.25887823310725006,200,0.4724123636210896,0.5460659759134248,0.6420294306727232,0.5500688252084369,200,0.7303663351724328,0.7814037019672285,0.8380087545325704,0.7826461350814031,200,0.04659941909128795,0.06919593198707805,0.09080434577130443,0.06799123738287476,200,10.0,11.0,12.0,11.31
|
||||
legacy,finetuned,fresh-zh-2,3,divergence,200,0.03335582065249128,0.055234106802345195,0.0975515551835997,0.06000915537145431,200,0.012812168989330531,0.027057777158915997,0.060208709910511964,0.03344982156530023,200,1.461007835670777,3.4090811996984236,5.629454009349147,3.5303832404016644,200,0.09131298972942356,0.21306757498115148,0.3518408755843217,0.22064895252510403,200,0.12281604708367172,0.24622318613007146,0.6634908311515196,0.31917421222504627,200,0.31226564507769916,0.5051161363833065,0.8250579284880163,0.5353766966979242,200,0.6908737676504688,0.8099698511246994,0.9269769180286297,0.8065751828684068,200,0.8831817398995889,0.9359377612686524,0.9768263555477565,0.9329543881439624,200,0.003533434856047367,0.011870002519207182,0.024069934335411117,0.012985240748814597,200,3.0,7.0,9.0,6.62
|
||||
legacy,finetuned,fresh-zh-2,3,QKV,200,1.0506348759518005,1.3205010847887024,1.8141025012359022,1.3623629112681375,200,0.23481560498476028,0.2849467843770981,0.364360299706459,0.2946363193541765,200,7.378763905024805,9.237427735194055,10.957295897362197,9.11420946208894,200,0.4611727440640503,0.5773392334496285,0.6848309935851373,0.5696380913805588,200,0.030681326541503847,0.04880580052949091,0.07789234769903004,0.05276801169533139,200,0.16860702187736584,0.22190147915664507,0.27742395537387393,0.22129421663478394,200,0.4805385423540358,0.5374710367400103,0.6124942939350668,0.5460016508471797,200,0.7463043254924909,0.807286543427975,0.8594660809972213,0.8068839259493401,200,0.01503658075823833,0.04829061620175803,0.07450159033636068,0.046810642270349104,200,10.0,11.0,12.0,10.715
|
||||
legacy,finetuned,fresh-zh-2,7,distance,200,0.34791452850331556,0.5842436215025373,0.755469014425762,0.5765081007956178,200,0.06990895196795464,0.11024657264351845,0.16875618398189546,0.11491135157644748,200,6.83058738430597,9.468264261682762,11.38152143336971,9.254030697996624,200,0.4269117115191231,0.5917665163551726,0.7113450895856068,0.578376918624789,200,0.027052508539943156,0.04599053348886208,0.08949795103561503,0.054619534165136444,200,0.14046964690456532,0.18878365274942394,0.302105770600801,0.20646632734016004,200,0.46815075245237775,0.552760207595342,0.6656094735265188,0.5585129063685008,200,0.7512854084572818,0.8030209299525926,0.8612620933806042,0.8029971120189139,200,0.03853862342292973,0.061173196536176755,0.08260915376331923,0.06122417197631635,200,10.0,11.0,12.0,10.97
|
||||
legacy,finetuned,fresh-zh-2,7,divergence,200,0.01586177180337245,0.036294161790465296,0.06322779567381075,0.03872669632329512,200,0.0048873211722821,0.012154559139162302,0.028478650003671648,0.014622196591226384,200,2.384145537713565,4.832235954629758,7.243530257911512,4.8975850122044235,200,0.1490090961070978,0.3020147471643599,0.4527206411194695,0.30609906326277647,200,0.08059154667547638,0.15407404105129333,0.3808959089371291,0.19689162412131242,200,0.22011953088420133,0.3365545656048119,0.6262449340704327,0.38013304440943374,200,0.6576382121810833,0.7893124683020918,0.9087894702199236,0.7835920523733239,200,0.8935100679701526,0.9446106883188845,0.9803103206709901,0.9395801255000679,200,0.003612556739359208,0.009668702948251751,0.02232947832427495,0.011685047670984881,200,4.0,7.0,9.0,6.6
|
||||
legacy,finetuned,fresh-zh-2,7,QKV,200,0.8220097047276795,1.0754010220989585,1.2480343207716942,1.0515228099207161,200,0.16327896118164062,0.20446515828371048,0.24075977951288222,0.20300967697054148,200,7.741014399451195,9.751923306449608,11.556816497118731,9.725217439097724,200,0.48381339996569966,0.6094952066531005,0.7223010310699207,0.6078260899436078,200,0.02563099331290628,0.042714122875598955,0.07112757576499078,0.04541391955618454,200,0.1459702630899728,0.19526984834985983,0.25014720620378866,0.19685174519383597,200,0.4620600850270701,0.5383931586864501,0.6192228322591956,0.5405384729679098,200,0.7315555349084455,0.7862210468027049,0.8414131572193515,0.7890305125923591,200,0.042624591021466415,0.06586279678424224,0.09151866217416357,0.06672040432505308,200,10.0,11.0,12.0,11.295
|
||||
legacy,finetuned,fresh-zh-2,11,distance,200,0.3420585579704493,0.5131511618383229,0.713656562473625,0.5225335752026876,200,0.05691137611865998,0.08349378407001495,0.1718100905418396,0.10528727192431689,200,7.681253580427333,11.279052369938732,12.967482964792687,10.579536465360189,200,0.4800783487767083,0.7049407731211708,0.8104676852995429,0.6612210290850118,200,0.015590366020866456,0.027903939912073496,0.0723471845403094,0.04621268910842716,200,0.12569014210102789,0.17407788963216458,0.2975221303063201,0.19856418710481435,200,0.4120056727311184,0.47806380537953563,0.6029942430896261,0.498979667101168,200,0.689054805552244,0.7334792641568139,0.8067899314461515,0.7433726751852313,200,0.06502782100650646,0.09902852144761567,0.12438353880501478,0.09656794263140889,200,11.0,12.0,13.0,12.27
|
||||
legacy,finetuned,fresh-zh-2,11,divergence,200,0.011059433381524287,0.02426894685595471,0.052717565604598345,0.03293156405778632,200,0.0032389172352850436,0.006971218157559633,0.029518900997936724,0.016552795204916038,200,2.0794303454495586,5.614880816496796,8.860161044736016,5.634656508113279,200,0.12996439659059741,0.35093005103104974,0.553760065296001,0.35216603175707994,200,0.053722808990206726,0.12330577278423767,0.44631369074853017,0.1894006739753562,200,0.20538983673393663,0.32389163611740346,0.6659314110917746,0.37903544058600985,200,0.5717176063285822,0.7052560747360435,0.8826833162724189,0.7198150855460601,200,0.8282908711086922,0.8922396706870965,0.9602976736953308,0.8941808798626144,200,0.009064373702619734,0.02811297857635471,0.051160229370062614,0.02960888802065809,200,5.0,9.0,10.0,8.14
|
||||
legacy,finetuned,fresh-zh-2,11,QKV,200,0.693144768057391,0.925412739161402,1.0844144532456994,0.9018172005563975,200,0.09452014788985252,0.14309297502040863,0.17988594323396684,0.1430305104330182,200,9.59221678321239,11.897780339103182,13.969053164984176,11.785146955494952,200,0.5995135489507744,0.7436112711939489,0.873065822811511,0.7365716847184345,200,0.009692708569774298,0.022985921630165766,0.04453695389104979,0.02547662729162824,200,0.11034158669582426,0.15192407330837232,0.22716460460584137,0.16085816778628753,200,0.38873206980341246,0.45733355202322384,0.5399848673342282,0.45634070699730134,200,0.654853644411828,0.7166054236764154,0.7708580624617111,0.7176292645091717,200,0.0794386079507975,0.10513414974915933,0.13910804631008994,0.10714215353806864,200,12.0,13.0,13.099999999999994,12.655
|
||||
legacy,finetuned,fresh-zh-2,15,distance,200,0.40148575452622026,0.5353431067196652,0.7288359694648534,0.5570754437823779,200,0.06496871262788773,0.10132881999015808,0.17385471910238262,0.11591398332267999,200,6.998241791622267,9.904091043395335,11.7235320030134,9.55062381385794,200,0.43739011197639166,0.6190056902122084,0.7327207501883375,0.5969139883661212,200,0.024318470797908335,0.04103326725745832,0.0857534624729684,0.05406945498375755,200,0.14092044038477514,0.18927598042600918,0.2869929993919656,0.20646882085739773,200,0.45667166788213037,0.5321077490866748,0.652046508860414,0.5473509684810581,200,0.7251698127079848,0.7780737283100773,0.8480635375040252,0.7857114990923222,200,0.046033044573599906,0.07355791560615572,0.1017823516909986,0.07328293315028936,200,10.0,12.0,13.0,11.4
|
||||
legacy,finetuned,fresh-zh-2,15,divergence,200,0.015386353456960933,0.030787485957262106,0.06639435047222832,0.03759260210411867,200,0.004220988508313895,0.010267541743814945,0.030228849872946727,0.01663703273748979,200,2.6208114298486787,4.887935679518047,7.176374653931292,4.898645882512158,200,0.16380071436554242,0.30549597996987793,0.44852341587070577,0.30616536765700986,200,0.08196955288835654,0.15157031986437242,0.34033264381925055,0.19542595604013857,200,0.23391811433266282,0.34548123809296316,0.5742619932399166,0.38149365815327757,200,0.6469275116346318,0.7725198325466691,0.9060159818523092,0.7771177806696287,200,0.8783252793322771,0.9303580370075455,0.9753561254099244,0.9290947852401775,200,0.004189107956975351,0.01413513952089996,0.03097651260894574,0.01632771950524364,200,4.0,7.0,9.0,6.955
|
||||
legacy,finetuned,fresh-zh-2,15,QKV,200,0.6564589522313327,0.7951376941055059,0.9426559056155384,0.8031179200496991,200,0.11046612486243249,0.1500706747174263,0.21138656735420228,0.15656439259648322,200,7.910676072093794,9.959421328027512,12.077662559631015,10.003086928424652,200,0.4944172545058621,0.6224638330017195,0.7548539099769385,0.6251929330265408,200,0.02165064762544552,0.04043460328269005,0.06817221023984384,0.04245827332131425,200,0.1420688429341677,0.187070424854491,0.26333826531420795,0.1961422744508669,200,0.46541646185225954,0.5314674379034885,0.5966275688671829,0.5306611896286543,200,0.7159910929926296,0.772359384329738,0.8161730361968065,0.7690099693689441,200,0.05164218566624216,0.07617044136534726,0.1059621977150445,0.07862689079905355,200,11.0,12.0,13.0,11.75
|
||||
legacy,finetuned,fresh-zh-2,19,distance,200,0.1827268737135455,0.2769849563483149,0.3535121880471706,0.27422260138962884,200,0.030771126970648766,0.05691741779446602,0.08558357656002044,0.058072913102805615,200,7.3364849727910855,9.410761282595335,11.125883976053357,9.36196764371635,200,0.45853031079944284,0.5881725801622084,0.6953677485033348,0.5851229777322718,200,0.029205871030537615,0.046678833702360104,0.0787253954191635,0.05027755219958218,200,0.1542806627643924,0.19841024570586085,0.2798267139455918,0.20967976032364555,200,0.49205028633656955,0.5590225404738877,0.6151211626957044,0.5548330756996085,200,0.7444541677655754,0.786959713172273,0.8255097431957068,0.7866341473714119,200,0.06060393643988146,0.0764710992643006,0.09324893184861585,0.07688870283582803,200,11.0,12.0,12.0,11.61
|
||||
legacy,finetuned,fresh-zh-2,19,divergence,200,0.003292508405593253,0.00892513455369226,0.014333238465405882,0.008831977008136392,200,0.0009468624542932957,0.0032396214082837105,0.007324549742043018,0.0038088414202502464,200,2.5442683022674983,4.454869170080942,6.299193837969226,4.459591070433442,200,0.15901676889171865,0.27842932313005886,0.39369961487307664,0.2787244419020901,200,0.10266725729447898,0.17277703261765753,0.35260866883240005,0.20614080096895634,200,0.25291617244509523,0.3758968008818321,0.5972195311742736,0.40526447045887326,200,0.6998517606574765,0.813644630850807,0.8872716300047747,0.8014246106845456,200,0.8985961608641307,0.9413523394928868,0.9678687036580598,0.9369035684160489,200,0.0074891438214883446,0.014369238637586884,0.02396606058627198,0.015387645803997523,200,5.0,7.0,9.0,6.715
|
||||
legacy,finetuned,fresh-zh-2,19,QKV,200,0.33192017376422883,0.39918337296694517,0.4895441006869078,0.40948685067589397,200,0.04968588799238205,0.061346208676695824,0.07916101217269894,0.06444869667291642,200,10.690122675011466,12.062780483924442,13.014230753081312,11.906889920190741,200,0.6681326671882166,0.7539237802452776,0.813389422067582,0.7441806200119213,200,0.015294912986231049,0.021759605140141015,0.03311428416057813,0.023427698769422824,200,0.13532596483110548,0.15056469705877007,0.19228276100300973,0.15748660964200795,200,0.42309754398903726,0.45906083537001413,0.5078269349181959,0.46131705980456517,200,0.6652248787338767,0.7141345494107241,0.7381062921374709,0.7089174719521076,200,0.10028069697377659,0.11804763495191425,0.14013822003752532,0.12099897896834592,200,13.0,13.0,14.0,13.02
|
||||
legacy,finetuned,fresh-zh-2,23,distance,200,0.07338529543922051,0.2414670392172411,0.3160045251308475,0.2209648924601788,200,0.017421136610209942,0.06330005079507828,0.09977171793580054,0.06207659718580544,200,5.2290207398709185,7.021789022684013,8.650294646381452,6.932208561949449,200,0.3268137962419324,0.4388618139177508,0.5406434153988408,0.43326303512184056,200,0.056643852673506155,0.08524157751433836,0.13732312699501648,0.09506130170038249,200,0.20859870316361598,0.26923810920800434,0.3530213046784713,0.2784985679798936,200,0.580686048193725,0.6586964453050699,0.7400747198071147,0.6586543285680552,200,0.8032500503863862,0.8537503033757541,0.8956561206059831,0.8514536371262375,200,0.02770087246947366,0.04495250297802926,0.06658438252944686,0.0461497460448712,200,9.0,10.0,11.0,9.985
|
||||
legacy,finetuned,fresh-zh-2,23,divergence,200,0.0007662264667533948,0.008118722435469294,0.017084476208844988,0.008700540317491194,200,0.0003034963912796229,0.004006900591775775,0.00995445419102907,0.004764970342694142,200,1.932624674018252,2.962164596124307,4.363309829018594,3.052005860638815,200,0.12078904212614075,0.18513528725776918,0.27270686431366215,0.19075036628992595,200,0.17779897366963568,0.2934321657900141,0.4852606591963128,0.3204193931523045,200,0.3564382943484241,0.49299653564145934,0.6994879474165675,0.5156516012161843,200,0.8386323336649901,0.9076601445487469,0.9575752783196655,0.8990067007682687,200,0.9520061435515556,0.977189678295757,0.9901835626877757,0.9733600193608092,200,0.0013355533585586853,0.004103805355702506,0.010740566479524422,0.005178498518691428,200,3.0,4.0,6.0,4.385
|
||||
legacy,finetuned,fresh-zh-2,23,QKV,200,0.3437072126194835,0.4453878866042942,0.5508965730434283,0.4513268984865863,200,0.07847670689225197,0.13102351874113083,0.21609251350164413,0.1349725304543972,200,5.573319209947487,7.541249589313933,9.757272365817684,7.591562790141317,200,0.3483324506217179,0.47132809933212083,0.6098295228636053,0.47447267438383234,200,0.0426535097465029,0.07477914690535012,0.12472167295127631,0.08116949913199921,200,0.21046579354997486,0.2898403198463084,0.380772710338647,0.2918187449555109,200,0.5248318705928383,0.5856093004815595,0.6468250753339695,0.5842077494991466,200,0.7658443751775019,0.7971321485802321,0.8329996761478033,0.7978583731229285,200,0.045102114169298095,0.06961806066610074,0.08506238984235735,0.06704439408232261,200,10.0,11.0,12.0,11.27
|
||||
legacy,finetuned,fresh-en-1,3,distance,200,0.5272722512017936,0.7194183696992695,0.9743251336738467,0.7391252413106849,200,0.13242788463830948,0.1896832063794136,0.3003763437271118,0.20750779908150435,200,4.755661911266672,8.165227266009387,10.61818189199978,7.984527831783422,200,0.297228869454167,0.5103267041255867,0.6636363682499863,0.49903298948646385,200,0.03379003141133197,0.06396868567576264,0.157627384225991,0.07871848944003759,200,0.18756904413914927,0.2686358954076478,0.42691730494892827,0.2848654260532374,200,0.4902523890947212,0.5694137971413097,0.6523153375448887,0.5678267887881185,200,0.7431423696518655,0.7902275994905299,0.8388646022855889,0.7896387598207204,200,0.049711527539978526,0.07250725802964661,0.09500215358170434,0.07297714465443478,200,10.0,12.0,12.0,11.44
|
||||
legacy,finetuned,fresh-en-1,3,divergence,200,0.036565147697774594,0.06467274387432553,0.13234159049534355,0.07649200170074096,200,0.017537165619432925,0.03597976639866829,0.09022602587938308,0.048124216631986204,200,1.3501809696060056,2.6593000141215892,4.924236122516526,2.955952139965542,200,0.08438631060037535,0.16620625088259933,0.3077647576572829,0.1847470087478464,200,0.14995747289839553,0.3344635277702088,0.723351235129084,0.37904829527579426,200,0.3776575588250152,0.5876378653827586,0.8588072291668455,0.5979473518721612,200,0.7267072524190924,0.8423827274589697,0.9379209706292867,0.8350244302800124,200,0.8998899367949802,0.9495680033204967,0.9806466564298388,0.9439328040780017,200,0.003919582900601124,0.011878258399321056,0.023956953624644655,0.013230403603091861,200,3.0,6.0,8.099999999999994,5.9
|
||||
legacy,finetuned,fresh-en-1,3,QKV,200,1.101589256786974,1.3604650683701038,1.8197576638660393,1.4185659695525827,200,0.2836529314517975,0.322201207280159,0.3997243970632553,0.33260651528835294,200,7.02414872773536,8.443907306299828,10.536824223425482,8.705533180957989,200,0.43900929548346,0.5277442066437392,0.6585515139640926,0.5440958238098743,200,0.03456598021300699,0.059657162335291405,0.0851904395980732,0.05928599886380974,200,0.19194071556219247,0.24181001278360525,0.28729170830666706,0.239780574705315,200,0.47353337125429484,0.560367223441776,0.6531334634736843,0.5588200431991017,200,0.745182627684237,0.8093316130164365,0.8931026090338946,0.8111210122446713,200,0.014373158241296832,0.04818839096856231,0.08155512220518964,0.04936921297324392,200,9.0,11.0,12.0,10.69
|
||||
legacy,finetuned,fresh-en-1,7,distance,200,0.4250159580260515,0.6076025643851608,0.7801178828347474,0.6061375473148655,200,0.08262168765068054,0.11914055794477463,0.20431101322174072,0.13078377526253462,200,5.2532538360542995,9.268233176060777,11.604547709811524,9.003903895996295,200,0.3283283647533937,0.5792645735037986,0.7252842318632202,0.5627439934997684,200,0.025251580787514525,0.04842180772984351,0.13638253532089065,0.06236009718236362,200,0.13567035273320738,0.20068831536270737,0.36299806609419144,0.2244645036701857,200,0.4637158553418997,0.5567978229807711,0.6511632351882545,0.5612567104606484,200,0.7458321693227228,0.7998711315239058,0.8553694485835988,0.8005046491180954,200,0.04312006034956634,0.06198862334697151,0.08536531987731405,0.06371953103155428,200,10.0,11.0,12.0,11.11
|
||||
legacy,finetuned,fresh-en-1,7,divergence,200,0.021563984626754976,0.041557265640221885,0.06729816531278628,0.04387818223659977,200,0.006826376309618354,0.01419452903792262,0.04174306392669678,0.019357659835368395,200,1.779192172671645,4.491042991854094,7.403387474552538,4.650809744349621,200,0.11119951079197782,0.2806901869908809,0.46271171715953363,0.2906756090218513,200,0.07741170064968506,0.17084342735055683,0.5328692940028164,0.23596103061262522,200,0.21226010808326046,0.3764283936365295,0.7410543292601097,0.4215556411394126,200,0.6384988102238026,0.8009278582260746,0.9323512579749245,0.7908747798412097,200,0.8907036042662825,0.9467876486584397,0.9790196379378455,0.9393647250149049,200,0.003639098158760709,0.010024509772087818,0.02216575277323116,0.011967058993235908,200,3.0,6.5,9.0,6.34
|
||||
legacy,finetuned,fresh-en-1,7,QKV,200,0.8682734288740903,1.0410724437097088,1.225852601416409,1.042333548606257,200,0.17561951130628586,0.21669606119394302,0.25753333866596223,0.21899689450860024,200,8.020542677842451,9.693252892662183,11.42880760078594,9.73077282953517,200,0.5012839173651532,0.6058283057913865,0.7143004750491212,0.6081733018459481,200,0.026664767769792392,0.04337551204817916,0.06632525403744392,0.04481765335664929,200,0.16745515775461686,0.20974062481912537,0.2776122001610138,0.2136284362990277,200,0.4599755658905811,0.5211612954339133,0.5602057576516388,0.5168763636680334,200,0.7359952985136846,0.7828767586609381,0.8181431166997297,0.7766373939491402,200,0.0406718450417487,0.0625043274426465,0.0860921764790434,0.06456192872557105,200,11.0,11.0,12.0,11.45
|
||||
legacy,finetuned,fresh-en-1,11,distance,200,0.3514061763882637,0.542330960277468,0.7291810931637883,0.5439334043615963,200,0.058625224977731705,0.09038844704627991,0.14859139025211332,0.1016827574186027,200,7.727504401455993,11.202174293758961,12.869242259069736,10.72097360271387,200,0.48296902509099954,0.7001358933599351,0.8043276411918585,0.6700608501696169,200,0.01621846735907664,0.02855349582761768,0.07136990810370429,0.03953769274941837,200,0.1272333890377786,0.16959872072082727,0.2759523912695496,0.18822883980643243,200,0.4257884219708893,0.48525904734082814,0.5824456007903207,0.4975578217182493,200,0.6902323315484893,0.7382498057851486,0.7991167988498163,0.7432049387712328,200,0.07390229492300462,0.09928203564131532,0.127450456669457,0.09969671588683114,200,12.0,12.0,13.0,12.36
|
||||
legacy,finetuned,fresh-en-1,11,divergence,200,0.012134489442723863,0.028195008846523706,0.053060469534830186,0.03131010133596646,200,0.0034369911300018434,0.008170092944055796,0.02207970935851335,0.01276566193002509,200,2.746220975521328,5.682244278588376,8.404739753999651,5.675554271473523,200,0.171638810970083,0.3551402674117735,0.5252962346249782,0.3547221419670952,200,0.06024595571203891,0.12105264980696771,0.3218243092323651,0.17055353927268999,200,0.20156114515564041,0.32350790500009574,0.5779746332150253,0.3603791282898823,200,0.6016095378392705,0.7165857080977105,0.8515295856905312,0.7217327996699807,200,0.8366562165506586,0.895347639746934,0.9524406006840342,0.8971091917053701,200,0.012345658896355107,0.02863307798196995,0.051752482475330056,0.030655109921468356,200,6.0,9.0,10.099999999999994,8.22
|
||||
legacy,finetuned,fresh-en-1,11,QKV,200,0.7346147155854851,0.92985531501472,1.1010335868224501,0.9208838701900095,200,0.09537210017442703,0.1451835259795189,0.18569873124361036,0.14433349043130875,200,10.111163714948324,12.20598833917959,13.509026101443906,11.964508036685697,200,0.6319477321842702,0.7628742711987244,0.8443141313402441,0.747781752292856,200,0.012292886646757699,0.020722195605964874,0.03882729278237194,0.023649286847107168,200,0.11187504691670837,0.15360727235556318,0.21284217949062365,0.15779329518879817,200,0.40094837093945856,0.44213933690458995,0.5113348119735558,0.45272263111998723,200,0.6733587369853662,0.7151631258197888,0.7540929722219253,0.7145255507076195,200,0.0818132698157194,0.11208900388226871,0.13568379319059687,0.11040720849881694,200,12.0,13.0,13.0,12.77
|
||||
legacy,finetuned,fresh-en-1,15,distance,200,0.4431339012691751,0.5756718516349792,0.7638036026619375,0.5836687245737994,200,0.06798388287425042,0.10282768309116364,0.15971038043498992,0.10994419045746326,200,7.579092934533321,10.07317387335631,11.848884719185758,9.890335718064309,200,0.47369330840833257,0.6295733670847694,0.7405552949491099,0.6181459823790193,200,0.023355907688915004,0.039225152011404696,0.07407181640100868,0.045299683448548934,200,0.1306535058215912,0.18221960729942283,0.2522290996978114,0.1892517662063465,200,0.45334409982166973,0.5268080663216634,0.6351295197710743,0.5372812603477048,200,0.7366014952400916,0.7875328630237065,0.8493314192178647,0.7881508507004463,200,0.046858098404466995,0.07088395488848198,0.09408030502051219,0.071111563342882,200,10.0,11.0,12.0,11.34
|
||||
legacy,finetuned,fresh-en-1,15,divergence,200,0.018031556456844556,0.03393463677980435,0.06049842149072901,0.037189093807259044,200,0.004621811024844646,0.010573532897979021,0.025507439672946927,0.013622189709567465,200,3.2873980168987904,5.159675299439181,7.746837643055929,5.3593126325730145,200,0.2054623760561744,0.3224797062149488,0.48417735269099554,0.3349570395358134,200,0.07102501432246136,0.1400674336625099,0.2578328555923481,0.15987293844024908,200,0.19998348233501217,0.3239273440207344,0.4964582860228307,0.34522699110985355,200,0.6267433581872277,0.7495421236037567,0.8901255776590328,0.7557961140524845,200,0.8799155832976748,0.9323523692082748,0.9740125738220705,0.9289802796595574,200,0.004777279122227702,0.01424462107683981,0.02792917941947638,0.015600117828741036,200,5.0,7.0,9.0,7.075
|
||||
legacy,finetuned,fresh-en-1,15,QKV,200,0.6216587599832565,0.7790311630815268,0.9221897925250232,0.7800927976498496,200,0.10176522359251976,0.12974239140748978,0.18153780251741408,0.13647626839578153,200,8.481116257880522,10.584310383712069,11.862960297185705,10.353245046228245,200,0.5300697661175326,0.6615193989820043,0.7414350185741065,0.6470778153892653,200,0.02324905785562129,0.03411144835663274,0.05910385662324606,0.038635150405044213,200,0.1317358299935914,0.1653660554540167,0.2399325535431688,0.1772794046226189,200,0.46053087955327254,0.5137588220752967,0.5867094794302404,0.5193171816987838,200,0.7319026585683902,0.7711308087499479,0.8112058090265407,0.7710650803994116,200,0.057212658838445465,0.07402865347765476,0.09486928156809636,0.07534460055998374,200,11.0,12.0,12.0,11.66
|
||||
legacy,finetuned,fresh-en-1,19,distance,200,0.30359728501643984,0.3841310947900638,0.45445832181721924,0.3838613071816508,200,0.06915641278028488,0.10723701491951942,0.15891841500997542,0.11214089913293719,200,5.329289373125406,7.209439566058007,8.909934698328625,7.110470137820557,200,0.3330805858203379,0.45058997287862546,0.5568709186455391,0.4444043836137848,200,0.053052179911152464,0.08128757270221895,0.133485097255449,0.09091237682385693,200,0.2068742797821117,0.2802964147877992,0.3728776807842335,0.28680777439955824,200,0.5697570297093443,0.6354621824938786,0.6993319054178007,0.6387354034183136,200,0.7892932153386175,0.8345280510070894,0.8729162404898674,0.8323474929524696,200,0.04605601814294414,0.05955638925002234,0.07596589366513551,0.06037074050850638,200,10.0,11.0,12.0,10.675
|
||||
legacy,finetuned,fresh-en-1,19,divergence,200,0.011492073349972999,0.020765161152667133,0.036630899116971705,0.023269616777525925,200,0.004782682890072465,0.011499778367578983,0.025255828909575933,0.014185365319717676,200,1.7335245882659873,2.685266609897477,4.2073425709122,2.864806890144664,200,0.10834528676662421,0.1678291631185923,0.2629589106820125,0.1790504306340415,200,0.18685970464079338,0.3305884444073367,0.5486526426849143,0.3468298288040292,200,0.37437480052847527,0.5608419064496872,0.7410464842776379,0.5593092795833611,200,0.8264031988882524,0.8953412520456905,0.9522508565068392,0.8931969726179488,200,0.945766036170025,0.97208689139446,0.9859356827155591,0.9695402131033469,200,0.003264189194508423,0.006650723066171652,0.013022969171699185,0.007446637951642807,200,3.0,5.0,6.0,4.555
|
||||
legacy,finetuned,fresh-en-1,19,QKV,200,0.3178175230044872,0.39397372817620635,0.5819798086769878,0.41805479087517594,200,0.04896724633872509,0.06418542936444283,0.08889304175972938,0.06665869036689401,200,11.005658205322389,12.062499955882974,13.40087260250099,12.078634394024093,200,0.6878536378326493,0.7539062472426858,0.8375545376563118,0.7549146496265058,200,0.012930146883040505,0.021761667381711095,0.03025317656495022,0.02221415442958647,200,0.13141033134861266,0.15936998920798262,0.19502528320287738,0.1603384746496464,200,0.3955336492476569,0.44526297358648265,0.4830717970557268,0.4453395444651131,200,0.6640090531022272,0.6969125214021773,0.7313196716884363,0.6975694646569471,200,0.09980629441264305,0.11816671337009832,0.1366261004705777,0.11816684915662776,200,12.0,13.0,13.0,12.955
|
||||
legacy,finetuned,fresh-en-1,23,distance,200,0.06444347239448689,0.22096599597716704,0.2720134000875987,0.1992423667586263,200,0.01873902715742588,0.04839377664029598,0.07468507438898087,0.04954179061111063,200,6.120997982896491,8.00320628379638,9.478901038209054,7.853321541068939,200,0.3825623739310307,0.5002003927372738,0.5924313148880659,0.4908325963168087,200,0.04586512367651623,0.06661325309902963,0.10759728010525364,0.07480297225971057,200,0.18667433017537752,0.23896982916353263,0.32647547293164836,0.25285415187246457,200,0.5476940746740195,0.6084407323573806,0.686575065626574,0.613616862480538,200,0.7760697821454198,0.8173559250007352,0.867769021214119,0.8194521883582482,200,0.04189657579981661,0.060328046925006984,0.08017943472755222,0.060437506743841254,200,10.0,11.0,12.0,10.825
|
||||
legacy,finetuned,fresh-en-1,23,divergence,200,0.0006712480984187864,0.0059191336069943645,0.010482519297943325,0.005975274568210436,200,0.00035116701910737903,0.002341957646422088,0.005577936675399541,0.00300833534605772,200,2.0555052924923305,3.3881103787301385,4.774292480191002,3.3726307430151565,200,0.12846908078077066,0.21175689867063366,0.2983932800119376,0.21078942143844728,200,0.1567524488839049,0.24816110070466546,0.45226844472497335,0.2843702530069091,200,0.3133903328849513,0.45682600335647594,0.666507841529774,0.48586328521584465,200,0.7995119104797707,0.8691324042017097,0.9380604135341445,0.8675093259482283,200,0.9318812584257284,0.9615664470758014,0.9824778538040244,0.9591612665679201,200,0.003531815450491743,0.008152683939735183,0.015220429235680271,0.008961995305834651,200,3.0,5.0,7.0,5.325
|
||||
legacy,finetuned,fresh-en-1,23,QKV,200,0.31983131847810004,0.395911613246426,0.4776430093217641,0.3983542170497822,200,0.06027193889021874,0.0892457403242588,0.136277437210083,0.09616722498089075,200,6.492465642251302,8.945897038203409,11.470586608349011,8.914386279338798,200,0.4057791026407064,0.559118564887713,0.7169116630218132,0.5571491424586749,200,0.02632481178519136,0.05256886202470028,0.09763049836154851,0.05951132808958844,200,0.15230965239354197,0.23146495672770012,0.3369538332933344,0.24074393978290387,200,0.47484775926030026,0.5415153237748254,0.6063780647184347,0.5412345283527542,200,0.7143886438541563,0.7726741472036641,0.8125849252470626,0.7673393923248937,200,0.05949098296046418,0.07837530431179185,0.0991402225916997,0.07982918066767032,200,11.0,12.0,12.099999999999994,11.8
|
||||
legacy,finetuned,fresh-en-2,3,distance,200,0.6166574957314879,0.8141454530414194,1.046656387206167,0.8169526239996776,200,0.13709777295589448,0.19382096827030182,0.33378102779388424,0.21289045501500367,200,4.982542948104248,9.030866412957597,10.92065836053741,8.49228037649921,200,0.3114089342565155,0.5644291508098498,0.6825411475335881,0.5307675235312006,200,0.03100858131362607,0.051448779285660676,0.14741444706351578,0.07002267046553157,200,0.17320621976225853,0.24006247400971648,0.4094818253348295,0.26370844140254374,200,0.48133558605488397,0.5476533549128088,0.6494499132870188,0.55697977871006,200,0.736646042676633,0.7791843125450724,0.829991429813349,0.7806353881634215,200,0.055689437451319046,0.07596397113377806,0.09902463230524486,0.07713462823006663,200,11.0,12.0,12.0,11.61
|
||||
legacy,finetuned,fresh-en-2,3,divergence,200,0.04502816855965648,0.07704945087607484,0.14902031926321796,0.08743148404967542,200,0.018795954063534737,0.03756812959909439,0.1114112928509712,0.05124732472235337,200,1.4122605002136803,3.2274726791186072,5.464600850126683,3.4045436887861977,200,0.08826628126335502,0.20171704244491295,0.34153755313291767,0.21278398054913736,200,0.12852912820902104,0.26382942297806466,0.6886271723063512,0.3308952130681858,200,0.31806441530841084,0.5225037696549911,0.8387357137170471,0.5457412441982007,200,0.7090788386443118,0.814855343263825,0.9393551526115795,0.8182802107959478,200,0.890405847997682,0.9402144274649751,0.9769223603880678,0.9366305689821631,200,0.004882352537920996,0.0137162397314246,0.02917251747840779,0.015638398447687926,200,3.0,7.0,9.0,6.285
|
||||
legacy,finetuned,fresh-en-2,3,QKV,200,1.1697979205986484,1.5072831648867577,2.0847270980942993,1.5538976829811872,200,0.2834134817123413,0.3396112769842148,0.4232092410326004,0.34764659851789476,200,7.280532879977844,9.518535659828046,11.079872451708715,9.32736895209475,200,0.4550333049986153,0.5949084787392529,0.6924920282317947,0.5829605595059218,200,0.029603996427938362,0.045395470967699886,0.07984292387440817,0.05072903510021248,200,0.18688191508997273,0.22394092979316133,0.28670410767195564,0.22930177057087384,200,0.46613775258199075,0.520359620320533,0.5953519228083024,0.5303342349854332,200,0.7303521670986313,0.7694774350693634,0.8573788484421119,0.7837672507609723,200,0.022835400728472934,0.06974563093079028,0.09559551107458694,0.06296759458803644,200,10.0,12.0,12.0,11.175
|
||||
legacy,finetuned,fresh-en-2,7,distance,200,0.4391549618449062,0.6387817830545828,0.8333876888267696,0.6372168826387496,200,0.08283431753516197,0.11212592199444771,0.1855333551764488,0.1231668438948691,200,6.948340926300195,10.099296834120095,11.909726463318341,9.687771542117154,200,0.43427130789376217,0.6312060521325059,0.7443579039573963,0.6054857213823222,200,0.02289600027804474,0.038951249823274844,0.08684728334352747,0.04951446429863594,200,0.13576294875165337,0.1786387421446336,0.3032472138323736,0.1997535682227801,200,0.4534867752687284,0.5334513769641793,0.6393484466164565,0.5422427598167721,200,0.7296622013060342,0.7916985947089823,0.84019370975143,0.7866254797639048,200,0.04968593796139011,0.06882213412470292,0.0944943933590673,0.07133392294202733,200,10.0,11.0,12.0,11.345
|
||||
legacy,finetuned,fresh-en-2,7,divergence,200,0.023945744896445832,0.04142454275097407,0.07359880574558701,0.04447252683692426,200,0.006861622259020806,0.012572225648909807,0.034422631934285165,0.016770282186334952,200,2.2363512803628267,5.368528576241769,7.693365622841078,5.184376281827735,200,0.13977195502267667,0.3355330360151106,0.4808353514275674,0.3240235176142334,200,0.07198812358112495,0.13202228908742547,0.41033219819249656,0.1866748876485769,200,0.21581255060066287,0.3189627003680643,0.6436123471329706,0.3707854254607011,200,0.637732129734201,0.7668593073810112,0.908997036810312,0.7678523017886164,200,0.8821290150676558,0.9350149002688278,0.974813041133116,0.9284730894925537,200,0.005062752892619349,0.013184901476386204,0.029658081295789993,0.016335806144513524,200,4.0,7.0,9.0,6.96
|
||||
legacy,finetuned,fresh-en-2,7,QKV,200,0.8433302800171077,1.0681711519137025,1.2643321176059543,1.0660207593708764,200,0.1535348430275917,0.190449520945549,0.2284102886915207,0.18982520438730716,200,8.801359123897486,10.740529314658463,12.205266874442943,10.609216862070758,200,0.5500849452435929,0.671283082166154,0.7628291796526839,0.6630760538794224,200,0.02072729887336766,0.03264563468448228,0.05452848206461737,0.035368380253222455,200,0.13293412494659212,0.1778774187245214,0.24612723791624602,0.1829742183374658,200,0.44523613678398827,0.49727005808493296,0.5487234423040579,0.4971848047301528,200,0.7182128437372867,0.7625714098937831,0.7923423204404847,0.7594788670627398,200,0.0532899962789328,0.0791534417561282,0.09955000290167769,0.07631093099550623,200,11.0,12.0,12.0,11.765
|
||||
legacy,finetuned,fresh-en-2,11,distance,200,0.35763208018615844,0.49708873638883233,0.6723948168568313,0.5064087439340074,200,0.056728119403123854,0.08283836022019386,0.14665604382753372,0.096560083553195,200,7.719858401625378,11.276525300195686,12.86471203935746,10.765599675460285,200,0.4824911501015861,0.7047828312622304,0.8040445024598413,0.6728499797162678,200,0.01624756352996027,0.027925362068720073,0.07150686444960697,0.03961797276911498,200,0.1278592709335898,0.16470345232423828,0.27718708473098563,0.18986204044619115,200,0.42346009527464534,0.48190886950750356,0.569355400527824,0.49490336376100125,200,0.6916757101017355,0.733491160006652,0.7917297601487987,0.7369016684049323,200,0.07372740638452209,0.10312483739580822,0.12647049413218373,0.10213516550747363,200,12.0,13.0,13.0,12.485
|
||||
legacy,finetuned,fresh-en-2,11,divergence,200,0.011597647679082002,0.023443774974111875,0.04386105069352197,0.02766854288174386,200,0.0032180840149521827,0.006862207315862179,0.021508429571986194,0.012111961698392406,200,2.6005598301221005,5.926545614421555,8.21960544632639,5.648492142230533,200,0.16253498938263128,0.3704091009013472,0.5137253403953994,0.3530307588894083,200,0.063104395731907,0.11331454851987634,0.34399007025239725,0.17508405145357048,200,0.20995553458302885,0.307906281777783,0.5900769685928914,0.3659681628663804,200,0.5994904977095188,0.7106710308116938,0.8491509370494156,0.7210517712271026,200,0.836483008512779,0.8918830393410093,0.9539174793394283,0.8919530646575864,200,0.013009166817958691,0.030474527099692443,0.05176064656167729,0.03202835756425997,200,6.0,9.0,10.0,8.365
|
||||
legacy,finetuned,fresh-en-2,11,QKV,200,0.736142901610583,0.9336117128841579,1.1446446764282883,0.9304587678611278,200,0.09372500777244568,0.14641409367322922,0.1857554942369461,0.14034467179328203,200,10.76302735425837,12.153705664100148,13.469098625678859,12.147145315478305,200,0.6726892096411481,0.7596066040062592,0.8418186641049287,0.7591965822173941,200,0.012526953492392018,0.02109806533312096,0.032438402809008116,0.02182219535097777,200,0.11471106604540908,0.14760091065668762,0.19605649610060777,0.1514852327619154,200,0.39562530317065847,0.4467852527934515,0.48553495734667457,0.4438113372250617,200,0.666906037502208,0.7136041276068257,0.7485929090500137,0.7114809481466493,200,0.07796174407972957,0.10717307683499433,0.13841882874122138,0.1071973882544781,200,12.0,13.0,13.0,12.66
|
||||
legacy,finetuned,fresh-en-2,15,distance,200,0.3893857566639781,0.5123732516076416,0.7095642678439618,0.5354797474609222,200,0.06399983912706375,0.09388342499732971,0.14143142104148865,0.10249617902562022,200,7.129906799688097,10.164287733917554,12.01837550144582,9.882814767838642,200,0.44561917498050607,0.6352679833698471,0.7511484688403638,0.6176759229899151,200,0.022086641311024002,0.038275932913003075,0.08293797465891208,0.04646750997935126,200,0.13696288948532184,0.17905109562796986,0.2601690238446028,0.19247619430895987,200,0.4519653588342448,0.5243720457478557,0.6335675442905345,0.5353472697908738,200,0.7278009681349972,0.7802456526919037,0.845765313675409,0.7829590703707638,200,0.048153341607249545,0.07328563494787355,0.10030384778828735,0.07305073896782927,200,10.0,12.0,13.0,11.45
|
||||
legacy,finetuned,fresh-en-2,15,divergence,200,0.014699611778996768,0.026913542755210074,0.05499471348048245,0.0319455651274609,200,0.004096038825809956,0.008814440574496984,0.02000285778194666,0.012032550415024162,200,2.904946610143894,5.2803576219903725,7.5707874248343945,5.292178953311995,200,0.18155916313399337,0.3300223513743983,0.47317421405214966,0.3307611845819997,200,0.07423023350798949,0.13533986768438255,0.3005365857273964,0.1669436008305771,200,0.21851514455284676,0.31792942151173376,0.523022073633723,0.35327379344293164,200,0.6342844471280604,0.7505610974766138,0.8830226151319276,0.7570477255245623,200,0.8786809205060654,0.9281424643548191,0.9742645656623652,0.9253445560263114,200,0.004420721859834475,0.01413050355874268,0.029537461358225256,0.016550421436246573,200,5.0,7.0,9.0,7.115
|
||||
legacy,finetuned,fresh-en-2,15,QKV,200,0.6568851702846586,0.7758694582153112,0.9126032256754115,0.7828560562100029,200,0.10984417721629143,0.12912584096193314,0.15805196464061738,0.13154210481792689,200,9.118029914713048,10.607539604479596,12.120508840837457,10.619193688126515,200,0.5698768696695655,0.6629712252799748,0.7575318025523411,0.6636996055079072,200,0.021338475965443558,0.03389086359851083,0.05031771357004958,0.03497513107658489,200,0.1361230114454724,0.16826137788591178,0.20861445347330088,0.17013575481957396,200,0.4514995730490878,0.509911870065376,0.5658428962964014,0.5085220883580714,200,0.7174495372657034,0.7606257579157769,0.8082136598210916,0.7623349409711326,200,0.054147033872225483,0.07275861011720035,0.10659562289413116,0.07665558381165638,200,11.0,12.0,13.0,11.74
|
||||
legacy,finetuned,fresh-en-2,19,distance,200,0.27011707895435394,0.334297260036692,0.41097497056471183,0.3398463001297205,200,0.06040669716894627,0.09904859960079193,0.16292995661497114,0.10570702897384762,200,4.539917428338253,6.862609499211548,9.712328464731547,7.040765390543772,200,0.2837448392711408,0.4289130937007217,0.6070205290457217,0.44004783690898575,200,0.043159382850639104,0.0887662538830303,0.1682902381776348,0.0976352622862768,200,0.20197415633539761,0.3002050649627519,0.42340836824683403,0.30444677791793395,200,0.5420347758068401,0.6217637451936775,0.7217581452475004,0.6227387699260132,200,0.7717537405184386,0.8181551526749198,0.8669905776683372,0.8185388401327242,200,0.041533698656560473,0.05993681128991736,0.08055259968323812,0.06054358897549889,200,10.0,11.0,12.0,10.86
|
||||
legacy,finetuned,fresh-en-2,19,divergence,200,0.008638520424983654,0.015480506729545596,0.03461979700932716,0.019133097133369005,200,0.0036490364233031872,0.009810625109821558,0.026548244431614867,0.01281754411465954,200,1.496321193343105,2.3369397680838357,4.31936236899563,2.693785087065936,200,0.09352007458394407,0.14605873550523973,0.2699601480622269,0.168361567941621,200,0.18028629368682475,0.389771591348276,0.6462088496641248,0.39595930236909554,200,0.39209000699609897,0.6277952925432667,0.8083931192523874,0.6048251062559737,200,0.7914638849229961,0.8955988156385131,0.9618245926208371,0.8832364882152802,200,0.9305642012417529,0.9668313418461765,0.9887932231831189,0.961739394530075,200,0.0024327094462009108,0.0074659578371432805,0.015965611626446042,0.008512918839292824,200,2.0,5.0,7.0,4.725
|
||||
legacy,finetuned,fresh-en-2,19,QKV,200,0.31858419461641463,0.39767567720264196,0.5434754455927759,0.4136390372226015,200,0.04179297387599945,0.0557518545538187,0.07704525887966156,0.05755256662145257,200,11.812924447870081,12.527160751416812,13.38900096540432,12.590071920519767,200,0.7383077779918801,0.7829475469635507,0.83681256033777,0.7868794950324854,200,0.013000726831702808,0.018481663470576982,0.02362991259063402,0.0183062309424758,200,0.11922272274282682,0.13792230782778594,0.16778187158955019,0.1399066367381323,200,0.3925706330574167,0.4293830035500479,0.4582359382391823,0.42584043833556884,200,0.6628204333349178,0.6943057305776232,0.7224460178268699,0.6929886382705683,200,0.08690999869979361,0.10432073114600417,0.12965843815991365,0.10564327265766835,200,12.0,13.0,13.0,12.63
|
||||
legacy,finetuned,fresh-en-2,23,distance,200,0.09241623702691867,0.24586023623123765,0.3123920375481248,0.22616958421123853,200,0.023460312187671663,0.058367643505334854,0.09541613683104515,0.06151874459348619,200,5.941321890596421,7.476787393113408,9.391444648106306,7.56401689901741,200,0.3713326181622763,0.467299212069588,0.5869652905066441,0.47275105618858815,200,0.04691207577561705,0.07599748085269029,0.11286986907888531,0.08011190028088624,200,0.19967226669227886,0.2730188588522259,0.33472058204083893,0.2711536933948573,200,0.5425140841515816,0.612046911788473,0.6859598889195546,0.6150029692975287,200,0.7735712522518483,0.8169547066737982,0.8647069212841386,0.8194335202592501,200,0.04196045164849739,0.06334539872572414,0.07989075592535506,0.06208206277211643,200,10.0,11.0,12.0,10.855
|
||||
legacy,finetuned,fresh-en-2,23,divergence,200,0.0013217035008324274,0.007763206415006607,0.014152832672311886,0.007867511124396742,200,0.0005504186789039523,0.0034068136010318995,0.009104298707097768,0.004508001925896679,200,1.8915807103814255,2.738969252545295,4.404116106456405,3.0327001149914974,200,0.1182237943988391,0.17118557828408093,0.2752572566535253,0.1895437571869686,200,0.17553110904271588,0.3227897000433947,0.4972356330987119,0.3269868071894562,200,0.35901490523557744,0.5496728450630675,0.7140317130501095,0.538658549520002,200,0.7991440867193729,0.8809147603837009,0.9371638763255814,0.8732507261389614,200,0.9354105432093297,0.9648151833998011,0.9841692147137114,0.9612391538317396,200,0.0031220422924096943,0.008264237335264178,0.015873777194686575,0.008841785175243118,200,3.0,5.0,7.0,5.12
|
||||
legacy,finetuned,fresh-en-2,23,QKV,200,0.37426581121981145,0.4715594795998186,0.6063961615785955,0.4780702057306189,200,0.06916943565011024,0.12009594589471817,0.2020330145955085,0.13168115781620146,200,5.594412070199879,8.21339846029483,11.058062158770133,8.329840699979016,200,0.34965075438749244,0.5133374037684268,0.6911288849231333,0.5206150437486885,200,0.029793905703938403,0.06320295880182317,0.12400000721729482,0.07009955360678863,200,0.1630356474359573,0.27138810373610767,0.3749615396133103,0.2682904829694202,200,0.48228680096591736,0.5572967873066916,0.6213333976386105,0.5542055268930944,200,0.7335977290490286,0.7773862556258981,0.8118857564233355,0.7754777225128362,200,0.06271669126897401,0.07989018631939213,0.09926565325655211,0.07968692089667818,200,11.0,12.0,12.0,11.78
|
||||
gsm8k,base,gsm8k-test-0000,3,distance,200,0.516678747953847,0.6785132046788931,0.9227323985658586,0.7069272202381398,200,0.11689556390047073,0.1856858804821968,0.2638332039117813,0.18942656625062226,200,4.900965351126817,8.285708287565464,10.784856416759018,8.097104910490343,200,0.30631033444542605,0.5178567679728415,0.6740535260474386,0.5060690569056464,200,0.032237468525962584,0.0620690593262882,0.15103733218738238,0.07649023943692765,200,0.1723901508212039,0.25789042217611546,0.4149418111514965,0.27454261463085705,200,0.4864112992151362,0.5683661853344546,0.664887659012562,0.5721560712432737,200,0.741650839641703,0.7973264357319474,0.843653561382101,0.7938484572379729,200,0.05120246890534877,0.0724068638672004,0.08921496156208622,0.07061951321965233,200,10.0,11.0,12.0,11.36
|
||||
gsm8k,base,gsm8k-test-0000,3,divergence,200,0.03579101385575996,0.06000158645110787,0.1021065945184091,0.06755024925041653,200,0.013664584327489139,0.03447924740612507,0.06960821002721786,0.039649535652715714,200,1.4171164219162298,2.8706144486813963,5.4723671102671005,3.2008796048505075,200,0.08856977636976436,0.17941340304258727,0.3420229443916938,0.20005497530315672,200,0.12825562362684972,0.3049147812565699,0.6860450488989607,0.3508201648172236,200,0.3218264407082991,0.5553132711714687,0.8370821716645966,0.564870031973424,200,0.7155803233562005,0.8477253813435732,0.941564932188296,0.832949561558885,200,0.8968765024235629,0.9509672732817143,0.9823091496147265,0.9450191859096051,200,0.003730255433633086,0.011173354475231372,0.020983618120287064,0.012220953581332919,200,3.0,6.0,9.0,5.89
|
||||
gsm8k,base,gsm8k-test-0000,3,QKV,200,1.1043512099800865,1.377722925040871,1.8542156467679887,1.4185510164734296,200,0.25767137110233307,0.31969912350177765,0.42113888561725615,0.33295733653008935,200,7.576222548699666,8.616110492309508,9.97339342573057,8.719631499590083,200,0.4735139092937291,0.5385069057693442,0.6233370891081607,0.5449769687243802,200,0.04028468060425209,0.05713242700690438,0.0741247208713094,0.057246588736246266,200,0.20110773229950624,0.23589166751041668,0.27564833742069655,0.23651886334367178,200,0.49928666645738584,0.5646221711159007,0.6112575958407938,0.561252379100387,200,0.7672626038146171,0.8052329696496674,0.8488569691244385,0.8090295903652063,200,0.020306417189204545,0.049187584132752704,0.0791536673105367,0.04952288434371465,200,10.0,11.0,12.0,10.725
|
||||
gsm8k,base,gsm8k-test-0000,7,distance,200,0.4117498789448291,0.6164979645982385,0.8880705389659852,0.6327356081767357,200,0.0833731509745121,0.11846722289919853,0.17724382579326628,0.12497165573760867,200,6.36848811965427,9.750391920260448,11.720804065623692,9.416019601448479,200,0.39803050747839186,0.609399495016278,0.7325502541014808,0.5885012250905299,200,0.024339661390520333,0.04273065655367837,0.10082468399621441,0.05390193852307066,200,0.14375563724897047,0.1854282892809463,0.3194614852185542,0.20999308192011518,200,0.4551932305227692,0.5386465439639994,0.6596410212807842,0.5512419853393127,200,0.7327922741887382,0.7878067943387836,0.8457413865610429,0.7873106411279062,200,0.04856124321156906,0.07237962016065265,0.09737545364700209,0.07304130542854702,200,10.0,12.0,12.0,11.42
|
||||
gsm8k,base,gsm8k-test-0000,7,divergence,200,0.02107444014563953,0.039906981437525246,0.07286599415310774,0.044948169894137496,200,0.0069511010311543945,0.014034503605216742,0.03141547404229641,0.016905079503776507,200,2.089724860636738,4.921035564437384,7.176928358847457,4.826291484870594,200,0.1306078037897961,0.3075647227773365,0.44855802242796605,0.3016432178044121,200,0.0819631326880205,0.15009494733040646,0.44380405355747254,0.20784857672393328,200,0.2373787634301123,0.34152455823277583,0.6826955340051502,0.39434536339284604,200,0.6471998379159037,0.7811402746458347,0.921029350795012,0.7843890833584346,200,0.8802248890377934,0.9358834589516543,0.9780062121500073,0.9311080185329385,200,0.0047477501852855,0.014402345058502205,0.030017191855513457,0.016308740472022666,200,4.0,7.0,9.0,6.81
|
||||
gsm8k,base,gsm8k-test-0000,7,QKV,200,0.9409899344667793,1.1430985634215176,1.5185112942010164,1.1927996050729417,200,0.1791912630200386,0.208652563393116,0.24431720972061158,0.2113325524330139,200,8.950190881051013,10.638217985447586,12.298730867620977,10.575375325693958,200,0.5593869300656883,0.6648886240904741,0.768670679226311,0.6609609578558724,200,0.020063160764548078,0.03360082317492799,0.05251203676562804,0.035779473966831815,200,0.1317128338485172,0.1772522463466704,0.24794192528913822,0.18334686202810294,200,0.441543648104141,0.5030934493702353,0.5490473815410768,0.499254577022779,200,0.7108707636215915,0.7730506364962701,0.8036931229857412,0.7620532260240731,200,0.051251696148189245,0.07277709063038461,0.10511838566154169,0.07616503586527297,200,11.0,12.0,13.0,11.675
|
||||
gsm8k,base,gsm8k-test-0000,11,distance,200,0.38847396508790555,0.5706635671667755,0.8512633473612368,0.5953381672830438,200,0.06197943761944771,0.09531774371862411,0.1824097007513046,0.11171729093417525,200,7.68935274838223,11.134665544472254,12.566229155320446,10.463790789092045,200,0.4805845467738894,0.6959165965295159,0.7853893222075279,0.6539869243182528,200,0.018216935118672437,0.02913038570205819,0.07205542031475673,0.04388963114822578,200,0.1243024342789327,0.15895042934850867,0.28363682774864607,0.1903440050385717,200,0.4346746415983759,0.49679950011676854,0.5938587431026503,0.5054174057693208,200,0.7062153674006837,0.7498650347687363,0.8080917506443216,0.7552479849767356,200,0.06625764134129486,0.09040325839500477,0.1129801305668519,0.08937630458552202,200,11.0,12.0,13.0,12.035
|
||||
gsm8k,base,gsm8k-test-0000,11,divergence,200,0.014188714587953656,0.03193297590223665,0.0718331566074994,0.03988979969090337,200,0.0038414865499362353,0.009085487108677626,0.03327351585030555,0.016614258942427115,200,2.3634518310105634,6.196657437617913,8.323803992200215,5.767340968018033,200,0.14771573943816022,0.38729108985111954,0.5202377495125134,0.36045881050112705,200,0.061480489255392165,0.10547000452527486,0.38473933770798574,0.17790096162608818,200,0.19321194561973487,0.2898987359096037,0.6281649659346475,0.3543655358951034,200,0.594367567251649,0.7123465903028078,0.8732249833022406,0.7249865948953501,200,0.8522053706310317,0.9060622835721595,0.961126892915543,0.9051355375625226,200,0.008278468222249508,0.023250441250487894,0.039585447229103954,0.02451156173520465,200,5.0,8.0,10.0,7.83
|
||||
gsm8k,base,gsm8k-test-0000,11,QKV,200,0.7695024954155087,0.9812722781207412,1.23104319088161,0.9932484547933563,200,0.10207027941942215,0.15329372137784958,0.20105945318937302,0.1521888054162264,200,10.202278749769372,12.262954245761676,13.357684759767515,11.972443459611007,200,0.6376424218605857,0.7664346403601048,0.8348552974854697,0.748277716225688,200,0.01318749485330839,0.020316224758140662,0.037885139197809614,0.023347086694890405,200,0.11246481136415583,0.14856250950737573,0.20514095529710816,0.15440848377227834,200,0.39488787155119665,0.4417614892255889,0.505258983635595,0.44687341623393295,200,0.6742765042541753,0.7153002287128911,0.7563830907631712,0.7168402192169355,200,0.07464166981057525,0.10049630226766897,0.12402531040144825,0.10013713132879339,200,12.0,13.0,13.0,12.525
|
||||
gsm8k,base,gsm8k-test-0000,15,distance,200,0.4274745984002948,0.5878968918696046,0.8400957179255782,0.6078716825312586,200,0.0707825779914856,0.10475524514913559,0.16399709582328795,0.11539929986000061,200,7.812126100024127,10.171356144598683,11.951061638606388,9.95699247974543,200,0.48825788125150793,0.6357097590374177,0.7469413524128993,0.6223120299840894,200,0.022588088449488397,0.03820299822914511,0.0698746007087678,0.044793972883039046,200,0.13322673266686508,0.17577810245709657,0.26288987505816747,0.19056921368724156,200,0.44918105297433963,0.5265387571725351,0.6112108022974331,0.5303695250037818,200,0.7221681889475137,0.7844177079497187,0.8373498309871539,0.7835646674635854,200,0.049820371822868616,0.07509844883634852,0.1016119552147572,0.07601610281503635,200,10.0,12.0,13.0,11.565
|
||||
gsm8k,base,gsm8k-test-0000,15,divergence,200,0.01831419549835118,0.03514782588899834,0.07342949221201707,0.04091369972163164,200,0.005010194005444646,0.01097367238253355,0.026895331591367717,0.015688442169921473,200,2.860149530698369,5.376359741314789,7.686802899815913,5.342996554831002,200,0.17875934566864807,0.3360224838321743,0.48042518123849454,0.3339372846769376,200,0.07209939046448671,0.13173300307565358,0.30627440062506145,0.16763727323177124,200,0.2023416568481539,0.30872510994961,0.5499105922897939,0.35299809346525807,200,0.6330608709190867,0.7532463168480725,0.868369510848064,0.7527512382554307,200,0.8727880948196338,0.9350875190838572,0.9686839139304252,0.9280192593165932,200,0.005891887677672996,0.015331589154774913,0.031062450414489157,0.017264340892382327,200,5.0,7.0,9.0,7.165
|
||||
gsm8k,base,gsm8k-test-0000,15,QKV,200,0.6804240548051894,0.796717185061425,1.02377725597471,0.829089741447242,200,0.09920224770903588,0.13420569151639938,0.18499674052000045,0.14118386790156365,200,8.977279756615207,10.948144282365785,12.732800351172026,10.882612649076815,200,0.5610799847884504,0.6842590176478616,0.7958000219482516,0.680163290567301,200,0.01710648197987278,0.030762332647103527,0.052152147566150546,0.03303973723836465,200,0.12431988341134276,0.16772126365263773,0.23190985122953234,0.17208086649942989,200,0.43166237859220685,0.49694566520407557,0.5601415477145087,0.4978998009893036,200,0.7110434521563684,0.756318833728655,0.797457015908113,0.7544563791015864,200,0.060634822202237884,0.08923290471148708,0.11457988855428096,0.08726182726806136,200,11.0,12.0,13.0,12.01
|
||||
gsm8k,base,gsm8k-test-0000,19,distance,200,0.2848799969884567,0.35066500899847597,0.4213353822473437,0.35235710674896836,200,0.06965448856353759,0.09752105921506882,0.13478369414806365,0.10067486859858037,200,5.645842432556155,7.515825633427342,9.364062730762933,7.500035564292251,200,0.3528651520347597,0.4697391020892089,0.5852539206726833,0.46875222276826567,200,0.0472440149407161,0.07525608674319667,0.12226406981382869,0.08135402559618429,200,0.21190509161655263,0.2816843206813584,0.3683174314945049,0.2854563252604616,200,0.5432484734207227,0.606994039961364,0.669942270155674,0.6047035404970391,200,0.772144612589219,0.8103066449684824,0.8477517576601235,0.8091631824825458,200,0.053389754552877956,0.06865802678058178,0.08805237612910685,0.07023603570327962,200,10.0,11.0,12.0,11.215
|
||||
gsm8k,base,gsm8k-test-0000,19,divergence,200,0.010469485508565413,0.016526168609289016,0.026711081705980178,0.017741218356901525,200,0.00485178530216217,0.009510356932878494,0.018167014420032497,0.010868591343169101,200,1.652054270950572,2.5544655239042378,4.05602787725606,2.754950651439141,200,0.10325339193441074,0.15965409524401486,0.25350174232850375,0.1721844157149463,200,0.19631655433959003,0.3509029148961372,0.5790049384488651,0.369018332346851,200,0.4017990133675195,0.5927135741983411,0.7704677028822696,0.5896953951997961,200,0.8058666107296056,0.8836714554937521,0.9347342450585858,0.8731654653401856,200,0.9373102997271794,0.9636588075671674,0.9789395847783625,0.9596614289421459,200,0.0047034757110214435,0.009606893270038488,0.017547911065917583,0.010550594347710766,200,3.0,5.0,7.0,5.065
|
||||
gsm8k,base,gsm8k-test-0000,19,QKV,200,0.35784564004279673,0.439322117716074,0.5295137697830796,0.43922689554747196,200,0.050774605572223665,0.06434449553489685,0.08748479336500167,0.06765936976298689,200,10.620075342739781,12.401541826036386,13.351109577579736,12.174861688429775,200,0.6637547089212363,0.7750963641272741,0.8344443485987335,0.760928855526861,200,0.013226852422162963,0.019344208347056642,0.033772044021100706,0.02159992663066638,200,0.1272450813248988,0.14799358083450787,0.18983737189848424,0.1541392678535334,200,0.4022711322266941,0.44304592217652133,0.5042821543040149,0.4473095703847343,200,0.6701738951928375,0.6996324657278779,0.7279575623049188,0.6993442357664899,200,0.10330630585379373,0.12025142188360821,0.13878991852910025,0.11934554127076638,200,13.0,13.0,13.0,12.975
|
||||
gsm8k,base,gsm8k-test-0000,23,distance,200,0.0859774901473429,0.24247226127772592,0.3115468222647905,0.22100177678250474,200,0.020143557339906693,0.05871833115816116,0.08728404641151429,0.05656428871443495,200,6.020657186991239,7.756322508679151,9.19959539193258,7.68429225077087,200,0.37629107418695246,0.48477015679244695,0.5749747119957862,0.4802682656731794,200,0.049280626732763635,0.0708559776411192,0.11050118934457508,0.07785165698568113,200,0.19264514689003703,0.24434106462375682,0.3253681330549431,0.25751102446473473,200,0.5611328332831799,0.6182548075651317,0.6977513898198393,0.6236736360895491,200,0.794373591069776,0.8221336251218114,0.8679262048269057,0.8272359319900764,200,0.040119649762336826,0.05724921460117974,0.07117066845099944,0.056648397011168805,200,10.0,11.0,11.0,10.64
|
||||
gsm8k,base,gsm8k-test-0000,23,divergence,200,0.0009918195865651568,0.007638052828383479,0.013023058803662479,0.007390125302575507,200,0.0004057639016536996,0.0034478427842259407,0.0076185435522347685,0.0037991140705162254,200,2.0573356015884428,3.2829369373595085,4.7210395093874675,3.3174200032406507,200,0.12858347509927767,0.20518355858496928,0.2950649693367167,0.20733875020254067,200,0.1592752198344575,0.25824577237887586,0.45180602562072397,0.2882147716360091,200,0.3386665389671859,0.4808401286042471,0.6798759910977102,0.49314375140452715,200,0.8063437190176992,0.8838581444627274,0.9403673060135335,0.8746582609430342,200,0.9433692011628815,0.9641418313912027,0.9833939284200156,0.9632459377314246,200,0.0029916092631083564,0.007304770697417879,0.012373656646281218,0.007872211720352427,200,3.0,5.0,7.0,5.065
|
||||
gsm8k,base,gsm8k-test-0000,23,QKV,200,0.3280119776725769,0.4029760400298983,0.5098655970301479,0.4114131474052556,200,0.07187108844518661,0.10643145442008972,0.1658969834446907,0.11296434864401818,200,5.981341118721267,8.083829089194786,10.211747532139889,8.15456667268445,200,0.3738338199200792,0.5052393180746741,0.638234220758743,0.5096604170427781,200,0.037788196225143796,0.06528402159327495,0.111666209276414,0.06915801409414389,200,0.20032637034618425,0.2694954974344086,0.3528808101716809,0.271215653152724,200,0.503143046978836,0.558541333730283,0.6221228583506108,0.5601401168382806,200,0.7507629484477409,0.7852734446797702,0.8275245601849299,0.7877898261545411,200,0.05650504218351432,0.07441094853766318,0.08939820750733604,0.0739321271713716,200,11.0,12.0,12.0,11.55
|
||||
gsm8k,base,gsm8k-test-0002,3,distance,200,0.5778809349983931,0.8139581922441721,1.0397058602422475,0.8147726438276004,200,0.12342397645115853,0.17571556568145752,0.2783580243587494,0.18995436742901803,200,5.585814770357019,9.34592834330294,12.144155892393636,9.190430018367815,200,0.3491134231473137,0.5841205214564338,0.7590097432746022,0.5744018761479884,200,0.021167289417881787,0.04746503897309742,0.12429339857926566,0.059049059747653974,200,0.14638022379267698,0.2245182736592162,0.35888193716325895,0.2408157590470933,200,0.4498876824164844,0.534394892221994,0.6251526512671801,0.5363251967599793,200,0.6974758815188615,0.7578101248109395,0.8168410356228315,0.7579047107065392,200,0.06158010453953037,0.09124034644605303,0.1256930044253247,0.09231627117570775,200,11.0,12.0,13.0,12.1
|
||||
gsm8k,base,gsm8k-test-0002,3,divergence,200,0.03922769215023436,0.0739228226702835,0.12326281644636765,0.07742392701949029,200,0.015234036836773159,0.030878269113600254,0.07748382538557051,0.03996294692857191,200,1.5959277076242524,3.583255662637402,7.060396164163792,3.9571265343593947,200,0.09974548172651578,0.22395347891483763,0.441274760260237,0.24732040839746217,200,0.08441093511076915,0.2310261044643796,0.6017322838789865,0.2886516702622793,200,0.25667038963979716,0.4828694838797152,0.7886935750733358,0.4970786274737604,200,0.6568301840028793,0.8020217545645113,0.926256003959145,0.7951792713675614,200,0.8534918690376757,0.9270139595359332,0.9720724199782839,0.9186501343273028,200,0.0069751700321484785,0.020207775445702354,0.046138344127879115,0.023777404273867722,200,3.0,7.0,10.0,7.08
|
||||
gsm8k,base,gsm8k-test-0002,3,QKV,200,1.2471024523023515,1.6848374158143997,2.0422420857474206,1.675778142453346,200,0.2787411868572235,0.3454492837190628,0.4103420972824096,0.346537334471941,200,7.512504956267244,9.26489683696256,11.843240819948452,9.439830053544425,200,0.46953155976670274,0.57905605231016,0.7402025512467783,0.5899893783465265,200,0.02339879641991042,0.048464659030014866,0.07531881955314076,0.04956681411365394,200,0.15473145536040728,0.21597481104999738,0.26266119312578723,0.21255231667474717,200,0.4621998863816477,0.5457939631008433,0.6296123551776766,0.5441618238959783,200,0.7104621257089849,0.7846317124977862,0.8517782138727126,0.7843664396954196,200,0.03067756102322136,0.0685446891062027,0.10736714783514326,0.06700567412239941,200,10.0,11.0,13.0,11.315
|
||||
gsm8k,base,gsm8k-test-0002,7,distance,200,0.4588568082312122,0.594299252377823,0.7913150947540998,0.6082002391468267,200,0.08101410940289498,0.12242840602993965,0.19604281187057496,0.13252689292654396,200,6.240606522411079,9.293470462490156,11.219858965694911,8.992650289789292,200,0.39003790765069246,0.5808419039056347,0.7012411853559319,0.5620406431118308,200,0.028402970969468963,0.04810930637582921,0.10425722642006854,0.057695739832514636,200,0.151806247298293,0.19824048061024813,0.3250660872730302,0.21995669318116648,200,0.4904520377850619,0.5623278343365258,0.6508166223675498,0.5681243098440669,200,0.7448179560270202,0.7994195231750447,0.838220519460707,0.7973244788955154,200,0.05491286658610531,0.07425678234764041,0.0954056762023492,0.07386324132943808,200,11.0,11.0,12.0,11.435
|
||||
gsm8k,base,gsm8k-test-0002,7,divergence,200,0.02133925435737183,0.040899452355915855,0.07314351725435697,0.044918261384087825,200,0.006563579104840756,0.014988765120506287,0.038432810455560684,0.019766522896825334,200,1.9985525714475059,4.465644484320402,6.466456994448759,4.365251803616908,200,0.12490953571546912,0.2791027802700251,0.40415356215304743,0.27282823772605674,200,0.09828724039118314,0.17219485050603964,0.4670530070312506,0.2273676206960406,200,0.2487065400295019,0.3732798250617877,0.6947117844970804,0.4193591984042902,200,0.7009787874127478,0.8117644558624368,0.9176442316720219,0.8107197192919051,200,0.9036231976528456,0.9479404227179655,0.9753227057730447,0.9441657081744377,200,0.00607985517104135,0.012337426117387038,0.02574915998088522,0.01429080802144564,200,4.0,6.0,8.0,6.175
|
||||
gsm8k,base,gsm8k-test-0002,7,QKV,200,0.911715642362833,1.111285433638841,1.3623870437964796,1.1235421768645757,200,0.1764332577586174,0.2297309786081314,0.27646832466125487,0.2293216209113598,200,8.256835903013751,9.892050328605478,11.303860561415847,9.788957524734386,200,0.5160522439383595,0.6182531455378424,0.7064912850884905,0.6118098452958991,200,0.027696380523480987,0.041164033037289396,0.06252025086722736,0.0441035225713798,200,0.16187901865082074,0.1970659313834993,0.2569650373880885,0.2058250561084426,200,0.4692899688702254,0.5184459170532822,0.5849363846665611,0.5251598571434153,200,0.734873658590213,0.7848732967018849,0.830486851702108,0.783699017591416,200,0.049254228193079695,0.07155021915056753,0.09238684445932373,0.07178566045159879,200,10.900000000000002,12.0,12.0,11.465
|
||||
gsm8k,base,gsm8k-test-0002,11,distance,200,0.3717869234737009,0.5517163160257041,0.7951661397702992,0.5744045477890176,200,0.06258412264287472,0.10153989493846893,0.18470735549926756,0.11871240390464663,200,6.216086996351456,10.583734529096978,12.286850370749091,10.092537412679818,200,0.388505437271966,0.6614834080685611,0.7679281481718182,0.6307835882924886,200,0.02014714923248563,0.03411692906339025,0.10493124787600414,0.04952281918193188,200,0.13796868245643307,0.17552783926454885,0.3316420171454854,0.2052406251323536,200,0.4426495089551375,0.50831109011817,0.6422774357729184,0.5226086298191379,200,0.7018934262376366,0.7591716561189159,0.8270206228729411,0.7590310118773342,200,0.06074325038737751,0.08934542359158021,0.1231553459461613,0.09157688456979163,200,11.0,12.0,13.0,12.075
|
||||
gsm8k,base,gsm8k-test-0002,11,divergence,200,0.013571130743912363,0.02989405481639551,0.07139487484819254,0.039562953546236485,200,0.003916811267845333,0.010310452431440353,0.03411883488297462,0.01898412890150212,200,1.949529789566126,5.348594046068695,7.517446511922116,5.184388625102257,200,0.12184561184788288,0.33428712787929343,0.46984040699513224,0.3240242890688911,200,0.07522548602906357,0.13276928941407398,0.48058931581150244,0.20263512558120544,200,0.2311240917807284,0.32918582692998255,0.6996814257349209,0.3902032244231103,200,0.6306614163532894,0.7499535396153776,0.9226869822490117,0.7538916615452572,200,0.8565389128126116,0.9134883227344689,0.9736931107620204,0.9104144486296516,200,0.006975389216219868,0.022761162176126806,0.046944547574914876,0.025880940861765397,200,4.0,8.0,10.0,7.575
|
||||
gsm8k,base,gsm8k-test-0002,11,QKV,200,0.7798966001719236,0.9306100197136402,1.126010161265731,0.9443512694630772,200,0.11035858392715454,0.1536755934357643,0.21309517771005632,0.16127233296632768,200,8.967965379822358,11.147546110307594,13.16153741867329,11.133210803219058,200,0.5604978362388974,0.6967216318942246,0.8225960886670807,0.6958256752011911,200,0.014377581732493781,0.02901975937377115,0.05227527296256638,0.031878474595679024,200,0.11501871895724478,0.1596913634428364,0.23733259587421146,0.17327599808487107,200,0.41375392800154315,0.48087633485685194,0.5597683903046374,0.48234859311074907,200,0.6806590198547666,0.7383426587906419,0.8048915726482105,0.7395463947340276,200,0.06289434437038191,0.09272360129230553,0.11881556549438901,0.09128685209709157,200,11.0,12.0,13.0,12.175
|
||||
gsm8k,base,gsm8k-test-0002,15,distance,200,0.3999343577655964,0.54410580242984,0.7712716283276677,0.5664035300011164,200,0.0685551829636097,0.09933661296963692,0.18621889054775237,0.11478499002754688,200,6.148039963131934,9.739882044966357,12.146497556842514,9.447871279722818,200,0.3842524976957459,0.6087426278103973,0.7591560973026571,0.5904919549826761,200,0.021150195230778556,0.04284949554502894,0.10683047319171673,0.05569465631777023,200,0.13411463109611388,0.18064117587798872,0.3093809262629546,0.20492748540401842,200,0.44677897974234376,0.5480681136311615,0.672802898811934,0.554306715680608,200,0.731257240893478,0.7958034373477175,0.8582682162117792,0.797407181979547,200,0.04377686299592411,0.06664996858729104,0.09774910403724539,0.06864411515479985,200,10.0,11.0,12.0,11.185
|
||||
gsm8k,base,gsm8k-test-0002,15,divergence,200,0.0168066328349596,0.0322431907222267,0.06949971609210479,0.03818456784576142,200,0.004699843283742666,0.009867845103144646,0.034678129851818076,0.015680436370894314,200,2.371436516223094,5.137114103149292,7.839786491635307,5.147250144321005,200,0.14821478226394338,0.32106963144683076,0.4899866557272067,0.3217031340200628,200,0.06939152204593305,0.14097334863484606,0.3832891697586219,0.19149737048774682,200,0.2133578736238559,0.3201947151607477,0.5933273532786095,0.3703992350199573,200,0.6231881629145388,0.7766455036505309,0.9286041692253579,0.7733454955507395,200,0.8730453436410366,0.9430758917388048,0.9815779715359356,0.9353535600639024,200,0.0037633778716239332,0.012065554946307823,0.02899584500390717,0.014411255036999533,200,4.0,7.0,9.0,6.635
|
||||
gsm8k,base,gsm8k-test-0002,15,QKV,200,0.6731397690018639,0.8251094203442335,0.9727590478956699,0.822807679377147,200,0.11363183557987214,0.15242495387792587,0.2278195798397064,0.16179404191672803,200,7.7821922962481604,9.850995433506405,11.728450729426815,9.813245403315463,200,0.48638701851551003,0.6156872145941503,0.7330281705891759,0.6133278377072164,200,0.024280308416693375,0.04161349471225567,0.07039839674149387,0.04463615752536985,200,0.14400442209413258,0.18589238477495845,0.2641369326476011,0.19804397968927476,200,0.46657169531552223,0.5394563517051518,0.6007028793718873,0.5354986080887606,200,0.7282230128042521,0.7793089698813548,0.822962266114176,0.7776686370328346,200,0.05138276972583985,0.07239528267727566,0.09918778946443979,0.07321879745493357,200,11.0,12.0,12.0,11.555
|
||||
gsm8k,base,gsm8k-test-0002,19,distance,200,0.26318493916187435,0.3480233011650853,0.4632515863515437,0.3558342826826265,200,0.06344504728913307,0.08565861359238625,0.12223782613873481,0.08936675632372498,200,6.58023039766261,7.864016583791619,9.381918138240696,7.964076006503982,200,0.4112643998539131,0.4915010364869762,0.5863698836400435,0.49775475040649886,200,0.04702825488161229,0.06897229693141407,0.09543509328183294,0.06977519596371157,200,0.19693475550589107,0.2510256706729836,0.3192279218010653,0.25368854744757235,200,0.5315064479560618,0.5989104394438203,0.6633564318228845,0.600597106338854,200,0.7775093095002485,0.8153247224750115,0.8468843719015107,0.8145685533674935,200,0.04942587310835667,0.0652728955821436,0.08408027551932994,0.06597197872975487,200,10.0,11.0,12.0,11.1
|
||||
gsm8k,base,gsm8k-test-0002,19,divergence,200,0.008319552923057928,0.014930177277392431,0.028071709721007206,0.0170691902814103,200,0.0040252932347357275,0.007337398128584027,0.014942117314785718,0.00848650029161945,200,2.105101912677889,3.081408528606336,4.5440998267938975,3.212835644087298,200,0.13156886954236807,0.192588033037896,0.2840062391746186,0.20080222775545611,200,0.16807238603027272,0.2794987243916909,0.44003905236891516,0.2911894799233538,200,0.358608866598447,0.5028344850703164,0.6729482026535009,0.5073381598164967,200,0.7855079803711392,0.8688632686792204,0.9193854834161709,0.8602169398818553,200,0.9379479197476167,0.9623528751130559,0.9769031080846019,0.95933306266242,200,0.004805746740439071,0.009145350421148564,0.01675882653615819,0.00991222424541032,200,4.0,5.0,7.0,5.435
|
||||
gsm8k,base,gsm8k-test-0002,19,QKV,200,0.3883025269955397,0.48424661345779896,0.5909120555967092,0.48922746131778694,200,0.05405767485499382,0.06794006004929543,0.09309958666563034,0.07158696992322802,200,10.948237133734878,12.4964405662761,13.33802596304126,12.248604706034179,200,0.6842648208584299,0.7810275353922562,0.8336266226900787,0.7655377941271362,200,0.013305260217294937,0.01869099813535863,0.030761613607515864,0.021126092568240634,200,0.11901502984105376,0.13830488124544768,0.18118734365656872,0.14722737022005344,200,0.4015710003063683,0.4420243924456849,0.5018916075303097,0.4487277504896065,200,0.6734679903282258,0.7011928846130578,0.7409413476323038,0.7037315512446769,200,0.09713503333510069,0.12035313577433807,0.13581042176827035,0.11849343583720792,200,12.0,13.0,13.0,12.935
|
||||
gsm8k,base,gsm8k-test-0002,23,distance,200,0.10958727307152004,0.2861355922068469,0.5407476099440828,0.3190064501683446,200,0.0294312147423625,0.07442718371748924,0.17424372285604475,0.0915224183909595,200,5.274077632715953,7.085277593638008,8.759572762561447,6.960968695767515,200,0.32962985204474704,0.4428298496023755,0.5474732976600905,0.4350605434854697,200,0.05510515638514999,0.08388031050993966,0.13558084075332383,0.09285956131320955,200,0.2041576868376657,0.2676839119304444,0.36749116561276907,0.2803433100032132,200,0.5895709994923499,0.6536785570283706,0.7232955507564917,0.6542702930720773,200,0.8121686607036298,0.8557761532442723,0.8980725263472019,0.8553252835350562,200,0.026412439205156846,0.041323540278234885,0.06160108910305274,0.04308461050046211,200,9.0,10.0,11.0,9.86
|
||||
gsm8k,base,gsm8k-test-0002,23,divergence,200,0.0019096716025927662,0.011176637987489357,0.049500706770936626,0.01990072268890051,200,0.0008661968051455915,0.005539476405829191,0.030361325107514853,0.01151464711445442,200,1.7753533751715274,2.9206226507931,4.465449951574323,3.018909444358251,200,0.11095958594822046,0.18253891567456876,0.2790906219733952,0.18868184027239068,200,0.1722044168333276,0.29855715379350056,0.5341541718386973,0.3282433748706869,200,0.35296567113925115,0.5166045743463012,0.7404387947326623,0.5293824428776983,200,0.8346503991286345,0.9063917147940977,0.9492950938674214,0.895843415732667,200,0.9541240063701711,0.9766413866988077,0.9906142973728801,0.974690483186132,200,0.0011530847456741114,0.0035456084038705056,0.008374254364169768,0.004312877988983412,200,3.0,4.0,6.0,4.485
|
||||
gsm8k,base,gsm8k-test-0002,23,QKV,200,0.363120306818746,0.5079717303160578,0.8828858155990019,0.6014205916930223,200,0.07070042490959168,0.13684461265802383,0.3057236313819885,0.16999507969245314,200,5.778022752758517,7.936816400701166,10.34817815877391,8.001633100769805,200,0.3611264220474073,0.49605102504382287,0.6467611349233694,0.5001020687981128,200,0.0364119008877611,0.067729932829502,0.11794097352568453,0.0733731395481363,200,0.17117794017764376,0.2711390995449628,0.3627376600247932,0.2659367446503703,200,0.5193991575712663,0.5868493934270956,0.6491736251586271,0.5870071138873729,200,0.756881004793214,0.8094099512324654,0.8612977322047606,0.8080553819304638,200,0.03768993391607685,0.05796910208243849,0.08168758312825014,0.05918773383738712,200,10.0,11.0,12.0,10.96
|
||||
gsm8k,base,gsm8k-test-0003,3,distance,200,0.6148790205828846,0.7874045209027827,1.0212811440229417,0.8028800293488894,200,0.13847081512212753,0.1931181401014328,0.3042105674743652,0.21091305892914533,200,5.330948369620039,8.66266560768636,10.956865916124428,8.4117588305163,200,0.3331842731012524,0.5414166004803975,0.6848041197577768,0.5257349269072688,200,0.030684833634568046,0.05646785455397221,0.1334228431458588,0.07061759919541265,200,0.18064044509514487,0.24678943698433947,0.3893697850834445,0.2657554771628546,200,0.48182154825395657,0.5604785427089676,0.6543843054003345,0.5621757243764183,200,0.7354445534144914,0.781620660489119,0.8328793724914949,0.7821154340882359,200,0.05910615940611908,0.08018443763970433,0.10246945505900713,0.0802283243027558,200,11.0,12.0,13.0,11.72
|
||||
gsm8k,base,gsm8k-test-0003,3,divergence,200,0.04555972157759243,0.07207538301008753,0.15029248573991932,0.08529381477180095,200,0.01917441450059414,0.03729468584060669,0.09254584237933157,0.05022315084934235,200,1.5234801429779299,3.0248059052906324,5.2845433254848215,3.2673925562927617,200,0.09521750893612062,0.18905036908066453,0.33028395784280135,0.2042120347682976,200,0.13518303968050316,0.2859732426336774,0.6334997982638673,0.33422623716620775,200,0.3424576741070622,0.5425915740558113,0.804981820905163,0.5523059623271203,200,0.7125713710566245,0.8294138406529887,0.930676451476751,0.8258855423032828,200,0.8954190621856893,0.9433746316363008,0.9769583460921459,0.9389629728698651,200,0.005357717275438934,0.014086558017285469,0.027123369030430312,0.016126465022773987,200,3.0,6.0,9.0,6.225
|
||||
gsm8k,base,gsm8k-test-0003,3,QKV,200,1.1608125476399436,1.4417741929646581,1.9687304812483488,1.5051811127745895,200,0.2656614869832993,0.32052019238471985,0.4454570531845093,0.3393246006965637,200,7.23365351752128,8.745186208756198,11.006552415528748,8.99895823960474,200,0.45210334484508,0.5465741380472624,0.6879095259705468,0.5624348899752962,200,0.030245314555290344,0.055305976977544774,0.08079243290412969,0.05492691859348362,200,0.1789198345547485,0.2268147435929962,0.275997383141086,0.22842450862765426,200,0.47745146031318936,0.5590080065077411,0.6431928720805071,0.5544374384740235,200,0.7390009992747721,0.8077221954158739,0.8790717552301434,0.8057352992786924,200,0.021740893211422353,0.052348190687101795,0.08833590503273382,0.05608449965443559,200,9.0,11.0,12.0,10.75
|
||||
gsm8k,base,gsm8k-test-0003,7,distance,200,0.42487007183954123,0.6324951644055545,0.856962319649756,0.6386413994658505,200,0.0741366595029831,0.11446922644972801,0.1777610197663307,0.12180069144815206,200,7.198150932443489,10.247867364214038,12.1812400148453,9.885285422515754,200,0.44988443327771804,0.6404917102633774,0.7613275009278313,0.6178303389072346,200,0.02089973318622392,0.037420040738868864,0.08152471435139345,0.04739271439624755,200,0.13513414261570153,0.1736845704515556,0.29407251699701153,0.19636065101808384,200,0.4518491183867284,0.5287988697092314,0.6248420845166458,0.5349372229815483,200,0.7248825369574607,0.7830724551064332,0.8276847213980669,0.7778340598077625,200,0.057878836559976186,0.07653809149787368,0.10359504635114765,0.07974162739259384,200,11.0,12.0,13.0,11.72
|
||||
gsm8k,base,gsm8k-test-0003,7,divergence,200,0.019469949923950482,0.03935822512721643,0.07831520501749764,0.044792808332805835,200,0.005496246507391334,0.013103326316922903,0.03159898221492767,0.01678036644880194,200,2.235954387218905,5.449684719917563,7.649419079502174,5.305522323116401,200,0.13974714920118156,0.3406052949948477,0.47808869246888586,0.33159514519477507,200,0.0727837017708198,0.12906335364494492,0.4105023435350707,0.18412634080614299,200,0.21327414689102908,0.3178084960467653,0.6533664760322253,0.3655591829249961,200,0.6448818887748203,0.760031954336003,0.8908937872566028,0.7622591347052873,200,0.8706528913574112,0.9308621594843492,0.9661691575286524,0.9248341772674591,200,0.0073854353150529135,0.015590740124614987,0.0339368029402314,0.01892030880804758,200,5.0,7.0,9.0,7.055
|
||||
gsm8k,base,gsm8k-test-0003,7,QKV,200,0.8821433159289882,1.0734853111207485,1.354620806593448,1.098048576307483,200,0.16758380979299545,0.2109229862689972,0.27060319781303405,0.21813257597386837,200,8.394958410154826,10.102838137265874,11.795969176245263,10.11441311792659,200,0.5246849006346767,0.6314273835791171,0.737248073515329,0.6321508198704119,200,0.023759746671255043,0.038914333296118285,0.06039404044923944,0.040810830702320854,200,0.150679537104305,0.19558964225628006,0.25719366827441126,0.20177692175621215,200,0.44702277056989215,0.5128740119106734,0.5746604333787128,0.5124529200387282,200,0.7272991032457127,0.773039740079098,0.8209962537683139,0.7714379500565856,200,0.04596363184067277,0.07106896477045065,0.09993287729165619,0.07282047517416729,200,10.0,12.0,12.0,11.51
|
||||
gsm8k,base,gsm8k-test-0003,11,distance,200,0.36322051696479324,0.5585828935727477,0.7560269128065555,0.5602555267553544,200,0.05534592792391777,0.0897398293018341,0.18744766861200327,0.11377193327993154,200,6.214548136512325,11.414524382416676,13.038357844017915,10.598616072300207,200,0.3884092585320203,0.7134077739010423,0.8148973652511197,0.6624135045187629,200,0.015143242801683368,0.026781627360852503,0.10505223767320872,0.0485112674317564,200,0.12299441002470836,0.16235991625203644,0.348967321944488,0.19745934662643955,200,0.41944621864449505,0.48153524468926606,0.6326585175300634,0.5048871925485678,200,0.675772604324555,0.7314264651479665,0.8213726622826826,0.7432743904497844,200,0.05996295943288118,0.1042113439127278,0.1346073343836346,0.10060767315205982,200,11.0,13.0,13.0,12.365
|
||||
gsm8k,base,gsm8k-test-0003,11,divergence,200,0.011794969367110752,0.027944314875639975,0.0634697668632725,0.03978869088838849,200,0.0030631720088422298,0.008053439669311047,0.03514739423990247,0.02075797386991326,200,1.7240609901228383,5.9967769019310335,8.777223569227946,5.787803875658766,200,0.1077538118826774,0.3747985563706896,0.5485764730767466,0.3617377422286729,200,0.05486017356309796,0.11120709393359868,0.5520483467124405,0.18885329471283577,200,0.19951327383225828,0.2947195219947224,0.7491916821933435,0.36589188620214924,200,0.5862657973549368,0.7065061634690535,0.9209511323053449,0.7242421409756449,200,0.8211474089568036,0.8925063791099227,0.9701443456677893,0.8932918271964158,200,0.0064851893980744525,0.03278920286562345,0.05930701820960026,0.03287480798357306,200,4.0,9.0,11.0,8.125
|
||||
gsm8k,base,gsm8k-test-0003,11,QKV,200,0.7511206613853574,0.9369289129972458,1.1453417709097264,0.9443349585030227,200,0.09952297583222389,0.1579308956861496,0.20963530987501144,0.15605784259736538,200,9.287599975982761,11.912442525938697,13.42788279695995,11.617885411404892,200,0.5804749984989226,0.7445276578711686,0.8392426748099969,0.7261178382128057,200,0.012771449205652105,0.022875865652862593,0.048182159841746654,0.026880241760596477,200,0.11057945837838544,0.16525345615348563,0.2323760070294224,0.16730132916616114,200,0.3995712972338297,0.46071357534576374,0.5382784767214714,0.4638339942468358,200,0.6628547425608147,0.7142510555552197,0.773247143227086,0.716886590764288,200,0.07923358430928196,0.10635479801648871,0.1358990157558667,0.10691076639985127,200,12.0,13.0,13.0,12.605
|
||||
gsm8k,base,gsm8k-test-0003,15,distance,200,0.42947639785707,0.5677564045181498,0.7429150291718541,0.582662995068822,200,0.06712151318788528,0.10545478761196136,0.1786970049142837,0.1175759301893413,200,6.345097408563426,10.074560555093885,11.885389219123697,9.679085746750495,200,0.3965685880352141,0.6296600346933678,0.7428368261952311,0.6049428591719059,200,0.023079386567919587,0.039210579005564454,0.10144246856997952,0.05011499508844878,200,0.13866049006033346,0.18123580582542423,0.30845495722708827,0.19856658483221207,200,0.4626001931493575,0.532854429148691,0.6656331294843857,0.5482433756268861,200,0.7283834060184198,0.7816096769637946,0.85489362071595,0.7871958646704801,200,0.04610905222816586,0.0785413642985443,0.10401804113328304,0.0761983489390483,200,10.0,12.0,13.0,11.525
|
||||
gsm8k,base,gsm8k-test-0003,15,divergence,200,0.016333204276270406,0.03305480521339632,0.07237946248023944,0.040300225753404535,200,0.004505337681621313,0.011120720766484737,0.03194103091955183,0.017019882659078577,200,2.3956978404660294,5.110345150061095,7.227892855466276,5.110177646730273,200,0.14973111502912684,0.3193965718788184,0.45174330346664227,0.31938610292064207,200,0.08090986125710498,0.14206035190940247,0.37863269470055744,0.18140391142737947,200,0.22443669949254114,0.32495387337801024,0.5852233943643683,0.3637464577156292,200,0.659423005025325,0.7677070052693644,0.9225855629471119,0.7752867433622777,200,0.883969374105989,0.9298071556307232,0.9785539499944101,0.9305453411626192,200,0.003599072923875008,0.01628218225311453,0.03355594150277399,0.017541318046117704,200,4.0,7.0,9.0,6.88
|
||||
gsm8k,base,gsm8k-test-0003,15,QKV,200,0.7038629116024822,0.834518953692168,0.9759014369919896,0.8421908039192203,200,0.12234886810183525,0.15557347238063812,0.21580820381641386,0.1632897013053298,200,8.140225232213067,9.903375140973612,11.382769156373813,9.829425298992959,200,0.5087640770133167,0.6189609463108507,0.7114230722733633,0.6143390811870599,200,0.027042338943789507,0.04104078043591525,0.0643708462110778,0.044075186205754575,200,0.14686838857149753,0.18547898581450123,0.2542354141292978,0.19606247070289817,200,0.48224028305988914,0.5439495016129292,0.5973584550495687,0.5407486954980848,200,0.7387095030787781,0.7762341643827307,0.8078529700876067,0.7749038321256182,200,0.05553970442673852,0.07700016094057346,0.09898913052219441,0.07738097246459963,200,11.0,12.0,12.0,11.675
|
||||
gsm8k,base,gsm8k-test-0003,19,distance,200,0.26619529486633836,0.3320698754978366,0.4182334843091667,0.3387854143412551,200,0.06789370179176331,0.09388939291238785,0.12296716049313546,0.0950566816329956,200,6.209796564591082,7.707897907706213,9.299256966721206,7.704407199462987,200,0.38811228528694264,0.48174361923163833,0.5812035604200754,0.4815254499664367,200,0.04803783981684321,0.0717197022588916,0.10510495388174895,0.07528181772045357,200,0.22380989386602493,0.27736476500071683,0.3426699644176706,0.28155919376801475,200,0.5349286349212538,0.5850540903441466,0.6392467250387909,0.5859112346990244,200,0.7661656960150847,0.8051672589467609,0.8400499727795008,0.8042676368554885,200,0.05210526657202767,0.06808561184363549,0.08594909534786987,0.06857229191126968,200,10.0,11.0,12.0,11.255
|
||||
gsm8k,base,gsm8k-test-0003,19,divergence,200,0.009005393478673795,0.014827692582002783,0.025136910100809474,0.015721694052077736,200,0.004609665367752314,0.0088152214884758,0.015120964031666516,0.0094826314173406,200,1.74156518092631,2.564033852242031,3.746854083250172,2.701157277237899,200,0.10884782380789437,0.16025211576512693,0.23417838020313575,0.1688223298273687,200,0.21805086957468378,0.349351731262932,0.5458115634879387,0.36639020204150713,200,0.43885352045991266,0.6016483814008501,0.7525613522820356,0.5976462179625228,200,0.7900045664311816,0.8624125812820218,0.9145291524175475,0.857378463747764,200,0.9314801749323778,0.9600750650081245,0.9748104074284162,0.9558848063213098,200,0.005170601989784313,0.009337534508607047,0.018113064158737115,0.010435394098801289,200,4.0,6.0,7.0,5.54
|
||||
gsm8k,base,gsm8k-test-0003,19,QKV,200,0.3661244992166758,0.47430233052000403,0.6061728548258543,0.4845273335056845,200,0.05154174380004406,0.06844192743301392,0.1288524016737938,0.08181358095258474,200,9.59739913406172,12.225028238210442,13.302546899491169,11.739475943046337,200,0.5998374458788575,0.7640642648881526,0.8314091812181981,0.733717246440396,200,0.013518656079257436,0.020586123975835265,0.044474589390997896,0.027038839650014804,200,0.1223281166540607,0.14458329371218043,0.22506733381876637,0.1651384214566029,200,0.40267475528029173,0.44778289435912333,0.5247696434911054,0.4585025489177223,200,0.6698881626017978,0.7079721275927737,0.7667970026481392,0.7128712200689944,200,0.08981335762995861,0.1153561560858797,0.13473406962615841,0.11437853560481663,200,12.0,13.0,13.0,12.845
|
||||
gsm8k,base,gsm8k-test-0003,23,distance,200,0.1003011952561792,0.28140300218365155,0.37208317801123486,0.25882489837160394,200,0.028136201575398447,0.07075439766049385,0.10220653414726256,0.06936437993310392,200,5.596464162518696,7.086121726203823,8.257859176737004,7.016909402303598,200,0.3497790101574185,0.4428826078877389,0.5161161985460627,0.43855683764397485,200,0.06250329830292727,0.08386235131726072,0.12393025998957653,0.09062142606346651,200,0.21025295041575354,0.26312476062739604,0.3372853093710498,0.2714332444115884,200,0.5904759070240881,0.6550394074336072,0.7243126539336138,0.6575214761993066,200,0.8155290782772441,0.8487322058078164,0.8859955046231855,0.8494924523967652,200,0.03344778543997255,0.04904845584597678,0.06323900440921304,0.04897043490513568,200,9.0,10.0,11.0,10.125
|
||||
gsm8k,base,gsm8k-test-0003,23,divergence,200,0.0016026902321712557,0.01091612074776549,0.02044312250054645,0.011040912722144051,200,0.000791652564657852,0.005006185499951243,0.01044619046151638,0.005584210277520469,200,2.1769577434392895,3.0395151381984107,4.059831326934231,3.065094796951168,200,0.1360598589649556,0.18996969613740067,0.2537394579333894,0.191568424809448,200,0.19607099211361767,0.28426651477949283,0.4233182132169203,0.30654671801564176,200,0.3427922266027834,0.4965068618856022,0.6540619192168082,0.5039631246185223,200,0.844069531762603,0.9045390863100984,0.9505665999128994,0.9016519401079189,200,0.9588864803985032,0.975213815351986,0.9887148701905456,0.9743922149585097,200,0.0017374977934091784,0.00484479755148025,0.008862244369235356,0.005313654309692083,200,3.0,4.0,6.0,4.365
|
||||
gsm8k,base,gsm8k-test-0003,23,QKV,200,0.34729341899510474,0.462055129930377,0.599712233711034,0.47176041034806987,200,0.06990195363759995,0.11934495344758034,0.17212649285793302,0.12279999125748872,200,6.352815690355091,8.22304524628338,9.913898512966854,8.204382674673846,200,0.3970509806471932,0.5139403278927113,0.6196186570604284,0.5127739171671154,200,0.04092639780780962,0.06305008231434027,0.10124111193160466,0.06680210803331559,200,0.1818415431351354,0.2582494952744951,0.3324070302853755,0.25726160877195803,200,0.5234402288967906,0.5735641184832156,0.6241638483063617,0.5737872727489737,200,0.7603808304551005,0.793775090207158,0.8207876132428946,0.7914395388561322,200,0.05817183908063483,0.06944485594006064,0.08752141695717973,0.0715660195195566,200,11.0,11.0,12.0,11.435
|
||||
gsm8k,base,gsm8k-test-0004,3,distance,200,0.5792004176881165,0.7580482575576752,0.9636388950049877,0.7545591866329778,200,0.11304593980312347,0.1627855747938156,0.24085893183946605,0.1723108423501253,200,6.086488290366591,9.365395765101743,11.095331781924857,8.993113403435832,200,0.38040551814791196,0.585337235318859,0.6934582363703036,0.5620695877147395,200,0.029469861593410123,0.04722779728878413,0.10858703575031495,0.058759275737432094,200,0.15893221235907684,0.21145478001762782,0.3550623909604303,0.2338035155358507,200,0.4867565780157185,0.5464843917906106,0.6303687750598865,0.5517287775375628,200,0.7417498237020429,0.784971221593463,0.83049802948535,0.7862586549226339,200,0.051620880688705524,0.07141713320996917,0.09309885902700947,0.07211285623454763,200,11.0,12.0,12.0,11.485
|
||||
gsm8k,base,gsm8k-test-0004,3,divergence,200,0.03759978450780182,0.058787019119790784,0.10532723854412324,0.06791999095056327,200,0.012779384292662144,0.026499359868466854,0.0580158296972513,0.03284973000176251,200,1.7499725518691005,3.952317957086018,6.170211801593906,4.0295962454929,200,0.10937328449181878,0.24701987231787612,0.3856382375996191,0.2518497653433063,200,0.10620774508878986,0.2032186811010026,0.5429133912102698,0.2615040084182768,200,0.2703525802915626,0.4247176516789044,0.7467505210353387,0.4690597813889357,200,0.7031770308461908,0.8018023483231047,0.9074762664029834,0.8016510013005024,200,0.8949356670813065,0.9383791733841811,0.9739600661200237,0.9363167555113848,200,0.004734640842932582,0.013220421800695499,0.02531514794639589,0.014314429238585941,200,4.0,7.0,9.0,6.61
|
||||
gsm8k,base,gsm8k-test-0004,3,QKV,200,1.1244378308067098,1.4537889009807259,1.9623313235118984,1.5036335841252002,200,0.24314792156219484,0.29436424374580383,0.3835132777690887,0.30825396470725536,200,7.257920496602356,9.02722657652132,11.505134965545864,9.282949553505343,200,0.45362003103764725,0.5642016610325825,0.7190709353466165,0.580184347094084,200,0.026045644146884307,0.05149474840227743,0.08029930905977338,0.051587787343675175,200,0.1526255080845054,0.2068252370516067,0.27503512747962927,0.21053609859809577,200,0.47041844413491857,0.5555374555223724,0.6287319849345606,0.5532910958910858,200,0.7297560879546188,0.8001007712342687,0.8590474263526497,0.8011424479898586,200,0.025361474854553912,0.05520252876358889,0.10066622941934814,0.05839256560330039,200,9.0,11.0,13.0,10.92
|
||||
gsm8k,base,gsm8k-test-0004,7,distance,200,0.3909899444319308,0.6113364188931882,0.791423321608454,0.6015714764402946,200,0.07735021784901619,0.11400501430034637,0.17518232613801954,0.12477644976228476,200,6.344703731536955,9.63427357894626,11.790516288374976,9.384389183654015,200,0.3965439832210597,0.6021420986841413,0.736907268023436,0.586524323978376,200,0.023801531856388166,0.04404937513040283,0.10145298677607069,0.05415821387934408,200,0.1406846635446987,0.18934397898958483,0.3321088257497976,0.21291537574052596,200,0.46209859659010993,0.5473832861479841,0.6366529962354807,0.5492680940134508,200,0.7351533716918269,0.7925148981932956,0.8391444745982216,0.7903521891287457,200,0.04907726208354567,0.07287504481723656,0.10174569296590978,0.0749133173160467,200,10.0,12.0,13.0,11.485
|
||||
gsm8k,base,gsm8k-test-0004,7,divergence,200,0.019471982200548153,0.03853671323759045,0.07060144576935272,0.04167258248771304,200,0.005983060458675027,0.012997205834835768,0.030689698271453375,0.01766557062219363,200,1.842829877336112,4.816991241916685,7.29653250015959,4.760898359196627,200,0.115176867333507,0.3010619526197928,0.4560332812599744,0.2975561474497892,200,0.07952149478852141,0.1547738316425291,0.5121670963554169,0.21505421137437084,200,0.22512338292528045,0.3437096735731879,0.7290398125580626,0.40538897448619976,200,0.6523923999469722,0.7930764619570758,0.9021157449797472,0.7834001024939102,200,0.8876672335267926,0.9461713192251989,0.9733672225530371,0.9359484653791961,200,0.004995205553849416,0.013721448158057658,0.0317072359461201,0.01584368719169472,200,4.0,7.0,9.0,6.63
|
||||
gsm8k,base,gsm8k-test-0004,7,QKV,200,0.9889560017734766,1.1344167906790972,1.3605736231431365,1.1596262183110229,200,0.1761748731136322,0.2098168209195137,0.24190850257873533,0.20973126865923405,200,9.346582902348464,10.7511043321819,11.847068913076528,10.614518588264957,200,0.584161431396779,0.6719440207613687,0.740441807067283,0.6634074117665598,200,0.023369671290712535,0.03254819599155967,0.04745706844354652,0.03474429169898237,200,0.14547116304553195,0.17839985373278183,0.22684837926428933,0.18308084023356616,200,0.4524901863009756,0.4902543822507114,0.5412330154452958,0.49384541128179527,200,0.7195140899150234,0.7553950634522862,0.8210526987993232,0.7616843373677383,200,0.049153526298293185,0.07965812418969276,0.1012339147198358,0.07915412465739814,200,11.0,12.0,13.0,11.7
|
||||
gsm8k,base,gsm8k-test-0004,11,distance,200,0.34612745861522853,0.543545635882765,0.7566914628259838,0.5477736511657713,200,0.053069109469652175,0.08510333299636841,0.19038386195898052,0.10219491980038584,200,6.718612219275393,11.430875692834068,13.006099782157222,10.886041747667214,200,0.41991326370471205,0.7144297308021292,0.8128812363848263,0.6803776092292009,200,0.015346137484919102,0.02664788912859472,0.09212277792755733,0.038319002692108854,200,0.12329330410766208,0.1606282485541144,0.32283090041392104,0.18474562252913976,200,0.41565567287952315,0.47641383289443223,0.5930605028422262,0.48773535348696173,200,0.6865336208697621,0.7340160874242943,0.8058007075073423,0.7415383983849131,200,0.06327485121200828,0.0971304067052187,0.12137023970592244,0.09496546327236084,200,11.0,12.0,13.0,12.22
|
||||
gsm8k,base,gsm8k-test-0004,11,divergence,200,0.011073461529031194,0.027465892086183885,0.06320796256914034,0.03242901019152498,200,0.002816353691741824,0.007242604857310653,0.036248474568128575,0.013577243965555682,200,1.9898847070739774,6.207999428026019,8.885609153856832,6.0333229411206775,200,0.12436779419212358,0.3879999642516262,0.555350572116052,0.37708268382004234,200,0.05338030840172123,0.10515468899239418,0.4693798878448038,0.16451013705310338,200,0.1942998616292948,0.2917213614746089,0.700244909056758,0.35134800907976316,200,0.574986224745178,0.6913539385148924,0.8860451864928333,0.700734816855986,200,0.8265446510462136,0.8884458371253272,0.9658244876673479,0.8896932487713406,200,0.007759829346792457,0.02897504686508123,0.046970690705422806,0.02952652093101389,200,5.0,9.0,11.0,8.415
|
||||
gsm8k,base,gsm8k-test-0004,11,QKV,200,0.7777908539399505,0.9525258196517825,1.154694497399032,0.9672909968695603,200,0.10568452253937721,0.14779021590948105,0.2015801191329956,0.15053427539765835,200,10.273404066523211,12.298729724676779,13.401531589103698,11.899798458412953,200,0.6420877541577007,0.7686706077922987,0.8375957243189811,0.7437374036508095,200,0.012926253275556873,0.020063162130634787,0.03716843143238328,0.024375272577493355,200,0.11982296428017453,0.14348341352915067,0.2042908856902427,0.1570619924622434,200,0.4040306730127515,0.4476674824761331,0.502541556897624,0.4545732507193758,200,0.6722454035225945,0.7156883745223157,0.7767706688538023,0.7191242351024241,200,0.07887482977190716,0.10410435393063484,0.1352320570678244,0.10648995116024261,200,12.0,13.0,13.0,12.6
|
||||
gsm8k,base,gsm8k-test-0004,15,distance,200,0.44527862106915567,0.5873221451183781,0.788975378125906,0.5936288708745269,200,0.0700216494500637,0.10915577039122581,0.17388439029455185,0.11667205737903714,200,7.3838719612687695,9.7927994338864,11.71656332378881,9.611411063894154,200,0.4614919975792981,0.6120499646179,0.7322852077368006,0.6007131914933846,200,0.024372811149067745,0.042256903785739125,0.077793153083107,0.04870994051848569,200,0.13960443438308823,0.1854369260519822,0.26834621424916033,0.19759716527741403,200,0.4659643644472217,0.5403995019209642,0.634890580065602,0.546518415701486,200,0.7413154650387686,0.7959277364364965,0.8584497171224917,0.7973515261691533,200,0.04484468737148872,0.06843694879671701,0.09670120387699477,0.06948993630135888,200,10.0,11.0,12.0,11.235
|
||||
gsm8k,base,gsm8k-test-0004,15,divergence,200,0.017716062335239258,0.03594773842542054,0.06382022233738098,0.039490088123296234,200,0.004903031419962645,0.011915088165551424,0.0302357817068696,0.015184876652201638,200,2.863451210799426,5.076977301239441,7.511317783952066,5.076401684068081,200,0.17896570067496412,0.31731108132746505,0.4694573614970041,0.31727510525425506,200,0.07534128909128072,0.14343212054938448,0.3058512177482714,0.17401864222064853,200,0.22419168815168358,0.33340907100540335,0.5537464881125271,0.36505724538590456,200,0.6475786826449622,0.7734482560535402,0.8779848364007438,0.7699743671984066,200,0.8896141042686991,0.9417891155092915,0.9774232290040054,0.9369176092874515,200,0.004163918449597326,0.012125564845067995,0.029435202416357793,0.014452939222995554,200,5.0,7.0,9.0,6.825
|
||||
gsm8k,base,gsm8k-test-0004,15,QKV,200,0.7088892538333311,0.8338137781247497,0.951790789142251,0.8323849217186217,200,0.10534566789865493,0.14268917590379715,0.20206736773252482,0.1508871993422508,200,8.291739129711384,10.551435912264246,12.06360810687533,10.333624508558714,200,0.5182336956069615,0.6594647445165154,0.7539755066797081,0.6458515317849196,200,0.021753673225465592,0.03442602099378057,0.0619771470718689,0.038776557079809613,200,0.12392575197341063,0.1786232933786873,0.2480530551158739,0.18345137932664193,200,0.4499923081167761,0.5063362503702049,0.5748826024873113,0.5104606380851129,200,0.7361381274387236,0.7759208399552562,0.8131404028676513,0.7765225103483183,200,0.053683737044375415,0.0735869496747158,0.09812841345334976,0.07516531068132563,200,11.0,12.0,12.099999999999994,11.615
|
||||
gsm8k,base,gsm8k-test-0004,19,distance,200,0.3052797598298639,0.3687430159188807,0.4465618295129389,0.37109544344071765,200,0.06991701573133469,0.09626937657594681,0.1369818553328514,0.10102631617337465,200,6.046244260441706,7.953100030049347,9.886787140202955,7.9557081851216624,200,0.37789026627760663,0.4970687518780842,0.6179241962626847,0.4972317615701039,200,0.04122149947136618,0.06745432610385044,0.10975233990885486,0.07225675435165231,200,0.20562764322086213,0.26338191878442097,0.34396706669912525,0.27091645701464545,200,0.5275866448753894,0.588879775673794,0.6602233227913404,0.589954125840829,200,0.7648383662114427,0.8013279754703422,0.8509889288439726,0.8038303825609173,200,0.04798226636911243,0.06985969973505188,0.0872860096915555,0.06959394464109336,200,10.0,11.0,12.0,11.265
|
||||
gsm8k,base,gsm8k-test-0004,19,divergence,200,0.010491945809292247,0.016923009554375312,0.02704198093260856,0.018484184598646605,200,0.004888390237465501,0.009267800021916628,0.018764030188322067,0.010950050727115013,200,1.8445317198290225,2.8493168707876495,4.347059432235944,2.9634965621662843,200,0.11528323248931391,0.1780823044242281,0.2716912145147465,0.18521853513539277,200,0.1787099902429682,0.3076922408161154,0.5116261286102058,0.3343524314909038,200,0.4047331349954002,0.5563304097996598,0.727136031193649,0.5652274656464091,200,0.7841912007969039,0.8613866474729172,0.9256721501443961,0.8553094078674736,200,0.923954032782994,0.9543418544220397,0.9804286319701143,0.9533207381913341,200,0.003881624518343729,0.010248924342557163,0.018999174719534984,0.011364127437668903,200,4.0,6.0,7.0,5.64
|
||||
gsm8k,base,gsm8k-test-0004,19,QKV,200,0.3612969814799726,0.4464109919499606,0.605773054715246,0.46444602181669326,200,0.049270202219486234,0.06572079658508301,0.09767986014485358,0.0703905713930726,200,11.004200261564744,12.079923579519015,13.148981199260973,12.065955288650667,200,0.6877625163477965,0.7549952237199384,0.8218113249538108,0.7541222055406667,200,0.01445495595600472,0.02163411627992829,0.030266132290618454,0.022372490276065016,200,0.12325160912376056,0.14850964504212594,0.17823317279871348,0.15156359497637265,200,0.4107833415192387,0.4495441953276743,0.4956458656507788,0.45328919302078163,200,0.673290182544824,0.7060657042622303,0.7397062052753858,0.7077379103396177,200,0.09218428264213878,0.1162935330315472,0.13204959977689318,0.11254387631634889,200,12.0,13.0,13.0,12.705
|
||||
gsm8k,base,gsm8k-test-0004,23,distance,200,0.0932104671257548,0.2608456803136505,0.3314726382959634,0.2373625426943181,200,0.027493463456630708,0.0553826242685318,0.08857744112610816,0.058086184593848886,200,5.618288000415849,7.880304654655083,9.485240803265388,7.749173267412741,200,0.3511430000259906,0.4925190409159427,0.5928275502040867,0.4843233292132963,200,0.04578879613077579,0.06869189158018398,0.12318951903698416,0.07666536883706691,200,0.18787214376454192,0.23637675024590574,0.3353541363419937,0.25117839595422004,200,0.5475872910062339,0.6243851944786016,0.7038033810613131,0.6247976638495137,200,0.7737568625622873,0.8193572200654271,0.8653962152618557,0.8205624374455641,200,0.04287442648656264,0.06007173126758192,0.08187163816351747,0.061042868350280916,200,10.0,11.0,12.0,10.81
|
||||
gsm8k,base,gsm8k-test-0004,23,divergence,200,0.0013466713247055396,0.007686647315495065,0.01478301872666634,0.008357886782981758,200,0.0007559725374449046,0.0030677232425659895,0.007846061885356902,0.0039925382925139275,200,2.0912730374271473,3.4043071381169216,4.715400373054549,3.417412324778478,200,0.1307045648391967,0.2127691961323076,0.2947125233159093,0.21358827029865488,200,0.1595424953689068,0.24666481343120794,0.4433912942248727,0.2776564798032005,200,0.3149986970568833,0.4548642625301523,0.6648314530760793,0.47323512706897747,200,0.8007324184898255,0.8864022104867433,0.9429750763636379,0.8785878231687501,200,0.9330957508506098,0.9628478133628557,0.9847574085695515,0.9605275203630516,200,0.00305184305798463,0.00760518278298465,0.01685175662193622,0.008734384200113281,200,3.0,5.0,7.0,5.015
|
||||
gsm8k,base,gsm8k-test-0004,23,QKV,200,0.37086151340045037,0.46105506108142436,0.5633907845476642,0.4636512024491094,200,0.07565892413258553,0.10913018137216568,0.15540633350610733,0.11056851116940379,200,6.67809137323285,8.806882022398,10.988120849012605,8.87553365997036,200,0.41738071082705314,0.550430126399875,0.6867575530632878,0.5547208537481475,200,0.03040799207193278,0.05445134121413221,0.09305962623523399,0.05803578137315549,200,0.16268262548550877,0.23491874572427568,0.32565025023295924,0.2410501117009491,200,0.4869504434289099,0.5516062707036851,0.5980352598473029,0.5449744033040812,200,0.7242243338271485,0.7662004441845254,0.7992394664485681,0.7630219588018947,200,0.06545397300517168,0.08434161165448863,0.10705124050940135,0.08542225769047086,200,11.0,12.0,13.0,12.04
|
||||
gsm8k,finetuned,gsm8k-test-0000,3,distance,200,0.5137887824326753,0.6829048923682421,0.9271006872411818,0.7091014222614467,200,0.11660449877381325,0.18243587017059326,0.26530554890632624,0.1897782825306058,200,4.804625045686464,8.159070098893773,10.705552735922032,8.080573054628813,200,0.300289065355404,0.5099418811808608,0.669097045995127,0.5050358159143008,200,0.032970564998946704,0.06406733514564407,0.15535734155864014,0.07680734341141594,200,0.16977948103364812,0.2578225234879742,0.417317512798289,0.2746604652362798,200,0.48807882989518475,0.5711036580121784,0.6666218306193985,0.5733086871270247,200,0.7428545951995732,0.7975376803725278,0.8446979431078111,0.7949074289403039,200,0.04994832102856141,0.07122009410624183,0.08862184307420676,0.07025540378278256,200,10.0,11.0,12.0,11.33
|
||||
gsm8k,finetuned,gsm8k-test-0000,3,divergence,200,0.03422472126330831,0.06031672264361987,0.10420513450953876,0.06801881569341958,200,0.013596609234809875,0.033282965421676636,0.07038977667689322,0.03975533865625039,200,1.417889538427648,2.8950909866529626,5.465951425604843,3.200217507487125,200,0.088618096151728,0.18094318666581016,0.3416219641003027,0.20001359421794532,200,0.12854292074148096,0.30177310746144703,0.6856300705879039,0.3508427138276203,200,0.31444821684685026,0.5542552872901246,0.8374885467281064,0.5641589005518022,200,0.7212956726333046,0.8502083361658607,0.9405083589499559,0.8338558514202533,200,0.901104910751331,0.9506390146337854,0.9825971947458089,0.9456858884590931,200,0.0035655402144047963,0.011106796471825596,0.020661390204291835,0.012092166907687303,200,3.0,6.0,8.0,5.86
|
||||
gsm8k,finetuned,gsm8k-test-0000,3,QKV,200,1.1064322639169404,1.3869650466367602,1.863009561994113,1.4204531571154075,200,0.25838682651519773,0.31861451268196106,0.41501420140266415,0.33231759279966355,200,7.592493065449164,8.635635904904982,10.00171036506336,8.733281681462651,200,0.4745308165905727,0.5397272440565614,0.62510689781646,0.5458301050914157,200,0.039981774694575485,0.05685251518792499,0.07382317851197842,0.057061649470920975,200,0.20189874544738343,0.2356853012059925,0.2756286392877329,0.2359013299298946,200,0.49996538589240674,0.5624370896192001,0.6110095642003098,0.5606519749961053,200,0.7677265756588135,0.8055701807210348,0.8484718093739896,0.8096396037040968,200,0.019674614093996894,0.04858313845421326,0.07793808723447494,0.0492117613985788,200,10.0,11.0,12.0,10.665
|
||||
gsm8k,finetuned,gsm8k-test-0000,7,distance,200,0.41216933124233035,0.6193190219346434,0.8919361499138176,0.6339980365196243,200,0.08321020528674125,0.11967325583100319,0.1789842739701271,0.12623456673696637,200,6.368854621026649,9.679616044025003,11.617005357339186,9.374384265328073,200,0.39805341381416554,0.6049760027515627,0.7260628348336992,0.5858990165830046,200,0.02515282921041312,0.043530803407231325,0.10081556718718943,0.05439714803580534,200,0.14524707275801885,0.18761773715113356,0.3193414526271858,0.21146090010542712,200,0.4605300880164791,0.5416269036694863,0.6576789061082089,0.5532664518131432,200,0.7327839111595168,0.7856808544212859,0.8445911946248387,0.7867110698934598,200,0.049482580536729416,0.07285505914941989,0.09759817930734023,0.07344587966826482,200,10.0,12.0,12.0,11.44
|
||||
gsm8k,finetuned,gsm8k-test-0000,7,divergence,200,0.021020242323538697,0.040746971037151525,0.07184405312546005,0.04541159678560973,200,0.006923937890678644,0.01432173140347004,0.03203540965914726,0.017282060706056656,200,2.0503103163991834,4.793894820997918,7.075673181889659,4.75060109120423,200,0.12814439477494896,0.2996184263123699,0.4422295738681037,0.29691256820026435,200,0.08408610834017706,0.1558386168927171,0.4536345876232863,0.21087575768027628,200,0.24103312459845716,0.34365296526023953,0.6899166549970154,0.3981667178071014,200,0.652743429178076,0.7878604627773513,0.9220096538559295,0.7888027075656169,200,0.8823514385692027,0.935706021763598,0.9778664069451563,0.9312582322840879,200,0.004641047849609289,0.014614217289072898,0.029026024638695944,0.01635732910989848,200,4.0,7.0,9.0,6.785
|
||||
gsm8k,finetuned,gsm8k-test-0000,7,QKV,200,0.939926034770906,1.1528743882663548,1.513593319989741,1.1919679949700366,200,0.18052826672792435,0.20980150997638702,0.24481084793806077,0.21218690864741802,200,8.916676060849678,10.591809022021618,12.30690987083714,10.553633268277785,200,0.5572922538031049,0.6619880638763511,0.7691818669273213,0.6596020792673616,200,0.02000555485829504,0.03404008370953031,0.052959384771438085,0.03599877049822368,200,0.1338730767036116,0.17928090910517525,0.2492586299426079,0.1841633030908455,200,0.4411994380478393,0.5066237154568978,0.5511021749177896,0.5008293922652732,200,0.7086207965042371,0.7725563997137657,0.8029782621201593,0.7610709288999269,200,0.052308822618168965,0.07353285128369859,0.10637565939843993,0.07684046071465335,200,11.0,12.0,13.0,11.705
|
||||
gsm8k,finetuned,gsm8k-test-0000,11,distance,200,0.3869037050986663,0.5717043336480856,0.8585314536467193,0.5969195567921269,200,0.06182914674282074,0.09682084619998932,0.1844987317919731,0.11262538600713015,200,7.726865181995363,11.080322651512809,12.535821446289235,10.434432591262512,200,0.4829290738747102,0.6925201657195506,0.7834888403930772,0.652152036953907,200,0.018422824421126874,0.02960012944790497,0.07138133843547476,0.04412959215172524,200,0.1277958981908042,0.1603594975011847,0.28174530343043264,0.1912551132188963,200,0.434700532190926,0.4985773769092274,0.5930190039917788,0.5065834846784255,200,0.7049396290775878,0.7509120105807692,0.8076918084391905,0.75554371911086,200,0.0645876336058173,0.09042310785870364,0.11206740649194932,0.08926024634438334,200,11.0,12.0,13.0,12.055
|
||||
gsm8k,finetuned,gsm8k-test-0000,11,divergence,200,0.014086014792701464,0.03208969739716849,0.07215499587437078,0.0401759376082083,200,0.0038228492019698026,0.009374356362968683,0.03404064439237117,0.0167880167911062,200,2.3566898500387983,6.195728044932361,8.242204625102632,5.715939762165587,200,0.1472931156274249,0.38723300280827255,0.5151377890689145,0.3572462351353492,200,0.0627498296387526,0.1054973673977157,0.38595103543021003,0.17928818657087964,200,0.1958767717597156,0.2910432520099121,0.6313687511124935,0.3569000249970407,200,0.6011565053669685,0.7162484329430397,0.8758189007202789,0.7273460479010996,200,0.8546851358256171,0.907759919966618,0.9596339594275295,0.905754600915178,200,0.00850312855565663,0.022997244169156203,0.04125937269268959,0.024389105145384504,200,5.0,8.0,10.0,7.82
|
||||
gsm8k,finetuned,gsm8k-test-0000,11,QKV,200,0.771041193883866,0.9841246260330081,1.2308062704280018,0.9974409051495604,200,0.10305022299289704,0.15472117811441422,0.2000311091542244,0.15297105424106122,200,10.216374569290236,12.244424076597324,13.349873752050817,11.973921914082156,200,0.6385234105806398,0.7652765047873328,0.834367109503176,0.7483701196301348,200,0.013234381452164502,0.020447831767237124,0.037740924173991726,0.02333847019754338,200,0.11293385663559857,0.14887180566470698,0.20481941828575945,0.15457544192928474,200,0.3941330573443272,0.44138470854280965,0.5040206372936196,0.446437095411672,200,0.6747309394112464,0.7143907195508912,0.7567381401787692,0.7166426704820946,200,0.07498067983222323,0.10032061433405015,0.1234606382400696,0.09992607843207751,200,12.0,13.0,13.0,12.51
|
||||
gsm8k,finetuned,gsm8k-test-0000,15,distance,200,0.42788795423693954,0.5911742807365954,0.8549222071655094,0.612383289533318,200,0.07089542448520661,0.10419977828860283,0.1687969535589218,0.1152957689575851,200,7.772925162703926,10.21375294640698,11.978990024834852,9.998632914904412,200,0.48580782266899536,0.6383595591504363,0.7486868765521782,0.6249145571815258,200,0.022381466399292716,0.03776768990772451,0.0705620833741223,0.04425980765675239,200,0.13189039448730028,0.1731871435031476,0.26056414892380986,0.18916881304496755,200,0.4490411356389033,0.5269029669185139,0.6098684323266313,0.5290017093439917,200,0.7244288490241232,0.783347708684281,0.8372955865118596,0.7826134314776709,200,0.052167365144201755,0.07531156903703631,0.10223143151014132,0.07657134553570391,200,10.0,12.0,13.0,11.57
|
||||
gsm8k,finetuned,gsm8k-test-0000,15,divergence,200,0.018093967449749473,0.03528215560436365,0.07178225568022752,0.041315003954979376,200,0.00502616842277348,0.010857594665139914,0.02849254999309778,0.01563766757841222,200,2.9245398721658833,5.44763519698443,7.777358637990407,5.391168290300195,200,0.1827837420103677,0.34047719981152685,0.4860849148744004,0.3369480181437622,200,0.0704839611302971,0.12914151726079584,0.29806314864756006,0.16531871863997202,200,0.20239262422769688,0.30635113931524594,0.541791768855199,0.34958727594978956,200,0.6329441406363002,0.7502461621375454,0.867691938785836,0.7506512845085035,200,0.8729641521008039,0.9343752703665466,0.9680491862393423,0.9272805733593014,200,0.006055528907706792,0.015660516106855708,0.031585151112165116,0.01755786348138479,200,5.0,7.0,9.0,7.195
|
||||
gsm8k,finetuned,gsm8k-test-0000,15,QKV,200,0.6790293713333085,0.7991825425997376,1.0295230954885484,0.8314329529437237,200,0.09906226843595504,0.13423892855644226,0.18417851775884628,0.14031065054237843,200,9.03359183620393,11.029991985047765,12.739461828170061,10.948826618660673,200,0.5645994897627457,0.6893744990654853,0.7962163642606288,0.6843016636662921,200,0.017062711177672805,0.030039367204700847,0.051411310009586764,0.032426655647038574,200,0.12362742741245293,0.1659484390774536,0.22968018565109696,0.1705901096562284,200,0.4298032351299414,0.49404662379930125,0.5578478864081995,0.4955141301711617,200,0.7083275287362393,0.7539103246800989,0.7947541434335399,0.7524325848396068,200,0.06098485516666979,0.09060203691745708,0.1154563087853332,0.088344990182642,200,11.0,12.0,13.0,12.05
|
||||
gsm8k,finetuned,gsm8k-test-0000,19,distance,200,0.28776293410919607,0.3528448115102947,0.42027654962148514,0.352766030569328,200,0.07002608180046081,0.09773524105548859,0.13450126051902772,0.10025284972041845,200,5.655039851496113,7.551513792739319,9.34815852967106,7.5297694372781026,200,0.35343999071850707,0.47196961204620747,0.5842599081044413,0.4706105898298814,200,0.047438143374112306,0.074585886429299,0.12195599656170894,0.08062846411120222,200,0.21149408801378783,0.27882249046194196,0.3667571217954982,0.28399186150949085,200,0.544989349117285,0.604979013784599,0.6698009569427449,0.6037621205188227,200,0.7709820855431966,0.8088915375541786,0.8481312374815173,0.8084972951128243,200,0.05401602363346609,0.06961874555059896,0.08690356510050475,0.07038745972859724,200,10.0,11.0,12.0,11.24
|
||||
gsm8k,finetuned,gsm8k-test-0000,19,divergence,200,0.010556567429466668,0.0165735462169323,0.026690395378500396,0.01767474908276313,200,0.004903788212686777,0.009552211500704288,0.018090599589049815,0.010768554646638222,200,1.660542426901209,2.5812235971860535,4.068609958763454,2.775453390128163,200,0.10378390168132556,0.16132647482412835,0.25428812242271587,0.17346583688301018,200,0.19550498325823013,0.3465821052184449,0.5757041875256946,0.36612389553645547,200,0.3975252148427294,0.5916052881463736,0.7698564600376582,0.5866200270190739,200,0.8046503756495478,0.8831686708166445,0.9332375787894223,0.8722419569167869,200,0.9378660288864844,0.9631801185348698,0.9786914103306827,0.9592505837131106,200,0.004762162822015517,0.00980054998662399,0.01734932526295648,0.010606560513872423,200,3.0,5.0,7.0,5.09
|
||||
gsm8k,finetuned,gsm8k-test-0000,19,QKV,200,0.36053648493252693,0.44148144219070673,0.5345079994760453,0.4411984587623738,200,0.05090355053544045,0.0639876015484333,0.08792313709855079,0.06736934358254075,200,10.63661351894396,12.432408245391814,13.344229405567667,12.207959484139714,200,0.6647883449339975,0.7770255153369884,0.8340143378479792,0.7629974677587321,200,0.01326803469232799,0.019130607414695137,0.03361591700856978,0.02135509778877081,200,0.12480940242298312,0.14639730104991533,0.1867669566087994,0.15279806945191,200,0.4025115210291777,0.43965113916836307,0.503380797714891,0.44638460167853794,200,0.6701884392161986,0.6977706213735708,0.7274201239395582,0.69818328568417,200,0.10418422397824159,0.12075792987363877,0.1389807847623245,0.11982322107205262,200,13.0,13.0,13.0,13.0
|
||||
gsm8k,finetuned,gsm8k-test-0000,23,distance,200,0.08689095607260243,0.24624597595538944,0.31511037147138266,0.22509488105326456,200,0.02134448867291212,0.06067569740116596,0.09079756438732148,0.05871284210123122,200,5.86245581609847,7.634926287806037,9.059163501921095,7.565342614022016,200,0.3664034885061544,0.4771828929878773,0.5661977188700684,0.472833913376376,200,0.05107793398829888,0.07304237801034388,0.11528211324086018,0.08027592822160401,200,0.19247874578869614,0.2530791423867309,0.3337929488594719,0.2620971179696672,200,0.5653289955179015,0.6230756072499696,0.7014891795964653,0.6278097502314276,200,0.7967304669972537,0.8240119208698415,0.8692722326059165,0.8294672266129092,200,0.039455186792941056,0.05683508429269762,0.07097411520199319,0.05592832735457696,200,10.0,11.0,11.0,10.605
|
||||
gsm8k,finetuned,gsm8k-test-0000,23,divergence,200,0.0010251708068068411,0.008168367431778734,0.013808873408106591,0.007813110242633843,200,0.00045558902493212375,0.0036815579514950514,0.008244198840111494,0.004096786811642232,200,1.9846017027538834,3.1844660822577144,4.582805371223617,3.240225297900504,200,0.12403760642211771,0.19902913014110715,0.28642533570147605,0.2025140811187815,200,0.16610265860385245,0.2682930262724501,0.4708048078373249,0.2973267446569682,200,0.3461321285166171,0.4897988951292913,0.6917976119545828,0.5023797976666744,200,0.8101782409410488,0.885480574668635,0.9421786814404575,0.8788130079587673,200,0.9444231215660662,0.965790649621475,0.9845948343272973,0.9646831425735336,200,0.002760330974417011,0.007140625227819402,0.012177537192671069,0.007585593908582266,200,3.0,5.0,7.0,4.96
|
||||
gsm8k,finetuned,gsm8k-test-0000,23,QKV,200,0.33303874123375865,0.41114774090237916,0.5206384780351072,0.41900573252118195,200,0.07587774619460105,0.11307193711400032,0.17213518023490904,0.11831605643033981,200,5.859410588257886,7.923732929444739,10.018769153110453,7.952405734377606,200,0.3662131617661179,0.4952333080902962,0.6261730720694033,0.49702535839860035,200,0.039800230390972746,0.06795002007626652,0.11537686853402274,0.07267576098196508,200,0.20818920632278942,0.2746775651155252,0.3583128293757965,0.2788082138694678,200,0.5069447040755837,0.565740509907004,0.6249368450652556,0.5652492788510353,200,0.7539361607602277,0.7885570391338602,0.8275213664192546,0.7898755582309511,200,0.057014868541076236,0.07311211769197473,0.08903118473155909,0.07311317009634209,200,11.0,12.0,12.0,11.53
|
||||
gsm8k,finetuned,gsm8k-test-0002,3,distance,200,0.5861232142196968,0.8125424887984991,1.0468116397038103,0.8137392610986717,200,0.12276827320456506,0.1763683408498764,0.2727173626422882,0.18947919707745314,200,5.790433494732773,9.312729420861988,12.094729476357148,9.178372504588232,200,0.3619020934207983,0.5820455888038742,0.7559205922723218,0.5736482815367645,200,0.021526264980820522,0.04787189919909646,0.11756857794412882,0.05909013915364713,200,0.14350690352453369,0.22256256747566427,0.3589717401310094,0.2406815077641975,200,0.4522574668074794,0.5357927070630306,0.6267500060279413,0.5374096337329535,200,0.6988096676317956,0.7582306521724114,0.817425701119141,0.7581393528014385,200,0.06001248628444951,0.09208181882582005,0.12500301147448084,0.09230073080231549,200,11.0,12.0,13.0,12.115
|
||||
gsm8k,finetuned,gsm8k-test-0002,3,divergence,200,0.0396255340352127,0.07400235132081434,0.12189901097444816,0.07722840687724557,200,0.015072087477892638,0.031107649207115173,0.07437710240483282,0.03966438201721758,200,1.5957363422052688,3.5390570683996545,6.979776899323456,3.942953649880491,200,0.0997335213878293,0.2211910667749784,0.436236056207716,0.24643460311753068,200,0.08615698031513222,0.23473220481068618,0.6017965206244686,0.28803060290442994,200,0.2529285351590543,0.47881176323517205,0.7887049349774404,0.49631828421363183,200,0.6572139516089451,0.8028838945621914,0.9230796180466687,0.7967706230779038,200,0.8556671037107774,0.9271576778759416,0.9723385511059871,0.919068910669648,200,0.007096391558800714,0.020067104140597418,0.04745355406847057,0.02370198388847919,200,3.0,7.0,10.0,7.09
|
||||
gsm8k,finetuned,gsm8k-test-0002,3,QKV,200,1.2482796687399969,1.675051489204634,2.0245829921215774,1.6684328539451234,200,0.27972609996795655,0.34436337649822235,0.40753168165683745,0.3448720436543226,200,7.549718131196697,9.31147534893811,11.813588695094776,9.459291960669702,200,0.4718573831997936,0.5819672093086319,0.7383492934434235,0.5912057475418564,200,0.023624860504911357,0.04788756473642425,0.07461896405073372,0.04925989641860884,200,0.15639100186651092,0.21483670573763175,0.25934721169381364,0.21232775045978522,200,0.45867920189016126,0.5431441901501792,0.6289159427013118,0.5424987118237518,200,0.7101398622510365,0.7838330213404009,0.8510744119411617,0.7833432075279603,200,0.03092649078577326,0.06839014794571513,0.10709274088282918,0.06702052970919674,200,10.0,11.0,13.0,11.305
|
||||
gsm8k,finetuned,gsm8k-test-0002,7,distance,200,0.4537952012149617,0.5945524559356272,0.785909054800868,0.6088061689754249,200,0.08263247311115265,0.12291320785880089,0.19778196066617965,0.13287273911759256,200,6.194914267264987,9.20682719959397,11.157312509380537,8.979731385188707,200,0.3871821417040617,0.5754266999746231,0.6973320318362836,0.5612332115742942,200,0.028935806224748322,0.049189436055321334,0.10551759711662008,0.057942562224144505,200,0.1527734954623412,0.20212673718728955,0.3266650881356609,0.2206080148708346,200,0.4952121936799912,0.5631304033363516,0.6502035211816096,0.5687100749710505,200,0.746205647707644,0.7968138433673022,0.8355857358973964,0.7962021046060027,200,0.05517347748468229,0.07433829210025994,0.09580754458460515,0.07418736123244571,200,11.0,11.0,12.0,11.43
|
||||
gsm8k,finetuned,gsm8k-test-0002,7,divergence,200,0.0216697800293332,0.04078332584049349,0.07380867860192665,0.04491347734506462,200,0.006828134134411812,0.015107664279639721,0.03911860287189483,0.01979441850329749,200,1.9835991139513436,4.385502937510623,6.425244827995628,4.337477115356744,200,0.12397494462195897,0.2740939335944139,0.40157780174972674,0.2710923197097965,200,0.09934516954918222,0.17656116128964056,0.47109036275109784,0.22888303508365127,200,0.25692889311605305,0.3727361806264593,0.6969460589615605,0.4208161511505435,200,0.7080261963219414,0.8156454313201535,0.9191487224737538,0.8125334790946299,200,0.9040166770435002,0.9481547886461319,0.9747663772545001,0.9436669518572022,200,0.006284752191646566,0.01243408657838738,0.025641946352840873,0.014326694018883469,200,4.0,6.0,8.099999999999994,6.17
|
||||
gsm8k,finetuned,gsm8k-test-0002,7,QKV,200,0.9111597992712632,1.1103207361884415,1.3555527718737719,1.1267244507954457,200,0.17975106537342073,0.22986891865730286,0.2760771572589874,0.23001387752592564,200,8.1974884080433,9.888602924366346,11.240393849474215,9.772745717409851,200,0.5123430255027063,0.6180376827728966,0.7025246155921384,0.6107966073381157,200,0.028229181129935665,0.04120162139582497,0.06345450724034143,0.044271091200699025,200,0.1623044820826516,0.1967594585398338,0.261317592014213,0.20634130755934152,200,0.4736455137216692,0.520605048269945,0.5861352352207299,0.5262978299282552,200,0.7346700299268577,0.7823419442257233,0.8302815777802597,0.7822053369192168,200,0.04735276325753003,0.07137541119297022,0.09398412498265116,0.0717595063190383,200,10.0,12.0,12.0,11.485
|
||||
gsm8k,finetuned,gsm8k-test-0002,11,distance,200,0.38104152944870295,0.5537373360712081,0.8018366364762186,0.5745382640243042,200,0.06187209114432335,0.10207236930727959,0.18672912567853925,0.11903586937114596,200,6.232400522461636,10.56598152907636,12.370373532959317,10.108475523609082,200,0.38952503265385224,0.6603738455672725,0.7731483458099573,0.6317797202255676,200,0.01956122523108486,0.034286270913657574,0.10448777280428469,0.049380538235694174,200,0.13717315829261179,0.17582688233605193,0.33263211479449944,0.2055579816105284,200,0.4392619666501249,0.5073269700347769,0.645989317312252,0.5216499541635321,200,0.7041626820522204,0.7563480953132673,0.8281080855079205,0.7580114345807548,200,0.06372903667018975,0.09001885054342931,0.12225186511345922,0.09210063116988668,200,11.0,12.0,13.0,12.125
|
||||
gsm8k,finetuned,gsm8k-test-0002,11,divergence,200,0.013565718445534003,0.03044252987820073,0.07159262634741026,0.039637577429862174,200,0.003828173130750656,0.010418769903481007,0.034870189800858487,0.01914324795419816,200,1.9734852446800775,5.360027339368774,7.55337961837928,5.181598478203898,200,0.12334282779250484,0.33500170871054835,0.472086226148705,0.3238499048877436,200,0.0745504976293778,0.13234021269643192,0.4741538451875007,0.2031654770638542,200,0.23151329096470005,0.33357180677596754,0.694161990784005,0.3917606636697022,200,0.6280021722994469,0.745573928562911,0.9245215742526532,0.753144721430234,200,0.8565618567690719,0.9113071827125432,0.9752340733878417,0.9096540076657783,200,0.006751521451150035,0.02293955575636012,0.04647061275202628,0.026084074801904777,200,4.0,8.0,10.0,7.595
|
||||
gsm8k,finetuned,gsm8k-test-0002,11,QKV,200,0.7801614590920508,0.9290395192801952,1.119946207664907,0.9439782392070629,200,0.1113031767308712,0.15465939790010452,0.211395525932312,0.16104358714073896,200,9.030305953682324,11.135696445193599,13.115575732648638,11.172776048786414,200,0.5643941221051453,0.6959810278245999,0.8197234832905399,0.6982985030491509,200,0.014661843996906253,0.02912138838741259,0.05145410132983209,0.03144127711311463,200,0.11468279177253382,0.15942887419520907,0.23793614645018332,0.1727699508139188,200,0.4124672231463437,0.4801097366732935,0.5576882141294954,0.48036057512207364,200,0.6780568505624295,0.7364264629116144,0.7998541784145804,0.7381750320771628,200,0.06340914004140126,0.0923759389783202,0.1198717998182387,0.09169304092557456,200,11.0,12.0,13.0,12.175
|
||||
gsm8k,finetuned,gsm8k-test-0002,15,distance,200,0.4060570724774152,0.5463368515484035,0.766747329942882,0.5685075975279323,200,0.06766543164849281,0.09934177994728088,0.18797669559717178,0.11519187405705451,200,6.11441098082357,9.772351256165031,12.204561304689628,9.458303633983634,200,0.38215068630147314,0.6107719535103144,0.7627850815431018,0.5911439771239771,200,0.020732495288529565,0.04248500635301056,0.10778466795244489,0.05581183833029138,200,0.1324048988880851,0.17941361259893635,0.30971083824096124,0.2048790594207675,200,0.4457851221263049,0.5477584013037184,0.6655776296742375,0.5542004872713909,200,0.7272143749479291,0.7950051048891209,0.8580751041510586,0.796454971504618,200,0.04441500835665265,0.06721331674785747,0.10091795110898294,0.06926473612226879,200,10.0,11.0,13.0,11.22
|
||||
gsm8k,finetuned,gsm8k-test-0002,15,divergence,200,0.017128099042747636,0.03195174681127355,0.0699212257630279,0.03848234567238592,200,0.004578654887154698,0.009868961293250322,0.03533524088561535,0.015871741864830254,200,2.3804247937913856,5.1392825940140785,8.021279476510939,5.165053006765314,200,0.1487765496119616,0.3212051621258799,0.5013299672819337,0.3228158129228321,200,0.06631407698000276,0.14088660149133347,0.38161486901024316,0.19234113640990322,200,0.20798799511074956,0.3150633226686022,0.6069161371301187,0.37004342282435426,200,0.6158879773427584,0.7828673923268968,0.9274658840389703,0.7732772281956426,200,0.8701822374752574,0.9430637983810273,0.9810145882620179,0.9345597817159162,200,0.003740321109017846,0.012175771840273342,0.031575447760714484,0.014786078921009782,200,4.0,7.0,9.0,6.615
|
||||
gsm8k,finetuned,gsm8k-test-0002,15,QKV,200,0.6731592075899243,0.8231188519857824,0.9803120773285627,0.826454773300793,200,0.11354329884052276,0.15255964547395706,0.22784739285707473,0.16134543169289828,200,7.875323315090874,9.891936990706805,11.760946830723494,9.852640454984858,200,0.49220770719317963,0.6182460619191753,0.7350591769202184,0.6157900284365536,200,0.024028980070569538,0.04116533160792814,0.0687776753224404,0.04416523083890798,200,0.14198736257013195,0.18561379406420242,0.2621633956450627,0.19657430115521812,200,0.4646327235586094,0.537298105375832,0.6001243206060949,0.5342391891760545,200,0.7263917332138875,0.7797120090392107,0.8260233853298817,0.777033390905159,200,0.052149912928848845,0.07173331446972436,0.1007340233773187,0.07388398539864639,200,11.0,12.0,13.0,11.555
|
||||
gsm8k,finetuned,gsm8k-test-0002,19,distance,200,0.26475379725452514,0.34800438658567145,0.4645815273746848,0.3556610992796777,200,0.06306288540363311,0.0855526551604271,0.12115095555782318,0.08929450599476695,200,6.541360863831491,7.872207552217631,9.422604409894936,7.9607246102511935,200,0.4088350539894682,0.49201297201360195,0.5889127756184335,0.4975452881406996,200,0.04653686563369193,0.06883125178296857,0.09639834298178927,0.06986144014477466,200,0.19750724191601504,0.25095366409161446,0.3181085427790002,0.2536137193363237,200,0.5323437147336876,0.6003479795246398,0.6667965434322238,0.6010962151427477,200,0.7752543114768877,0.8155707741413295,0.8469171714172957,0.814177703859857,200,0.04958283910704623,0.06422397999448098,0.08430567961310605,0.06605635473716197,200,10.0,11.0,12.0,11.105
|
||||
gsm8k,finetuned,gsm8k-test-0002,19,divergence,200,0.00835859241072967,0.014735271476183698,0.028717368985962822,0.017072368665432693,200,0.003976927557960153,0.007319261785596609,0.014677634835243224,0.008472429901594296,200,2.1017749046172507,3.1217345487207697,4.468489357068774,3.216433270115006,200,0.13136093153857817,0.1951084092950481,0.27928058481679835,0.20102707938218786,200,0.17204661541515429,0.2750239351075041,0.44084159226657543,0.29101456135321535,200,0.3460608063588007,0.5067687337224627,0.6702640608597662,0.5068024850506281,200,0.7868839116230926,0.8685812100035051,0.9213304678842182,0.8607401444833731,200,0.93657324866675,0.962343434358178,0.9769810118064723,0.9591441574143349,200,0.004802899011164982,0.009107277499373699,0.016457819818486635,0.009967703536885255,200,4.0,5.0,7.0,5.4
|
||||
gsm8k,finetuned,gsm8k-test-0002,19,QKV,200,0.3894812896382064,0.48542622406966984,0.5922858995385468,0.49007118737208655,200,0.05403104387223721,0.06778312847018242,0.09543900638818739,0.07225100632756948,200,10.921499543488093,12.472106333270315,13.355794569614366,12.25357655143313,200,0.6825937214680058,0.7795066458293947,0.8347371606008979,0.7658485344645706,200,0.013199028331645166,0.018857513319431588,0.031000694207107774,0.021091000529934803,200,0.12088181247196166,0.13961024814055295,0.18057263656452402,0.1480048821605874,200,0.4037660274097866,0.4412877375374373,0.503635359923236,0.4485056941187529,200,0.6721001563125266,0.7009107288941473,0.7417557522334214,0.7026308186328609,200,0.09671150860041725,0.12057594332833499,0.13897748569325663,0.11902019253577882,200,12.0,13.0,13.0,12.94
|
||||
gsm8k,finetuned,gsm8k-test-0002,23,distance,200,0.11417096609366127,0.29247323190793395,0.5432869527488946,0.3236989406025532,200,0.031285582855343816,0.07620572298765182,0.1781864359974861,0.09419189762789756,200,5.251493006292796,6.96979016901763,8.639623210369514,6.862238232186728,200,0.3282183128932997,0.43561188556360186,0.5399764506480946,0.4288898895116705,200,0.056795652662024805,0.08637513258058607,0.13645257839969754,0.0950497842703618,200,0.206286671784145,0.2720204892211292,0.3707572960133434,0.28361620092423956,200,0.5945321701899796,0.658635279748913,0.729585733857283,0.6590544790348564,200,0.8152265751891514,0.8578488994363358,0.8994590424821101,0.8572814917180139,200,0.026499161772505123,0.040851272513952284,0.06169679883472952,0.04256740327766741,200,9.0,10.0,11.0,9.78
|
||||
gsm8k,finetuned,gsm8k-test-0002,23,divergence,200,0.002108765179198713,0.011849882519868515,0.05122904014584719,0.020772531239084294,200,0.000978788163047284,0.005807358305901289,0.0317505408078432,0.01217435457801912,200,1.7653600671989818,2.890499606225258,4.3824255327321575,2.964953184182739,200,0.11033500419993636,0.18065622538907863,0.27390159579575984,0.18530957401142117,200,0.17672998462870826,0.30235936072830705,0.5375551671651226,0.33392322256692686,200,0.3553800270234976,0.5196541577655585,0.7409633380224621,0.5344528751198684,200,0.8405622006348089,0.9065767788400789,0.9526223672804104,0.9002421648560924,200,0.9570498262886615,0.9776974826116318,0.9914644887444619,0.9757633724786788,200,0.0011547096361998596,0.00336566598514217,0.008290811890694109,0.004147029453827352,200,3.0,4.0,6.0,4.4
|
||||
gsm8k,finetuned,gsm8k-test-0002,23,QKV,200,0.36844358816742895,0.5207858956418931,0.8871487533207982,0.6067873523867456,200,0.07295355424284936,0.1436539813876152,0.31165820658206933,0.17300018211826682,200,5.655707825656688,7.817935963303986,10.334859208719864,7.906982004190271,200,0.353481739103543,0.48862099770649914,0.6459287005449915,0.49418637526189196,200,0.036544301558503633,0.06977189027286623,0.12193343402254743,0.0751000512620191,200,0.17476870703429853,0.2693581857604333,0.36769740481776186,0.26882550435231023,200,0.5229222651815116,0.5917162789729111,0.6525789954461101,0.591082719014178,200,0.7592072172839182,0.8107200566521299,0.8648066097970674,0.8094300015351663,200,0.037524456131933316,0.057856188390043695,0.08113893661881796,0.05916277301374807,200,10.0,11.0,12.0,10.925
|
||||
gsm8k,finetuned,gsm8k-test-0003,3,distance,200,0.6086965966969728,0.7824259013868868,1.027777993865311,0.8030686871474608,200,0.13826231211423876,0.19456008821725845,0.3066834986209869,0.21028283581137658,200,5.31537871928187,8.724109894074221,10.9310942033902,8.435071360862223,200,0.33221116995511685,0.5452568683796388,0.6831933877118875,0.5271919600538889,200,0.030914333555205013,0.055600858800983886,0.13400916571009786,0.07018360821123992,200,0.18316506526330004,0.2459639144061499,0.39106890580209,0.2652570881527407,200,0.4830310179906012,0.5577121700682642,0.6546935926563907,0.560988562029371,200,0.7353599557733667,0.7813789835019498,0.831757274763109,0.7815511149019835,200,0.0590283112741357,0.07950116652996303,0.10171427303511092,0.08020797662182311,200,11.0,12.0,13.0,11.695
|
||||
gsm8k,finetuned,gsm8k-test-0003,3,divergence,200,0.04509253361466108,0.07284721633550362,0.15090872583677992,0.0849461912284869,200,0.01911676675081253,0.03785364516079426,0.09405550360679625,0.04977683682460338,200,1.5286577314911594,3.0897551057422588,5.158945455406302,3.2747952447226965,200,0.09554110821819746,0.19310969410889117,0.3224340909628939,0.20467470279516853,200,0.14009872729650352,0.2785604023652375,0.6311798894986462,0.3328970846704615,200,0.3472591188395531,0.5299493701967724,0.8040188660743106,0.5519393749761801,200,0.7146318328448183,0.8302931797123971,0.9309244121736511,0.8244151065150757,200,0.892915147358979,0.9429531776127401,0.9771724019817045,0.9383891513134032,200,0.005265659054314583,0.013843601665828523,0.02721159016151252,0.016161377387003163,200,3.0,7.0,9.0,6.245
|
||||
gsm8k,finetuned,gsm8k-test-0003,3,QKV,200,1.1601755222538486,1.4414845602586865,1.9473883109167218,1.5024173459268058,200,0.26494814455509186,0.3201083391904831,0.43996088206768036,0.3375805613398552,200,7.286980987928941,8.774540438684227,11.029008682188271,9.031203570330865,200,0.4554363117455588,0.5484087774177642,0.689313042636767,0.564450223145679,200,0.030047989829052865,0.05489715540010105,0.07971344200032814,0.054408720497524155,200,0.17964277520000735,0.22721773127753697,0.2742685611125878,0.2277320928379336,200,0.4778474675741422,0.5561466139433291,0.6374524562180436,0.553055903284296,200,0.7386499438545446,0.8055708737457294,0.8762702553584748,0.8053379232991031,200,0.021138561097721997,0.052235159128563094,0.08956306550384299,0.05588265844197913,200,9.0,11.0,12.0,10.755
|
||||
gsm8k,finetuned,gsm8k-test-0003,7,distance,200,0.42035067188553515,0.6352679359260947,0.8556057389825582,0.6415638547309208,200,0.07491196990013123,0.11543427407741547,0.17760915905237198,0.12255760833621025,200,7.259760112918753,10.220952156877956,12.17460264609117,9.868461438825522,200,0.45373500705742204,0.6388095098048723,0.7609126653806981,0.6167788399265951,200,0.020947508797017903,0.037694130125039305,0.08027579982431017,0.047611572939534774,200,0.1347171522196126,0.1757149133476738,0.3025644384168367,0.19677983851258068,200,0.4542642409895017,0.5296353367609623,0.6255202285964548,0.536013875919284,200,0.7221874631149243,0.7829062282548997,0.8278655915787146,0.7777008803374094,200,0.057775246172752016,0.07633148289685929,0.10506650012426627,0.08007700830323206,200,11.0,12.0,13.0,11.7
|
||||
gsm8k,finetuned,gsm8k-test-0003,7,divergence,200,0.01995215111019206,0.039971969430553145,0.07903079791667551,0.045352178050594175,200,0.005611807201057672,0.013325146865099669,0.03154510594904423,0.01702943189709913,200,2.132473576276976,5.401660620470924,7.619656466316989,5.288907319724546,200,0.133279598517311,0.33760378877943276,0.4762285291448118,0.33055670748278415,200,0.07332492874125819,0.13080375839763417,0.4335975002888347,0.18521342308681826,200,0.21028769054733548,0.31995038567828926,0.6638289539917692,0.36642104438952017,200,0.6477888804354844,0.7650705230992367,0.8934562232487728,0.764082903832014,200,0.8677516666298916,0.9327473888038164,0.9667025450064309,0.9248491124107359,200,0.006739429457379717,0.015157028702213252,0.03465993666939036,0.019072102964539442,200,5.0,7.0,9.0,7.035
|
||||
gsm8k,finetuned,gsm8k-test-0003,7,QKV,200,0.8851492325309664,1.0757438333239406,1.3625152982771396,1.1015632723388262,200,0.1691047340631485,0.21289841830730438,0.27311553955078127,0.21963061183691024,200,8.305662500191483,10.126223350040569,11.738663157182973,10.085188231009257,200,0.5191039062619677,0.6328889593775355,0.7336664473239358,0.6303242644380785,200,0.024201310821819103,0.038670400299169454,0.06176072858319361,0.0410723526063046,200,0.15253650244626055,0.19528800029596965,0.25891822231198924,0.20251407253723933,200,0.44851275806943863,0.5158436130277453,0.5754737659563364,0.5143003179256656,200,0.7262258495003491,0.7736885853824522,0.8209211474189947,0.7717665659954064,200,0.04831826507426933,0.07113632048348492,0.10328642553636823,0.07336504465567113,200,10.0,12.0,13.0,11.57
|
||||
gsm8k,finetuned,gsm8k-test-0003,11,distance,200,0.3698225241620094,0.5619712678017095,0.7631645567249505,0.5638118235388538,200,0.055651943385601046,0.09112667292356491,0.1947927117347717,0.11403326166793704,200,6.285324378057054,11.4903935211725,13.101374320780804,10.62300854895889,200,0.3928327736285659,0.7181495950732812,0.8188358950488003,0.6639380343099306,200,0.014749759921117612,0.02616450596265508,0.1031986887445329,0.04817750889731121,200,0.1233101430367506,0.1613588334032628,0.3461211441531291,0.1968313449309949,200,0.41833322771935444,0.4814029099411827,0.6238077602444437,0.5037681858983822,200,0.67229529909467,0.7324198788915366,0.8205961560224786,0.7423177481686448,200,0.06140100038051367,0.10555386994836177,0.1341868217701027,0.10098014745511026,200,11.0,13.0,13.0,12.385
|
||||
gsm8k,finetuned,gsm8k-test-0003,11,divergence,200,0.012090600941701269,0.029079316166644276,0.06479491739592048,0.04005027275096495,200,0.0030971389263868333,0.008304112125188112,0.03794658705592154,0.02075930146733299,200,1.752171389456498,6.087494753142284,8.738320590790535,5.819001268052179,200,0.10951071184103113,0.38046842207139275,0.5461450369244084,0.3636875792532612,200,0.055401075464950114,0.1085563763291517,0.5422118357610629,0.1877648192653377,200,0.19496825573411575,0.29262047590010776,0.747166907083447,0.36448098245703575,200,0.5843116242282206,0.7016656202421432,0.913569825932818,0.7226050361869099,200,0.8159682137501002,0.8910714007365074,0.9691361571890349,0.8923148482975695,200,0.006723197825254624,0.03322221202127263,0.05967067303578807,0.03305582224158826,200,4.0,9.0,11.0,8.145
|
||||
gsm8k,finetuned,gsm8k-test-0003,11,QKV,200,0.7563514536712319,0.9393407981842756,1.1461985245347022,0.9479450973623897,200,0.10101253762841225,0.1592702493071556,0.21245212256908416,0.157736291103065,200,9.270882680749194,11.867156630713868,13.473537968956421,11.587095662914578,200,0.5794301675468246,0.7416972894196168,0.8420961230597763,0.7241934789321611,200,0.012501382655389326,0.02321728412038378,0.04838891883389339,0.027170381179980198,200,0.10956141992359565,0.16595287288384297,0.23324230310653973,0.16840136109520426,200,0.4001905395683069,0.4601116910586197,0.5363396197387118,0.46437827818468613,200,0.6664082269322316,0.7158185488281712,0.774714241792502,0.7174756168535806,200,0.07874648081487465,0.10563340826814678,0.13476727613613287,0.10663141540056405,200,12.0,13.0,13.0,12.61
|
||||
gsm8k,finetuned,gsm8k-test-0003,15,distance,200,0.4295443454757333,0.5734104805160314,0.7498907979577779,0.586723312784452,200,0.0669890098273754,0.10519488155841827,0.18502816706895828,0.11733639573678374,200,6.441997227143791,10.188401411961056,11.927975457275181,9.759906026883222,200,0.4026248266964869,0.636775088247566,0.7454984660796988,0.6099941266802014,200,0.022758960803838088,0.03802758720791913,0.098914655754623,0.04892331612127565,200,0.13906741259307187,0.18016665798977263,0.30137146659235003,0.19692711815030733,200,0.46017086838638616,0.529213735214271,0.664829681188094,0.5448050350931983,200,0.7251853995493751,0.7795307944042655,0.8505275980063635,0.7839610746947693,200,0.047963016920335846,0.08012342640208574,0.10684274799122943,0.0776805659537021,200,10.0,12.0,13.0,11.565
|
||||
gsm8k,finetuned,gsm8k-test-0003,15,divergence,200,0.016662569431628072,0.032940639666776406,0.07202167410214315,0.04041818663077223,200,0.004487527906894684,0.011065971571952105,0.03423575907945633,0.01683708645578008,200,2.3880326495705138,5.094403414238526,7.237603939171033,5.159227939741238,200,0.1492520405981571,0.31840021338990787,0.4523502461981896,0.3224517462338274,200,0.08072475094011035,0.1427134746676346,0.38000675004228024,0.17866622301538818,200,0.2233922026312352,0.3255782244901946,0.5999357249866467,0.36147216849491587,200,0.6527752226777753,0.759032867785312,0.9208308752005923,0.7718301087065623,200,0.8823638834628412,0.9273556421641516,0.9771034802109194,0.9281737917076157,200,0.004561893489529149,0.016760037784479258,0.034994067920115855,0.0183144217122206,200,4.0,7.0,9.0,6.985
|
||||
gsm8k,finetuned,gsm8k-test-0003,15,QKV,200,0.7072985640726983,0.8374831294640899,0.9885779065079987,0.8459054864535573,200,0.12306834831833839,0.15337084233760834,0.2151373878121376,0.16215382486581803,200,8.229837776641793,9.962016470677362,11.46495936234377,9.929909616200078,200,0.514364861040112,0.6226260294173351,0.7165599601464856,0.6206193510125049,200,0.026370460071159302,0.04040673209619527,0.06294309390407367,0.042900470182744836,200,0.14700419391719274,0.18259052984941754,0.25072556322180956,0.19377289930641317,200,0.47664278657960246,0.5399913382763476,0.5934678073821278,0.5368823604050937,200,0.734197317264381,0.7732465097933849,0.8052768190056181,0.772119632637532,200,0.05602510628972675,0.07933882539236736,0.10086108143229155,0.07911344996908057,200,11.0,12.0,13.0,11.755
|
||||
gsm8k,finetuned,gsm8k-test-0003,19,distance,200,0.26771306976443154,0.33227533265016973,0.42634317113552245,0.34049642127007246,200,0.06704426109790802,0.0932806208729744,0.12459056600928306,0.09490817861631512,200,6.1627355882223585,7.656634332381184,9.239830117794005,7.74571019934502,200,0.3851709742638974,0.478539645773824,0.5774893823621253,0.4841068874590638,200,0.04877788315619018,0.07264636302158933,0.10641684045650113,0.07462865704582351,200,0.21640599652820125,0.27603830270217417,0.3458514770991962,0.2796695664758131,200,0.5334560534920725,0.583725513540845,0.6371992595892969,0.5855955728955797,200,0.762581454099703,0.8050825206772072,0.8396489189629126,0.8035771955465697,200,0.052888284976666415,0.06820387527564459,0.08610631869644798,0.06874589292229309,200,10.900000000000002,11.0,12.0,11.27
|
||||
gsm8k,finetuned,gsm8k-test-0003,19,divergence,200,0.008871455248299753,0.014897460416250397,0.025881484209958214,0.01583732152597918,200,0.004494985658675433,0.008701286278665066,0.015522809978574515,0.009481437861686572,200,1.76100288150933,2.6083581372347813,3.862338176544516,2.7426088570788125,200,0.11006268009433312,0.16302238357717383,0.24139613603403226,0.17141305356742578,200,0.20952486659475578,0.3422752141335961,0.5390519094859183,0.3615454608865301,200,0.42948213405154256,0.5922372245861517,0.7481893822593269,0.592418977029386,200,0.793222175357056,0.8631009089563675,0.9148346307423885,0.8565486226659675,200,0.930158294095365,0.9591043125355214,0.9749798786364245,0.9552843880520487,200,0.005194491364800838,0.009480395985958116,0.01826286210287631,0.010579097658450992,200,4.0,6.0,7.0,5.555
|
||||
gsm8k,finetuned,gsm8k-test-0003,19,QKV,200,0.3676547828130424,0.4775542584247887,0.6094445672817528,0.4884013429749757,200,0.052512463927268986,0.06865526363253593,0.12836642414331437,0.08194584969431162,200,9.685653864338414,12.26922870691209,13.312573741082057,11.762377719956175,200,0.6053533665211509,0.7668267941820056,0.8320358588176285,0.735148607497261,200,0.013458568431858322,0.020271697033684293,0.04346187672183603,0.02685877411371549,200,0.12258908556823382,0.14453808728932344,0.22267996067044754,0.16432823919338574,200,0.4046657372802793,0.44753749859090836,0.526123743115573,0.4582862929478651,200,0.6699232575962171,0.707017986830023,0.7666821117759691,0.7120786809526352,200,0.08992306584961218,0.1168957798288247,0.136410570490733,0.11491187811483546,200,12.0,13.0,13.0,12.85
|
||||
gsm8k,finetuned,gsm8k-test-0003,23,distance,200,0.09935817614896222,0.28417803591582924,0.37923222952522334,0.2643441467973025,200,0.028821804933249952,0.07255387306213379,0.10645385906100273,0.07205177498050035,200,5.6202495526281675,6.9784230474642435,8.170355042125614,6.91612025107939,200,0.35126559703926047,0.4361514404665152,0.5106471901328509,0.4322575156924619,200,0.06388669507990437,0.0861855036940814,0.12312515080602256,0.09301512532322931,200,0.21332083433161755,0.2688029065534113,0.3430496911996064,0.275501185802812,200,0.5953768763458235,0.6593786348832855,0.7283359604714308,0.6615319214745645,200,0.8162381778791976,0.8489601026540176,0.8867810881208819,0.8508956509350392,200,0.032794261991790356,0.04859955569709458,0.06247326392672216,0.048734351565403566,200,9.0,10.0,11.0,10.09
|
||||
gsm8k,finetuned,gsm8k-test-0003,23,divergence,200,0.0016503044036355163,0.011521586967660369,0.0217239799672825,0.011740049440451159,200,0.0008308223972562702,0.005264068720862269,0.01133242743089795,0.006043745023926021,200,2.1142692154736813,2.9568628027901234,3.930511188290089,3.0036580202970202,200,0.13214182596710508,0.1848039251743827,0.24565694926813056,0.18772862626856376,200,0.20471480071058587,0.29408644525987093,0.4378420082602333,0.3140360683805292,200,0.35946126944941964,0.5018650810562272,0.6644975820122394,0.5112961403648437,200,0.8510560435639289,0.9100205497905488,0.9504723484661567,0.9053185485430089,200,0.9602096467702326,0.9763991044098504,0.9893247394467959,0.9752851464289312,200,0.0016607714111884528,0.004858256726883034,0.008637761765902977,0.00519518554467693,200,3.0,4.0,6.0,4.25
|
||||
gsm8k,finetuned,gsm8k-test-0003,23,QKV,200,0.34861789874266835,0.46796738798730075,0.6063007142860442,0.4789219241408864,200,0.07236264199018479,0.12218262255191803,0.1776002138853073,0.12679029785096646,200,6.26281897940081,8.095873729676617,9.803698756549137,8.096748938075045,200,0.3914261862125506,0.5059921081047886,0.6127311722843211,0.5060468086296903,200,0.04213650903026502,0.06508811261636015,0.10365370843686234,0.06877332581287648,200,0.1835115049445634,0.26368743089418345,0.33788105704136245,0.26130129865598223,200,0.5287599584236525,0.5789400767564707,0.6264378636545932,0.5772727630543738,200,0.7612311038252128,0.7942320673751508,0.821991571696186,0.7923559946386356,200,0.05784129274955745,0.06968791991604886,0.08734769807273109,0.07172628297071418,200,11.0,11.0,12.0,11.435
|
||||
gsm8k,finetuned,gsm8k-test-0004,3,distance,200,0.5756832751911134,0.7499919068068266,0.9677668931894005,0.7544594633276575,200,0.1142282247543335,0.16310137510299683,0.24235956668853756,0.17249117325991392,200,6.0316345880289015,9.384994519848487,11.111017274866331,8.975273461885028,200,0.37697716175180634,0.5865621574905304,0.6944385796791457,0.5609545913678142,200,0.029334160643042596,0.046989985965247796,0.11017990053060392,0.05898431722032111,200,0.157975964019326,0.21188413162834444,0.3539765316276721,0.23408385651066257,200,0.48908237925968284,0.5478114641331859,0.6296967266495378,0.55278621145791,200,0.7431249267264494,0.7860700607188297,0.8307309762619826,0.7866860941400933,200,0.05135562642108581,0.0711500076899122,0.09266480281613501,0.07197949183999697,200,11.0,12.0,12.0,11.5
|
||||
gsm8k,finetuned,gsm8k-test-0004,3,divergence,200,0.03809460926349857,0.05897535532403708,0.10592873184250493,0.06803958878985895,200,0.01304808733984828,0.026602058671414852,0.05874049514532088,0.03291173319565132,200,1.7365589988094015,3.9643298794330244,6.135239582911343,4.01704630578849,200,0.10853493742558759,0.24777061746456402,0.3834524739319589,0.25106539411178064,200,0.10719242931264207,0.20240263504127082,0.5475894816374566,0.2620421635714175,200,0.27008484619485185,0.4272469203090723,0.7491976381459611,0.46933982745287683,200,0.7080179135936912,0.8032069533527112,0.907389616437403,0.8029881585732767,200,0.8959748050643362,0.9388639639272056,0.9735247909387836,0.9367554925971178,200,0.0046558222313032825,0.012898225211093201,0.025043779993389383,0.014240304577339017,200,4.0,7.0,9.0,6.585
|
||||
gsm8k,finetuned,gsm8k-test-0004,3,QKV,200,1.1191820678766817,1.4518953387159854,1.9597307241056114,1.5018172950684674,200,0.24420124739408494,0.29378268122673035,0.3823872923851013,0.30761374153196813,200,7.264334458593829,9.030420913158412,11.479837922869994,9.289689728119999,200,0.45402090366211434,0.5644013070724008,0.7174898701793746,0.5806056080074999,200,0.026249860805769052,0.05145260512303765,0.0801695555315307,0.05145342578226913,200,0.15338677660633768,0.20636714610502782,0.2746146924537192,0.21032911764523834,200,0.47150421032261525,0.5551044741111731,0.6273329036926905,0.5529156562688198,200,0.7294694552044237,0.8010460938784545,0.8591347599111356,0.8006998310323533,200,0.025309134483376027,0.05513372526399209,0.101480739474427,0.05852195387788422,200,9.0,11.0,13.0,10.92
|
||||
gsm8k,finetuned,gsm8k-test-0004,7,distance,200,0.3922450354322791,0.6164290828164667,0.7942018928006291,0.605319182073581,200,0.07778256982564927,0.11393243819475174,0.17568108588457104,0.1253366791829467,200,6.263656119379643,9.63355281908866,11.908039052647277,9.412655302116377,200,0.3914785074612277,0.6020970511930412,0.7442524407904548,0.5882909563822736,200,0.022908683408180072,0.044057463703671265,0.10362792690251416,0.0537973748764424,200,0.14240012140668212,0.18838928703135394,0.3335819282201322,0.21266128039904725,200,0.45882577118571977,0.54694757275765,0.6347649335657862,0.548071841437539,200,0.732416298597809,0.7915579474594951,0.8392757120793444,0.7887364892165257,200,0.0505098239604359,0.07375148083819807,0.1007410653532591,0.0755842584777364,200,10.0,12.0,13.0,11.495
|
||||
gsm8k,finetuned,gsm8k-test-0004,7,divergence,200,0.019507185577276688,0.03862954997657653,0.069919358221523,0.04204623736435849,200,0.00605012821033597,0.012980866711586714,0.03086655940860509,0.017786418852047063,200,1.829583206929137,4.838482380493215,7.20957746662085,4.765589438411994,200,0.11434895043307106,0.3024051487808259,0.4505985916638031,0.29784933990074963,200,0.08128629690885146,0.1537892032386743,0.5163506079485244,0.21481115213520127,200,0.23311444506249146,0.34428156690860073,0.7304888813856234,0.4055184455290943,200,0.6482606794988804,0.7890288452156442,0.9053011538986285,0.7826642558691151,200,0.8844422789062003,0.9439373175509643,0.9731969075950471,0.9347875820039658,200,0.005157594860332973,0.0140839858443475,0.03165205660885028,0.016140449883359137,200,4.0,7.0,9.0,6.67
|
||||
gsm8k,finetuned,gsm8k-test-0004,7,QKV,200,0.9931196695193648,1.137864707969129,1.3605041407980025,1.1627634145203047,200,0.17549821138381957,0.2092011272907257,0.24060453474521634,0.20995012134313584,200,9.38076589603196,10.758191303206583,11.8190112386637,10.642399282779506,200,0.5862978685019975,0.6723869564504115,0.7386882024164813,0.6651499551737191,200,0.02358411799361814,0.03248265708334869,0.047041187967856335,0.03447842316808716,200,0.14548637417633886,0.1780432151803173,0.22644966544376066,0.18271423868411663,200,0.4542748473281149,0.48844992899987816,0.5381890054160203,0.49356033784948655,200,0.718519531311371,0.7538134703564812,0.8170713692857914,0.760040134782075,200,0.0501008365457001,0.08002014732871784,0.10190007251828063,0.08023039393300743,200,11.0,12.0,13.0,11.795
|
||||
gsm8k,finetuned,gsm8k-test-0004,11,distance,200,0.3446411787066609,0.5500103977974504,0.7538010963238775,0.5486787442339118,200,0.0533649779856205,0.08664434403181076,0.17741657197475424,0.10220501048490405,200,6.699066025376323,11.476824575536579,13.110653230039306,10.91597717481231,200,0.41869162658602016,0.7173015359710362,0.8194158268774566,0.6822485734257694,200,0.014692107855341236,0.026274323436599804,0.09260026036537027,0.038184725526432775,200,0.1222254816447473,0.1587926109879515,0.32427746127054735,0.1846807828501383,200,0.41063786398006236,0.4741947752849516,0.5925289880815718,0.48678183739823233,200,0.6848621350426006,0.7332422279681599,0.8073580406084817,0.7399096607856438,200,0.06357082076645575,0.09813610681154025,0.12387664588790143,0.09566654213514893,200,11.0,12.0,13.0,12.25
|
||||
gsm8k,finetuned,gsm8k-test-0004,11,divergence,200,0.010573522247068468,0.02709890310143237,0.06256201373107613,0.032448091242506506,200,0.002847854048013687,0.007507256232202053,0.031496046297252146,0.013569634797022445,200,1.949256580720226,6.247068669185887,9.088707827673533,6.066583656424812,200,0.12182853629501413,0.3904417918241179,0.5680442392295958,0.37916147852655074,200,0.0506952825783607,0.10408008905948454,0.4805523833944269,0.16446522326313215,200,0.19331236196707358,0.28965202744763474,0.7067138058924629,0.35140248322384776,200,0.5703452026835969,0.6887771052765679,0.8861214715565024,0.6996282058581562,200,0.8249623116560955,0.8869970802315508,0.9643489921895702,0.8880392452373451,200,0.008049799106913634,0.029463760025357236,0.049684298665120266,0.03008501938774778,200,5.0,9.0,11.0,8.415
|
||||
gsm8k,finetuned,gsm8k-test-0004,11,QKV,200,0.7807283835485578,0.9548521544784307,1.149836900830269,0.9675446304655634,200,0.1059288427233696,0.1474798172712326,0.20215321332216263,0.15088109008967876,200,10.061375315553175,12.336644789775654,13.390385335910425,11.902125859015653,200,0.6288359572220734,0.7710402993609784,0.8368990834944016,0.7438828661884783,200,0.012992733984683098,0.01979660502691609,0.03934983284929576,0.0243787029099852,200,0.12006352438842197,0.14401452574272866,0.20500393802418979,0.15745110872392065,200,0.401190699084348,0.4468455916768499,0.5093806662421958,0.4540370730666702,200,0.6739182157350955,0.7164334872181359,0.7756651787918942,0.7185702375773516,200,0.0783728919709507,0.10441698493544654,0.1340015910757365,0.10650022038126203,200,12.0,13.0,13.0,12.62
|
||||
gsm8k,finetuned,gsm8k-test-0004,15,distance,200,0.44083303215447817,0.588351801969111,0.7867047629319132,0.5944030670146457,200,0.07011627703905106,0.10753919556736946,0.17311558723449705,0.11631072660908102,200,7.312007702910328,9.852794303821298,11.742237926149206,9.658510811389618,200,0.4570004814318955,0.6157996439888311,0.7338898703843254,0.6036569257118511,200,0.02417353965364648,0.04159367183707042,0.07921220004897153,0.04815417779255547,200,0.14246494227833326,0.1840295091472801,0.2670327233128917,0.19684011000932672,200,0.46597368783077664,0.5374318878444485,0.6340241352683883,0.5445933391813845,200,0.7374009238000921,0.795149692432118,0.8585741559883213,0.7954948571222101,200,0.04516442829826715,0.06890139433431011,0.09765832967016902,0.07005500403844361,200,10.0,11.0,12.0,11.285
|
||||
gsm8k,finetuned,gsm8k-test-0004,15,divergence,200,0.018280529956791726,0.03572839953812945,0.06533679007698083,0.03943159756418879,200,0.004916293919086457,0.011564681772142649,0.02996946424245834,0.0150954563729465,200,2.850898444422446,5.131046211388287,7.369400044898987,5.103441175834548,200,0.17818115277640287,0.32069038821176793,0.4605875028061867,0.31896507348965925,200,0.07807707511442417,0.14121865325842203,0.307484402716816,0.17264751554429508,200,0.227038650189954,0.3320236057479202,0.5494237611526755,0.3638754864943645,200,0.6437374845307772,0.7704235772696899,0.8868746632277896,0.7680688105487624,200,0.8871795838320551,0.9403136514292346,0.976660502051696,0.9354806654393326,200,0.0043974943265127206,0.012459584794230798,0.029608474964594882,0.01475064685106209,200,5.0,7.0,9.0,6.94
|
||||
gsm8k,finetuned,gsm8k-test-0004,15,QKV,200,0.7122904821764677,0.8381739845499396,0.9551435619592666,0.8345206865045475,200,0.10450245887041092,0.14302034676074982,0.19997365176677698,0.1508120831474662,200,8.319808489915104,10.605316728401084,12.05317734089146,10.382929091569522,200,0.519988030619694,0.6628322955250677,0.7533235838057163,0.6489330682230952,200,0.021830164428878134,0.033912684727520855,0.06154213832378587,0.038270018657699974,200,0.12455257540796226,0.17905611192147655,0.24509992743281567,0.18279059295327188,200,0.44632796277065495,0.5064215706827015,0.5754003301449845,0.5085877814563154,200,0.7347463446652296,0.7742280249839504,0.811655367651109,0.7746879104383936,200,0.05360052669159922,0.07482765490303375,0.09995070086206517,0.0761512117558978,200,11.0,12.0,12.099999999999994,11.655
|
||||
gsm8k,finetuned,gsm8k-test-0004,19,distance,200,0.30828953003510834,0.37099468137603253,0.44660065032076085,0.3733190082694637,200,0.07051588594913483,0.09632141143083572,0.13741239458322524,0.10136590443551541,200,6.000130704998263,8.001926151257493,9.908720056232141,7.967192276692669,200,0.3750081690623914,0.5001203844535933,0.6192950035145088,0.4979495172932918,200,0.04098289226240607,0.06663475222921428,0.11110737737404704,0.07215691799759255,200,0.2061257947508828,0.262537761235126,0.34491015025649324,0.2704571537236373,200,0.5262321988808908,0.587500763614398,0.6619725910633351,0.5897823000428066,200,0.7628773286537666,0.8007662645898905,0.8508088822992783,0.8033677686289002,200,0.04753409145609897,0.07023263774181075,0.08833781700253777,0.06959970856481984,200,10.0,11.0,12.0,11.275
|
||||
gsm8k,finetuned,gsm8k-test-0004,19,divergence,200,0.010500463201208277,0.017314820954197785,0.02746363611968263,0.018665262287283893,200,0.004972490109503269,0.009277816861867905,0.01888216771185398,0.011013422089163213,200,1.8406731876750908,2.8533396382580043,4.299535668437,2.971239025356192,200,0.11504207422969318,0.17833372739112527,0.2687209792773125,0.185702439084762,200,0.18142315224412567,0.3071698108255384,0.512834606807871,0.33316490462375936,200,0.4113642260930746,0.5543392055653752,0.7215012419326087,0.563771223665337,200,0.7814981458891114,0.8622473022444368,0.92736856337255,0.8549538211370731,200,0.9230621638960219,0.9548391748120878,0.9802091670587428,0.9529188732310474,200,0.003915117761291869,0.010415227352510601,0.0197192982122433,0.011413864081149715,200,4.0,6.0,7.0,5.67
|
||||
gsm8k,finetuned,gsm8k-test-0004,19,QKV,200,0.36403866540640595,0.44772964669391513,0.6097449773922563,0.46817739385878665,200,0.04970967210829258,0.06649107486009598,0.098834590613842,0.07065259017050267,200,11.030094140944712,12.122524922263915,13.17271207841385,12.090744029512997,200,0.6893808838090445,0.7576578076414947,0.8232945049008656,0.7556715018445623,200,0.014308816696613777,0.021323904111726445,0.030038571736742358,0.022198962268575907,200,0.12330951220396662,0.14820269242231127,0.17752802038894624,0.15100469400909533,200,0.40954655448029,0.44876064437528973,0.49573765335257797,0.4526655152481838,200,0.6713474843538065,0.7040545136600656,0.7388951440833018,0.7067119444199474,200,0.09215335153997367,0.11725525204701003,0.1331890154203083,0.11294295948168662,200,12.0,13.0,13.0,12.715
|
||||
gsm8k,finetuned,gsm8k-test-0004,23,distance,200,0.09471694757812657,0.26578445394989103,0.3411585597321391,0.24245402665721486,200,0.027876036614179613,0.05792309530079365,0.09403472021222115,0.06014243890531361,200,5.555129591657033,7.766940178452877,9.339663468585192,7.66147814158342,200,0.34719559947856454,0.48543376115330483,0.5837289667865745,0.47884238384896377,200,0.04754158850097682,0.07066760085749252,0.1253480589631668,0.07829853345996467,200,0.19105026062667346,0.2419698470506887,0.338331129430703,0.2541039431088737,200,0.553663349572356,0.6273426997202768,0.7060342237379488,0.6283594997142591,200,0.7778767719651125,0.820821796627609,0.8637641558618221,0.8222334163740791,200,0.042407448324610665,0.05951974179697801,0.08123648742173223,0.060489309193147794,200,10.0,11.0,12.0,10.78
|
||||
gsm8k,finetuned,gsm8k-test-0004,23,divergence,200,0.001414323423176711,0.008126621263045308,0.01564682163220823,0.008844431355144806,200,0.000777118344558403,0.003355089691467583,0.008842604793608188,0.004283871050502057,200,2.066591841640581,3.3796688550088385,4.649042222746643,3.361946728813872,200,0.12916199010253632,0.2112293034380524,0.2905651389216652,0.210121670550867,200,0.16277301908924843,0.2489510901914282,0.4494887872235451,0.28311412843832956,200,0.32396723558814605,0.45403680393117685,0.6783667764426525,0.47884090887646624,200,0.8093973272190494,0.8895813124790914,0.9443782130807168,0.8823427377201208,200,0.9350008663064203,0.9638642016019726,0.9846245595410553,0.9618179175546815,200,0.0029936978609438284,0.0073919495789090605,0.015839011544442116,0.008460309216775081,200,3.0,5.0,7.0,4.93
|
||||
gsm8k,finetuned,gsm8k-test-0004,23,QKV,200,0.3733385583967902,0.4674871617462486,0.5743780856020748,0.4721059119142592,200,0.07522632703185082,0.11245330423116684,0.1619580239057541,0.1147119796089828,200,6.5512164623204,8.70580804466519,10.908337748407254,8.756348008284348,200,0.409451028895025,0.5441130027915744,0.6817711092754534,0.5472717505177718,200,0.03111865779595822,0.05586344436097393,0.09615296735193839,0.060068979000868,200,0.16380887821681717,0.23857915719068645,0.33245504509249607,0.24565029165819954,200,0.48858004042812936,0.5547735674141244,0.6014285793661123,0.5482254707636758,200,0.7254773396842396,0.7654038994241463,0.8013996857321585,0.7641423080063015,200,0.06646098895196735,0.08383203628442756,0.10621306354218488,0.08507041971740975,200,11.0,12.0,13.0,12.02
|
||||
|
@@ -0,0 +1,62 @@
|
||||
{
|
||||
"primary": "same six full-attention receivers and original4x32 token cohort as published8.931822/16",
|
||||
"adapter_repo": "Avifenesh/gdn-beta-gate-qwen3.5-2b-grpo",
|
||||
"adapter_revision": "038d8be9bd0edd3883fa019651b02bdf65609468",
|
||||
"adapter_file": "lora/seed0/adapter.safetensors",
|
||||
"adapter_sha256": "93a3f43a4356f84da0d13ecc622fee13e41ba15d237cdaad6094f243899c62d6",
|
||||
"base_revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
|
||||
"scale": 2,
|
||||
"rank": 16,
|
||||
"arm": "pure LoRA",
|
||||
"training": "publisher-reported GSM8K GRPO; we do no training",
|
||||
"sample_choice": "four original texts; first4 canonical GSM8K test questions with >=32tokens, length selection only, first32tokens; question only",
|
||||
"source_layers": [
|
||||
2,
|
||||
6,
|
||||
10,
|
||||
14,
|
||||
18,
|
||||
22
|
||||
],
|
||||
"primary_queries": [
|
||||
7,
|
||||
8,
|
||||
9,
|
||||
10,
|
||||
11,
|
||||
12,
|
||||
13,
|
||||
14,
|
||||
15,
|
||||
16,
|
||||
17,
|
||||
18,
|
||||
19,
|
||||
20,
|
||||
21,
|
||||
22,
|
||||
23,
|
||||
24,
|
||||
25,
|
||||
26,
|
||||
27,
|
||||
28,
|
||||
29,
|
||||
30,
|
||||
31
|
||||
],
|
||||
"main_metric": "concentration(sqrt(single_JS)); same FP32 tensors and sequence-wide donor-head deletion as prior run",
|
||||
"cohort_pairing": "each comparison uses same question/tokenIDs before and after; base legacy arrays reused with elementwise checks",
|
||||
"runtime": "read-only FP32 effective W+2BA; no BF16 re-rounding or quantization",
|
||||
"controls": "head sum, null mask, explicit low-rank versus effective-weight full forward relL2<1e-4",
|
||||
"budget_seconds": 7200,
|
||||
"keep_all_weights_and_caches": true,
|
||||
"limitations": [
|
||||
"one existing training seed",
|
||||
"4 texts per cohort, 32 tokens each",
|
||||
"dependent query/head observations, not iid textual samples",
|
||||
"no generation or accuracy measurement",
|
||||
"GDN targets only; ordinary attention weights stay unchanged",
|
||||
"no pruning or functional interchangeability claim"
|
||||
]
|
||||
}
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user