Validate repeated learned RMSNorm functions with CPU evidence

Reviewed-on: #2

.gitattributes

@@ -3,3 +3,4 @@ results/** -text
 # Generated CSV uses RFC 4180 line endings; Matplotlib SVG paths contain spaces.
 results/**/*.csv -whitespace
 results/**/*.svg -whitespace
+results/**/*.log -whitespace

.gitignore

@@ -3,3 +3,8 @@ __pycache__/
 weights/
 *.partial
 *.log
+
+# Large reproducible functional activations remain on the experiment server.
+results/cpu-qwen35-2b/baseline-activations.npz
+results/cpu-qwen35-2b/neuron-reuse-activations.npz
+results/cpu-qwen35-2b/norm-repetition-input-profiles.npz

README.md

@@ -1,9 +1,16 @@
 # Model parameter similarity experiments
 
-Bounded, inference-free analysis of official unquantized model weights. The first experiment
-is **Qwen3.5-2B**. Model weights are downloaded and analyzed on the authorized remote host,
-then deleted after archived results pass validation. No model weights are committed here.
+Research on repeated learned functions in **Qwen3.5-2B**: a completed static weight baseline
+and a training-free CPU functional pilot. The CPU study found similar existing RMSNorm
+functions at layer indices 12/13, with a uniform analytic error bound and heldout substitution checks.
+Cost is not a success condition. The completed static pilot was inference-free and its
+weights were deleted after archival; the functional pilot executes streaming FP32 text forwards.
+The current CPU functional checkpoint is retained until the user explicitly requests deletion.
+No model weights are committed here.
 
+- [CPU learned-function repetition findings](docs/CPU_FUNCTIONAL_FINDINGS.md)
+- [Current learned-function repetition criterion](docs/LEARNED_FUNCTION_REPETITION.md)
+- [CPU-first feasibility and validation](docs/CPU_FIRST_VALIDATION.md)
 - [Training-free functional reuse plan (proposed; not executed)](docs/TRAINING_FREE_FUNCTIONAL_PLAN.md)
 - [Primary-source literature and applicability](docs/FUNCTIONAL_REUSE_LITERATURE.md)
 - [Procedure and limits](docs/SOP.md)
@@ -18,9 +25,10 @@ then deleted after archived results pass validation. No model weights are commit
 - `pilot/run.py`: one-model supervisor with bounded download wait and analysis timeouts.
 - `tests/test_pilot.py`: synthetic numerical and 3D correctness tests.
 
-The primary result is **verified net byte savings versus direct relative weight error**.
-This does not demonstrate preserved task capability or faster inference. Different block
-scales are alternative partitions and their savings must not be summed.
+The static baseline reports **verified net byte savings versus direct relative weight error**.
+The functional study separately reports same-input error, an all-input RMSNorm function bound
+and tiny heldout NLL changes; these are not broad task-accuracy guarantees. Different static
+block scales are alternative partitions and their savings must not be summed.
 
 Dependencies are pinned in `requirements.txt`; install only prebuilt wheels on the remote host.
 Full resolved versions and run provenance accompany each result. See the SOP for commands.

docs/CPU_FIRST_VALIDATION.md

@@ -0,0 +1,75 @@
+# CPU 先导：先找值得继续验证的信号
+
+更新：2026-11-09。目的不是在 CPU 上完成广泛能力及高吞吐评测，
+而是低成本判断实际输入分布上是否存在值得 GPU 进一步验证的复用候选。
+全程不训练，不求 R。
+
+## 已执行的容量检查
+
+服务器 8.211.154.134：2 核 x86_64，约 1.7 GiB RAM；检查时可用约 892 MiB，
+swap 已用约 2.2 GiB，根盘可用约 73 GiB。共享主机快照不是实验独占资源。
+NumPy 2.5.4，OPENBLAS_NUM_THREADS=1、OMP_NUM_THREADS=1，远程 timeout 60 秒。
+随机种子 42、float32、X 为 32×2048；每种 W 随机生成后连续执行 4 次 X@W，
+第 1 次作为冷调用，其余 3 次单独报告。随机输入仅用于容量测量。
+
+| W 形状 | W 大小 | 首次乘法 | 后续三次乘法（秒） |
+|---|---|---|---|
+| 2048×2048 | 16 MiB | 0.027816 | 0.027035、0.027207、0.025878 |
+| 2048×6144 | 48 MiB | 0.086039 | 0.082543、0.110806、0.075336 |
+
+输出均为有限数，进程峰值 RSS 88,220 KiB（约 86.2 MiB）。
+耗时只包含矩阵乘法，不包含权重生成、磁盘读取、BF16 转换、模型其余操作及框架开销。
+它支持继续评估小批量 CPU 投影计算，不足以外推完整模型时延或宣称发现可复用性。
+没有模型权重参与，没有编译、下载或模型前向。
+
+## 执行路线与证据等级
+
+1. **实现与资源可行性**：核对固定 revision 的位置编码、归一化、GQA 和门控，
+   使用受信任的算子实现及小数组正确性检查；模型仓库代码不直接执行。
+   先确认首个短样本能以受控内存运行。不用不正确的“简化 Qwen”产生看似有价值的结果。
+2. **真实分布上的候选**：batch=1、32 tokens，校准和留出各 4 段真实文本。
+   按顺序加载、计算、释放每层，保留很小的入口激活；探测预先固定的 3 个 full-attention 层对、2 个 MLP 对，以及 6 个 full-attention 层内的 Q/K 归一化模块对，共 11 对。
+   每对在两个来源层的隐藏输入上分别测试，记录 JS、top-k、完整分支输出误差。
+3. **最小干预证据**：最多对 1 个候选做单处替换；冻结参数，在留出文本上看输出变化，
+   条件允许时流式计算 token NLL。词表输出头分块归约 logsumexp，避免一次物化大 logits。
+   若没有正确且可承受的干预实现，就止于“候选”，不称为验证成功。
+4. **GPU 细化**：候选通过才扩展长度、样本、线性注意力、多处共享及质量—成本曲线。
+
+先导主证据来自真实文本产生的隐藏状态，而不是高维空间上的均匀随机向量。
+直接按真实样本出现频率聚合，就能近似按目标使用分布加权，不先引入昂贵的高维密度估计。
+保留逐文本及尾部误差，少量低频但关键用例不被平均值掩盖。
+
+## CPU 内存与停止约束
+
+- 不同时加载完整模型的 BF16/FP32 副本；模型原始权重约 4.55 GB，大于当前 RAM。
+- 单层/子矩阵流式读取，及时解除映射或释放页面；先设本进程 RSS 512 MiB 软停止线，
+  系统可用内存低于 256 MiB 或持续 swap 活动就停止并保存进度，不改全局 swap 配置。
+  软停止线不替代实际峰值检测，分配前须计算临时内存。
+- 先实现最小正确路径；线性状态递推、长上下文和生成速度不塞入首轮。
+- 总计算 120 分钟，阶段预算见功能方案；无候选、超时或数值/实现核对失败均停止。
+- 容量探测已完成，真实模型阶段尚未执行。后续需要恢复本实验自有的固定 revision 权重，
+  权重持续保留，只有用户明确要求删除时才清理；不增加模型规模或自动租用 GPU。
+
+CPU 上“找到输出接近的候选”能支持继续投入 GPU；
+“在指定小样本中单处复用保持质量且节省明确资源”才是初步正例。
+两者都不能替代大规模能力验证。CPU 时延不外推为 GPU 加速率。
+
+## 当前权重保留规则
+
+用户于 2026-11-09 明确要求：本轮模型权重保留，只有用户明确要求删除时才删除。
+试验完成、超时、失败、归档或结果校验均不得自动触发清理。该要求覆盖先前逐模型分析后删除的约定。
+上一轮已删除的历史记录保留；本轮恢复下载的权重受新规则保护。
+
+## 本轮执行补充
+
+实际实验分支为 `codex/cpu-reuse-experiment`。在读取真实权重/功能指标前，新增固定的
+6 个层内 Q/K RMSNorm 增益共享候选，以检查较小粒度的功能复用；不是看过留出结果后扩充搜索。
+最多选择一个候选，必须分别验证两个替换方向；完整模块失败不会被小模块通过所掩盖。
+归一化共享最多节省少量字节，不预设能显著减少计算。
+采集真实隐藏输入必须正确执行前置线性注意力；它不进入首轮复用候选，但其递推计算纳入官方参考对照。
+
+## 用户最新目标修订
+
+不再以成本/加速作为成功条件；主问题是现有学习部分的功能重复。
+新增一次固定四层的 MLP 神经元函数搜索、独立留出和双向替换，
+见 [功能重复判据](LEARNED_FUNCTION_REPETITION.md)。解析增益合并单独归类，不冒充学习部分重复。

docs/CPU_FUNCTIONAL_FINDINGS.md

@@ -0,0 +1,86 @@
+# CPU 先导结论：存在已训练归一化模块的相似功能重复
+
+本轮主目标按用户最新要求为功能重复的存在性，不以成本/加速为成功条件。
+全部计算在 2 核、约 1.7 GiB RAM 的 CPU 服务器完成，无训练、缩放拟合或旋转搜索。
+层编号均为从 0 开始的索引。
+
+## 一个具体正例
+
+固定官方 Qwen3.5-2B checkpoint 的两个独立已训练模块：
+
+- `model.language_model.layers.12.input_layernorm`
+- `model.language_model.layers.13.input_layernorm`
+
+二者均为 2048→2048 的 RMSNorm，eps=1e-6，使用训练后的有效增益 gamma=1+weight。
+增益标准差分别约 0.1901、0.1685，最小值约 0.5547、0.5703；不是零函数或恒定增益。
+
+| 证据 | 结果 |
+|---|---|
+| 全输入域数学界（相对于层12输出） | 输出相对 L2 误差≤10.1852% |
+| 反方向全输入域数学界（相对于层13输出） | 输出相对 L2 误差≤11.3402% |
+| 4 条留出文本、两个来源层输入的最坏双向误差 | 4.5285% |
+| 相同留出比较中的最低余弦 | 0.999487 |
+| 层12使用层13，留出平均 token NLL 增量 | +0.0000325651 nat/token |
+| 层13使用层12，留出平均 token NLL 增量 | −0.0003639730 nat/token |
+
+两方向逐文本 NLL 增量 P95 分别约 0.00098464、0.00070972，均通过原定小样本质量门槛。
+所有数值来自保存的逐文本、逐 token 原始数据。留出为 4 条不同的实验原创短文本，
+每条截取 32 tokens；校准另有 4 条。它们不是广泛任务能力评测。
+
+## 为什么不只是“权重看起来接近”
+
+对相同 eps 和维数，f_a(x)=diag(a)x/sqrt(mean(x²)+eps)。任意非零实数输入满足：
+
+||f_a(x)-f_b(x)|| / ||f_a(x)|| ≤ max_j |a_j-b_j|/|a_j|。
+
+共同归一化分母消去；坐标基向量可达到最大值，因而该界是精确的最坏情况。
+反向分母换成 |b_j|。本轮先穷举 24 层同角色 input/post-attention norm 的 552 对，
+仅按最小双向最坏界选定一对，未使用任何输入样本作选择。≤20% 界有 18 对，≤10% 无配对；
+本轮只验证最优的一对，不根据留出结果换选。
+
+这个确定性界直接给出两个已训练函数整体接近的证据。真实输入上的误差更小，
+与只关心目标输入分布的目标一致。浮点实测与实数数学界分别报告。
+重新采集输入时，未修改的完整前向逐 bit 复现了最初保存的基线输出。
+输入探测分别使用两个来源层，对相同输入同时应用两套增益，避免比较不同输入的输出。
+从已加增益的激活还原共同归一化输入时有额外 FP32 舍入，未将其称为 bitwise 等价。
+
+结论限定为两个已有 RMSNorm 模块实现相似映射；不把共同包含 RMS 操作本身当作证据。
+二者学习增益不同且非恒定，输出非零。完整 attention/MLP 主体的重复仍未得到正例。
+
+## 已完成的阴性搜索
+
+1. 原定 3 对完整 attention、2 对完整 MLP、6 对层内 Q/K norm 均未通过预设门槛。
+   attention 检索分布 JS 范围约 0.145～0.318，也未通过 0.01 的门槛；
+   未进行头排列、连续坐标变换或训练对齐，不能外推为不存在其他可复用表示。
+2. 在层 5、6、20、21 各 6144 个 MLP scalar 子函数中，排除低活跃和近常量单元后，
+   共精确筛查 7,450,139 个唯一同层配对。要求双向相对误差≤20%、余弦≥0.98、
+   Pearson≥0.95，无通过者。没有针对留出集不断更换候选或放宽门槛。
+3. 这些结果只覆盖声明的模块、输入和阈值，不否定其他粒度/角色的功能重复。
+
+## 单独归类的解析恒等式
+
+Q/K 未参与 RoPE 的 192 个维度，其增益只以乘积影响点积，可合并到任一侧。
+96 组真实输入检查最大分数差 3.8147e-6、最大概率差 8.0466e-7；固定第3层两个方向
+留出 NLL 变化接近 FP32 舍入。详见 [推导](QK_GAIN_FACTORIZATION.md)。
+这是重参数化等价，不作为“两个已有学习部分功能重复”的正例。先前性能测试原始数据
+保留用于溯源，成本不属于当前结论的成功条件。
+
+## 正确性、资源与保留规则
+
+- 固定模型 revision：15852e8c16360a2fea060d615a32b45270f8a8fc，原始权重 SHA256 校验通过。
+- 流式读取原始 BF16，逐块转为 FP32，不同时载入全模型。
+- 与固定 Transformers 官方定义比较的 22 项检查通过，最大差约 5.4e-7。
+- 保留早期参考适配器错误、临时分区空间不足及两次过于保守的换页停止记录。
+  只归还自身临时堆页，未改变系统 swap 设置；后续有界前向完成。
+- 计算预算跨重启累计，初始保守计入 180 秒先前检查预留；具体时长及峰值见归档日志。
+- 当前权重按用户最新指令持续保留，只在用户明确要求删除时清理。
+
+原始表、样本/token IDs、激活、NLL、证明界与图表见 `results/cpu-qwen35-2b/`。
+后续 GPU 阶段可增加长度、语言和任务来验证正例的稳健性；本轮不自动启动新模型或扩展搜索。
+
+## Git 交付与保留缓存
+
+全部图表源 CSV/JSON、逐 token NLL、证明界和日志已交付。三份较大的中间激活缓存
+保留在原服务器及本地完整归档，路径、大小、SHA256 见结果目录的 external-activation-cache.json。
+其完整原实验校验表为 checksums-full-server.json；Git交付包校验表为 checksums-functional.json。
+没有删除模型或缓存；这些缓存可由固定代码和样本重新生成，也可从服务器按哈希取回。

docs/LEARNED_FUNCTION_REPETITION.md

@@ -0,0 +1,58 @@
+# 当前目标：已有学习部分是否重复实现相似函数
+
+用户于 2026-11-09 明确：不以成本或加速作为成功条件，只关心相似功能重复是否存在。
+权重继续保留，只有用户明确要求删除才清理；不训练、不拟合旋转或缩放。
+
+## 什么可以作为正例
+
+两个不同的已训练部分，在同一真实输入上产生接近、且并非恒定/接近零的输出；
+独立留出输入仍满足预先固定的门槛。实际双向替换只作补充验证。
+不能仅因替换一小部分后整模型 NLL 几乎不变，就认定它们实现相同函数。
+
+Q/K 增益的解析合并证明同一计算的重参数化自由度，不等于两个现成学习部分功能重复。
+前一轮 11 对完整/归一化模块未通过门槛，也不能否定更小粒度的重复。
+不将两个模块都包含 SiLU、矩阵乘法等共同运算本身作为重复功能证据。
+
+## 新增一次有界搜索：MLP 学习神经元
+
+在已保存的第 5、6、20、21 层真实 MLP 入口输入上，研究
+
+f_(l,j)(h) = SiLU(w_gate_(l,j)^T h) × (w_up_(l,j)^T h)。
+
+每层 6,144 个神经元具有不同学习参数。比较同一层内两个 scalar 子函数，
+不是比较来自不同输入的两个观测值，也不是把整层或整 tensor 称作重复。
+下游 down-projection 的列保持各自原样；替换发生在两个投影和 SiLU 之后的激活值处。
+
+1. 4 条校准文本、4 条留出文本，每条 32 tokens，中文/英文各半，沿用先前冻结样本。
+2. 校准阶段只保留 RMS 至少为该层神经元中位数、std/RMS≥0.25、能量有效 token 数≥16
+   的非平凡单元。能量有效样本数定义为 (sum f²)² / sum f⁴，不是独立统计样本量。
+3. 对这些单元做分块穷举；实际配对数必须记录。不用随机 hidden 向量作主要证据。
+4. 预先固定门槛：双向直接相对 L2≤20%、余弦≥0.98、Pearson≥0.95；同时输出 5/10/20%
+   三档校准配对数量。没有拟合线性缩放、偏置或 R。
+5. 全部层中仅按校准的最小最坏方向误差选择 1 对，冻结选择；无通过者停止。
+6. 留出集要求同样的汇总函数门槛及活跃度条件，逐文本双向误差≤35%。不根据留出失败换候选。
+7. 只有函数验证通过，才做两个方向的独立网络替换；留出平均 NLL 增量≤0.01 nat/token，
+   逐文本增量 P95≤0.05。它不是主要的功能重复判据，防止因单元贡献很小而得到假正例。
+
+本阶段使用先前两小时累计预算的剩余额度，复用现有隐藏输入；不增加模型、样本规模或成本测试。
+神经元层面的经验正例不证明完整层可以互换、不证明全输入空间等价，也不能推断独立任务能力。
+留出只有 4 条文本，结论必须限定到此短文本先导分布；后续扩展文本才能检验稳健性。
+
+## 同角色学习 RMSNorm：确定性函数误差界
+
+静态试验已经发现部分语言层 post-attention norm 的参数接近。新增一次独立检验：
+比较 24 层各自的 input/post-attention RMSNorm（同角色内共 552 对），
+仅按训练后增益的全输入最坏误差界选 1 对，完全不依赖校准或留出输入。
+增益需非零且不是常量向量（std≥0.001），不把没有学习参数的共同归一化运算当作正例。
+
+相同 eps、相同形状下，f_a(x)=diag(a)x/sqrt(mean(x²)+eps)。对任意非零 x，
+
+||f_a(x)-f_b(x)|| / ||f_a(x)|| ≤ max_j |a_j-b_j|/|a_j|。
+
+反方向分母换为 |b_j|。该上界由相应坐标基向量达到，是实数算术下的精确最坏界。
+因此可先证明两个现成学习模块的函数整体接近，而不只比较原始参数 w；实际增益为 1+w。
+选择门槛：最坏双向界≤20%；同时记录≤5%、≤10%、≤20% 配对数量。只选最优界的一对。
+随后重新采集其实际输入，要求留出逐文本直接双向输出误差≤5%，再测两个方向网络替换。
+函数接近与替换后网络 NLL 分开报告；后者沿用已有 0.01/0.05 门槛，不决定函数存在性本身。
+选定后不根据留出结果换配对；不搜索视觉模块、不增加模型、不以成本为条件。
+归一化层正例不能推广为 attention 或 MLP 主体存在同样重复。

docs/QK_GAIN_FACTORIZATION.md

@@ -0,0 +1,46 @@
+# Q/K 未旋转维度的确定性增益合并（已数值验证，归为重参数化）
+
+这是从已核对架构直接推导的恒等式，不是看留出指标后扩大近邻搜索。
+原定 11 对候选保留原协议；本项是独立的解析对照，不训练、不求 R，也不选择层。
+预先固定完整网络干预层为第 3 层，分别验证两个移动方向；不根据数据换层。
+
+## 恒等式与前提
+
+以完成 RMS 单位化、尚未乘可学习增益的 q/k 为 u/v，Q/K 增益向量为 a/b。
+固定 checkpoint 的 norm 参数实际使用 a=1+w_Q，b=1+w_K。
+模型 head_dim=256，partial_rotary_factor=0.25：前 64 维旋转，后 192 维不旋转。
+逐头共享的增益向量与 GQA 结构已由官方代码确认。
+
+对于后 192 维，任意 query/key 位置 p/s：
+
+sum_j (a_j u_pj)(b_j v_sj) = sum_j u_pj ((a_j b_j) v_sj)
+                         = sum_j ((a_j b_j) u_pj) v_sj。
+
+前 64 维保留原增益和位置旋转。因而完整 QK 点积在实数算术中相等，softmax、V、
+输出门控和 O 不变时，完整 attention 分支也是同一个函数。这个结论不依赖某一输入分布。
+浮点乘法次序改变，不能承诺逐 bit 相等；需报告实际误差并验证留出 NLL。
+不能把这一规则用于旋转维度，或未经核对的逐头独立增益架构。
+
+## 验证与成本边界
+
+1. 在已有真实 q/k 输入上，检查 6 个 full-attention 层的原始/两个合并方向分数、概率及输出。
+2. 固定第 3 层进行两个方向的独立网络替换，不叠加，使用原留出文本和质量阈值。
+3. 记录预先缓存增益后、相同 CPU 环境下原版与合并版局部归一化耗时；不外推 GPU 性能。
+4. 双精度数学推导和 FP32 实测分开。没有消除 Q/K 投影或 RMS 方差计算。
+
+将 Q 的未旋转增益移入 K 时，每 token、每层少做 8×192=1,536 次逐元素增益乘法；
+反方向少做 2×192=384 次。它们只是归一化里的小部分操作，不是整个模型的百分比加速。
+NumPy 切片/拼接可能抵消这点收益，必须报告测得的局部时延。
+
+参数量从两个 256 维增益变为两个 64 维旋转增益及一个 192 维联合增益，即 512→320 个数。
+但原增益以 BF16 存储、联合乘积以 FP32 保留时，载荷仍是 1,024 字节，没有 checkpoint
+净压缩；元数据还可能增加。FP32 运行时缓存增益的理论载荷 2,048→1,280 字节，
+不等于已测得整个模型 RSS 降低。另行 BF16 量化乘积会增加误差，不属于本次恒等式验证。
+
+这是同一注意力计算内部的重参数化与冗余消除，不是两个现成 attention 层能互换的证明。
+若得到正例，应将它作为小粒度确定性优化机会；较大模块互换仍由原候选试验单独回答。
+
+## 完成状态
+
+96 组真实输入分数/概率检查及固定第3层双向网络检查均通过。
+该结果独立归类；主目标的已训练函数重复正例见 [CPU 功能结论](CPU_FUNCTIONAL_FINDINGS.md)。

docs/SOP.md

@@ -121,3 +121,20 @@ validation. See the [bounded plan](TRAINING_FREE_FUNCTIONAL_PLAN.md) and
 [primary-source literature](FUNCTIONAL_REUSE_LITERATURE.md). No forward pass, new download,
 or new model is started by adopting this document. The original static pilot remains
 inference-free; the proposed follow-up requires a separately confirmed execution environment.
+
+## Current functional-run retention override
+
+On 2026-11-09 the user explicitly changed retention: keep the CPU functional pilot
+checkpoint until the user explicitly requests deletion. Completion, timeout, failure
+and archive validation do not authorize cleanup. The earlier pilot cleanup remains
+a historical action; do not apply its automatic cleanup step to the current run.
+
+## CPU functional follow-up outcome
+
+The executed CPU study found a learned RMSNorm function-repetition example at text
+layer indices 12/13. Use same-input function error and a uniform analytic bound as
+primary evidence, with separate heldout network substitution checks. Costs are not
+a success condition under the latest user instruction. See the
+[executed findings](CPU_FUNCTIONAL_FINDINGS.md) and
+[function-repetition criterion](LEARNED_FUNCTION_REPETITION.md).
+Keep whole-module, neuron, normalization and algebraic-reparameterization results distinct.

docs/TRAINING_FREE_FUNCTIONAL_PLAN.md

@@ -1,6 +1,6 @@
 # 免训练功能复用：下一阶段有界方案
 
-状态：研究设计，尚未执行。原始参数试验已完成；没有新增前向、下载或训练。
+状态：研究设计，真实模型功能探测尚未执行。原始参数试验已完成；另已完成单线程合成 CPU 容量探测，未新增模型前向、下载或训练。
 目标：寻找并验证至少一个在留出数据上保持约定质量、同时降低明确运行成本的复用实例；
 若没有通过者，报告有界搜索的阴性结果，不保证一定能找到。
 
@@ -64,26 +64,32 @@ CKA 仅作辅助筛选；共享位置模板、attention sink 和共同残差都
 
 ## 5. 资源与阶段预算
 
-当前服务器实际约 1.7 GiB RAM，不按“60 GB 内存”安排前向；60 GB 是磁盘扩容。
-后续需要用户确认可执行前向的远程环境；建议先评估至少 16 GiB RAM/显存的单模型环境，
-这不是已验证最低配置。先做内存估算及一次短样本容量探测，不足即停止，不自动换机器。
-不在本机编译、不自动重下已删除权重。本轮只交付设计。
+采用 CPU 先导、GPU 后续细化的两阶段路径；GPU 不作为寻找初步证据的前置条件。
+当前服务器为 2 核 CPU、约 1.7 GiB RAM；60 GB 是磁盘扩容。详见
+[CPU 先导设计及实测](CPU_FIRST_VALIDATION.md)。
 
-后续若获执行授权，首轮固定模型/revision/dtype、batch=1、长度最多 256，
-校准和留出各 16 条固定且去重的文本序列（中文/英文各半），只处理文本路径。
-数据来源、许可、tokenizer revision、截断、样本 hash 和抽样 seed 必须先入 manifest。
-每序列至多 32 个固定 query 位置，保留这些行的全部合法 key，避免采样 key 改变 softmax。
-流式聚合，不保存全部层的 N×N attention，不调用生成式聊天作替代评测。
+CPU 阶段仅处理文本，batch=1、长度 32，校准/留出各 4 条真实且去重的文本；
+仅检查 3 个预先固定、覆盖浅中深的 full-attention 层对和 2 个 MLP 层对。
+完整保留合法 key，逐头比较，不通过采样 key 改变 softmax。
+模型依赖实现一致性检查和容量探测完成后，才实施真实前向。
+权重按层/子矩阵流式读取并释放，禁止全模型转换成 FP32；不把依赖换页的完整框架加载视为可行方案。
+数据来源、许可、tokenizer revision、截断、样本 hash 和 seed 在运行前写入 manifest。
 
-| 阶段 | 上限 | 完成条件 / 停止条件 |
+| CPU 阶段 | 上限 | 完成条件 / 停止条件 |
 |---|---|---|
-| 容量探测及基线 | 15 分钟 | 无 OOM，固定输出重复测量可复现；否则停止 |
-| 同输入探测与筛选 | 45 分钟 | 固定 27 对候选；每类别最多保留 1 对，总计最多 3 对；无通过者即停止后续干预 |
-| 单处干预与质量 | 45 分钟 | 逐个候选，不叠加；留出集测 token NLL 和逐序列变化；失败不追加搜索 |
-| 成本测量和归档 | 15 分钟 | 保存基线/干预的质量、内存和计时，超时则输出已完成覆盖率 |
+| 实现一致性及短样本容量探测 | 20 分钟 | 模块计算语义核对，内存峰值可控；OOM、持续换页或实现核对失败即停止 |
+| 真实隐藏输入采集 | 40 分钟 | 流式文本前向采集固定入口，不保存所有层权重/注意力图；超时保存覆盖率 |
+| 同输入功能比较 | 30 分钟 | 固定 5 对候选；无通过者停止；最多留下 1 个干预候选 |
+| 单处干预及归档 | 30 分钟 | 在 4 条留出文本上检查输出或流式 token NLL；不足以完成则只交付候选证据 |
 
-总计算上限 120 分钟；下载单独记录，不触发无限重试。16 条留出序列只是先导证据，
-不能据此宣布广泛能力保持。首轮通过后才另立更大留出任务验证计划。
+CPU 首轮计算上限 120 分钟，下载单独计时。当前仅执行了约 88 MiB 峰值 RSS 的
+合成矩阵乘法探测，没有重下权重或执行真实前向。容量探测不能保证完整模型实现的峰值内存。
+线性注意力需另行完成正确性核对，首轮先不加入，避免状态递推实现拖延初步判断。
+短文本小样本无法覆盖长上下文或低频关键能力；阴性结果也不能排除更广泛候选。
+
+只有发现值得跟进的候选，GPU 阶段才扩展到原定 27 对候选、长度 256、
+校准/留出各 16 条及更多独立任务，细化单处/多处干预与真实性能。
+届时由用户提供并确认 GPU 执行环境；当前不租用或假定已有 GPU。
 
 ## 6. 干预必须对应可节省的对象
 

functional/README.md

@@ -0,0 +1,91 @@
+# Bounded CPU functional pilot
+
+This is the execution implementation of the CPU-first plan, not the completed static
+parameter pilot. Frozen Qwen3.5-2B weights are decoded to FP32 a projection block at a
+time. CPU only, no autograd/optimization, no rotation fitting, no model-hub Python code.
+The user explicitly requires retaining this checkpoint until they request deletion.
+
+## Fixed scope
+
+- Batch of eight distinct experiment-authored plain texts, first 32 tokens each:
+  four calibration and four heldout, two Chinese/two English in each split.
+- Full attention pairs: (3,7), (11,15), (19,23).
+- MLP pairs: (5,6), (20,21).
+- Smaller-granularity Q/K RMSNorm pairs inside layers 3,7,11,15,19,23.
+  Added before observing real weights/functions for this pilot; these are small storage
+  opportunities, not a claim of substantial compute savings.
+- At most one calibration-selected candidate, two separate substitution directions.
+  Selection prefers component size, then worst symmetric output error. No heldout-driven retry.
+- Same-input relative L2 <=0.05 in both directions and token P95 <=0.1 in both directions;
+  attention additionally requires mean JS <=0.01 and top-5 overlap >=0.8.
+- Heldout mean NLL increase <=0.01 nat/token and per-sequence P95 increase <=0.05.
+  These are pilot decision criteria, not mathematical capability guarantees.
+
+## Source and checks
+
+`reference.json` pins the official Transformers Qwen3.5 source commit, URL and hash.
+`reference_check.py` extracts only named mathematical definitions from that exact file,
+removes optional fused-kernel decorators, and compares the NumPy implementation against
+Torch operators, full toy decoder layers and a two-layer text forward. Chunked delta and
+recurrent delta implementations are both checked, including sequence padding boundaries.
+The official source is Apache-2.0, copyright Qwen Team and HuggingFace; the reference file
+is fetched for testing and is not bundled here. FP32 equivalence is checked, not equality
+to every deployed BF16 kernel.
+
+`tests/test_functional.py` independently checks streamed projection/embedding/NLL,
+causality, causal convolution, normalization conventions and basic metric identities.
+These tests use tiny synthetic arrays. Real model evidence is produced only by `run.py`.
+
+## Remote execution
+
+Use the previously authorized CPU host and existing NumPy/Matplotlib environment. Install
+only binary wheels: tokenizers==0.23.3 from HTTPS PyPI; torch==2.15.0+cpu from the official
+CPU wheel index for independent reference checks. Use an experiment-owned on-disk TMPDIR:
+the shared host's /tmp tmpfs was too small during the first wheel download. Do not change
+system swap or shared temporary files.
+
+```sh
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_functional -v
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m functional.reference_check --source runs/cpu-qwen35-2b/reference.py --output runs/cpu-qwen35-2b/reference-check.json
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m functional.supervise --root runs/cpu-qwen35-2b
+```
+
+The supervisor waits at most one hour for binary dependencies and four hours for the
+already started pinned, verified checkpoint download. It does not install, retry downloads,
+search extra candidates, mutate Git or delete weights. After reference checks, it runs one
+8-token real text capacity check, then the bounded functional pilot. Reference, preflight
+and functional compute together are capped at two hours with a reporting reserve;
+download wait is recorded separately. Projection memory is checked, RSS soft stop is
+512 MiB, available-system-memory floor 256 MiB; sustained host swap IO combined with low available memory or >128 MiB process swap also stops the run.
+A persistent ledger counts computation across restarts; early checks/preflight receive a conservative 180-second reserve.
+
+`preflight.py` is a capacity smoke check, not a semantic oracle. `run.py` records exact
+sample/token IDs, fixed configuration, source hashes, activations, per-text/per-head metrics,
+selection, per-token NLL, intervention results and truthful stop coverage. `report.py` reads
+saved data to produce PNG/PDF/SVG figures without reopening weights.
+
+No generation throughput is measured in this short-text pilot. Any parameter-tying storage
+benefit must be accounted for separately, including metadata. Neither same-input similarity
+nor favorable tiny heldout NLL demonstrates broad mutual interchangeability or GPU speedup.
+
+## Current primary result and completed extensions
+
+The latest user goal is learned-function repetition, independent of costs. Existing
+input RMSNorm modules at layer indices 12/13 have a proven worst two-direction real-arithmetic
+error bound of 11.3402%, worst observed heldout relative error 4.5285%, minimum cosine 0.999487,
+and small bidirectional heldout NLL changes. See [findings](../docs/CPU_FUNCTIONAL_FINDINGS.md).
+
+- `neuron_reuse.py`: fixed four-layer scalar MLP function search, nontrivial activity
+  gates, calibration-only selection and independent heldout/substitution logic; no pair passed.
+- `norm_repetition.py`: same-role learned RMSNorm bound across 552 pairs, one deterministic
+  selection independent of any input samples, real probes and two full-network substitutions.
+- `gain_factorization.py`: architecture-derived identity on non-RoPE coordinates; a separate
+  reparameterization result, not a learned-function repetition example.
+- `benchmark_factorization.py`: historical one-attempt CPU allocation optimization and
+  bitwise equivalence check. Performance is no longer a success condition.
+- `storage.py`: explicit component layout accounting; the original candidate run selected
+  no pair. This does not affect the primary function-repetition conclusion.
+
+Do not expand the final positive claim to full attention or MLP bodies. Retain the
+fixed checkpoint until the user explicitly requests deletion. Full tests and audit are
+run remotely; no local build or model execution is required to inspect the archived results.

functional/__init__.py

@@ -0,0 +1 @@
+"""CPU-only, training-free functional reuse pilot."""

functional/audit.py

@@ -0,0 +1,86 @@
+"""Audit saved functional evidence and archive only outputs, never checkpoint weights."""
+import argparse
+import csv
+import hashlib
+import json
+from pathlib import Path
+import platform
+import subprocess
+import time
+import numpy as np
+from .model import StreamingLM
+from .norm_repetition import uniform_bounds
+from .run import write_json
+
+EXCLUDE={'tokenizer.json','reference.py','checksums-functional.json','audit-validation.json','archive-files.json'}
+
+
+def archive_files(root):
+    files=[p for p in root.iterdir() if p.is_file() and p.name not in EXCLUDE and not p.name.endswith(('.partial','.safetensors'))]
+    files += [p for p in (root/'figures').glob('*') if p.is_file()]
+    return sorted(files)
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    manifest=json.loads((root/'manifest.json').read_text())
+    assert manifest['revision']=='15852e8c16360a2fea060d615a32b45270f8a8fc'
+    for f in manifest['files']:
+        if f['name'].endswith('.safetensors'):
+            path=root/'weights'/f['name']
+            assert path.is_file() and path.stat().st_size==f['bytes']
+            assert f['sha256']==f['upstream_sha256']
+    retention=json.loads((root/'retention-policy.json').read_text());assert retention['automatic_cleanup'] is False
+    reference=json.loads((root/'reference-check.json').read_text())
+    assert reference['passed'] and len(reference['tests'])==22
+    assert reference['implementation_sha256']==hashlib.sha256(Path('functional/model.py').read_bytes()).hexdigest()
+    tests=(root/'final-tests.txt').read_text();assert 'Ran 12 tests' in tests and '\nOK\n' in tests
+    result=json.loads((root/'norm-repetition-summary.json').read_text());assert result['state']=='completed'
+    assert result['learned_function_repetition_positive'] and result['bidirectional_network_quality_pass']
+    cfg=json.loads((root/'model_config.json').read_text());lm=StreamingLM(root/'weights',cfg,seconds=60)
+    c=result['selected'];ga=1+lm.small(f'layers.{c["a"]}.{c["role"]}.weight');gb=1+lm.small(f'layers.{c["b"]}.{c["role"]}.weight')
+    bounds=uniform_bounds(ga,gb)
+    np.testing.assert_allclose(bounds,[c['a_to_b_uniform_bound'],c['b_to_a_uniform_bound']],atol=1e-14,rtol=1e-14)
+    with (root/'norm-repetition-uniform-bounds.csv').open() as f:assert len(list(csv.DictReader(f)))==552
+    baseline=np.load(root/'baseline-heldout-token-nll.npy');assert baseline.shape==(4,31)
+    for rec in result['interventions']:
+        nll=np.load(root/f'norm-repetition-{rec["target"]}-uses-{rec["donor"]}-token-nll.npy')
+        delta=nll.mean(1)-baseline.mean(1)
+        np.testing.assert_allclose(delta,rec['sequence_nll_deltas'],atol=1e-13,rtol=1e-13)
+        assert abs(delta.mean()-rec['mean_nll_delta'])<1e-13
+    with (root/'norm-repetition-function-metrics.csv').open() as f:rows=list(csv.DictReader(f))
+    held=[r for r in rows if r['split']=='heldout'];assert len(held)==8
+    worst=max(max(float(r['relative_l2']),float(r['reverse_relative_l2'])) for r in held)
+    assert abs(worst-result['worst_heldout_relative_l2'])<1e-14
+    neuron=json.loads((root/'neuron-reuse-search-summary.json').read_text());assert sum(x['eligible_unique_pairs_checked'] for x in neuron)==7450139
+    ledger=json.loads((root/'compute-budget-ledger.json').read_text());assert ledger['spent_seconds']+600<7200
+    environment={'python':platform.python_version(),'platform':platform.platform(),'numpy':np.__version__,
+                 'packages':subprocess.run([str(Path(__import__('sys').executable)),'-m','pip','freeze'],capture_output=True,text=True,check=True).stdout.splitlines(),
+                 'weights_retained':True,'no_automatic_cleanup':True,'ledger_seconds_including_initial_reserve':ledger['spent_seconds'],
+                 'final_tests_report_audit_additional_budget_upper_reserve_seconds':600,
+                 'historical_error_logs':'Early reference adapter errors and preflight pressure stops; latest summaries/tests are authoritative.'}
+    write_json(root/'functional-environment.json',environment)
+    source_paths=[*Path('functional').glob('*.py'),Path('functional/reference.json'),Path('pilot/tensors.py'),Path('tests/test_functional.py')]
+    write_json(root/'final-source-hashes.json',{str(p):hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(source_paths)})
+    files=archive_files(root)
+    # Every numeric CSV value must be finite. Empty diagnostic fields stay unknown.
+    for path in files:
+        if path.suffix=='.csv':
+            with path.open() as f:
+                for row in csv.DictReader(f):
+                    for value in row.values():
+                        if value in (None,''):continue
+                        try:number=float(value)
+                        except ValueError:continue
+                        assert np.isfinite(number),(path,value)
+    checksums={str(p.relative_to(root)):hashlib.sha256(p.read_bytes()).hexdigest() for p in files}
+    write_json(root/'checksums-functional.json',checksums)
+    selected=[*checksums,'checksums-functional.json','audit-validation.json','archive-files.json']
+    write_json(root/'archive-files.json',selected)
+    write_json(root/'audit-validation.json',{'passed':True,'validated_unix':time.time(),'checksummed_files':len(checksums),
+               'proof_recomputed_from_retained_weights':True,'heldout_nll_recomputed_from_saved_arrays':True,
+               'weights_present_and_retained':True,'archive_excludes_weights_and_tokenizer':True})
+    print(json.dumps({'passed':True,'checksummed_files':len(checksums),'weights_retained':True}))
+
+
+if __name__=='__main__':main()

functional/benchmark_factorization.py

@@ -0,0 +1,83 @@
+"""One allocation-reduction attempt, with a similarly optimized baseline.
+
+Does not change weights or run the network again. Checks bitwise equality to the
+network-tested factorization on every captured real Q/K input before timing.
+"""
+import argparse
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM
+from .gain_factorization import consolidate,normalized,LAYERS
+from .run import save_csv,write_json
+
+
+def baseline_inplace(q,k,a,b):
+    q=normalized(q);q*=a
+    k=normalized(k);k*=b
+    return q,k
+
+
+def factored_inplace(q,k,a,b,r,combined,move='q_to_k'):
+    q,k=normalized(q),normalized(k)
+    if move=='q_to_k':
+        q[...,:r]*=a[:r];k*=combined
+    else:
+        q*=combined;k[...,:r]*=b[:r]
+    return q,k
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);a=p.parse_args();root=a.root
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text());remaining=int(7200-ledger['spent_seconds']-20)
+    if remaining<60:raise TimeoutError('Cumulative budget insufficient')
+    start=time.monotonic();result={'state':'running'}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());lm=StreamingLM(root/'weights',cfg,seconds=min(remaining,300))
+        r=64;data=np.load(root/'baseline-activations.npz');checks=[]
+        for layer in LAYERS:
+            lm.guard();pfx=f'layers.{layer}.self_attn.'
+            ag,bg=1+lm.small(pfx+'q_norm.weight'),1+lm.small(pfx+'k_norm.weight')
+            q,k=data[f'qraw_{layer}'],data[f'kraw_{layer}']
+            for move in ('q_to_k','k_to_q'):
+                combined=np.concatenate([(bg if move=='q_to_k' else ag)[:r],ag[r:]*bg[r:]])
+                expected=consolidate(q,k,ag,bg,r,move)
+                actual=factored_inplace(q,k,ag,bg,r,combined,move)
+                for x,y in zip(expected,actual):np.testing.assert_array_equal(x,y)
+                checks.append({'layer':layer,'direction':move,'query_elements':int(q.size),'key_elements':int(k.size),'bitwise_equal':True})
+        q,k=data['qraw_3'][:1],data['kraw_3'][:1];pfx='layers.3.self_attn.'
+        ag,bg=1+lm.small(pfx+'q_norm.weight'),1+lm.small(pfx+'k_norm.weight')
+        combined=np.concatenate([bg[:r],ag[r:]*bg[r:]])
+        funcs={'baseline_legacy':lambda:(normalized(q)*ag,normalized(k)*bg),
+               'baseline_inplace':lambda:baseline_inplace(q,k,ag,bg),
+               'factored_inplace':lambda:factored_inplace(q,k,ag,bg,r,combined)}
+        for _ in range(3):
+            for fn in funcs.values():fn()
+        timing=[];names=list(funcs)
+        for repeat in range(30):
+            order=names[repeat%3:]+names[:repeat%3]
+            for name in order:
+                lm.guard();t=time.perf_counter()
+                for _ in range(50):funcs[name]()
+                timing.append({'repeat':repeat,'implementation':name,'seconds_per_call':(time.perf_counter()-t)/50,'calls_per_block':50})
+        save_csv(root/'gain-factorization-optimized-timing.csv',timing)
+        values={name:np.array([row['seconds_per_call'] for row in timing if row['implementation']==name]) for name in names}
+        ratio=values['baseline_inplace']/values['factored_inplace']
+        medians={name:float(np.median(v)) for name,v in values.items()}
+        result.update(state='completed',bitwise_checks=checks,medians_seconds=medians,
+                      paired_baseline_over_factored_median=float(np.median(ratio)),
+                      paired_ratio_p10=float(np.quantile(ratio,.1)),paired_ratio_p90=float(np.quantile(ratio,.9)),
+                      fraction_blocks_factored_faster=float(np.mean(ratio>1)),
+                      descriptive_local_gain=bool(np.median(ratio)>1.05 and np.mean(ratio>1)>=.8 and np.quantile(ratio,.1)>1),
+                      claim_limit='Hot-array CPU normalization microbenchmark against equally allocation-optimized baseline; no full-model speedup or independent statistical significance claim.')
+    except Exception as exc:
+        result.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-start;result.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.benchmark_factorization','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'gain-factorization-optimized-summary.json',result)
+
+
+if __name__=='__main__':main()

functional/gain_factorization.py

@@ -0,0 +1,135 @@
+"""Architecture-derived Q/K gain consolidation on non-RoPE dimensions only.
+
+Fixed layer3 interventions; no fitting, layer selection or learned alignment.
+Run only after baseline/candidate pilot finishes, within the same cumulative budget.
+"""
+import argparse
+import csv
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM,norm,rope
+from .run import probability_metrics,write_json,save_csv
+
+LAYERS=[3,7,11,15,19,23]
+
+
+def normalized(x):
+    return x/np.sqrt(np.mean(x*x,axis=-1,keepdims=True)+np.float32(1e-6))
+
+
+def consolidate(q,k,a,b,r,move,joint=None):
+    q,k=normalized(q),normalized(k)
+    c=a[r:]*b[r:] if joint is None else joint
+    if move=='q_to_k':
+        q=np.concatenate([q[...,:r]*a[:r],q[...,r:]],axis=-1)
+        k=np.concatenate([k[...,:r]*b[:r],k[...,r:]*c],axis=-1)
+    else:
+        q=np.concatenate([q[...,:r]*a[:r],q[...,r:]*c],axis=-1)
+        k=np.concatenate([k[...,:r]*b[:r],k[...,r:]],axis=-1)
+    return q,k
+
+
+class FactoredLM(StreamingLM):
+    def configure(self,layer,move):
+        self.factor_layer=layer;self.move=move
+        p=f'layers.{layer}.self_attn.'
+        self.a=1+self.small(p+'q_norm.weight');self.b=1+self.small(p+'k_norm.weight')
+        self.r=int(self.config['head_dim']*self.config['rope_parameters']['partial_rotary_factor'])
+        self.joint=self.a[self.r:]*self.b[self.r:]
+
+    def normalization(self,x,name):
+        p=f'layers.{self.factor_layer}.self_attn.'
+        if name not in (p+'q_norm',p+'k_norm'):return super().normalization(x,name)
+        # This path does not multiply the unit-gain tail, unlike a full-vector mask.
+        unit=x/np.sqrt(np.mean(x*x,axis=-1,keepdims=True)+self.eps)
+        isq=name==p+'q_norm';head=self.a if isq else self.b
+        skip=(isq and self.move=='q_to_k') or (not isq and self.move=='k_to_q')
+        tail=unit[...,self.r:] if skip else unit[...,self.r:]*self.joint
+        return np.concatenate([unit[...,:self.r]*head[:self.r],tail],axis=-1)
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);a=p.parse_args();root=a.root
+    # Protocol is fixed before inspecting functional measurements.
+    protocol={'fixed_intervention_layer':3,'directions':['q_to_k','k_to_q'],'diagnostic_layers':LAYERS,'no_selection':True,'no_training':True,'precision':'FP32 product; no BF16 requantization','quality_gate':{'mean_nll_delta':.01,'sequence_p95_nll_delta':.05}}
+    protocol_path=root/'gain-factorization-protocol.json'
+    if protocol_path.exists():
+        old=json.loads(protocol_path.read_text());assert all(old[k]==v for k,v in protocol.items()), 'Protocol differs from preregistration'
+    else:write_json(protocol_path,{**protocol,'registered_unix':time.time()})
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text())
+    remaining=int(7200-ledger['spent_seconds']-30)
+    if remaining<60:raise TimeoutError('No remaining cumulative budget for deterministic identity verification')
+    start=time.monotonic()
+    report={'state':'running','records':[],'interventions':[]}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());c=cfg['text_config'];r=int(c['head_dim']*c['rope_parameters']['partial_rotary_factor'])
+        assert c['rms_norm_eps']==1e-6
+        assert c['head_dim']==256 and r==64 and c['num_attention_heads']==8 and c['num_key_value_heads']==2
+        lm=FactoredLM(root/'weights',cfg,seconds=remaining)
+        data=np.load(root/'baseline-activations.npz');samples=json.loads((root/'samples-tokenized.json').read_text())['samples']
+        rp=c['rope_parameters'];repeats=c['num_attention_heads']//c['num_key_value_heads']
+        records=[]
+        for layer in LAYERS:
+            lm.guard()
+            pfx=f'layers.{layer}.self_attn.'
+            aw=lm.small(pfx+'q_norm.weight');bw=lm.small(pfx+'k_norm.weight');ag,bg=1+aw,1+bw
+            q,k=data[f'qraw_{layer}'],data[f'kraw_{layer}']
+            qr,kr=rope(norm(q,aw),rp['rope_theta'],rp['partial_rotary_factor']),rope(norm(k,bw),rp['rope_theta'],rp['partial_rotary_factor'])
+            ref=np.einsum('bthd,bshd->bhts',qr,np.repeat(kr,repeats,axis=2))/np.float32(np.sqrt(c['head_dim']))
+            mask=np.triu(np.ones(ref.shape[-2:],dtype=bool),1)
+            def probs(s):
+                s=np.where(mask,-np.inf,s);e=np.exp(s-s.max(-1,keepdims=True));return e/e.sum(-1,keepdims=True)
+            pref=probs(ref)
+            for move in ('q_to_k','k_to_q'):
+                qq,kk=consolidate(q,k,ag,bg,r,move)
+                qq,kk=rope(qq,rp['rope_theta'],rp['partial_rotary_factor']),rope(kk,rp['rope_theta'],rp['partial_rotary_factor'])
+                changed=np.einsum('bthd,bshd->bhts',qq,np.repeat(kk,repeats,axis=2))/np.float32(np.sqrt(c['head_dim']))
+                pp=probs(changed)
+                for bi,s in enumerate(samples):
+                    diff=changed[bi].astype(np.float64)-ref[bi]
+                    records.append({'layer':layer,'direction':move,'sample_id':s['id'],'split':s['split'],
+                                    'score_max_abs_error':float(np.abs(diff).max()),'score_relative_l2':float(np.linalg.norm(diff)/max(np.linalg.norm(ref[bi].astype(np.float64)),1e-12)),
+                                    'probability_max_abs_error':float(np.abs(pp[bi]-pref[bi]).max()),**probability_metrics(pref[bi:bi+1],pp[bi:bi+1])})
+            save_csv(root/'gain-factorization-score-metrics.csv',records)
+        held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout']);ids=np.array([s['token_ids'] for s in samples],dtype=np.int64)
+        base=np.load(root/'baseline-heldout-token-nll.npy').mean(1)
+        for move in ('q_to_k','k_to_q'):
+            lm.configure(3,move)
+            h,_=lm.forward(ids[held])
+            nll=lm.nll(h,ids[held]);np.save(root/f'gain-factorization-{move}-token-nll.npy',nll)
+            delta=nll.mean(1)-base
+            report['interventions'].append({'layer':3,'direction':move,'sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_delta':float(np.quantile(delta,.95)),'passed':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)})
+            write_json(root/'gain-factorization-summary.json',report)
+        # Local normalization timing uses precomputed gains for both implementations.
+        q,k=data['qraw_3'][:1],data['kraw_3'][:1];pfx='layers.3.self_attn.'
+        ag,bg=1+lm.small(pfx+'q_norm.weight'),1+lm.small(pfx+'k_norm.weight')
+        def baseline():return normalized(q)*ag,normalized(k)*bg
+        joint=ag[r:]*bg[r:]
+        def factored():return consolidate(q,k,ag,bg,r,'q_to_k',joint=joint)
+        for _ in range(3):baseline();factored()
+        timing=[]
+        for repeat in range(10):
+            for label,fn in (('baseline',baseline),('factored',factored)) if repeat%2==0 else (('factored',factored),('baseline',baseline)):
+                lm.guard();t=time.perf_counter()
+                for _ in range(20):fn()
+                timing.append({'repeat':repeat,'implementation':label,'seconds_per_call':(time.perf_counter()-t)/20})
+        save_csv(root/'gain-factorization-kernel-timing.csv',timing)
+        report.update(state='completed',maximum_score_abs_error=max(x['score_max_abs_error'] for x in records),maximum_probability_abs_error=max(x['probability_max_abs_error'] for x in records),
+                      bidirectional_heldout_pass=all(x['passed'] for x in report['interventions']),
+                      elementwise_gain_multiplies_saved_per_token_layer={'q_to_k':1536,'k_to_q':384},
+                      original_bf16_payload_bytes=1024,factored_lossless_mixed_payload_bytes=1024,
+                      fp32_cached_gain_payload_before=2048,fp32_cached_gain_payload_after=1280,
+                      claim_limit='Exact real-arithmetic identity only on non-RoPE coordinates; empirical FP32/NLL checks on tiny text pilot. Not evidence that two existing full layers interchange, not model-wide speedup.')
+    except Exception as exc:
+        report.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-start
+        report.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.gain_factorization','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'gain-factorization-summary.json',report)
+
+
+if __name__=='__main__':main()

functional/model.py

@@ -0,0 +1,247 @@
+"""Streaming FP32 text-only Qwen3.5 reference, original BF16 weights.
+
+Mathematical semantics checked against Transformers Qwen3.5, commit
+fd3cf0140de03f7619ac95b5cfed05b711ff9e22 (Apache-2.0, Qwen Team/HuggingFace).
+No cached decode, padding, images, generation, fused kernels or autograd.
+Full-sequence gated delta recurrence replaces the algebraically equivalent
+chunk algorithm; both are independently compared in remote tests.
+"""
+import ctypes
+import json
+import time
+from pathlib import Path
+import numpy as np
+from pilot.tensors import load_tensors, decode
+
+PREFIX = 'model.language_model.'
+
+
+def sigmoid(x):
+    return np.exp(-np.logaddexp(np.float32(0), -x))
+
+
+def silu(x):
+    return x * sigmoid(x)
+
+
+def norm(x, weight, eps=1e-6, centered=True):
+    return (x / np.sqrt(np.mean(x*x, axis=-1, keepdims=True) + eps)
+            * (1 + weight if centered else weight)).astype(np.float32)
+
+
+def rope(x, theta=10000000., fraction=.25):
+    # Text-only: all three MRoPE position axes coincide.
+    dim = int(x.shape[-1] * fraction)
+    inv = np.float32(1) / np.float32(theta) ** (np.arange(0, dim, 2, dtype=np.float32) / dim)
+    freq = np.arange(x.shape[1], dtype=np.float32)[:, None] * inv[None, :]
+    angle = np.concatenate([freq, freq], axis=-1)[None, :, None, :]
+    a = x[..., :dim]
+    rotated = np.concatenate([-a[..., dim//2:], a[..., :dim//2]], axis=-1)
+    return np.concatenate([a*np.cos(angle) + rotated*np.sin(angle), x[..., dim:]], axis=-1)
+
+
+def delta_rule(q, k, v, g, beta):
+    q = q / np.sqrt(np.sum(q*q, axis=-1, keepdims=True) + 1e-6)
+    k = k / np.sqrt(np.sum(k*k, axis=-1, keepdims=True) + 1e-6)
+    q = q / np.float32(np.sqrt(q.shape[-1]))
+    state = np.zeros((q.shape[0], q.shape[2], q.shape[3], v.shape[3]), dtype=np.float32)
+    out = np.empty_like(v)
+    for t in range(q.shape[1]):
+        state *= np.exp(g[:, t, :, None, None])
+        predicted = np.sum(state * k[:, t, :, :, None], axis=-2)
+        delta = (v[:, t] - predicted) * beta[:, t, :, None]
+        state += k[:, t, :, :, None] * delta[:, :, None, :]
+        out[:, t] = np.sum(state * q[:, t, :, :, None], axis=-2)
+    return out, state
+
+
+def causal_conv(x, kernel):
+    out = np.zeros_like(x)
+    width = kernel.shape[-1]
+    for j in range(width):
+        lag = width - 1 - j
+        if lag < x.shape[1]:
+            out[:, lag:] += x[:, :x.shape[1]-lag] * kernel[:, j]
+    return silu(out)
+
+
+def attention(q, k, v):
+    repeats = q.shape[2] // k.shape[2]
+    k, v = np.repeat(k, repeats, axis=2), np.repeat(v, repeats, axis=2)
+    scores = np.einsum('bthd,bshd->bhts', q, k) / np.float32(np.sqrt(q.shape[-1]))
+    mask = np.triu(np.ones(scores.shape[-2:], dtype=bool), 1)
+    scores = np.where(mask, -np.inf, scores)
+    exp = np.exp(scores - scores.max(axis=-1, keepdims=True))
+    p = exp / exp.sum(axis=-1, keepdims=True)
+    return np.einsum('bhts,bshd->bthd', p, v), p
+
+
+class StreamingLM:
+    def __init__(self, weights, config, seconds=7200, enforce_memory=True):
+        self.tensors = load_tensors(Path(weights))
+        self.config = config.get('text_config', config)
+        self.deadline = time.monotonic() + seconds
+        self.enforce_memory = enforce_memory
+        self.eps = self.config['rms_norm_eps']
+        self.norm_replacement = None
+        self.capture_qk = False
+        self.qk_inputs = {}
+        self.swap_observation = None
+        self.swap_pressure_windows = 0
+        self.malloc_trim = getattr(ctypes.CDLL(None), 'malloc_trim', None)
+
+    def guard(self):
+        if time.monotonic() > self.deadline:
+            raise TimeoutError('CPU computation deadline reached')
+        if not self.enforce_memory:
+            return
+        status = dict(line.split(':', 1) for line in Path('/proc/self/status').read_text().splitlines() if ':' in line)
+        rss_kib = int(status['VmRSS'].split()[0])
+        available = int(next(line.split()[1] for line in Path('/proc/meminfo').read_text().splitlines() if line.startswith('MemAvailable:')))
+        now = time.monotonic()
+        if self.swap_observation is None or now-self.swap_observation[0] >= 5:
+            counters = dict(line.split() for line in Path('/proc/vmstat').read_text().splitlines())
+            count = int(counters['pswpin'])+int(counters['pswpout'])
+            if self.swap_observation is not None:
+                elapsed = now-self.swap_observation[0]
+                rate = (count-self.swap_observation[1])*4096/elapsed
+                self.swap_pressure_windows = self.swap_pressure_windows+1 if rate > 4*1024**2 else 0
+                own_swap_kib = int(status.get('VmSwap', '0 kB').split()[0])
+                if self.swap_pressure_windows >= 3 and (available < 384*1024 or own_swap_kib > 128*1024):
+                    raise MemoryError(f'Sustained host swap pressure plus constrained memory: available={available} KiB, process_swap={own_swap_kib} KiB')
+            self.swap_observation = (now,count)
+        if rss_kib > 512*1024 or available < 256*1024:
+            raise MemoryError(f'Soft memory stop: RSS {rss_kib} KiB, available {available} KiB')
+
+    def small(self, name):
+        t = self.tensors[PREFIX + name]
+        if t.numel > 1048576:
+            raise ValueError('Use blockwise linear for large tensors')
+        x = t.read().reshape(t.shape)
+        t.evict()
+        return x
+
+    def linear(self, x, name, rows=256):
+        t = self.tensors[PREFIX + name + '.weight']
+        outdim, indim = t.shape
+        flat = x.reshape(-1, indim)
+        result = np.empty((flat.shape[0], outdim), dtype=np.float32)
+        for start in range(0, outdim, rows):
+            self.guard()
+            end = min(start + rows, outdim)
+            w = t.read(slice(start*indim, end*indim)).reshape(end-start, indim)
+            result[:, start:end] = flat @ w.T
+            del w
+            t.evict()
+        if self.malloc_trim is not None:
+            self.malloc_trim(0)  # Release only this process's freed temporary heap pages.
+        return result.reshape(*x.shape[:-1], outdim)
+
+    def embedding(self, ids):
+        t = self.tensors[PREFIX + 'embed_tokens.weight']
+        raw = t.raw.reshape(t.shape)
+        result = decode(raw[ids], t.dtype)
+        t.evict()
+        return result
+
+    def normalization(self, x, name):
+        if getattr(self, 'norm_replacement', None) and name == self.norm_replacement[0]:
+            name = self.norm_replacement[1]
+        return norm(x, self.small(name + '.weight'), self.eps)
+
+    def mlp(self, h, layer):
+        p = f'layers.{layer}.mlp.'
+        gate = silu(self.linear(h, p+'gate_proj'))
+        up = self.linear(h, p+'up_proj')
+        return self.linear(gate*up, p+'down_proj')
+
+    def full_attention(self, h, layer, shared_p=None):
+        p = f'layers.{layer}.self_attn.'
+        c = self.config
+        heads, kvheads, dim = c['num_attention_heads'], c['num_key_value_heads'], c['head_dim']
+        # Q and output gate are interleaved per head, not globally split.
+        mixed = self.linear(h, p+'q_proj').reshape(*h.shape[:-1], heads, 2*dim)
+        q, gate = mixed[..., :dim], mixed[..., dim:]
+        k = self.linear(h, p+'k_proj').reshape(*h.shape[:-1], kvheads, dim)
+        v = self.linear(h, p+'v_proj').reshape(*h.shape[:-1], kvheads, dim)
+        if getattr(self, 'capture_qk', False):
+            self.qk_inputs[layer] = (q.copy(), k.copy())
+        q = self.normalization(q, p+'q_norm')
+        k = self.normalization(k, p+'k_norm')
+        rp = c['rope_parameters']
+        q, k = (rope(x, rp['rope_theta'], rp['partial_rotary_factor']) for x in (q,k))
+        out, probs = attention(q,k,v)
+        if shared_p is not None:
+            out = np.einsum('bhts,bshd->bthd', shared_p, np.repeat(v, heads//kvheads, axis=2))
+        out = (out*sigmoid(gate)).reshape(*h.shape[:-1], -1)
+        return self.linear(out, p+'o_proj'), probs
+
+    def linear_attention(self, h, layer):
+        p = f'layers.{layer}.linear_attn.'
+        c = self.config
+        kh, vh = c['linear_num_key_heads'], c['linear_num_value_heads']
+        kd, vd = c['linear_key_head_dim'], c['linear_value_head_dim']
+        qkv = self.linear(h, p+'in_proj_qkv')
+        qkv = causal_conv(qkv, self.small(p+'conv1d.weight')[:,0,:])
+        q, k, v = np.split(qkv, [kh*kd, 2*kh*kd], axis=-1)
+        q = q.reshape(*h.shape[:-1], kh, kd)
+        k = k.reshape(*h.shape[:-1], kh, kd)
+        v = v.reshape(*h.shape[:-1], vh, vd)
+        if vh != kh:
+            q, k = np.repeat(q, vh//kh, axis=2), np.repeat(k, vh//kh, axis=2)
+        z = self.linear(h, p+'in_proj_z').reshape(*h.shape[:-1], vh, vd)
+        beta = sigmoid(self.linear(h, p+'in_proj_b'))
+        a = self.linear(h, p+'in_proj_a')
+        g = -np.exp(self.small(p+'A_log')) * np.logaddexp(np.float32(0), a+self.small(p+'dt_bias'))
+        out, _ = delta_rule(q,k,v,g,beta)
+        out = norm(out, self.small(p+'norm.weight'), self.eps, centered=False)*silu(z)
+        return self.linear(out.reshape(*h.shape[:-1], -1), p+'out_proj')
+
+    def forward(self, ids, capture=None, replacement=None, progress=None):
+        h = self.embedding(ids)
+        saved = {}
+        capture = set(capture or [])
+        for layer, kind in enumerate(self.config['layer_types']):
+            self.guard()
+            raw = h
+            n = self.normalization(h, f'layers.{layer}.input_layernorm')
+            if ('attn', layer) in capture:
+                saved[f'attn_{layer}'] = n.copy()
+            donor = replacement[2] if replacement and replacement[:2] == ('attn',layer) else layer
+            if kind == 'full_attention':
+                out, _ = self.full_attention(n,donor)
+            else:
+                out = self.linear_attention(n,donor)
+            h = raw + out
+            n = self.normalization(h, f'layers.{layer}.post_attention_layernorm')
+            if ('mlp',layer) in capture:
+                saved[f'mlp_{layer}'] = n.copy()
+            donor = replacement[2] if replacement and replacement[:2] == ('mlp',layer) else layer
+            h = h + self.mlp(n,donor)
+            if not np.isfinite(h).all():
+                raise FloatingPointError(f'Nonfinite hidden states at layer {layer}')
+            if progress:
+                progress(layer)
+        return self.normalization(h,'norm'), saved
+
+    def nll(self, h, ids, rows=512):
+        # Tied output embedding; online logsumexp avoids full vocabulary logits.
+        x = h[:,:-1].reshape(-1,h.shape[-1])
+        targets = ids[:,1:].reshape(-1)
+        t = self.tensors[PREFIX+'embed_tokens.weight']
+        mx = np.full(x.shape[0],-np.inf,dtype=np.float64)
+        total = np.zeros(x.shape[0],dtype=np.float64)
+        correct = np.empty(x.shape[0],dtype=np.float64)
+        for start in range(0,t.shape[0],rows):
+            self.guard()
+            end=min(start+rows,t.shape[0])
+            w=t.read(slice(start*t.shape[1],end*t.shape[1])).reshape(end-start,t.shape[1])
+            scores=x@w.T
+            m=np.maximum(mx,scores.max(axis=-1))
+            total=total*np.exp(mx-m)+np.exp(scores-m[:,None]).sum(axis=-1)
+            mx=m
+            which=(targets>=start)&(targets<end)
+            correct[which]=scores[np.flatnonzero(which),targets[which]-start]
+            del w,scores
+            t.evict()
+        return (mx+np.log(total)-correct).reshape(ids.shape[0],-1)

functional/neuron_reuse.py

@@ -0,0 +1,134 @@
+"""Find repeated learned scalar MLP functions on real hidden inputs, no training.
+
+Four fixed layers; calibration-only selection; one frozen pair; independent heldout
+functional test and two actual network substitutions. No scale fitting or R.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM,silu
+from .run import write_json,save_csv
+
+LAYERS=[5,6,20,21]
+
+
+def pair_metrics(x,y):
+    x,y=x.astype(np.float64),y.astype(np.float64)
+    xx,yy=float(x@x),float(y@y);delta=float((x-y)@(x-y))
+    xc,yc=x-x.mean(),y-y.mean()
+    return {'relative_l2_a_to_b':float(np.sqrt(delta/max(xx,1e-30))),
+            'relative_l2_b_to_a':float(np.sqrt(delta/max(yy,1e-30))),
+            'cosine':float(x@y/max(np.sqrt(xx*yy),1e-30)),
+            'pearson':float(xc@yc/max(np.linalg.norm(xc)*np.linalg.norm(yc),1e-30)),
+            'rms_a':float(np.sqrt(xx/x.size)),'rms_b':float(np.sqrt(yy/y.size)),
+            'std_over_rms_a':float(np.std(x)/max(np.sqrt(xx/x.size),1e-30)),
+            'std_over_rms_b':float(np.std(y)/max(np.sqrt(yy/y.size),1e-30)),
+            'effective_energy_tokens_a':float(xx*xx/max(float((x**4).sum()),1e-30)),
+            'effective_energy_tokens_b':float(yy*yy/max(float((y**4).sum()),1e-30))}
+
+
+class NeuronReuseLM(StreamingLM):
+    def configure(self,layer,target,donor):self.reused_layer,self.target,self.donor=layer,target,donor
+    def mlp(self,h,layer):
+        p=f'layers.{layer}.mlp.'
+        z=silu(self.linear(h,p+'gate_proj'))*self.linear(h,p+'up_proj')
+        if layer==self.reused_layer:z[...,self.target]=z[...,self.donor]
+        return self.linear(z,p+'down_proj')
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    protocol={'layers':LAYERS,'unit':'post_silu_gate_times_up_scalar_function; keep each output column',
+              'thresholds':[.05,.10,.20],'maximum_directional_relative_l2':.20,'cosine_min':.98,'pearson_min':.95,
+              'rms_floor':'at least within-layer median calibration neuron RMS','std_over_rms_min':.25,
+              'effective_energy_tokens_min':16,'max_selected_pairs':1,'selection':'smallest worst directional calibration error; then layer/id',
+              'heldout_per_text_error_max':.35,'heldout_quality_mean_nll_delta_max':.01,'heldout_quality_sequence_p95_delta_max':.05,
+              'training':False,'scale_fitting':False,'cost_required':False}
+    protocol_path=root/'neuron-reuse-protocol.json'
+    if protocol_path.exists():
+        old=json.loads(protocol_path.read_text());assert all(old[k]==v for k,v in protocol.items())
+    else:write_json(protocol_path,{**protocol,'registered_unix':time.time()})
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text());remaining=int(7200-ledger['spent_seconds']-30)
+    if remaining<120:raise TimeoutError('Remaining cumulative budget insufficient')
+    started=time.monotonic();result={'state':'running','selected':None,'function_pass':False,'interventions':[]}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());lm=NeuronReuseLM(root/'weights',cfg,seconds=remaining)
+        data=np.load(root/'baseline-activations.npz');samples=json.loads((root/'samples-tokenized.json').read_text())['samples']
+        cal=np.array([i for i,s in enumerate(samples) if s['split']=='calibration']);held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+        activations={};summaries=[];best=None
+        for layer in LAYERS:
+            h=data[f'mlp_{layer}'];pfx=f'layers.{layer}.mlp.'
+            z=silu(lm.linear(h,pfx+'gate_proj'))*lm.linear(h,pfx+'up_proj');activations[layer]=z
+            # Profiles use all calibration tokens; never use heldout in selection.
+            x=np.ascontiguousarray(z[cal].reshape(-1,z.shape[-1]).T,dtype=np.float64)
+            n=x.shape[1];normsq=(x*x).sum(1);means=x.mean(1);variance=np.maximum(normsq-n*means**2,0)
+            rms=np.sqrt(normsq/n);floor=float(np.median(rms));effective=normsq**2/np.maximum((x**4).sum(1),1e-30)
+            active=(rms>=floor)&(np.sqrt(variance/np.maximum(normsq,1e-30))>=.25)&(effective>=16)
+            counts={str(t):0 for t in protocol['thresholds']};tested=0;layerbest=None
+            for start in range(0,x.shape[0],256):
+                lm.guard();end=min(start+256,x.shape[0]);dot=x[start:end]@x.T
+                delta=np.maximum(normsq[start:end,None]+normsq[None,:]-2*dot,0)
+                error=np.sqrt(delta/np.maximum(np.minimum(normsq[start:end,None],normsq[None,:]),1e-30))
+                cosine=dot/np.maximum(np.sqrt(normsq[start:end,None]*normsq[None,:]),1e-30)
+                pearson=(dot-n*means[start:end,None]*means[None,:])/np.maximum(np.sqrt(variance[start:end,None]*variance[None,:]),1e-30)
+                unique=np.arange(start,end)[:,None]<np.arange(x.shape[0])[None,:]
+                allowed=unique&active[start:end,None]&active[None,:]&(cosine>=.98)&(pearson>=.95)
+                tested+=int((unique&active[start:end,None]&active[None,:]).sum())
+                for threshold in protocol['thresholds']:counts[str(threshold)]+=int((allowed&(error<=threshold)).sum())
+                mask=allowed&(error<=.20)
+                if mask.any():
+                    score=np.where(mask,error,np.inf);row,col=np.unravel_index(np.argmin(score),score.shape)
+                    candidate={'layer':layer,'a':int(start+row),'b':int(col),'calibration_worst_relative_l2':float(score[row,col])}
+                    key=lambda c:(c['calibration_worst_relative_l2'],c['layer'],c['a'],c['b'])
+                    if layerbest is None or key(candidate)<key(layerbest):layerbest=candidate
+                    if best is None or key(candidate)<key(best):best=candidate
+            summaries.append({'layer':layer,'neurons':int(x.shape[0]),'eligible_neurons':int(active.sum()),'eligible_unique_pairs_checked':tested,'rms_floor':floor,'calibration_pair_counts':counts,'best':layerbest})
+            write_json(root/'neuron-reuse-search-summary.json',summaries)
+            print(json.dumps(summaries[-1]),flush=True)
+        np.savez(root/'neuron-reuse-activations.npz',**{f'layer{k}':v for k,v in activations.items()})
+        write_json(root/'neuron-reuse-selection.json',{'selected':best,'selection_uses_heldout':False})
+        result['selected']=best
+        if best is not None:
+            layer,i,j=best['layer'],best['a'],best['b'];z=activations[layer]
+            metrics=[]
+            for split,index in [('calibration',cal),('heldout',held)]:
+                metrics.append({'split':split,'scope':'pooled',**pair_metrics(z[index,:,i].reshape(-1),z[index,:,j].reshape(-1))})
+                for idx in index:metrics.append({'split':split,'scope':samples[idx]['id'],**pair_metrics(z[idx,:,i],z[idx,:,j])})
+            save_csv(root/'neuron-reuse-function-metrics.csv',metrics)
+            pooled=next(r for r in metrics if r['split']=='heldout' and r['scope']=='pooled')
+            floor=next(r['rms_floor'] for r in summaries if r['layer']==layer)
+            function_pass=(max(pooled['relative_l2_a_to_b'],pooled['relative_l2_b_to_a'])<=.20 and pooled['cosine']>=.98 and pooled['pearson']>=.95 and
+                           min(pooled['std_over_rms_a'],pooled['std_over_rms_b'])>=.25 and min(pooled['effective_energy_tokens_a'],pooled['effective_energy_tokens_b'])>=16 and
+                           min(pooled['rms_a'],pooled['rms_b'])>=floor and
+                           all(max(r['relative_l2_a_to_b'],r['relative_l2_b_to_a'])<=.35 for r in metrics if r['split']=='heldout' and r['scope']!='pooled'))
+            result.update(function_pass=bool(function_pass),heldout_function_metrics=pooled)
+            # Show parameter similarity separately from actual functional similarity.
+            wd=[]
+            for role in ('gate_proj','up_proj'):
+                t=lm.tensors[f'model.language_model.layers.{layer}.mlp.{role}.weight'];d=t.shape[1]
+                wa,wb=t.read(slice(i*d,(i+1)*d)),t.read(slice(j*d,(j+1)*d))
+                wd.append({'role':role,'weight_cosine':float(wa.astype(np.float64)@wb/max(np.linalg.norm(wa.astype(np.float64))*np.linalg.norm(wb.astype(np.float64)),1e-30))});t.evict()
+            result['weight_diagnostics']=wd
+            if function_pass:
+                ids=np.array([s['token_ids'] for s in samples],dtype=np.int64);base=np.load(root/'baseline-heldout-token-nll.npy').mean(1)
+                for target,donor in ((i,j),(j,i)):
+                    lm.configure(layer,target,donor);h,_=lm.forward(ids[held]);nll=lm.nll(h,ids[held])
+                    np.save(root/f'neuron-reuse-{target}-uses-{donor}-token-nll.npy',nll)
+                    delta=nll.mean(1)-base
+                    result['interventions'].append({'target':target,'donor':donor,'sequence_nll_deltas':delta.tolist(),'mean_nll_delta':float(delta.mean()),'sequence_p95_delta':float(np.quantile(delta,.95)),'passed':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)})
+                    write_json(root/'neuron-reuse-summary.json',result)
+            # No fallback after looking at heldout.
+        result.update(state='completed',empirical_repeated_function_positive=bool(result['function_pass'] and len(result['interventions'])==2 and all(r['passed'] for r in result['interventions'])),
+                      claim_limit='Distinct learned scalar MLP subfunctions on the specified short-text distribution; not whole-layer duplication, global equivalence or broad task-capability proof. No performance requirement.')
+    except Exception as exc:result.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-started;result.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.neuron_reuse','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'neuron-reuse-summary.json',result)
+
+
+if __name__=='__main__':main()

functional/norm_repetition.py

@@ -0,0 +1,108 @@
+"""Existing learned RMSNorm modules: uniform function bound + independent probes.
+
+Deterministic exhaustive selection by a mathematical all-input error bound, not
+calibration/heldout fitting. Same role, shape and eps; one pair only.
+"""
+import argparse
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM,norm
+from .run import write_json,save_csv,error_rows
+
+ROLES=['input_layernorm','post_attention_layernorm']
+
+
+def uniform_bounds(a,b):
+    if np.any(a==0) or np.any(b==0):return None
+    diff=np.abs(a.astype(np.float64)-b.astype(np.float64))
+    return float(np.max(diff/np.abs(a))),float(np.max(diff/np.abs(b)))
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);args=p.parse_args();root=args.root
+    protocol={'roles':ROLES,'layers':list(range(24)),'same_role_only':True,'same_eps':1e-6,
+              'uniform_worst_error_limit':.20,'empirical_function_worst_relative_error_limit':.05,
+              'selection':'smallest maximum two-direction uniform bound, then role/layer; no input data used',
+              'minimum_gain_standard_deviation':.001,'max_selected_pairs':1,'heldout_nll_mean_delta_limit':.01,
+              'heldout_nll_sequence_p95_delta_limit':.05,'training':False,'cost_required':False}
+    path=root/'norm-repetition-protocol.json'
+    if path.exists():
+        existing=json.loads(path.read_text());assert all(existing[k]==v for k,v in protocol.items())
+    else:write_json(path,{**protocol,'registered_unix':time.time()})
+    ledger_path=root/'compute-budget-ledger.json';ledger=json.loads(ledger_path.read_text());remaining=int(7200-ledger['spent_seconds']-30)
+    if remaining<120:raise TimeoutError('Insufficient cumulative computation budget')
+    started=time.monotonic();result={'state':'running','selected':None,'interventions':[]}
+    try:
+        cfg=json.loads((root/'model_config.json').read_text());lm=StreamingLM(root/'weights',cfg,seconds=remaining)
+        assert lm.eps==1e-6
+        candidates=[];gains={}
+        for role in ROLES:
+            for layer in range(24):gains[(role,layer)]=1+lm.small(f'layers.{layer}.{role}.weight')
+            for i in range(24):
+                for j in range(i+1,24):
+                    a,b=gains[(role,i)],gains[(role,j)]
+                    bounds=uniform_bounds(a,b)
+                    if bounds is None:continue
+                    eligible=min(float(np.std(a)),float(np.std(b)))>=.001
+                    candidates.append({'role':role,'a':i,'b':j,'a_to_b_uniform_bound':bounds[0],'b_to_a_uniform_bound':bounds[1],
+                                       'worst_uniform_bound':max(bounds),'eligible_nonconstant_gains':eligible,
+                                       'gain_std_a':float(np.std(a)),'gain_std_b':float(np.std(b)),
+                                       'gain_min_a':float(np.min(a)),'gain_min_b':float(np.min(b))})
+        save_csv(root/'norm-repetition-uniform-bounds.csv',candidates)
+        feasible=[c for c in candidates if c['eligible_nonconstant_gains'] and c['worst_uniform_bound']<=.20]
+        counts={str(t):sum(c['eligible_nonconstant_gains'] and c['worst_uniform_bound']<=t for c in candidates) for t in (.05,.10,.20)}
+        selected=sorted(feasible,key=lambda c:(c['worst_uniform_bound'],c['role'],c['a'],c['b']))[:1]
+        write_json(root/'norm-repetition-selection.json',{'selected':selected,'uniform_bound_counts':counts,'selection_reads_no_input_data':True})
+        result['uniform_bound_counts']=counts
+        if selected:
+            c=selected[0];result['selected']=c
+            # Selection is frozen before reading tokenized inputs or their activations.
+            samples=json.loads((root/'samples-tokenized.json').read_text())['samples'];ids=np.array([s['token_ids'] for s in samples],dtype=np.int64)
+            held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+            kind='attn' if c['role']=='input_layernorm' else 'mlp'
+            captures=[(kind,c['a']),(kind,c['b'])]
+            h,saved=lm.forward(ids,capture=captures)
+            # The untouched recomputation must reproduce the saved real baseline.
+            original=np.load(root/'baseline-activations.npz')['final']
+            np.testing.assert_array_equal(h,original)
+            np.savez(root/'norm-repetition-input-profiles.npz',**saved)
+            ga,gb=gains[(c['role'],c['a'])],gains[(c['role'],c['b'])]
+            rows=[]
+            for source in (c['a'],c['b']):
+                n=saved[f'{kind}_{source}'];g=gains[(c['role'],source)]
+                # n = u * gamma_source. Reconstruct common normalized input u;
+                # source gains are nonzero. Additional rounding is ~FP32 only.
+                unit=n/g
+                fa,fb=unit*ga,unit*gb
+                rec=error_rows(fa,fb,samples,{'source_layer':source,'role':c['role'],'a':c['a'],'b':c['b']})
+                rows.extend(rec)
+            save_csv(root/'norm-repetition-function-metrics.csv',rows)
+            heldrows=[r for r in rows if r['split']=='heldout']
+            function_pass=all(max(r['relative_l2'],r['reverse_relative_l2'])<=.05 for r in heldrows)
+            result.update(uniform_function_similarity_proven=True,heldout_function_pass=bool(function_pass),
+                          worst_heldout_relative_l2=max(max(r['relative_l2'],r['reverse_relative_l2']) for r in heldrows),
+                          worst_heldout_cosine=min(r['cosine'] for r in heldrows),untouched_full_forward_bitwise_matches_baseline=True)
+            if function_pass:
+                baseline=np.load(root/'baseline-heldout-token-nll.npy').mean(1)
+                for target,donor in ((c['a'],c['b']),(c['b'],c['a'])):
+                    lm.norm_replacement=(f'layers.{target}.{c["role"]}',f'layers.{donor}.{c["role"]}')
+                    changed,_=lm.forward(ids[held]);nll=lm.nll(changed,ids[held]);np.save(root/f'norm-repetition-{target}-uses-{donor}-token-nll.npy',nll)
+                    delta=nll.mean(1)-baseline
+                    result['interventions'].append({'target':target,'donor':donor,'sequence_nll_deltas':delta.tolist(),
+                         'mean_nll_delta':float(delta.mean()),'sequence_p95_nll_delta':float(np.quantile(delta,.95)),
+                         'passed_quality':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)})
+                    write_json(root/'norm-repetition-summary.json',result)
+            result['learned_function_repetition_positive']=bool(function_pass)
+            result['bidirectional_network_quality_pass']=bool(len(result['interventions'])==2 and all(x['passed_quality'] for x in result['interventions']))
+        result.update(state='completed',claim_limit='Existing learned RMSNorm module functions with identical eps and shape; uniform real-arithmetic relative bound and tiny real-text probes. Does not establish attention/MLP-body duplication or broad task capability.')
+    except Exception as exc:result.update(state='stopped',error_type=type(exc).__name__,error=str(exc));raise
+    finally:
+        elapsed=time.monotonic()-started;result.update(elapsed_seconds=elapsed,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        ledger=json.loads(ledger_path.read_text());ledger['spent_seconds']+=elapsed;ledger['calls'].append({'module':'functional.norm_repetition','seconds':elapsed});write_json(ledger_path,ledger)
+        write_json(root/'norm-repetition-summary.json',result)
+
+
+if __name__=='__main__':main()

functional/package_delivery.py

@@ -0,0 +1,46 @@
+"""Package code-independent scientific outputs, retain large activation caches externally.
+
+The experiment directory remains immutable and complete. The delivery directory
+contains all figure source tables/NLL arrays plus hashes/locations for three large
+reproducible activation caches. Never deletes model weights or cache files.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import shutil
+from .run import write_json
+
+CACHES={'baseline-activations.npz','neuron-reuse-activations.npz','norm-repetition-input-profiles.npz'}
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);p.add_argument('--output',required=True,type=Path);a=p.parse_args()
+    root,out=a.root.resolve(),a.output.resolve();assert root!=out
+    out.mkdir(parents=True,exist_ok=True)
+    full=json.loads((root/'checksums-functional.json').read_text())
+    files=json.loads((root/'archive-files.json').read_text())
+    for name in files:
+        src=root/name
+        if name in CACHES:continue
+        dest=out/name;dest.parent.mkdir(parents=True,exist_ok=True);shutil.copy2(src,dest)
+    external=[]
+    for name in sorted(CACHES):
+        src=root/name;digest=hashlib.sha256(src.read_bytes()).hexdigest();assert digest==full[name]
+        external.append({'name':name,'server_path':str(src),'bytes':src.stat().st_size,'sha256':digest,'retained':True,'reproducible_by':'functional.run / functional.neuron_reuse / functional.norm_repetition'})
+    write_json(out/'external-activation-cache.json',{'host':'8.211.154.134','files':external,'not_model_weights':True,
+               'reason':'Large intermediate caches retained on the authorized server; all figure source tables and token NLL arrays are delivered in Git.'})
+    shutil.copy2(root/'checksums-functional.json',out/'checksums-full-server.json')
+    (out/'DELIVERY.md').write_text('# Scientific delivery and retained caches\n\nAll figure CSV/JSON source data, per-token NLL, protocols, reports, metadata and historical logs are included.\nThree large reproducible intermediate activation caches remain on 8.211.154.134 and in the original full local archive;\n`external-activation-cache.json` records exact locations, sizes and SHA256. No weight or cache was deleted.\nThe original full experiment manifest is `checksums-full-server.json`; the delivery-only manifest is\n`checksums-functional.json`. No model checkpoint or tokenizer payload is in this Git delivery.\n')
+    validation=json.loads((out/'audit-validation.json').read_text());validation.update(delivery_excludes_external_activation_caches=True,full_experiment_checksums='checksums-full-server.json',figure_source_tables_and_token_nll_complete=True)
+    write_json(out/'audit-validation.json',validation)
+    checks={}
+    for file in sorted(out.rglob('*')):
+        if file.is_file() and file.name not in ('checksums-functional.json','audit-validation.json','archive-files.json'):
+            checks[str(file.relative_to(out))]=hashlib.sha256(file.read_bytes()).hexdigest()
+    write_json(out/'checksums-functional.json',checks)
+    write_json(out/'archive-files.json',[*checks,'checksums-functional.json','audit-validation.json','archive-files.json'])
+    print(json.dumps({'delivery_files':len(checks)+3,'checksummed_files':len(checks),'external_caches':len(external),'weights_and_caches_retained':True}))
+
+
+if __name__=='__main__':main()

functional/preflight.py

@@ -0,0 +1,39 @@
+"""One 8-token real text, bounded memory/semantic capacity smoke check."""
+import argparse
+import json
+from pathlib import Path
+import resource
+import time
+import numpy as np
+from .model import StreamingLM
+from .run import tokenize,write_json
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',required=True,type=Path);a=p.parse_args()
+    samples,ids=tokenize(a.root)
+    start=time.time()
+    lm=StreamingLM(a.root/'weights',json.loads((a.root/'model_config.json').read_text()),seconds=1200)
+    snapshots=[]
+    result={'passed':False,'sample_id':samples[0]['id'],'tokens':8}
+    def progress(layer):
+        status=dict(line.split(':',1) for line in Path('/proc/self/status').read_text().splitlines() if ':' in line)
+        snapshots.append({'layer':layer,'seconds':time.time()-start,'rss_kib':int(status['VmRSS'].split()[0]),
+                          'process_swap_kib':int(status.get('VmSwap','0 kB').split()[0]),
+                          'available_kib':int(next(line.split()[1] for line in Path('/proc/meminfo').read_text().splitlines() if line.startswith('MemAvailable:')))})
+        print(json.dumps(snapshots[-1]),flush=True)
+    try:
+        h,_=lm.forward(ids[:1,:8],progress=progress)
+        nll=lm.nll(h,ids[:1,:8])
+        if not np.isfinite(nll).all():raise FloatingPointError('Nonfinite NLL')
+        result.update(passed=True,mean_nll=float(nll.mean()),token_nll=nll.tolist())
+    except Exception as exc:
+        result.update(error_type=type(exc).__name__,error=str(exc))
+        raise
+    finally:
+        result.update(elapsed_seconds=time.time()-start,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
+                      resource_snapshots=snapshots,limitation='Real-weight capacity check, not an independent correctness oracle or quality benchmark')
+        write_json(a.root/'preflight.json',result)
+
+
+if __name__=='__main__':main()

functional/reference.json

@@ -0,0 +1,5 @@
+{
+  "commit": "fd3cf0140de03f7619ac95b5cfed05b711ff9e22",
+  "url": "https://raw.githubusercontent.com/huggingface/transformers/fd3cf0140de03f7619ac95b5cfed05b711ff9e22/src/transformers/models/qwen3_5/modeling_qwen3_5.py",
+  "sha256": "4463b8a108f678c166755101d95f188179f54143305953d78147de0389b80ebc"
+}

functional/reference_check.py

@@ -0,0 +1,142 @@
+"""Remote-only numerical comparison with pinned official Torch definitions.
+
+Reads only a known official Transformers source file; extracts named mathematical
+operators/classes via AST, removing optional kernel decorators. Never imports the
+complete model or executes checkpoint repository code. No compilation/build.
+"""
+import argparse
+import ast
+import hashlib
+import json
+from pathlib import Path
+from types import SimpleNamespace
+import numpy as np
+from .model import StreamingLM, delta_rule, rope, norm
+
+EXPECTED_COMMIT = 'fd3cf0140de03f7619ac95b5cfed05b711ff9e22'
+NAMES = {'apply_mask_to_padding_states', 'l2norm', 'torch_chunk_gated_delta_rule', 'torch_recurrent_gated_delta_rule',
+         'causal_conv1d_fn', 'Qwen3_5RMSNormGated', 'Qwen3_5GatedDeltaNet',
+         'rotate_half', 'apply_rotary_pos_emb', 'repeat_kv', 'eager_attention_forward',
+         'Qwen3_5Attention', 'Qwen3_5MLP', 'Qwen3_5RMSNorm', 'Qwen3_5DecoderLayer',
+         'Qwen3_5TextRotaryEmbedding'}
+
+
+def official_namespace(source):
+    import torch
+    import torch.nn.functional as F
+    tree = ast.parse(source)
+    nodes = []
+    for node in tree.body:
+        if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in NAMES:
+            for child in ast.walk(node):
+                if isinstance(child, (ast.FunctionDef, ast.ClassDef)):
+                    child.decorator_list = [d for d in child.decorator_list if isinstance(d, ast.Name) and d.id in ('staticmethod','classmethod')]
+            nodes.append(node)
+    assert {x.name for x in nodes} == NAMES
+    tree = ast.Module(body=[ast.ImportFrom(module='__future__',names=[ast.alias(name='annotations')],level=0),*nodes],type_ignores=[])
+    ns = {'torch':torch, 'nn':torch.nn, 'F':F, 'ACT2FN':{'silu':F.silu},
+          'GradientCheckpointingLayer':torch.nn.Module,
+          'is_torchdynamo_exporting':lambda:False,
+          'ALL_ATTENTION_FUNCTIONS':SimpleNamespace(get_interface=lambda name, fallback:fallback)}
+    exec(compile(ast.fix_missing_locations(tree),'pinned_transformers_reference','exec'),ns)
+    return ns
+
+
+class ArrayLM(StreamingLM):
+    def __init__(self, arrays, config):
+        self.arrays, self.config, self.eps = arrays,config,config['rms_norm_eps']
+    def guard(self):
+        pass
+    def small(self,name):
+        return self.arrays[name]
+    def linear(self,x,name,rows=256):
+        return x@self.arrays[name+'.weight'].T
+    def embedding(self,ids):
+        return self.arrays['embed_tokens.weight'][ids]
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--source',type=Path,required=True);p.add_argument('--output',type=Path,required=True)
+    a=p.parse_args()
+    import torch
+    torch.set_num_threads(1)
+    torch.manual_seed(42)
+    provenance=json.loads(Path(__file__).with_name('reference.json').read_text())
+    assert hashlib.sha256(a.source.read_bytes()).hexdigest()==provenance['sha256'], 'Reference source hash mismatch'
+    ns=official_namespace(a.source.read_text())
+    cfg={'hidden_size':16,'num_attention_heads':2,'num_key_value_heads':1,'head_dim':8,
+         'intermediate_size':24,'hidden_act':'silu','attention_bias':False,'attention_dropout':0.,
+         'linear_num_value_heads':2,'linear_num_key_heads':2,'linear_key_head_dim':4,
+         'linear_value_head_dim':4,'linear_conv_kernel_dim':4,'rms_norm_eps':1e-6,
+         'layer_types':['linear_attention','full_attention'],'max_position_embeddings':4096,
+         'rope_parameters':{'rope_type':'default','rope_theta':10000000.,'partial_rotary_factor':.5,'mrope_section':[1,0,0]},
+         '_attn_implementation':'eager'}
+    config=SimpleNamespace(**cfg)
+    layers=[ns['Qwen3_5DecoderLayer'](config,i).eval() for i in range(2)]
+    for layer in layers:
+        for name,t in layer.named_parameters():
+            with torch.no_grad():
+                t.uniform_(-.2,.2)
+                if name=='linear_attn.norm.weight':t.add_(1.)
+    arrays={f'layers.{i}.'+k:v.detach().numpy().copy() for i,l in enumerate(layers) for k,v in l.state_dict().items()}
+    lm=ArrayLM(arrays,cfg)
+    rng=np.random.default_rng(123)
+    x=rng.standard_normal((2,7,16),dtype=np.float32)
+    tx=torch.from_numpy(x)
+    positions=torch.arange(7)[None,:]
+    rotary=ns['Qwen3_5TextRotaryEmbedding'](config)
+    cos,sin=rotary(tx,positions)
+    mask=torch.triu(torch.full((7,7),-float('inf')),diagonal=1)[None,None]
+    evidence=[]
+    def check(name,actual,expected,atol=2e-5,rtol=2e-5):
+        actual=np.asarray(actual);expected=expected.detach().numpy() if hasattr(expected,'detach') else np.asarray(expected)
+        np.testing.assert_allclose(actual,expected,atol=atol,rtol=rtol)
+        evidence.append({'test':name,'shape':list(actual.shape),'max_abs_error':float(np.max(np.abs(actual-expected))),'atol':atol,'rtol':rtol,'passed':True})
+    with torch.no_grad():
+        for i,layer in enumerate(layers):
+            n=lm.normalization(x,f'layers.{i}.input_layernorm')
+            tn=layer.input_layernorm(torch.from_numpy(x))
+            check(f'layer{i}_rmsnorm',n,tn)
+            if i==0:
+                out=lm.linear_attention(n,i)
+                tout=layer.linear_attn(tn)
+            else:
+                out,pr=lm.full_attention(n,i)
+                tout,tpr=layer.self_attn(tn,(cos,sin),mask)
+                check('attention_probabilities',pr,tpr)
+            check(f'layer{i}_mixer',out,tout)
+            h=x+out
+            n=lm.normalization(h,f'layers.{i}.post_attention_layernorm')
+            check(f'layer{i}_mlp',lm.mlp(n,i),layer.mlp(layer.post_attention_layernorm(torch.from_numpy(h))))
+            actual=h+lm.mlp(n,i)
+            expected=layer(torch.from_numpy(x),(cos,sin),attention_mask=mask if i else None)
+            check(f'layer{i}_complete',actual,expected)
+        # Independent recurrence/chunk equality, including nonmultiple sequence length.
+        for length in (1,7,33):
+            q=rng.standard_normal((2,length,2,4),dtype=np.float32)
+            k=rng.standard_normal(q.shape,dtype=np.float32)
+            v=rng.standard_normal((2,length,2,5),dtype=np.float32)
+            g=-np.abs(rng.standard_normal((2,length,2),dtype=np.float32))
+            beta=rng.uniform(size=g.shape).astype(np.float32)
+            out,state=delta_rule(q,k,v,g,beta)
+            args=[torch.from_numpy(z) for z in (q,k,v,g,beta)]
+            for kind in ('torch_recurrent_gated_delta_rule','torch_chunk_gated_delta_rule'):
+                ref,refstate=ns[kind](*args,output_final_state=True,use_qk_l2norm_in_kernel=True)
+                check(f'{kind}_length{length}_output',out,ref)
+                check(f'{kind}_length{length}_state',state,refstate)
+        # Full two-layer residual forward, final norm and embedding lookup.
+        arrays['embed_tokens.weight']=rng.standard_normal((31,16),dtype=np.float32)
+        arrays['norm.weight']=rng.standard_normal((16,),dtype=np.float32)*.1
+        ids=rng.integers(0,31,size=(2,7))
+        h=torch.from_numpy(arrays['embed_tokens.weight'][ids])
+        for i,l in enumerate(layers):h=l(h,(cos,sin),attention_mask=mask if i else None)
+        expected=h*torch.rsqrt(h.square().mean(-1,keepdim=True)+1e-6)*(1+torch.from_numpy(arrays['norm.weight']))
+        actual,_=lm.forward(ids)
+        check('complete_two_layer_text_forward',actual,expected)
+    result={'passed':True,'reference_commit':EXPECTED_COMMIT,'source_sha256':hashlib.sha256(a.source.read_bytes()).hexdigest(),
+            'torch':torch.__version__,'numpy':np.__version__,'implementation_sha256':hashlib.sha256(Path(__file__).with_name('model.py').read_bytes()).hexdigest(),'tests':evidence,'limitations':'FP32 CPU semantics on tiny synthetic configurations; not full-checkpoint BF16 kernel equivalence'}
+    a.output.write_text(json.dumps(result,indent=2))
+    print(json.dumps({'passed':True,'checks':len(evidence),'maximum_absolute_discrepancy':max(x['max_abs_error'] for x in evidence)}))
+
+
+if __name__=='__main__':main()

functional/report.py

@@ -0,0 +1,104 @@
+"""Render scientific pilot figures using saved data; never reopen model weights."""
+import argparse
+import csv
+import json
+from pathlib import Path
+import numpy as np
+import matplotlib
+matplotlib.use('Agg')
+import matplotlib.pyplot as plt
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    out=root/'figures';out.mkdir(exist_ok=True)
+    plt.rcParams.update({'font.size':10,'savefig.bbox':'tight','pdf.fonttype':42,'svg.fonttype':'none'})
+    def save(fig,name):
+        for ext in ('png','pdf','svg'):fig.savefig(out/f'{name}.{ext}',dpi=180)
+        plt.close(fig)
+    status=json.loads((root/'functional-status.json').read_text())
+    if (root/'candidate-summary.json').exists():
+        c=json.loads((root/'candidate-summary.json').read_text())
+        fig,ax=plt.subplots(figsize=(10,4.5))
+        ax.bar(np.arange(len(c)),[max(v['maximum_symmetric_relative_error'],1e-8) for v in c],color=['#238b45' if v['passed'] else '#9e9e9e' for v in c])
+        ax.axhline(.05,color='#b2182b',linestyle='--',label='Relative L2 screen = 0.05; other gates also apply')
+        ax.set_yscale('log');ax.set_xticks(np.arange(len(c)),[v['id'] for v in c],rotation=45,ha='right')
+        ax.set_ylabel('Worst calibration directional relative L2')
+        ax.set_title('Fixed candidates; same-input function comparison\n4 calibration texts, 32 tokens; no training')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.38),fontsize=8);fig.tight_layout();save(fig,'candidate_function_errors')
+    if (root/'interventions.json').exists():
+        interventions=json.loads((root/'interventions.json').read_text())
+        fig,ax=plt.subplots(figsize=(8,4.5))
+        for idx,r in enumerate(interventions):
+            values=r['sequence_nll_deltas'];x=np.arange(len(values))
+            ax.plot(x,values,'o-',label=r['direction'])
+        ax.axhline(0,color='gray',linewidth=.7);ax.axhline(.01,color='#b2182b',linestyle='--',label='Mean-delta budget (not per-sequence bound)')
+        ax.set_xticks(np.arange(4),['heldout zh1','heldout zh2','heldout en1','heldout en2'])
+        ax.set_ylabel('Token NLL change (nat/token)');ax.set_title('Single-site interventions; frozen candidate\n4 heldout texts, 32 tokens; limited pilot evidence')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.18),fontsize=8);fig.tight_layout();save(fig,'heldout_intervention_nll')
+    norm_path=root/'norm-repetition-summary.json'
+    repetition=json.loads(norm_path.read_text()) if norm_path.exists() else {}
+    if (root/'norm-repetition-function-metrics.csv').exists():
+        with (root/'norm-repetition-function-metrics.csv').open() as f:rows=list(csv.DictReader(f))
+        sample_ids=list(dict.fromkeys(row['sample_id'] for row in rows))
+        fig,ax=plt.subplots(figsize=(9,4.8))
+        for source in sorted({row['source_layer'] for row in rows},key=int):
+            rr=[r for r in rows if r['source_layer']==source]
+            ax.plot(range(len(rr)),[max(float(r['relative_l2']),float(r['reverse_relative_l2'])) for r in rr],'o-',label=f'Input captured at layer {source}')
+        ax.axhline(.05,color='#b2182b',ls='--',label='Heldout empirical screen = 0.05')
+        chosen=repetition.get('selected')
+        if chosen:ax.axhline(chosen['worst_uniform_bound'],color='#6a51a3',ls=':',label='Proven uniform all-input bound')
+        ax.axvline(3.5,color='gray',linewidth=.8)
+        ax.set_xticks(range(len(sample_ids)),sample_ids,rotation=30,ha='right');ax.set_ylabel('Worst of two directional relative L2 errors')
+        ax.set_title('Learned input RMSNorm at layer indices 12/13: same-input comparison\nCalibration (left) / independent heldout texts (right), 32 tokens each')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.25),fontsize=8);fig.tight_layout();save(fig,'learned_norm_function_errors')
+    if repetition.get('interventions'):
+        fig,ax=plt.subplots(figsize=(8,4.5))
+        for v in repetition['interventions']:
+            ax.plot(range(4),v['sequence_nll_deltas'],'o-',label=f'Layer {v["target"]} uses {v["donor"]}; mean={v["mean_nll_delta"]:.5g}')
+        ax.axhline(0,color='gray',linewidth=.8);ax.set_xticks(range(4),['zh1','zh2','en1','en2'])
+        ax.set_xlabel('Independent heldout text (32 tokens)');ax.set_ylabel('Mean token NLL change (nat/token)')
+        ax.set_title('Bidirectional single-site substitution of existing learned RMSNorm modules')
+        ax.legend(loc='upper left',bbox_to_anchor=(0,-.18),fontsize=8);fig.tight_layout();save(fig,'learned_norm_network_substitution')
+    if (root/'gain-factorization-score-metrics.csv').exists():
+        with (root/'gain-factorization-score-metrics.csv').open() as f:rr=list(csv.DictReader(f))
+        layers=sorted({int(v['layer']) for v in rr});fig,axes=plt.subplots(1,2,figsize=(10,4))
+        for ax,key,label in zip(axes,['score_max_abs_error','probability_max_abs_error'],['Maximum absolute score error','Maximum absolute probability error']):
+            for direction in ('q_to_k','k_to_q'):
+                ax.plot(layers,[max(float(v[key]) for v in rr if int(v['layer'])==layer and v['direction']==direction) for layer in layers],'o-',label=direction)
+            ax.set_xlabel('Full-attention layer index');ax.set_xticks(layers);ax.set_ylabel(label);ax.ticklabel_format(axis='y',style='sci',scilimits=(0,0));ax.legend(loc='upper left',fontsize=8)
+        fig.suptitle('Supplement: algebraic reparameterization, separate from learned-function repetition');fig.tight_layout();save(fig,'algebraic_gain_identity')
+    neuron_path=root/'neuron-reuse-summary.json'
+    neuron=json.loads(neuron_path.read_text()) if neuron_path.exists() else {}
+    text=['# CPU learned-function repetition pilot','',
+          'Primary question: do two distinct trained parts implement similar functions? Cost is not a success condition.',
+          f'- Existing learned RMSNorm function positive: {repetition.get("learned_function_repetition_positive",False)}',
+          f'- Bidirectional network quality screen passed: {repetition.get("bidirectional_network_quality_pass",False)}',
+          f'- Learned scalar MLP neuron search positive: {neuron.get("empirical_repeated_function_positive",False)}',
+          f'- Original 11 candidate run state: {status["state"]}; selected: {status.get("selected_candidate",[])}',
+          '- All model weights are retained on the server until the user explicitly requests deletion.','']
+    if repetition.get('selected'):
+        c=repetition['selected'];text += ['## Existing learned module evidence','',
+          f'Role: {c["role"]}; layers {c["a"]} and {c["b"]}; each maps 2048-dimensional input to output.',
+          f'Uniform real-arithmetic relative-error bound, first-to-second: {c["a_to_b_uniform_bound"]:.12g}; reverse: {c["b_to_a_uniform_bound"]:.12g}.',
+          f'Maximum heldout directional output relative L2: {repetition.get("worst_heldout_relative_l2")}; minimum cosine: {repetition.get("worst_heldout_cosine")}.',
+          'Same epsilon and nonzero gains: the common RMS denominator cancels. For any nonzero input x,',
+          '`||f_a(x)-f_b(x)|| / ||f_a(x)|| <= max_j |a_j-b_j|/|a_j|`; swap a/b for the reverse direction.',
+          'Selection used only the tight all-input bound across 552 same-role pairs, before inspecting inputs.',
+          'Real probes use four calibration and four heldout experiment-authored texts, 32 tokens each, and both source layers.',
+          'These are learned, nonconstant gain vectors. The claim concerns RMSNorm modules; it does not establish attention/MLP-body duplication.','']
+        for v in repetition.get('interventions',[]):text += [f'- Layer {v["target"]} uses {v["donor"]}: mean heldout NLL delta {v["mean_nll_delta"]:.12g}; sequence P95 delta {v["sequence_p95_nll_delta"]:.12g}; screen pass={v["passed_quality"]}.']
+    text += ['', '## Negative results and separate algebraic result','',
+             'The original whole-attention, whole-MLP and within-layer Q/K norm candidates did not pass their gates.',
+             'The scalar neuron search excludes low-RMS/near-constant profiles and selects at most one pair without heldout feedback; consult its search summary for coverage.',
+             'Q/K non-RoPE gain consolidation is a separately verified real-arithmetic identity, not evidence that two existing learned modules duplicate functions.',
+             'Earlier local CPU timing records are preserved for provenance; performance is outside the current success criterion.','',
+             '## Limits and provenance','',
+             'Original BF16 weights were decoded into streaming FP32 CPU operators; no training, scale fitting, R fitting or GPU.',
+             'The numerical reference includes 22 independent official Torch comparisons. Early adapter errors and two resource-guard stops are preserved as historical logs.',
+             'The persistent computation ledger includes a conservative 180-second reserve for early checks; no budget reset after that reserve.',
+             'The text sample is tiny and short. Network NLL checks are not broad task-accuracy validation.','']
+    (root/'FUNCTIONAL_REPORT.md').write_text('\n'.join(text))
+
+
+if __name__=='__main__':main()

functional/run.py

@@ -0,0 +1,227 @@
+"""Bounded CPU pilot. One fixed model; calibration selects, heldout only validates."""
+import argparse
+import csv
+import hashlib
+import json
+import os
+from pathlib import Path
+import resource
+import time
+import traceback
+import urllib.request
+import numpy as np
+from .model import StreamingLM, norm
+
+ATTN_PAIRS=[(3,7),(11,15),(19,23)]
+MLP_PAIRS=[(5,6),(20,21)]
+NORM_LAYERS=[3,7,11,15,19,23]
+REVISION='15852e8c16360a2fea060d615a32b45270f8a8fc'
+
+
+def write_json(path,obj):
+    tmp=path.with_suffix('.tmp')
+    tmp.write_text(json.dumps(obj,indent=2,allow_nan=False)+'\n')
+    tmp.replace(path)
+
+
+def error_rows(a,b,samples,metadata):
+    records=[]
+    for x,y,sample in zip(a,b,samples):
+        xf,yf=x.astype(np.float64).reshape(-1),y.astype(np.float64).reshape(-1)
+        xn,yn=np.linalg.norm(xf),np.linalg.norm(yf)
+        delta=np.linalg.norm(xf-yf)
+        tokenerr=np.linalg.norm((x-y).reshape(x.shape[0],-1).astype(np.float64),axis=-1)/np.maximum(np.linalg.norm(x.reshape(x.shape[0],-1).astype(np.float64),axis=-1),1e-12)
+        records.append({**metadata,'sample_id':sample['id'],'split':sample['split'],
+                        'relative_l2':float(delta/max(xn,1e-12)),
+                        'reverse_relative_l2':float(delta/max(yn,1e-12)),
+                        'token_relative_p95':float(np.quantile(tokenerr,.95)),
+                        'reverse_token_relative_p95':float(np.quantile(np.linalg.norm((x-y).reshape(x.shape[0],-1).astype(np.float64),axis=-1)/np.maximum(np.linalg.norm(y.reshape(x.shape[0],-1).astype(np.float64),axis=-1),1e-12),.95)),
+                        'cosine':float(xf@yf/max(xn*yn,1e-30)),
+                        'low_norm_tokens':int((np.linalg.norm(x.reshape(x.shape[0],-1),axis=-1)<1e-8).sum())})
+    return records
+
+
+def probability_metrics(a,b):
+    m=(a+b)*.5
+    def term(p):
+        return np.where(p>0,p*np.log(np.maximum(p,1e-30)/np.maximum(m,1e-30)),0)
+    js=(term(a)+term(b)).sum(axis=-1)*.5
+    overlap=[]
+    for row in range(a.shape[-2]):
+        k=min(5,row+1)
+        ai=np.argsort(a[:,:,row,:row+1],axis=-1)[...,-k:]
+        bi=np.argsort(b[:,:,row,:row+1],axis=-1)[...,-k:]
+        overlap.append(np.mean((ai[...,None]==bi[...,None,:]).any(axis=-1),axis=-1))
+    # Attention-sink control: remove key zero, renormalize valid remaining mass.
+    aa,bb=a[...,1:],b[...,1:]
+    mass_a,mass_b=aa.sum(-1),bb.sum(-1)
+    valid=(mass_a>1e-12)&(mass_b>1e-12)
+    aa=aa/np.maximum(mass_a[...,None],1e-30)
+    bb=bb/np.maximum(mass_b[...,None],1e-30)
+    mm=(aa+bb)*.5
+    conditional_js=.5*(np.where(aa>0,aa*np.log(np.maximum(aa,1e-30)/np.maximum(mm,1e-30)),0)+np.where(bb>0,bb*np.log(np.maximum(bb,1e-30)/np.maximum(mm,1e-30)),0)).sum(-1)
+    return {'first_key_mass_a':float(a[...,0].mean()),'first_key_mass_b':float(b[...,0].mean()),
+            'js_without_first_key':float(conditional_js[valid].mean()) if valid.any() else None,
+            'without_first_key_valid_rows':int(valid.sum()),'js_mean':float(js.mean()),'js_p95':float(np.quantile(js,.95)),
+            'js_without_first_query':float(js[...,1:].mean()),
+            'top5_overlap':float(np.stack(overlap,axis=-1).mean())}
+
+
+def save_csv(path,rows):
+    if not rows:return
+    keys=list(dict.fromkeys(k for r in rows for k in r))
+    with path.open('w',newline='') as f:
+        w=csv.DictWriter(f,fieldnames=keys);w.writeheader();w.writerows(rows)
+
+
+def tokenize(root):
+    from tokenizers import Tokenizer
+    path=root/'tokenizer.json'
+    if not path.exists():
+        url=f'https://huggingface.co/Qwen/Qwen3.5-2B/resolve/{REVISION}/tokenizer.json'
+        with urllib.request.urlopen(url,timeout=180) as r:path.write_bytes(r.read())
+    tokenizer=Tokenizer.from_file(str(path))
+    samples=json.loads(Path(__file__).with_name('samples.json').read_text())['samples']
+    ids=[]
+    for s in samples:
+        tokens=tokenizer.encode(s['text'],add_special_tokens=False).ids
+        if len(tokens)<32:raise ValueError(f'Text too short: {s["id"]}')
+        s['text_sha256']=hashlib.sha256(s['text'].encode()).hexdigest()
+        s['full_token_count']=len(tokens);s['token_ids']=tokens[:32]
+        ids.append(tokens[:32])
+    write_json(root/'samples-tokenized.json',{'revision':REVISION,'tokenizer_sha256':hashlib.sha256(path.read_bytes()).hexdigest(),'samples':samples})
+    return samples,np.array(ids,dtype=np.int64)
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);p.add_argument('--seconds',type=int,default=7200)
+    args=p.parse_args();root=args.root;root.mkdir(exist_ok=True,parents=True)
+    write_json(root/'retention-policy.json',{'weights':'retain_until_explicit_user_delete_request','automatic_cleanup':False,'instruction_date':'2026-11-09'})
+    samples,ids=tokenize(root)
+    cal=np.array([i for i,s in enumerate(samples) if s['split']=='calibration'])
+    held=np.array([i for i,s in enumerate(samples) if s['split']=='heldout'])
+    # Freeze all candidates/thresholds before opening weights or measuring functions.
+    experiment={'revision':REVISION,'attention_pairs':ATTN_PAIRS,'mlp_pairs':MLP_PAIRS,'norm_layers':NORM_LAYERS,
+                'seconds':args.seconds,'output_relative_l2_limit':.05,'token_p95_limit':.1,
+                'attention_js_limit':.01,'attention_top5_min':.8,'heldout_nll_mean_delta_limit':.01,
+                'heldout_nll_sequence_p95_delta_limit':.05,'max_intervention_candidates':1,
+                'directions_per_selected_candidate':2,'training':False,'search_R':False,'automatic_weight_cleanup':False,
+                'dtype':'original weights decoded to FP32; not a BF16 backend benchmark',
+                'blas_threads':os.environ.get('OPENBLAS_NUM_THREADS')}
+    write_json(root/'experiment.json',experiment)
+    started=time.time();status={'state':'running','started_unix':started,'phase':'baseline'}
+    write_json(root/'functional-status.json',status)
+    rows=[];headrows=[];interventions=[]
+    try:
+        config=json.loads((root/'model_config.json').read_text())
+        lm=StreamingLM(root/'weights',config,seconds=args.seconds)
+        overall_deadline=lm.deadline
+        lm.deadline=min(overall_deadline,time.monotonic()+2400)
+        lm.capture_qk=True
+        capture=[('attn',i) for pair in ATTN_PAIRS for i in pair]+[('mlp',i) for pair in MLP_PAIRS for i in pair]
+        def progress(layer):
+            status.update(last_completed_layer=layer,elapsed_seconds=time.time()-started)
+            write_json(root/'functional-status.json',status)
+            print(json.dumps({'phase':status['phase'],'layer':layer,'elapsed':time.time()-started}),flush=True)
+        h,saved=lm.forward(ids,capture=capture,progress=progress)
+        np.savez(root/'baseline-activations.npz',**saved,**{f'qraw_{k}':v[0] for k,v in lm.qk_inputs.items()},**{f'kraw_{k}':v[1] for k,v in lm.qk_inputs.items()},final=h)
+        # Baseline heldout NLL uses no selection information.
+        nll=lm.nll(h[held],ids[held]);np.save(root/'baseline-heldout-token-nll.npy',nll)
+        baseline_nll=nll.mean(axis=1)
+        lm.capture_qk=False
+        lm.deadline=min(overall_deadline,time.monotonic()+1800)
+        status['phase']='calibration';write_json(root/'functional-status.json',status)
+        candidates=[]
+        for kind,pairs in [('attn',ATTN_PAIRS),('mlp',MLP_PAIRS)]:
+            for i,j in pairs:
+                pairid=f'{kind}_{i}_{j}';pairrows=[]
+                for source in (i,j):
+                    x=saved[f'{kind}_{source}'][cal]
+                    if kind=='attn':
+                        a,pa=lm.full_attention(x,i);b,pb=lm.full_attention(x,j)
+                        pm={}
+                        for bi,si in enumerate(cal):
+                            for hi in range(pa.shape[1]):
+                                hpm=probability_metrics(pa[bi:bi+1,hi:hi+1],pb[bi:bi+1,hi:hi+1])
+                                headrows.append({'candidate':pairid,'source':source,'sample_id':samples[si]['id'],'head':hi,**hpm})
+                        save_csv(root/'attention-head-metrics.csv',headrows)
+                    else:
+                        a,b=lm.mlp(x,i),lm.mlp(x,j);pm={}
+                    rec=error_rows(a,b,[samples[t] for t in cal],{'candidate':pairid,'kind':kind,'source':source,**pm})
+                    if kind=='attn':
+                        for bi,r in enumerate(rec):r.update(probability_metrics(pa[bi:bi+1],pb[bi:bi+1]))
+                    rows.extend(rec);pairrows.extend(rec)
+                    save_csv(root/'calibration-metrics.csv',rows)
+                passing=all(r['relative_l2']<=.05 and r['reverse_relative_l2']<=.05 and r['token_relative_p95']<=.1 and r['reverse_token_relative_p95']<=.1 for r in pairrows)
+                if kind=='attn':passing=passing and all(r['js_mean']<=.01 and r['top5_overlap']>=.8 for r in pairrows)
+                prefix=f'model.language_model.layers.{i}.'+('self_attn.' if kind=='attn' else 'mlp.')
+                size=sum(t.bytes for name,t in lm.tensors.items() if name.startswith(prefix))
+                candidates.append({'id':pairid,'kind':kind,'i':i,'j':j,'passed':passing,'maximum_symmetric_relative_error':max(max(r['relative_l2'],r['reverse_relative_l2']) for r in pairrows),'donor_component_bytes':size})
+        for layer in NORM_LAYERS:
+            pairid=f'qk_norm_{layer}';pfx=f'layers.{layer}.self_attn.'
+            qw,kw=lm.small(pfx+'q_norm.weight'),lm.small(pfx+'k_norm.weight')
+            pairrows=[]
+            for source,x in zip(('q','k'),lm.qk_inputs[layer]):
+                a,b=norm(x[cal],qw,lm.eps),norm(x[cal],kw,lm.eps)
+                rec=error_rows(a,b,[samples[t] for t in cal],{'candidate':pairid,'kind':'qk_norm','source':source})
+                pairrows.extend(rec);rows.extend(rec)
+            passing=all(r['relative_l2']<=.05 and r['reverse_relative_l2']<=.05 and r['token_relative_p95']<=.1 and r['reverse_token_relative_p95']<=.1 for r in pairrows)
+            candidates.append({'id':pairid,'kind':'qk_norm','i':layer,'j':layer,'passed':passing,'maximum_symmetric_relative_error':max(max(r['relative_l2'],r['reverse_relative_l2']) for r in pairrows),'donor_component_bytes':lm.tensors['model.language_model.'+pfx+'q_norm.weight'].bytes})
+        save_csv(root/'calibration-metrics.csv',rows)
+        write_json(root/'candidate-summary.json',candidates)
+        passed=[c for c in candidates if c['passed']]
+        # Prefer larger potential savings, then smaller worst calibration error; never consult heldout.
+        selected=sorted(passed,key=lambda c:(-c['donor_component_bytes'],c['maximum_symmetric_relative_error'],c['id']))[:1]
+        write_json(root/'selection.json',{'selected':selected,'rule':'calibration pass; descending component bytes, ascending maximum symmetric error; maximum one candidate'})
+        if selected:
+            c=selected[0]
+            heldrows=[]
+            if c['kind']=='qk_norm':
+                pfx=f'layers.{c["i"]}.self_attn.'
+                qw,kw=lm.small(pfx+'q_norm.weight'),lm.small(pfx+'k_norm.weight')
+                for source,x in zip(('q','k'),lm.qk_inputs[c['i']]):
+                    heldrows.extend(error_rows(norm(x[held],qw,lm.eps),norm(x[held],kw,lm.eps),[samples[t] for t in held],{'candidate':c['id'],'kind':'qk_norm','source':source}))
+            else:
+                for source in (c['i'],c['j']):
+                    x=saved[f'{c["kind"]}_{source}'][held]
+                    if c['kind']=='attn':
+                        a,_=lm.full_attention(x,c['i']);b,_=lm.full_attention(x,c['j'])
+                    else:a,b=lm.mlp(x,c['i']),lm.mlp(x,c['j'])
+                    heldrows.extend(error_rows(a,b,[samples[t] for t in held],{'candidate':c['id'],'kind':c['kind'],'source':source}))
+            save_csv(root/'heldout-function-metrics.csv',heldrows)
+            held_function_pass=all(r['relative_l2']<=.05 and r['reverse_relative_l2']<=.05 and r['token_relative_p95']<=.1 and r['reverse_token_relative_p95']<=.1 for r in heldrows)
+            status['heldout_function_pass']=held_function_pass
+            lm.deadline=min(overall_deadline,time.monotonic()+1800)
+            status['phase']='intervention';write_json(root/'functional-status.json',status)
+            for reverse in (False,True):
+                replacement=None;lm.norm_replacement=None
+                if c['kind']=='qk_norm':
+                    pfx=f'layers.{c["i"]}.self_attn.'
+                    target,donor=('q_norm','k_norm') if not reverse else ('k_norm','q_norm')
+                    lm.norm_replacement=(pfx+target,pfx+donor)
+                    label=target+'_uses_'+donor
+                else:
+                    target,donor=(c['i'],c['j']) if not reverse else (c['j'],c['i'])
+                    replacement=(c['kind'],target,donor);label=f'{target}_uses_{donor}'
+                t=time.monotonic();modified,_=lm.forward(ids[held],replacement=replacement,progress=progress)
+                changed=lm.nll(modified,ids[held]);np.save(root/f'intervention-{label}-token-nll.npy',changed)
+                delta=changed.mean(axis=1)-baseline_nll
+                rec={'candidate':c['id'],'direction':label,'elapsed_seconds':time.monotonic()-t,
+                     'nll_mean_delta':float(delta.mean()),'nll_sequence_p95_delta':float(np.quantile(delta,.95)),
+                     'sequence_nll_deltas':delta.tolist(),'passed_quality':bool(delta.mean()<=.01 and np.quantile(delta,.95)<=.05)}
+                interventions.append(rec);write_json(root/'interventions.json',interventions)
+            # No fallback candidate if heldout fails.
+        status.update(state='completed',phase='done',selected_candidate=selected,
+                      positive_bidirectional_quality=bool(interventions and status.get('heldout_function_pass') and all(x['passed_quality'] for x in interventions)),
+                      elapsed_seconds=time.time()-started,peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss,
+                      claim_limit='Small text-only heldout pilot; no global interchangeability proof, no measured GPU speedup; component storage savings require explicit layout accounting.')
+    except Exception as exc:
+        status.update(state='stopped',error_type=type(exc).__name__,error=str(exc),elapsed_seconds=time.time()-started,
+                      peak_rss_kib=resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)
+        (root/'functional-error.txt').write_text(traceback.format_exc())
+        raise
+    finally:
+        write_json(root/'functional-status.json',status)
+
+
+if __name__=='__main__':main()

functional/samples.json

@@ -0,0 +1,13 @@
+{
+  "origin": "Experiment-authored plain text, no external corpus; fixed before functional metrics. Chinese/English balanced in each split. No chat template. First 32 tokenizer tokens retained.",
+  "samples": [
+    {"id":"cal-zh-1","split":"calibration","language":"zh","text":"清晨，研究员打开实验记录本，检查昨天保存的数据。他先确认文件完整，再比较两组观测结果。为了避免偶然误差，他为每一步留下时间和条件，并将尚未验证的推测单独记录。"},
+    {"id":"cal-zh-2","split":"calibration","language":"zh","text":"一辆列车从山间驶向海边，经过村庄、桥梁和开阔的田野。旅客望着窗外变化的景色，计划抵达后先步行到旅馆，再去市场购买晚餐需要的食材。天气预报说傍晚可能下雨。"},
+    {"id":"cal-en-1","split":"calibration","language":"en","text":"A small library opens every morning beside the river. The librarian checks returned books, repairs damaged covers, and places new arrivals on a wooden table. Visitors often stop to read before continuing their journey across the old bridge."},
+    {"id":"cal-en-2","split":"calibration","language":"en","text":"To compare two measurements, first record their units and the conditions under which they were collected. Compute the difference, inspect unusual observations, and repeat the procedure with independent examples. A short report should separate observations from explanations and unresolved questions."},
+    {"id":"test-zh-1","split":"heldout","language":"zh","text":"午后，工程师检查温室中的传感器。温度记录出现一个异常值，她没有立刻删除，而是查看设备状态并补做一次测量。确认原因以后，她调整了采样计划，在报告中说明这次变化。"},
+    {"id":"test-zh-2","split":"heldout","language":"zh","text":"周末，家人沿着湖边的小路散步。他们带了水和简单的午餐，打算在树林旁休息。途中遇到一处正在维修的桥，于是参考路标选择另一条道路，终于在日落前回到车站。"},
+    {"id":"test-en-1","split":"heldout","language":"en","text":"The gardener planted several rows of beans behind the house. Each evening she checked the soil and wrote down how much water the plants received. After a week of warm weather, the first green shoots appeared near the fence beside the apple tree."},
+    {"id":"test-en-2","split":"heldout","language":"en","text":"When a program reads a table, it should check column names and handle missing entries explicitly. The developer tested a small example, compared the output with a hand calculation, and saved the input alongside the result so another person could reproduce the check."}
+  ]
+}

functional/storage.py

@@ -0,0 +1,62 @@
+"""Measure an explicit two-component alias storage layout, not inference speed.
+
+Payloads remain on the server under weights/ and must not be committed or cleaned
+without the user's explicit deletion request. Manifest/byte/hash evidence may be archived.
+"""
+import argparse
+import hashlib
+import json
+from pathlib import Path
+import numpy as np
+from pilot.tensors import load_tensors,decode
+from .run import write_json
+
+
+def payload(t,path):
+    digest=hashlib.sha256()
+    with path.open('wb') as f:
+        for start in range(0,t.raw.size,262144):
+            b=t.raw[start:start+262144].tobytes();f.write(b);digest.update(b)
+            t.evict()
+    return {'file':path.name,'dtype':t.dtype,'shape':list(t.shape),'bytes':path.stat().st_size,'sha256':digest.hexdigest()}
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);a=p.parse_args();root=a.root
+    selected=json.loads((root/'selection.json').read_text())['selected']
+    if not selected:
+        write_json(root/'storage-evidence.json',{'measured':False,'reason':'No calibration candidate selected'});return
+    c=selected[0]
+    if c['kind']!='qk_norm':
+        write_json(root/'storage-evidence.json',{'measured':False,'reason':'Only two-gain alias layout implemented in this bounded pilot; larger-module savings remain theoretical'});return
+    tensors=load_tensors(root/'weights')
+    prefix=f'model.language_model.layers.{c["i"]}.self_attn.'
+    names=[prefix+'q_norm.weight',prefix+'k_norm.weight']
+    # Original pair plus two alternative layouts; each is counted independently.
+    path=root/'component-layout'/'weights';path.mkdir(parents=True,exist_ok=True)
+    original={'format':'two_component_raw_v1','tensors':{}}
+    for short,name in zip(('q','k'),names):original['tensors'][name]=payload(tensors[name],path/f'{short}.bin')
+    original_bytes=json.dumps(original,separators=(',',':'),sort_keys=True).encode()
+    (path/'original-layout.json').write_bytes(original_bytes)
+    evidence=[]
+    for target,donor in ((names[0],names[1]),(names[1],names[0])):
+        entry=original['tensors'][donor]
+        layout={'format':'two_component_raw_v1','tensors':{donor:entry},'aliases':{target:donor}}
+        encoded=json.dumps(layout,separators=(',',':'),sort_keys=True).encode()
+        file=path/('share-'+('k' if donor==names[1] else 'q')+'.json');file.write_bytes(encoded)
+        t=tensors[donor]
+        raw=np.fromfile(path/entry['file'],dtype=t.raw.dtype)
+        reconstructed=decode(raw,t.dtype).reshape(t.shape)
+        np.testing.assert_array_equal(reconstructed,t.read().reshape(t.shape));t.evict()
+        before=sum(e['bytes'] for e in original['tensors'].values())+len(original_bytes)
+        after=entry['bytes']+len(encoded)
+        evidence.append({'target':target,'donor':donor,'original_payload_bytes':sum(e['bytes'] for e in original['tensors'].values()),
+                         'shared_payload_bytes':entry['bytes'],'original_descriptor_bytes':len(original_bytes),
+                         'shared_descriptor_bytes':len(encoded),'original_total_bytes':before,'shared_total_bytes':after,
+                         'net_saved_bytes':before-after,'shared_descriptor_sha256':hashlib.sha256(encoded).hexdigest(),
+                         'decoded_donor_exactly_matches_intervention_gain':True})
+    write_json(root/'storage-evidence.json',{'measured':True,'candidate':c['id'],'layouts':evidence,
+               'claim_limit':'Measured standalone component layout including descriptors, not an integrated compressed checkpoint, latency gain or full-model RSS reduction. Two sharing directions are alternatives and must not be added.'})
+
+
+if __name__=='__main__':main()

functional/supervise.py

@@ -0,0 +1,95 @@
+"""One bounded CPU run; waits for verified download and binary dependencies.
+
+No installation, retry search, cleanup, training, GPU selection or Git mutation.
+"""
+import argparse
+import hashlib
+import importlib.util
+import json
+from pathlib import Path
+import subprocess
+import sys
+import time
+import traceback
+from .run import write_json
+
+
+def main():
+    p=argparse.ArgumentParser();p.add_argument('--root',type=Path,required=True);p.add_argument('--download-wait-seconds',type=int,default=14400)
+    a=p.parse_args();root=a.root.resolve();start=time.monotonic()
+    state={'state':'waiting','phase':'dependencies','weight_retention':'until_explicit_user_delete_request'}
+    def status(**kwargs):
+        state.update(kwargs,supervisor_elapsed_seconds=time.monotonic()-start)
+        write_json(root/'supervisor-status.json',state)
+    def execute(module,args,timeout,log):
+        ledger_path=root/'compute-budget-ledger.json'
+        ledger=json.loads(ledger_path.read_text()) if ledger_path.exists() else {'spent_seconds':0.,'calls':[]}
+        timeout=min(timeout,int(7200-ledger['spent_seconds']))
+        if timeout<1:raise TimeoutError('Cumulative two-hour compute budget exhausted')
+        t=time.monotonic()
+        try:
+            with (root/log).open('w') as out:
+                r=subprocess.run([sys.executable,'-B','-m',module,*args],stdout=out,stderr=subprocess.STDOUT,timeout=timeout)
+        finally:
+            elapsed=time.monotonic()-t
+            ledger['spent_seconds']+=elapsed
+            ledger['calls'].append({'module':module,'seconds':elapsed})
+            write_json(ledger_path,ledger)
+        if r.returncode:raise RuntimeError(f'{module} failed: exit {r.returncode}, see {log}')
+    status()
+    try:
+        while True:
+            ready=False
+            if all(importlib.util.find_spec(name) for name in ('torch','tokenizers')):
+                try:
+                    r=subprocess.run([sys.executable,'-B','-c','import torch, tokenizers; torch.empty(1)'],capture_output=True,timeout=30)
+                    ready=r.returncode==0
+                except subprocess.TimeoutExpired:pass
+            if ready:break
+            if time.monotonic()-start>3600:raise TimeoutError('Binary reference dependencies not ready within one hour')
+            status()
+            time.sleep(15)
+        compute_start=time.monotonic()
+        status(state='running',phase='reference_check')
+        provenance={str(p):hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(Path('functional').glob('*')) if p.is_file()}
+        write_json(root/'functional-code-hashes.json',provenance)
+        execute('functional.reference_check',['--source',str(root/'reference.py'),'--output',str(root/'reference-check.json')],300,'reference-check.log')
+        reference_seconds=time.monotonic()-compute_start
+        status(state='waiting',phase='verified_weights',reference_seconds=reference_seconds)
+        while True:
+            path=root/'manifest.json'
+            try:
+                manifest=json.loads(path.read_text())
+                weights=[f for f in manifest.get('files',[]) if f['name'].endswith('.safetensors')]
+                ready=(manifest.get('completed_unix') is not None and weights and
+                       all((root/'weights'/f['name']).is_file() and (root/'weights'/f['name']).stat().st_size==f['bytes'] and f['sha256']==f['upstream_sha256'] for f in weights))
+            except (OSError,ValueError,KeyError):ready=False
+            if ready:break
+            if time.monotonic()-start>a.download_wait_seconds:raise TimeoutError('Verified weight download not ready before wait deadline')
+            status()
+            time.sleep(15)
+        status(state='running',phase='real_weight_preflight')
+        compute_start=time.monotonic()
+        execute('functional.preflight',['--root',str(root)],1200,'preflight.log')
+        preflight_seconds=time.monotonic()-compute_start
+        remaining=int(7200-json.loads((root/'compute-budget-ledger.json').read_text())['spent_seconds']-60)
+        if remaining<60:raise TimeoutError('No remaining bounded functional computation budget')
+        status(phase='functional_analysis',preflight_seconds=preflight_seconds,remaining_seconds=remaining)
+        execute('functional.run',['--root',str(root),'--seconds',str(remaining)],remaining+15,'functional-run.log')
+        status(phase='storage_accounting')
+        execute('functional.storage',['--root',str(root)],120,'storage-accounting.log')
+        status(phase='report')
+        execute('functional.report',['--root',str(root)],120,'functional-report.log')
+        status(state='completed',phase='done')
+    except Exception as exc:
+        status(state='stopped',error_type=type(exc).__name__,error=str(exc))
+        (root/'supervisor-error.txt').write_text(traceback.format_exc())
+        # Retain all weights/results. No automatic retries or candidate expansion.
+        if not (root/'functional-status.json').exists():
+            write_json(root/'functional-status.json',{'state':'stopped','phase':state['phase'],'error_type':type(exc).__name__,'error':str(exc)})
+        try:execute('functional.report',['--root',str(root)],120,'functional-report.log')
+        except Exception:pass
+        raise
+
+
+if __name__=='__main__':main()

requirements-functional.txt

@@ -0,0 +1,5 @@
+# Existing requirements.txt supplies NumPy and plotting dependencies.
+# Install binary wheels only. Torch is a reference-check dependency, not the streaming runtime.
+tokenizers==0.23.3
+# Install separately with --index-url https://download.pytorch.org/whl/cpu:
+# torch==2.15.0+cpu

results/cpu-qwen35-2b/DELIVERY.md

@@ -0,0 +1,7 @@
+# Scientific delivery and retained caches
+
+All figure CSV/JSON source data, per-token NLL, protocols, reports, metadata and historical logs are included.
+Three large reproducible intermediate activation caches remain on 8.211.154.134 and in the original full local archive;
+`external-activation-cache.json` records exact locations, sizes and SHA256. No weight or cache was deleted.
+The original full experiment manifest is `checksums-full-server.json`; the delivery-only manifest is
+`checksums-functional.json`. No model checkpoint or tokenizer payload is in this Git delivery.

results/cpu-qwen35-2b/FUNCTIONAL_REPORT.md

@@ -0,0 +1,36 @@
+# CPU learned-function repetition pilot
+
+Primary question: do two distinct trained parts implement similar functions? Cost is not a success condition.
+- Existing learned RMSNorm function positive: True
+- Bidirectional network quality screen passed: True
+- Learned scalar MLP neuron search positive: False
+- Original 11 candidate run state: completed; selected: []
+- All model weights are retained on the server until the user explicitly requests deletion.
+
+## Existing learned module evidence
+
+Role: input_layernorm; layers 12 and 13; each maps 2048-dimensional input to output.
+Uniform real-arithmetic relative-error bound, first-to-second: 0.101851851852; reverse: 0.113402061856.
+Maximum heldout directional output relative L2: 0.04528510757179746; minimum cosine: 0.9994870299724158.
+Same epsilon and nonzero gains: the common RMS denominator cancels. For any nonzero input x,
+`||f_a(x)-f_b(x)|| / ||f_a(x)|| <= max_j |a_j-b_j|/|a_j|`; swap a/b for the reverse direction.
+Selection used only the tight all-input bound across 552 same-role pairs, before inspecting inputs.
+Real probes use four calibration and four heldout experiment-authored texts, 32 tokens each, and both source layers.
+These are learned, nonconstant gain vectors. The claim concerns RMSNorm modules; it does not establish attention/MLP-body duplication.
+
+- Layer 12 uses 13: mean heldout NLL delta 3.25651368766e-05; sequence P95 delta 0.000984639618287; screen pass=True.
+- Layer 13 uses 12: mean heldout NLL delta -0.000363972997404; sequence P95 delta 0.000709721155035; screen pass=True.
+
+## Negative results and separate algebraic result
+
+The original whole-attention, whole-MLP and within-layer Q/K norm candidates did not pass their gates.
+The scalar neuron search excludes low-RMS/near-constant profiles and selects at most one pair without heldout feedback; consult its search summary for coverage.
+Q/K non-RoPE gain consolidation is a separately verified real-arithmetic identity, not evidence that two existing learned modules duplicate functions.
+Earlier local CPU timing records are preserved for provenance; performance is outside the current success criterion.
+
+## Limits and provenance
+
+Original BF16 weights were decoded into streaming FP32 CPU operators; no training, scale fitting, R fitting or GPU.
+The numerical reference includes 22 independent official Torch comparisons. Early adapter errors and two resource-guard stops are preserved as historical logs.
+The persistent computation ledger includes a conservative 180-second reserve for early checks; no budget reset after that reserve.
+The text sample is tiny and short. Network NLL checks are not broad task-accuracy validation.

results/cpu-qwen35-2b/archive-files.json

@@ -0,0 +1,88 @@
+[
+  "DELIVERY.md",
+  "FUNCTIONAL_REPORT.md",
+  "attention-head-metrics.csv",
+  "baseline-heldout-token-nll.npy",
+  "calibration-metrics.csv",
+  "candidate-summary.json",
+  "checksums-full-server.json",
+  "compute-budget-ledger.json",
+  "download.log",
+  "experiment.json",
+  "external-activation-cache.json",
+  "figures/algebraic_gain_identity.pdf",
+  "figures/algebraic_gain_identity.png",
+  "figures/algebraic_gain_identity.svg",
+  "figures/candidate_function_errors.pdf",
+  "figures/candidate_function_errors.png",
+  "figures/candidate_function_errors.svg",
+  "figures/learned_norm_function_errors.pdf",
+  "figures/learned_norm_function_errors.png",
+  "figures/learned_norm_function_errors.svg",
+  "figures/learned_norm_network_substitution.pdf",
+  "figures/learned_norm_network_substitution.png",
+  "figures/learned_norm_network_substitution.svg",
+  "final-report.log",
+  "final-source-hashes.json",
+  "final-tests.txt",
+  "functional-code-hashes.json",
+  "functional-environment.json",
+  "functional-report.log",
+  "functional-run.log",
+  "functional-status.json",
+  "gain-factorization-k_to_q-token-nll.npy",
+  "gain-factorization-kernel-timing.csv",
+  "gain-factorization-optimized-summary.json",
+  "gain-factorization-optimized-timing.csv",
+  "gain-factorization-optimized.log",
+  "gain-factorization-protocol.json",
+  "gain-factorization-q_to_k-token-nll.npy",
+  "gain-factorization-score-metrics.csv",
+  "gain-factorization-summary.json",
+  "gain-factorization.log",
+  "install-torch-disk-tmp.log",
+  "install-torch.log",
+  "manifest.json",
+  "model_config.json",
+  "neuron-reuse-protocol.json",
+  "neuron-reuse-search-summary.json",
+  "neuron-reuse-selection.json",
+  "neuron-reuse-summary.json",
+  "neuron-reuse.log",
+  "norm-repetition-12-uses-13-token-nll.npy",
+  "norm-repetition-13-uses-12-token-nll.npy",
+  "norm-repetition-function-metrics.csv",
+  "norm-repetition-protocol.json",
+  "norm-repetition-selection.json",
+  "norm-repetition-summary.json",
+  "norm-repetition-uniform-bounds.csv",
+  "norm-repetition.log",
+  "preflight-first-attempt.log",
+  "preflight-second-attempt.json",
+  "preflight-second-attempt.log",
+  "preflight.json",
+  "preflight.log",
+  "reference-check-first-attempt.log",
+  "reference-check-second-attempt.log",
+  "reference-check.json",
+  "reference-check.log",
+  "retention-policy.json",
+  "samples-tokenized.json",
+  "selection.json",
+  "source_metadata.json",
+  "storage-accounting.log",
+  "storage-evidence.json",
+  "supervisor-error.txt",
+  "supervisor-first-attempt-error.txt",
+  "supervisor-first-attempt.json",
+  "supervisor-second-attempt.json",
+  "supervisor-status.json",
+  "supervisor-third-attempt.json",
+  "supervisor.log",
+  "tokenizer-download.log",
+  "tokenizers-wheel.log",
+  "unit-tests.txt",
+  "checksums-functional.json",
+  "audit-validation.json",
+  "archive-files.json"
+]

results/cpu-qwen35-2b/attention-head-metrics.csv

@@ -0,0 +1,193 @@
+candidate,source,sample_id,head,first_key_mass_a,first_key_mass_b,js_without_first_key,without_first_key_valid_rows,js_mean,js_p95,js_without_first_query,top5_overlap
+attn_3_7,3,cal-zh-1,0,0.178242027759552,0.44051653146743774,0.2887135446071625,31,0.2728434205055237,0.42193013429641724,0.2816448211669922,0.5875
+attn_3_7,3,cal-zh-1,1,0.14070242643356323,0.6567835807800293,0.154605433344841,31,0.273810476064682,0.45863786339759827,0.2826431095600128,0.5375
+attn_3_7,3,cal-zh-1,2,0.11500269174575806,0.4787790775299072,0.3218191862106323,31,0.35086748003959656,0.46183231472969055,0.3621857762336731,0.53125
+attn_3_7,3,cal-zh-1,3,0.17224811017513275,0.6523524522781372,0.22310931980609894,31,0.28077495098114014,0.43149834871292114,0.289832204580307,0.5
+attn_3_7,3,cal-zh-1,4,0.11534158885478973,0.6260660886764526,0.19559672474861145,31,0.2952086329460144,0.470927894115448,0.30473145842552185,0.53125
+attn_3_7,3,cal-zh-1,5,0.0896417498588562,0.34022626280784607,0.22164911031723022,31,0.2483729124069214,0.3841992914676666,0.256384938955307,0.54375
+attn_3_7,3,cal-zh-1,6,0.1247040405869484,0.7327346801757812,0.26632776856422424,31,0.37061089277267456,0.5524277091026306,0.38256609439849854,0.525
+attn_3_7,3,cal-zh-1,7,0.05083201080560684,0.44692301750183105,0.04695317894220352,31,0.20246444642543793,0.4956357181072235,0.20899559557437897,0.65625
+attn_3_7,3,cal-zh-2,0,0.14007353782653809,0.6036521196365356,0.2504415214061737,31,0.30694448947906494,0.4879894554615021,0.31684595346450806,0.59375
+attn_3_7,3,cal-zh-2,1,0.14963164925575256,0.7174584865570068,0.16050270199775696,31,0.2995006740093231,0.44737952947616577,0.3091619610786438,0.58125
+attn_3_7,3,cal-zh-2,2,0.1916198432445526,0.6128076314926147,0.24314163625240326,31,0.28848910331726074,0.42006126046180725,0.29779520630836487,0.63125
+attn_3_7,3,cal-zh-2,3,0.11550820618867874,0.6839271783828735,0.22851790487766266,31,0.3617430627346039,0.4926488697528839,0.373412162065506,0.43125
+attn_3_7,3,cal-zh-2,4,0.1432671993970871,0.714910089969635,0.16698074340820312,31,0.29421237111091614,0.44453173875808716,0.30370306968688965,0.55
+attn_3_7,3,cal-zh-2,5,0.15627557039260864,0.4676961600780487,0.22456194460391998,31,0.24842627346515656,0.35863032937049866,0.25644001364707947,0.53125
+attn_3_7,3,cal-zh-2,6,0.10968590527772903,0.8703562021255493,0.2677042484283447,31,0.44428783655166626,0.6032188534736633,0.45861977338790894,0.4625
+attn_3_7,3,cal-zh-2,7,0.07148394733667374,0.5658342838287354,0.050552818924188614,31,0.24003271758556366,0.5258023738861084,0.24777570366859436,0.6875
+attn_3_7,3,cal-en-1,0,0.11021165549755096,0.7443452477455139,0.2832939028739929,31,0.4321710467338562,0.5805379152297974,0.4461120665073395,0.5
+attn_3_7,3,cal-en-1,1,0.14157001674175262,0.6908999681472778,0.15555045008659363,31,0.3069734573364258,0.46220970153808594,0.3168758153915405,0.5562499999999999
+attn_3_7,3,cal-en-1,2,0.22632445394992828,0.6502443552017212,0.3039384186267853,31,0.33335912227630615,0.4760979115962982,0.34411266446113586,0.575
+attn_3_7,3,cal-en-1,3,0.1636783331632614,0.749056875705719,0.18588410317897797,31,0.31960076093673706,0.4827522933483124,0.3299104571342468,0.5062500000000001
+attn_3_7,3,cal-en-1,4,0.18975511193275452,0.5745540857315063,0.20487342774868011,31,0.2522231340408325,0.39599278569221497,0.26035937666893005,0.60625
+attn_3_7,3,cal-en-1,5,0.1436786651611328,0.45787984132766724,0.1883384883403778,31,0.2209610939025879,0.3852173388004303,0.2280888706445694,0.6187499999999999
+attn_3_7,3,cal-en-1,6,0.08422182500362396,0.8810427784919739,0.24220214784145355,31,0.4833376407623291,0.6262145638465881,0.49892911314964294,0.45625
+attn_3_7,3,cal-en-1,7,0.09002429246902466,0.49231916666030884,0.053437694907188416,31,0.19592277705669403,0.4695172905921936,0.20224282145500183,0.6624999999999999
+attn_3_7,3,cal-en-2,0,0.09620732069015503,0.7015095949172974,0.2738996148109436,31,0.3950675129890442,0.5772150158882141,0.40781161189079285,0.5874999999999999
+attn_3_7,3,cal-en-2,1,0.1168593019247055,0.7301980257034302,0.160212442278862,31,0.3232686221599579,0.4884706735610962,0.3336966037750244,0.55625
+attn_3_7,3,cal-en-2,2,0.21824826300144196,0.655203104019165,0.3244048058986664,31,0.3314599394798279,0.489634245634079,0.3421522378921509,0.49375
+attn_3_7,3,cal-en-2,3,0.15237370133399963,0.7099946737289429,0.253733366727829,31,0.34891438484191895,0.4875939190387726,0.3601696491241455,0.4
+attn_3_7,3,cal-en-2,4,0.20830751955509186,0.6707655191421509,0.16886863112449646,31,0.23844638466835022,0.41475459933280945,0.24613820016384125,0.58125
+attn_3_7,3,cal-en-2,5,0.20089492201805115,0.4360402226448059,0.20298565924167633,31,0.19701015949249268,0.3117577135562897,0.20336531102657318,0.5875
+attn_3_7,3,cal-en-2,6,0.08693217486143112,0.8322523832321167,0.28768372535705566,31,0.45854610204696655,0.604308545589447,0.47333794832229614,0.4625
+attn_3_7,3,cal-en-2,7,0.12079077959060669,0.5237122178077698,0.08786331117153168,31,0.1976630836725235,0.4211302399635315,0.2040393203496933,0.71875
+attn_3_7,7,cal-zh-1,0,0.08316324651241302,0.1214274913072586,0.22053609788417816,31,0.2072795033454895,0.32793501019477844,0.21396592259407043,0.5625
+attn_3_7,7,cal-zh-1,1,0.11647684127092361,0.187703475356102,0.09203154593706131,31,0.09238047897815704,0.16611215472221375,0.09536049515008926,0.66875
+attn_3_7,7,cal-zh-1,2,0.06550130248069763,0.07431766390800476,0.28803399205207825,31,0.2721514105796814,0.46315261721611023,0.28093045949935913,0.47500000000000003
+attn_3_7,7,cal-zh-1,3,0.10703553259372711,0.24631237983703613,0.1426287740468979,31,0.150299072265625,0.26700741052627563,0.1551474630832672,0.56875
+attn_3_7,7,cal-zh-1,4,0.04052526503801346,0.28515881299972534,0.11374201625585556,31,0.1819339096546173,0.2913167476654053,0.1878027319908142,0.5875
+attn_3_7,7,cal-zh-1,5,0.03951618820428848,0.12756675481796265,0.12323599308729172,31,0.13915526866912842,0.22263947129249573,0.14364413917064667,0.6062500000000001
+attn_3_7,7,cal-zh-1,6,0.09479819983243942,0.28205406665802,0.14449526369571686,31,0.16065017879009247,0.30267569422721863,0.16583245992660522,0.6375000000000001
+attn_3_7,7,cal-zh-1,7,0.031424712389707565,0.10597768425941467,0.029422910884022713,31,0.05383709818124771,0.10532202571630478,0.055573780089616776,0.60625
+attn_3_7,7,cal-zh-2,0,0.0902467742562294,0.11627861112356186,0.21221575140953064,31,0.19495421648025513,0.317206472158432,0.201243057847023,0.6187499999999999
+attn_3_7,7,cal-zh-2,1,0.11616441607475281,0.1248721182346344,0.0952492356300354,31,0.08693543076515198,0.16702036559581757,0.08973980695009232,0.5874999999999999
+attn_3_7,7,cal-zh-2,2,0.11578349769115448,0.08946777880191803,0.2738611102104187,31,0.2561575174331665,0.45070335268974304,0.26442068815231323,0.5
+attn_3_7,7,cal-zh-2,3,0.11586448550224304,0.16896863281726837,0.10842260718345642,31,0.10158580541610718,0.21354182064533234,0.1048627570271492,0.60625
+attn_3_7,7,cal-zh-2,4,0.044329218566417694,0.22534678876399994,0.15060655772686005,31,0.19093912839889526,0.3356643319129944,0.19709844887256622,0.61875
+attn_3_7,7,cal-zh-2,5,0.05062376335263252,0.12924723327159882,0.1047384962439537,31,0.11341066658496857,0.2176104635000229,0.1170690730214119,0.6312500000000001
+attn_3_7,7,cal-zh-2,6,0.08418147265911102,0.28032630681991577,0.1446688026189804,31,0.16730237007141113,0.24755442142486572,0.1726992279291153,0.5125
+attn_3_7,7,cal-zh-2,7,0.0315190926194191,0.11244469881057739,0.02086171880364418,31,0.04779163748025894,0.10416773706674576,0.04933330416679382,0.65
+attn_3_7,7,cal-en-1,0,0.08185911923646927,0.10868857800960541,0.24351894855499268,31,0.22597578167915344,0.35052549839019775,0.23326534032821655,0.66875
+attn_3_7,7,cal-en-1,1,0.12180840969085693,0.12996530532836914,0.1148008182644844,31,0.10937604308128357,0.1990041583776474,0.11290428787469864,0.6625
+attn_3_7,7,cal-en-1,2,0.12156520038843155,0.07304315268993378,0.32663044333457947,31,0.31218329071998596,0.4966602325439453,0.3222537338733673,0.4375
+attn_3_7,7,cal-en-1,3,0.09531049430370331,0.21274563670158386,0.13099102675914764,31,0.14007462561130524,0.2635233998298645,0.14459317922592163,0.5874999999999999
+attn_3_7,7,cal-en-1,4,0.051348961889743805,0.27256035804748535,0.17027397453784943,31,0.22669382393360138,0.33905521035194397,0.23400652408599854,0.59375
+attn_3_7,7,cal-en-1,5,0.05210518091917038,0.14343655109405518,0.11446496844291687,31,0.12723194062709808,0.23429512977600098,0.13133619725704193,0.64375
+attn_3_7,7,cal-en-1,6,0.09278014302253723,0.3040335178375244,0.20964911580085754,31,0.21833369135856628,0.34637004137039185,0.22537672519683838,0.53125
+attn_3_7,7,cal-en-1,7,0.031563661992549896,0.11319160461425781,0.02739104814827442,31,0.053969405591487885,0.1002146303653717,0.055710360407829285,0.5625000000000001
+attn_3_7,7,cal-en-2,0,0.07943743467330933,0.1426258534193039,0.21740850806236267,31,0.21001917123794556,0.35335251688957214,0.2167939990758896,0.58125
+attn_3_7,7,cal-en-2,1,0.13507506251335144,0.15443003177642822,0.10350806266069412,31,0.09826720505952835,0.18229186534881592,0.10143710672855377,0.575
+attn_3_7,7,cal-en-2,2,0.07290980964899063,0.08657881617546082,0.26718389987945557,31,0.259122759103775,0.4706645905971527,0.26748156547546387,0.45625
+attn_3_7,7,cal-en-2,3,0.06336543709039688,0.2253912389278412,0.11350395530462265,31,0.1445191204547882,0.30314192175865173,0.1491810381412506,0.5875
+attn_3_7,7,cal-en-2,4,0.04387465864419937,0.2312193661928177,0.14195290207862854,31,0.1880926638841629,0.3466520309448242,0.19416013360023499,0.5125
+attn_3_7,7,cal-en-2,5,0.05407806113362312,0.1308087259531021,0.15231285989284515,31,0.16046105325222015,0.3131849467754364,0.16563721001148224,0.5874999999999999
+attn_3_7,7,cal-en-2,6,0.10013270378112793,0.30412447452545166,0.17993809282779694,31,0.19388216733932495,0.34072187542915344,0.2001364380121231,0.5437500000000001
+attn_3_7,7,cal-en-2,7,0.03161787986755371,0.14472994208335876,0.04679951071739197,31,0.0830649882555008,0.13866016268730164,0.08574451506137848,0.65625
+attn_11_15,11,cal-zh-1,0,0.2780076861381531,0.40727436542510986,0.08935041725635529,31,0.08769044280052185,0.1605687141418457,0.09051916748285294,0.6937500000000001
+attn_11_15,11,cal-zh-1,1,0.599311351776123,0.1917644888162613,0.1712544858455658,31,0.22863143682479858,0.39283493161201477,0.23600666224956512,0.5750000000000001
+attn_11_15,11,cal-zh-1,2,0.15545204281806946,0.5101379156112671,0.08800674974918365,31,0.15655069053173065,0.3642641007900238,0.16160069406032562,0.73125
+attn_11_15,11,cal-zh-1,3,0.11032958328723907,0.4199199676513672,0.08535198122262955,31,0.14290741086006165,0.27922654151916504,0.14751732349395752,0.7
+attn_11_15,11,cal-zh-1,4,0.18725907802581787,0.3965998888015747,0.12378748506307602,31,0.1280820369720459,0.24588972330093384,0.13221368193626404,0.6937500000000001
+attn_11_15,11,cal-zh-1,5,0.3307006359100342,0.43053850531578064,0.12539583444595337,31,0.1220257431268692,0.27262142300605774,0.1259620636701584,0.6187499999999999
+attn_11_15,11,cal-zh-1,6,0.17290356755256653,0.4159870147705078,0.10268668830394745,31,0.12616759538650513,0.27286267280578613,0.13023751974105835,0.625
+attn_11_15,11,cal-zh-1,7,0.12546084821224213,0.4914926290512085,0.11725502461194992,31,0.17813995480537415,0.28658801317214966,0.18388642370700836,0.6625000000000001
+attn_11_15,11,cal-zh-2,0,0.2187591940164566,0.42793723940849304,0.09548557549715042,31,0.12088388949632645,0.2345586121082306,0.12478335946798325,0.6875
+attn_11_15,11,cal-zh-2,1,0.5546900033950806,0.1175224781036377,0.16409580409526825,31,0.25557559728622437,0.37492382526397705,0.26381993293762207,0.6125
+attn_11_15,11,cal-zh-2,2,0.11856245994567871,0.44309669733047485,0.08432265371084213,31,0.1634930670261383,0.37660637497901917,0.16876700520515442,0.65
+attn_11_15,11,cal-zh-2,3,0.09019850194454193,0.3149072527885437,0.09793759882450104,31,0.1334252804517746,0.24160735309123993,0.13772933185100555,0.71875
+attn_11_15,11,cal-zh-2,4,0.09969134628772736,0.20464956760406494,0.10986322164535522,31,0.11441972851753235,0.2605915665626526,0.11811067909002304,0.7
+attn_11_15,11,cal-zh-2,5,0.16705867648124695,0.412482887506485,0.20107370615005493,31,0.21316100656986237,0.44814038276672363,0.22003716230392456,0.6000000000000001
+attn_11_15,11,cal-zh-2,6,0.126181960105896,0.38109123706817627,0.10754827409982681,31,0.14080098271369934,0.2554462254047394,0.14534294605255127,0.6687500000000001
+attn_11_15,11,cal-zh-2,7,0.11085975170135498,0.5181387662887573,0.10885247588157654,31,0.2060571014881134,0.35480910539627075,0.2127041071653366,0.66875
+attn_11_15,11,cal-en-1,0,0.2882137894630432,0.45660895109176636,0.07201986759901047,31,0.08352888375520706,0.1487027406692505,0.08622336387634277,0.73125
+attn_11_15,11,cal-en-1,1,0.597124457359314,0.13483837246894836,0.22424918413162231,31,0.29736971855163574,0.529407799243927,0.3069622814655304,0.51875
+attn_11_15,11,cal-en-1,2,0.17265087366104126,0.4521681070327759,0.08010748028755188,31,0.13888895511627197,0.29661378264427185,0.14336924254894257,0.7375
+attn_11_15,11,cal-en-1,3,0.10834433138370514,0.319663405418396,0.08455760031938553,31,0.11557236313819885,0.24554768204689026,0.11930052191019058,0.84375
+attn_11_15,11,cal-en-1,4,0.157099187374115,0.3456929326057434,0.10397545993328094,31,0.11958309262990952,0.20836150646209717,0.12344060838222504,0.73125
+attn_11_15,11,cal-en-1,5,0.30003029108047485,0.3683816194534302,0.11273742467164993,31,0.11081759631633759,0.26781079173088074,0.11439235508441925,0.6875
+attn_11_15,11,cal-en-1,6,0.16862526535987854,0.5013703107833862,0.12122618407011032,31,0.1663668006658554,0.29310786724090576,0.17173348367214203,0.65
+attn_11_15,11,cal-en-1,7,0.14973583817481995,0.5537562966346741,0.1512605994939804,31,0.22106251120567322,0.3912945091724396,0.22819358110427856,0.64375
+attn_11_15,11,cal-en-2,0,0.3351830840110779,0.4993361234664917,0.08562158793210983,31,0.08981436491012573,0.19977226853370667,0.09271162003278732,0.675
+attn_11_15,11,cal-en-2,1,0.5792163610458374,0.1499037742614746,0.1789509803056717,31,0.2536986768245697,0.41500288248062134,0.2618825435638428,0.55
+attn_11_15,11,cal-en-2,2,0.20357520878314972,0.4823607802391052,0.11572320759296417,31,0.16091257333755493,0.35024091601371765,0.1661033034324646,0.6000000000000001
+attn_11_15,11,cal-en-2,3,0.11477383971214294,0.4060356318950653,0.08803554624319077,31,0.14013110101222992,0.2660136818885803,0.14465147256851196,0.75
+attn_11_15,11,cal-en-2,4,0.1895177811384201,0.2564787268638611,0.1033802479505539,31,0.10677468031644821,0.19931447505950928,0.11021901667118073,0.7125
+attn_11_15,11,cal-en-2,5,0.2725520133972168,0.3527078926563263,0.1301337331533432,31,0.12496069073677063,0.22729209065437317,0.12899169325828552,0.6312500000000001
+attn_11_15,11,cal-en-2,6,0.17815515398979187,0.4918685555458069,0.08680415898561478,31,0.1395310014486313,0.22975514829158783,0.14403200149536133,0.68125
+attn_11_15,11,cal-en-2,7,0.1471988707780838,0.4305167496204376,0.15007153153419495,31,0.1933930218219757,0.31820762157440186,0.1996314972639084,0.60625
+attn_11_15,15,cal-zh-1,0,0.20194272696971893,0.35520243644714355,0.1205381229519844,31,0.13211002945899963,0.2777846157550812,0.1363716572523117,0.63125
+attn_11_15,15,cal-zh-1,1,0.23890593647956848,0.12377866357564926,0.21396537125110626,31,0.21463359892368317,0.41781342029571533,0.22155724465847015,0.6062500000000001
+attn_11_15,15,cal-zh-1,2,0.13280686736106873,0.27719253301620483,0.1377590000629425,31,0.14551271498203278,0.2994697093963623,0.15020668506622314,0.65
+attn_11_15,15,cal-zh-1,3,0.04135940596461296,0.23285867273807526,0.21063239872455597,31,0.24926505982875824,0.45159438252449036,0.2573058605194092,0.6937500000000001
+attn_11_15,15,cal-zh-1,4,0.07648462802171707,0.4026046395301819,0.1620333045721054,31,0.23594452440738678,0.4158128798007965,0.24355560541152954,0.6500000000000001
+attn_11_15,15,cal-zh-1,5,0.14696139097213745,0.41105949878692627,0.1549629122018814,31,0.18883343040943146,0.3439426124095917,0.19492481648921967,0.6187499999999999
+attn_11_15,15,cal-zh-1,6,0.0764666199684143,0.34140151739120483,0.11151057481765747,31,0.16509875655174255,0.31560462713241577,0.17042450606822968,0.6625
+attn_11_15,15,cal-zh-1,7,0.10250869393348694,0.5306729078292847,0.16751603782176971,31,0.25549793243408203,0.3858420252799988,0.26373982429504395,0.6062500000000001
+attn_11_15,15,cal-zh-2,0,0.22696512937545776,0.35936564207077026,0.14069660007953644,31,0.13378489017486572,0.28451812267303467,0.1381005346775055,0.65
+attn_11_15,15,cal-zh-2,1,0.25547248125076294,0.10696703940629959,0.23399728536605835,31,0.24166268110275269,0.39466142654418945,0.24945826828479767,0.5875
+attn_11_15,15,cal-zh-2,2,0.17822568118572235,0.2864110469818115,0.08877820521593094,31,0.10718775540590286,0.2511947453022003,0.11064542829990387,0.6937500000000001
+attn_11_15,15,cal-zh-2,3,0.04604611545801163,0.23604938387870789,0.22704336047172546,31,0.2641073167324066,0.4154779016971588,0.2726269066333771,0.70625
+attn_11_15,15,cal-zh-2,4,0.08757959306240082,0.2918083965778351,0.22551555931568146,31,0.24939756095409393,0.48551467061042786,0.2574426531791687,0.6000000000000001
+attn_11_15,15,cal-zh-2,5,0.11609973013401031,0.3603805601596832,0.21242640912532806,31,0.23965227603912354,0.4984361231327057,0.2473829984664917,0.51875
+attn_11_15,15,cal-zh-2,6,0.0702231377363205,0.29952526092529297,0.12570251524448395,31,0.17031171917915344,0.32249733805656433,0.17580564320087433,0.6125
+attn_11_15,15,cal-zh-2,7,0.0856853500008583,0.4777143895626068,0.1792273223400116,31,0.27432966232299805,0.47331610321998596,0.28317901492118835,0.65625
+attn_11_15,15,cal-en-1,0,0.2137582153081894,0.36318275332450867,0.121327243745327,31,0.12573851644992828,0.2549639940261841,0.12979458272457123,0.64375
+attn_11_15,15,cal-en-1,1,0.31107524037361145,0.1114300861954689,0.24769999086856842,31,0.28352850675582886,0.4685184955596924,0.29267460107803345,0.53125
+attn_11_15,15,cal-en-1,2,0.17990511655807495,0.29312068223953247,0.12414757162332535,31,0.12930452823638916,0.3116541802883148,0.1334756314754486,0.74375
+attn_11_15,15,cal-en-1,3,0.048217516392469406,0.23240914940834045,0.20335303246974945,31,0.2404388189315796,0.3818458914756775,0.2481948882341385,0.70625
+attn_11_15,15,cal-en-1,4,0.10281141847372055,0.38790467381477356,0.18024703860282898,31,0.22299107909202576,0.4459393620491028,0.23018434643745422,0.70625
+attn_11_15,15,cal-en-1,5,0.15091386437416077,0.3693487346172333,0.1916365921497345,31,0.21596358716487885,0.39266589283943176,0.2229301631450653,0.53125
+attn_11_15,15,cal-en-1,6,0.0790158361196518,0.32175618410110474,0.11456245183944702,31,0.16404178738594055,0.30913954973220825,0.16933345794677734,0.6875
+attn_11_15,15,cal-en-1,7,0.10605430603027344,0.5771000385284424,0.18373095989227295,31,0.28234609961509705,0.4291541278362274,0.2914540469646454,0.625
+attn_11_15,15,cal-en-2,0,0.251964807510376,0.31455427408218384,0.10559824854135513,31,0.11457027494907379,0.24910703301429749,0.11826609075069427,0.675
+attn_11_15,15,cal-en-2,1,0.31838059425354004,0.11868949234485626,0.257511168718338,31,0.2719646394252777,0.44276586174964905,0.2807376980781555,0.5687500000000001
+attn_11_15,15,cal-en-2,2,0.26772984862327576,0.33754318952560425,0.0924711525440216,31,0.09108234941959381,0.191604882478714,0.09402050077915192,0.7375
+attn_11_15,15,cal-en-2,3,0.06395735591650009,0.2933264374732971,0.20108558237552643,31,0.2424471080303192,0.4563605785369873,0.25026798248291016,0.66875
+attn_11_15,15,cal-en-2,4,0.08209535479545593,0.3278740644454956,0.1982748657464981,31,0.24514394998550415,0.3975651264190674,0.2530518174171448,0.56875
+attn_11_15,15,cal-en-2,5,0.09166160225868225,0.44504809379577637,0.17847460508346558,31,0.2476060837507248,0.3910382390022278,0.2555933892726898,0.4875
+attn_11_15,15,cal-en-2,6,0.07513932138681412,0.45416805148124695,0.12279381603002548,31,0.22479453682899475,0.34578394889831543,0.2320459932088852,0.59375
+attn_11_15,15,cal-en-2,7,0.1007680892944336,0.5731154084205627,0.18980540335178375,31,0.3043324649333954,0.41584962606430054,0.3141496479511261,0.5375000000000001
+attn_19_23,19,cal-zh-1,0,0.0955502837896347,0.03435931354761124,0.08661527186632156,31,0.10083474218845367,0.1792067289352417,0.10408748686313629,0.73125
+attn_19_23,19,cal-zh-1,1,0.10103033483028412,0.04235740005970001,0.16347354650497437,31,0.16801373660564423,0.37647515535354614,0.17343351244926453,0.6125
+attn_19_23,19,cal-zh-1,2,0.047133591026067734,0.03135885298252106,0.47210854291915894,31,0.46042412519454956,0.6502387523651123,0.47527655959129333,0.5375000000000001
+attn_19_23,19,cal-zh-1,3,0.07069262862205505,0.06008598208427429,0.12148955464363098,31,0.11885681003332138,0.26471832394599915,0.12269090116024017,0.6625000000000001
+attn_19_23,19,cal-zh-1,4,0.10007724165916443,0.044495031237602234,0.16073106229305267,31,0.16553831100463867,0.3016175925731659,0.17087827622890472,0.68125
+attn_19_23,19,cal-zh-1,5,0.06431633979082108,0.040679484605789185,0.0816011056303978,31,0.08355634659528732,0.1771341860294342,0.08625171333551407,0.8
+attn_19_23,19,cal-zh-1,6,0.08145896345376968,0.03135719522833824,0.4502938687801361,31,0.44716787338256836,0.6495819091796875,0.4615926444530487,0.5
+attn_19_23,19,cal-zh-1,7,0.08670078217983246,0.03888973593711853,0.09126521646976471,31,0.09830940514802933,0.18466481566429138,0.10148067027330399,0.70625
+attn_19_23,19,cal-zh-2,0,0.06336566060781479,0.03736908361315727,0.07764910906553268,31,0.08064413070678711,0.16660116612911224,0.08324555307626724,0.8187500000000001
+attn_19_23,19,cal-zh-2,1,0.0888693630695343,0.06530671566724777,0.1942276507616043,31,0.18916305899620056,0.33451300859451294,0.19526511430740356,0.6125
+attn_19_23,19,cal-zh-2,2,0.07010967284440994,0.031316906213760376,0.3875632584095001,31,0.3861002027988434,0.6248269081115723,0.3985550105571747,0.65
+attn_19_23,19,cal-zh-2,3,0.08096092939376831,0.08616353571414948,0.08690906316041946,31,0.08501926809549332,0.19463855028152466,0.08776183426380157,0.7937500000000001
+attn_19_23,19,cal-zh-2,4,0.0936945378780365,0.04400797560811043,0.11036926507949829,31,0.11584575474262238,0.2122059315443039,0.11958272755146027,0.8187500000000001
+attn_19_23,19,cal-zh-2,5,0.056567154824733734,0.03972743824124336,0.06345413625240326,31,0.06394530087709427,0.16273385286331177,0.066008061170578,0.7937500000000001
+attn_19_23,19,cal-zh-2,6,0.08575102686882019,0.031342219561338425,0.4025454819202423,31,0.4032740592956543,0.5980108976364136,0.4162829518318176,0.60625
+attn_19_23,19,cal-zh-2,7,0.10350999981164932,0.0646558329463005,0.10814103484153748,31,0.10938012599945068,0.23185187578201294,0.11290851980447769,0.7375
+attn_19_23,19,cal-en-1,0,0.22517549991607666,0.09088313579559326,0.11986515671014786,31,0.1366778463125229,0.2391858845949173,0.14108680188655853,0.74375
+attn_19_23,19,cal-en-1,1,0.23973168432712555,0.11891130357980728,0.17766058444976807,31,0.16800963878631592,0.2676515281200409,0.17342931032180786,0.66875
+attn_19_23,19,cal-en-1,2,0.21452879905700684,0.03307320922613144,0.3807314932346344,31,0.4102427363395691,0.5871645212173462,0.42347636818885803,0.66875
+attn_19_23,19,cal-en-1,3,0.21709948778152466,0.14377164840698242,0.0770971029996872,31,0.08144749701023102,0.15173088014125824,0.08407483249902725,0.8125
+attn_19_23,19,cal-en-1,4,0.23791193962097168,0.19146007299423218,0.12041598558425903,31,0.11442764848470688,0.22197961807250977,0.11811887472867966,0.66875
+attn_19_23,19,cal-en-1,5,0.2485853135585785,0.10372771322727203,0.08068431913852692,31,0.11313603073358536,0.23292472958564758,0.11678558588027954,0.7562500000000001
+attn_19_23,19,cal-en-1,6,0.31094634532928467,0.031405214220285416,0.39337676763534546,31,0.45084744691848755,0.6603637933731079,0.4653908908367157,0.55
+attn_19_23,19,cal-en-1,7,0.2927525043487549,0.15012584626674652,0.12347250431776047,31,0.12942291796207428,0.2467067539691925,0.13359786570072174,0.68125
+attn_19_23,19,cal-en-2,0,0.19307208061218262,0.055295493453741074,0.1307787448167801,31,0.15591834485530853,0.3169764280319214,0.1609479784965515,0.5875
+attn_19_23,19,cal-en-2,1,0.1491451859474182,0.10034040361642838,0.14188656210899353,31,0.1398725062608719,0.2532970607280731,0.14438451826572418,0.59375
+attn_19_23,19,cal-en-2,2,0.11011949181556702,0.033411312848329544,0.4344772696495056,31,0.4410479664802551,0.6305621266365051,0.4552752673625946,0.5375
+attn_19_23,19,cal-en-2,3,0.11554233729839325,0.08585710823535919,0.1385604739189148,31,0.13558831810951233,0.3369371294975281,0.13996215164661407,0.5874999999999999
+attn_19_23,19,cal-en-2,4,0.1721733808517456,0.10996092110872269,0.1335446536540985,31,0.13328826427459717,0.24826908111572266,0.13758789002895355,0.625
+attn_19_23,19,cal-en-2,5,0.18817278742790222,0.06915841996669769,0.11790769547224045,31,0.13643309473991394,0.2266169786453247,0.14083418250083923,0.65625
+attn_19_23,19,cal-en-2,6,0.2252667248249054,0.03137722611427307,0.45010486245155334,31,0.4878306984901428,0.6629999876022339,0.503567099571228,0.49375
+attn_19_23,19,cal-en-2,7,0.19858212769031525,0.07953452318906784,0.1386381983757019,31,0.15347915887832642,0.2583900988101959,0.15843011438846588,0.5687500000000001
+attn_19_23,23,cal-zh-1,0,0.18854272365570068,0.06906979531049728,0.08094117045402527,31,0.10190774500370026,0.19865523278713226,0.10519509017467499,0.61875
+attn_19_23,23,cal-zh-1,1,0.12637057900428772,0.094294473528862,0.11472687125205994,31,0.11000899970531464,0.20533926784992218,0.11355767399072647,0.60625
+attn_19_23,23,cal-zh-1,2,0.06566040217876434,0.0320066399872303,0.3936012387275696,31,0.3894338607788086,0.6095475554466248,0.40199628472328186,0.53125
+attn_19_23,23,cal-zh-1,3,0.10236738622188568,0.07628713548183441,0.0909285917878151,31,0.09045256674289703,0.16597077250480652,0.09337039291858673,0.65625
+attn_19_23,23,cal-zh-1,4,0.11772166937589645,0.09994609653949738,0.11158378422260284,31,0.1082884818315506,0.2333190143108368,0.11178164929151535,0.6000000000000001
+attn_19_23,23,cal-zh-1,5,0.1445631980895996,0.05731264501810074,0.06436286121606827,31,0.07913714647293091,0.15170255303382874,0.08168993890285492,0.6625000000000001
+attn_19_23,23,cal-zh-1,6,0.13692563772201538,0.03156186267733574,0.42265841364860535,31,0.43705910444259644,0.6045969128608704,0.45115774869918823,0.5249999999999999
+attn_19_23,23,cal-zh-1,7,0.17913174629211426,0.0774894654750824,0.055066999047994614,31,0.07003086805343628,0.11456935852766037,0.07228992879390717,0.6875
+attn_19_23,23,cal-zh-2,0,0.08711955696344376,0.08616629242897034,0.09776511043310165,31,0.09617017209529877,0.2418135702610016,0.09927243739366531,0.7125
+attn_19_23,23,cal-zh-2,1,0.1001269742846489,0.1142689436674118,0.11429562419652939,31,0.11118914932012558,0.19028690457344055,0.1147758960723877,0.6625000000000001
+attn_19_23,23,cal-zh-2,2,0.13165870308876038,0.03296923264861107,0.32159483432769775,31,0.34172311425209045,0.5507174134254456,0.35274645686149597,0.675
+attn_19_23,23,cal-zh-2,3,0.10557965934276581,0.08546337485313416,0.10869138687849045,31,0.10796000063419342,0.267814040184021,0.11144257336854935,0.71875
+attn_19_23,23,cal-zh-2,4,0.0753588005900383,0.0625741258263588,0.09596671164035797,31,0.09504459798336029,0.18062597513198853,0.09811054915189743,0.6937500000000001
+attn_19_23,23,cal-zh-2,5,0.0798945426940918,0.0568334236741066,0.07381810247898102,31,0.0762210339307785,0.13966575264930725,0.07867977768182755,0.74375
+attn_19_23,23,cal-zh-2,6,0.0759878009557724,0.031536251306533813,0.39428046345710754,31,0.3959207534790039,0.5929224491119385,0.408692330121994,0.6375
+attn_19_23,23,cal-zh-2,7,0.11138643324375153,0.07136214524507523,0.07186198234558105,31,0.07652410864830017,0.15722090005874634,0.07899262756109238,0.7312500000000001
+attn_19_23,23,cal-en-1,0,0.12444712221622467,0.0752042829990387,0.10921256989240646,31,0.10872898995876312,0.22416618466377258,0.11223636567592621,0.6875
+attn_19_23,23,cal-en-1,1,0.10719993710517883,0.08359251916408539,0.12698784470558167,31,0.12113931775093079,0.22020384669303894,0.12504702806472778,0.7000000000000001
+attn_19_23,23,cal-en-1,2,0.13445816934108734,0.03345245122909546,0.3665089011192322,31,0.3763067126274109,0.5767470002174377,0.3884456157684326,0.65625
+attn_19_23,23,cal-en-1,3,0.10536500066518784,0.07050738483667374,0.07684175670146942,31,0.07960161566734314,0.14911630749702454,0.08216940611600876,0.78125
+attn_19_23,23,cal-en-1,4,0.0911213681101799,0.17202866077423096,0.10298387706279755,31,0.10929879546165466,0.20484577119350433,0.11282456666231155,0.61875
+attn_19_23,23,cal-en-1,5,0.18703660368919373,0.12409132719039917,0.09463418275117874,31,0.09024524688720703,0.15500988066196442,0.09315639734268188,0.70625
+attn_19_23,23,cal-en-1,6,0.13999314606189728,0.03152035176753998,0.4262007474899292,31,0.43630746006965637,0.6232555508613586,0.450381875038147,0.5187499999999999
+attn_19_23,23,cal-en-1,7,0.11322036385536194,0.10936357825994492,0.07323051989078522,31,0.07213389873504639,0.1506829559803009,0.07446079701185226,0.6125
+attn_19_23,23,cal-en-2,0,0.11863207072019577,0.13078393042087555,0.09141960740089417,31,0.08733218908309937,0.1577986180782318,0.09014936536550522,0.59375
+attn_19_23,23,cal-en-2,1,0.07288858294487,0.1372600495815277,0.1089932844042778,31,0.11363835632801056,0.1698836386203766,0.11730410903692245,0.50625
+attn_19_23,23,cal-en-2,2,0.07288709282875061,0.04088134318590164,0.376287043094635,31,0.3685654401779175,0.5480253100395203,0.3804546594619751,0.5375
+attn_19_23,23,cal-en-2,3,0.06728185713291168,0.07526300847530365,0.09231049567461014,31,0.08894366025924683,0.15198129415512085,0.09181280434131622,0.6187499999999999
+attn_19_23,23,cal-en-2,4,0.09862713515758514,0.12817975878715515,0.09404788166284561,31,0.08950259536504745,0.16771778464317322,0.09238976985216141,0.5562499999999999
+attn_19_23,23,cal-en-2,5,0.07161959260702133,0.0855005532503128,0.10532733052968979,31,0.10144606977701187,0.1908321976661682,0.10471852868795395,0.5625
+attn_19_23,23,cal-en-2,6,0.08469876646995544,0.03168217092752457,0.4412229359149933,31,0.43726012110710144,0.6215424537658691,0.45136526226997375,0.5437500000000001
+attn_19_23,23,cal-en-2,7,0.10539951920509338,0.08276306837797165,0.07327506691217422,31,0.0738329142332077,0.13249149918556213,0.07621461898088455,0.63125

results/cpu-qwen35-2b/audit-validation.json

@@ -0,0 +1,12 @@
+{
+  "passed": true,
+  "validated_unix": 1794204667.0190518,
+  "checksummed_files": 83,
+  "proof_recomputed_from_retained_weights": true,
+  "heldout_nll_recomputed_from_saved_arrays": true,
+  "weights_present_and_retained": true,
+  "archive_excludes_weights_and_tokenizer": true,
+  "delivery_excludes_external_activation_caches": true,
+  "full_experiment_checksums": "checksums-full-server.json",
+  "figure_source_tables_and_token_nll_complete": true
+}

results/cpu-qwen35-2b/calibration-metrics.csv

@@ -0,0 +1,89 @@
+candidate,kind,source,sample_id,split,relative_l2,reverse_relative_l2,token_relative_p95,reverse_token_relative_p95,cosine,low_norm_tokens,first_key_mass_a,first_key_mass_b,js_without_first_key,without_first_key_valid_rows,js_mean,js_p95,js_without_first_query,top5_overlap
+attn_3_7,attn,3,cal-zh-1,calibration,1.75886973383732,1.2170194989630583,2.3326619869739242,1.3829599318796777,-0.001709405865837512,0,0.12333933264017105,0.5467976927757263,0.21484676003456116,248,0.2868691682815552,0.4794144630432129,0.2961229979991913,0.5515625
+attn_3_7,attn,3,cal-zh-2,calibration,2.0666622735799773,1.1397562572129543,2.6608388071181586,1.2319401946881372,0.004627105486643174,0,0.1346932351589203,0.6545802354812622,0.1990504413843155,248,0.3104545772075653,0.512997567653656,0.3204692304134369,0.55859375
+attn_3_7,attn,3,cal-en-1,calibration,1.9684586970451095,1.1687255334183004,2.5460869484043305,1.3209705033413297,-0.011291865355731309,0,0.1436830461025238,0.6550427675247192,0.2021898329257965,248,0.31806862354278564,0.5598381161689758,0.3283289074897766,0.56015625
+attn_3_7,attn,3,cal-en-2,calibration,2.1157833337453664,1.1413057345289206,2.896573536355606,1.279392467480941,-0.010752011897128681,0,0.1500767469406128,0.6574594974517822,0.2199564427137375,248,0.3112969994544983,0.5577057003974915,0.32133883237838745,0.5484375
+attn_3_7,attn,7,cal-zh-1,calibration,1.3340187814155438,1.42344327376934,1.4289276609634798,1.638356412458448,0.05265559345262384,0,0.07230515778064728,0.17881479859352112,0.144265815615654,248,0.15721087157726288,0.3267841339111328,0.16228218376636505,0.5890625
+attn_3_7,attn,7,cal-zh-2,calibration,1.3371723133481364,1.418275178551954,1.4336050300809013,1.6659403627988982,0.05349503065318622,0,0.081089086830616,0.1558690071105957,0.13882805407047272,248,0.1448846012353897,0.3248489499092102,0.14955829083919525,0.590625
+attn_3_7,attn,7,cal-en-1,calibration,1.405999268639987,1.3246539988272832,1.532711571208276,1.4346161336828325,0.07054514898566913,0,0.08104264736175537,0.16970808804035187,0.1672150194644928,248,0.17672982811927795,0.38887327909469604,0.18243078887462616,0.5859375
+attn_3_7,attn,7,cal-en-2,calibration,1.366473684595312,1.4009631878257847,1.4664096677417215,1.59944923669573,0.0431210172272288,0,0.07256138324737549,0.1774885654449463,0.1528259664773941,248,0.16717864573001862,0.3629882335662842,0.17257151007652283,0.5625
+attn_11_15,attn,11,cal-zh-1,calibration,2.7606058538571316,0.9703348793344666,2.4183673572578086,1.0857279648648543,0.2588922604193846,0,0.24492809176445007,0.4079643785953522,0.11288609355688095,248,0.1462744027376175,0.3241320848464966,0.15099292993545532,0.6625
+attn_11_15,attn,11,cal-zh-2,calibration,2.5044424857870546,0.9843061509576916,2.592796177795181,1.1478459097975344,0.23612970738935357,0,0.18575024604797363,0.3524782657623291,0.12114741653203964,248,0.16847708821296692,0.3656128942966461,0.17391183972358704,0.66328125
+attn_11_15,attn,11,cal-en-1,calibration,2.649979511635331,0.9795891219874814,2.4632888494810317,1.1094207865752836,0.23948147120552113,0,0.24272800981998444,0.39156001806259155,0.11876673251390457,248,0.15664874017238617,0.337306410074234,0.16170193254947662,0.6929687500000001
+attn_11_15,attn,11,cal-en-2,calibration,2.7191409308620766,0.9759398005295231,2.5703854656788505,1.1257459370791087,0.24568690002591262,0,0.2525215446949005,0.3836510181427002,0.11734011769294739,248,0.15115201473236084,0.3331157863140106,0.1560278832912445,0.65078125
+attn_11_15,attn,15,cal-zh-1,calibration,3.7267750977169203,0.9943195388381949,1.8571948100572258,1.4659544762265972,0.15463241076860404,0,0.12717953324317932,0.33434635400772095,0.15986472368240356,248,0.19836202263832092,0.40538591146469116,0.20476080477237701,0.6398437499999999
+attn_11_15,attn,15,cal-zh-2,calibration,3.484117411382163,0.9958762023786318,1.9555333671772042,1.5034310789069971,0.15731411020281283,0,0.13328716158866882,0.30227771401405334,0.17917340993881226,248,0.21005423367023468,0.41705968976020813,0.21683017909526825,0.628125
+attn_11_15,attn,15,cal-en-1,calibration,3.5714882610996845,0.9941664364521509,1.8393702843739006,1.5570787013529828,0.16007660497298756,0,0.14896893501281738,0.3320315480232239,0.17083808779716492,248,0.2080441117286682,0.4197816848754883,0.21475520730018616,0.646875
+attn_11_15,attn,15,cal-en-2,calibration,3.783713712256275,0.9885545400787986,2.0440754255967826,1.5107488942763196,0.17418988407724514,0,0.15646213293075562,0.35803988575935364,0.1682518720626831,248,0.2177426666021347,0.419617235660553,0.22476662695407867,0.6046875
+attn_19_23,attn,19,cal-zh-1,calibration,3.1071992972679547,1.080818751446137,3.9981135079246157,1.1800456131328154,-0.06780946108706973,0,0.08087001740932465,0.04044787213206291,0.20344728231430054,248,0.20533767342567444,0.5970298051834106,0.21196146309375763,0.65390625
+attn_19_23,attn,19,cal-zh-2,calibration,3.5223723842603682,1.066376496758468,4.7369591160509135,1.1555630164896329,-0.07515431162135316,0,0.08035354316234589,0.04998621344566345,0.17885737121105194,248,0.17917148768901825,0.5459575057029724,0.18495121598243713,0.72890625
+attn_19_23,attn,19,cal-en-1,calibration,3.7338041826396506,1.0390406833474568,4.724166114393082,1.1075162239519407,-0.0038920109670231525,0,0.24834147095680237,0.10791976749897003,0.18416298925876617,248,0.20052647590637207,0.5714813470840454,0.20699506998062134,0.69375
+attn_19_23,attn,19,cal-en-2,calibration,3.3431946049591024,1.0618542149778405,4.853161886124916,1.1462103851521934,-0.04195946512998055,0,0.169009268283844,0.07061692327260971,0.21073731780052185,248,0.22293230891227722,0.6151081323623657,0.23012366890907288,0.58125
+attn_19_23,attn,23,cal-zh-1,calibration,2.981238336078164,1.0546738615923779,3.342676305565292,1.0914181503369393,0.018114203094761517,0,0.1326604187488556,0.06724601984024048,0.1667337417602539,248,0.17328983545303345,0.5646554827690125,0.17887982726097107,0.6109375
+attn_19_23,attn,23,cal-zh-2,calibration,3.0395249971461062,1.0612041520781252,3.9775899363194043,1.111839148442333,-0.006099502317895858,0,0.09588906913995743,0.06764671951532364,0.15978427231311798,248,0.16259410977363586,0.5136446356773376,0.16783908009529114,0.6968749999999999
+attn_19_23,attn,23,cal-en-1,calibration,3.044278410194487,1.074729515389283,4.006931562831888,1.1811940201100057,-0.04307182449714221,0,0.12535521388053894,0.08747006952762604,0.1720750480890274,248,0.1742202639579773,0.5411514639854431,0.17984026670455933,0.66015625
+attn_19_23,attn,23,cal-en-2,calibration,2.9710652272259535,1.0799822705908066,3.68900214533588,1.1411985546671786,-0.0470831335221732,0,0.08650433272123337,0.08903923630714417,0.17286045849323273,248,0.17006517946720123,0.538245439529419,0.1755511462688446,0.56875
+mlp_5_6,mlp,5,cal-zh-1,calibration,1.2343440431264474,1.2448870136879528,1.4262659492232905,1.3330956688296154,0.23172673358878942,0,,,,,,,,
+mlp_5_6,mlp,5,cal-zh-2,calibration,1.2339654450576971,1.2535104800781527,1.32561843838562,1.329338987748564,0.22672917563938177,0,,,,,,,,
+mlp_5_6,mlp,5,cal-en-1,calibration,1.2263457192811411,1.2317267194400943,1.3474685872410954,1.321453653023078,0.2447481896291194,0,,,,,,,,
+mlp_5_6,mlp,5,cal-en-2,calibration,1.2077976165854318,1.2358845908873308,1.3068781752715062,1.3085733890221656,0.2539150140171328,0,,,,,,,,
+mlp_5_6,mlp,6,cal-zh-1,calibration,1.4662701939974856,1.3822306441896342,1.7481556583915787,1.7624732809638852,-0.011619417250216499,0,,,,,,,,
+mlp_5_6,mlp,6,cal-zh-2,calibration,1.4798445991262743,1.3637763123448376,1.6959117760237137,1.4894484186140093,-0.005750877367744503,0,,,,,,,,
+mlp_5_6,mlp,6,cal-en-1,calibration,1.464591579412775,1.3766590597797903,1.689543163635455,1.5032948778211077,-0.006204174235411236,0,,,,,,,,
+mlp_5_6,mlp,6,cal-en-2,calibration,1.4368842673613405,1.4008394919323996,1.6226235618647424,1.6896559214993705,-0.006099379702393146,0,,,,,,,,
+mlp_20_21,mlp,20,cal-zh-1,calibration,1.3631629826891347,1.3480373135423591,1.4508876157140895,1.4575901158218467,0.0812649689016906,0,,,,,,,,
+mlp_20_21,mlp,20,cal-zh-2,calibration,1.4305121300612613,1.314068906091469,1.5976490501921767,1.4075788062559882,0.06371076852261268,0,,,,,,,,
+mlp_20_21,mlp,20,cal-en-1,calibration,1.366192568032983,1.35226347926907,1.5789533371800022,1.5827466919073778,0.07632635231947486,0,,,,,,,,
+mlp_20_21,mlp,20,cal-en-2,calibration,1.4386161527552375,1.3312892565009653,1.5821573752114773,1.4214461315124072,0.045400137548793094,0,,,,,,,,
+mlp_20_21,mlp,21,cal-zh-1,calibration,1.389882929510158,1.3382556516005442,1.4917751564635304,1.4560073575192687,0.07070714893962673,0,,,,,,,,
+mlp_20_21,mlp,21,cal-zh-2,calibration,1.4299639779406776,1.3285677793009314,1.6132811807602088,1.41117608068315,0.052803821787486716,0,,,,,,,,
+mlp_20_21,mlp,21,cal-en-1,calibration,1.3453233062044447,1.4107978765350015,1.5399413812441296,1.7867758701030254,0.052139705952592985,0,,,,,,,,
+mlp_20_21,mlp,21,cal-en-2,calibration,1.4554092594533905,1.3180744101503734,1.5831740681011888,1.3865901398082663,0.045747088042376524,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-zh-1,calibration,0.22945597860473388,0.23195008932522895,0.24221562711880595,0.24569969899481256,0.973447272274363,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-zh-2,calibration,0.2290088724166579,0.23181448107574626,0.24542894690924066,0.24779202863840974,0.9735303499099761,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-en-1,calibration,0.23455492648208914,0.2376969550882275,0.24989139477418182,0.2529845119820021,0.972212040535478,0,,,,,,,,
+qk_norm_3,qk_norm,q,cal-en-2,calibration,0.22901516213757628,0.23221465328764443,0.24944281350466996,0.2558284584833273,0.9735059067182225,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-zh-1,calibration,0.19278924917702345,0.19044985719492102,0.21696829135487192,0.21613622029669263,0.9817161843232514,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-zh-2,calibration,0.1914737037557284,0.18949712918171538,0.22059518754606067,0.22263213250570094,0.98191197888193,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-en-1,calibration,0.19741689848184946,0.1948868605635172,0.25540160556856767,0.2544707934700031,0.9808462075453858,0,,,,,,,,
+qk_norm_3,qk_norm,k,cal-en-2,calibration,0.21059744905193553,0.20938264759903177,0.24188706740832974,0.24203851319432876,0.9779690077710401,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-zh-1,calibration,0.20847388940801903,0.20369709176427747,0.2170900495487928,0.21347731005429094,0.9790359000979272,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-zh-2,calibration,0.19935697980676284,0.19546641114417262,0.21545417284813181,0.20622001465458153,0.9807104227014257,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-en-1,calibration,0.20377890669282792,0.1992044511742432,0.2180618268213074,0.21066534853291913,0.9799609125477599,0,,,,,,,,
+qk_norm_7,qk_norm,q,cal-en-2,calibration,0.2049050681850306,0.20143020818672328,0.21865291387428726,0.20943815578722963,0.9795092387917065,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-zh-1,calibration,0.11058006301891404,0.10891256044231754,0.12484384056818072,0.12331584938512573,0.9940936589516192,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-zh-2,calibration,0.1025377174596076,0.10115362014926518,0.1190380009492926,0.11772165381976195,0.994906319610355,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-en-1,calibration,0.11061820718242668,0.10874403924660925,0.1246694172087446,0.12218803487494587,0.9941314653674382,0,,,,,,,,
+qk_norm_7,qk_norm,k,cal-en-2,calibration,0.1097129827694824,0.10824015260116268,0.12658812743768616,0.1257629217271933,0.9941536582549061,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-zh-1,calibration,0.13162251097797606,0.1336688160421926,0.14723992137490324,0.1503968566479842,0.9913220883632566,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-zh-2,calibration,0.1293519958781259,0.1314802843900349,0.15550840977823716,0.15903465263092675,0.9916295485881612,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-en-1,calibration,0.12647077725692374,0.12852946057966494,0.14803436556570124,0.1502912423902487,0.9920027531738658,0,,,,,,,,
+qk_norm_11,qk_norm,q,cal-en-2,calibration,0.1325318326855699,0.13489053569820655,0.15675391452711515,0.16029285586480232,0.9912169569277748,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-zh-1,calibration,0.19235633022629203,0.19080289992858507,0.2290122037106061,0.22871738277610806,0.9816818019088915,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-zh-2,calibration,0.20373761974053708,0.2026516990230578,0.22610646515999447,0.22641810348752162,0.9793703931467622,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-en-1,calibration,0.20072570704403014,0.1997337419816918,0.23210824690537907,0.232000375512668,0.9799664235115935,0,,,,,,,,
+qk_norm_11,qk_norm,k,cal-en-2,calibration,0.21292543194773159,0.2114735746425023,0.2540111544217496,0.253548340715113,0.9775093552656579,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-zh-1,calibration,0.15982234891287764,0.1668593847865133,0.1806429713396301,0.18971996556779416,0.9875945267208645,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-zh-2,calibration,0.15740332806536778,0.1630453842176067,0.17808143035469418,0.1865293914125642,0.987788243570594,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-en-1,calibration,0.16051291705164175,0.16606195948802505,0.18552144269008958,0.19215103895989022,0.9872500535949923,0,,,,,,,,
+qk_norm_15,qk_norm,q,cal-en-2,calibration,0.1646608571816797,0.17165005121760066,0.18445606954569943,0.19141765315408663,0.9867321292200623,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-zh-1,calibration,0.12891191745886205,0.12454504053986937,0.1385991791084394,0.1331904813118359,0.9925662013931714,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-zh-2,calibration,0.12161598809109211,0.11777013931789594,0.1368420974285716,0.1315377420603533,0.9933549654835544,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-en-1,calibration,0.12197566460111624,0.11819082207382924,0.135322184241321,0.13196489880351644,0.9932886289657402,0,,,,,,,,
+qk_norm_15,qk_norm,k,cal-en-2,calibration,0.12441025575886772,0.12064470117171276,0.1406080640306027,0.13504984145616641,0.9929676304409041,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-zh-1,calibration,0.35607606695379634,0.35506507510461943,0.4419932151127893,0.43353427677209583,0.9367889544420241,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-zh-2,calibration,0.3050017336245941,0.3063513991209177,0.35702892926241475,0.35499913749231526,0.9532908937392158,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-en-1,calibration,0.3004172668571062,0.30248734097466856,0.3551434800248225,0.3574678230240773,0.9545873680313379,0,,,,,,,,
+qk_norm_19,qk_norm,q,cal-en-2,calibration,0.332120216475189,0.33442277896716227,0.39234612989053624,0.3869494866775299,0.9444895843396817,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-zh-1,calibration,0.1494891933687725,0.1453919459232531,0.19873146648565732,0.1911951771845603,0.9895189305032556,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-zh-2,calibration,0.14909792210432973,0.14536281895078332,0.19307308117224836,0.18807685138750432,0.9894852004735976,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-en-1,calibration,0.1529571262250341,0.14878974314439253,0.19553431973346902,0.18740449249037416,0.9890023265976378,0,,,,,,,,
+qk_norm_19,qk_norm,k,cal-en-2,calibration,0.1608915089597201,0.15621975475373284,0.19992876686197072,0.1909273809291801,0.9878669550506567,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-zh-1,calibration,0.5472715903763039,0.4942115602598659,0.5959124252670717,0.5246077344510891,0.8699706486172409,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-zh-2,calibration,0.5066278025120974,0.4652273156892467,0.5698812253775749,0.505450409971441,0.8857874771365101,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-en-1,calibration,0.5396383470456222,0.4881273164223247,0.6104980922233472,0.5338668240636818,0.8733304573488526,0,,,,,,,,
+qk_norm_23,qk_norm,q,cal-en-2,calibration,0.5484944291943057,0.4948734608679084,0.6117866340678192,0.5354951226175159,0.8695786360146698,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-zh-1,calibration,0.1313144920837702,0.13005524261632884,0.18219490064170027,0.1762085222381826,0.9915073560564274,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-zh-2,calibration,0.12151622394280005,0.12124252528158447,0.1793544881596151,0.17483197862503747,0.9926360753692314,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-en-1,calibration,0.12730373699904377,0.12630314407300633,0.17809052202752992,0.17316093196082644,0.9919917024971001,0,,,,,,,,
+qk_norm_23,qk_norm,k,cal-en-2,calibration,0.15218433424490216,0.1492986535261055,0.1960834395064235,0.18960021577591676,0.9888227905277445,0,,,,,,,,

results/cpu-qwen35-2b/candidate-summary.json

@@ -0,0 +1,101 @@
+[
+  {
+    "id": "attn_3_7",
+    "kind": "attn",
+    "i": 3,
+    "j": 7,
+    "passed": false,
+    "maximum_symmetric_relative_error": 2.1157833337453664,
+    "donor_component_bytes": 29361152
+  },
+  {
+    "id": "attn_11_15",
+    "kind": "attn",
+    "i": 11,
+    "j": 15,
+    "passed": false,
+    "maximum_symmetric_relative_error": 3.783713712256275,
+    "donor_component_bytes": 29361152
+  },
+  {
+    "id": "attn_19_23",
+    "kind": "attn",
+    "i": 19,
+    "j": 23,
+    "passed": false,
+    "maximum_symmetric_relative_error": 3.7338041826396506,
+    "donor_component_bytes": 29361152
+  },
+  {
+    "id": "mlp_5_6",
+    "kind": "mlp",
+    "i": 5,
+    "j": 6,
+    "passed": false,
+    "maximum_symmetric_relative_error": 1.4798445991262743,
+    "donor_component_bytes": 75497472
+  },
+  {
+    "id": "mlp_20_21",
+    "kind": "mlp",
+    "i": 20,
+    "j": 21,
+    "passed": false,
+    "maximum_symmetric_relative_error": 1.4554092594533905,
+    "donor_component_bytes": 75497472
+  },
+  {
+    "id": "qk_norm_3",
+    "kind": "qk_norm",
+    "i": 3,
+    "j": 3,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.2376969550882275,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_7",
+    "kind": "qk_norm",
+    "i": 7,
+    "j": 7,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.20847388940801903,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_11",
+    "kind": "qk_norm",
+    "i": 11,
+    "j": 11,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.21292543194773159,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_15",
+    "kind": "qk_norm",
+    "i": 15,
+    "j": 15,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.17165005121760066,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_19",
+    "kind": "qk_norm",
+    "i": 19,
+    "j": 19,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.35607606695379634,
+    "donor_component_bytes": 512
+  },
+  {
+    "id": "qk_norm_23",
+    "kind": "qk_norm",
+    "i": 23,
+    "j": 23,
+    "passed": false,
+    "maximum_symmetric_relative_error": 0.5484944291943057,
+    "donor_component_bytes": 512
+  }
+]

results/cpu-qwen35-2b/checksums-full-server.json

@@ -0,0 +1,85 @@
+{
+  "FUNCTIONAL_REPORT.md": "6bfde7e8843e4c87186f8229bf15ab1db919f08ddd8b40fd04aecdec20c7390d",
+  "attention-head-metrics.csv": "732ada825914f8db49b72083cb129cf1305434c84c88253cc32b499ca1a9a119",
+  "baseline-activations.npz": "a851b489a8917a63b07cb6615a47fd1399b36e62f62d6140be618c4966d204ef",
+  "baseline-heldout-token-nll.npy": "303853deb86cb09848b63e2f8b62673db5e730c3b7984c7e65fef3f61c33cda1",
+  "calibration-metrics.csv": "fff8d1e638c6dcfaed88df580de348083fb01621bae7cda7ca2f04ec29636137",
+  "candidate-summary.json": "dc1d738760fa87ef681e33a72caf67cdb68db7413b66f364f80a67b0233fdbdc",
+  "compute-budget-ledger.json": "33dac0f3e9aaf6a82a01c67c22f9a755dff9be8092710f5d3fa8236ce551b255",
+  "download.log": "9f5b76650a94f15729e949605b50ab172ff0fc3a1d791b35dd2a88d4ace02dd0",
+  "experiment.json": "911ba54b489547373b724bd0e719717d8f836807d8fad8aa221eadb5a525d05a",
+  "figures/algebraic_gain_identity.pdf": "b9c44ed9f2df66f7b9172e9a1650db31ec3915290fef3d59b77320a0510ee6de",
+  "figures/algebraic_gain_identity.png": "2e637bb3362ac1592959fbe8f84e67ba06b1ea00ec71dfeec80c45a7ce7520b1",
+  "figures/algebraic_gain_identity.svg": "553ac72babfc005f67a91564f8b737b5edd279d62fb013b8d5d87f6884c4f1ac",
+  "figures/candidate_function_errors.pdf": "370f9b888a6cff64d69bb5f52541e71c66695139cc0b1a45ca8115eae44aae33",
+  "figures/candidate_function_errors.png": "a469f33e4f0a8ba81d4cbcaa333ddcf822f6fa2e4f33ea2c0a6d7da3cbd814b9",
+  "figures/candidate_function_errors.svg": "a647cfcf58bd5d96770daed29cf14ffd245aef3a0c463706f3724bf84b800469",
+  "figures/learned_norm_function_errors.pdf": "cba6296e9297d8f14b1f3346b4ab4015845a54bc880d0f153c8d4b586fe16f90",
+  "figures/learned_norm_function_errors.png": "85b1bce927e1c152778b814629a52996bfdb4a0828995492f56e1fb12f68648f",
+  "figures/learned_norm_function_errors.svg": "ce867a8b8aedfd941dde51ae53ad2541f4e407235c2572092dca6446e71d8f8c",
+  "figures/learned_norm_network_substitution.pdf": "6d3440323d276b4a70209f71cd31e016d8eb78d257bdaa41ffc0468d2813a428",
+  "figures/learned_norm_network_substitution.png": "e0dca3ce8082abf3e904aaa27aa581bd1c7ed1ff7a19285dde613dd88b3738ad",
+  "figures/learned_norm_network_substitution.svg": "7753abfa3f85cda50fbc3a00491f54a597cbbb54fdb75f0c0b2fe7a7fcc131c5",
+  "final-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "final-source-hashes.json": "52398645bce397827c36048c417f721a1cd8f76dc38eeaee530b8a5031b09e91",
+  "final-tests.txt": "3eb63c08f7b8de12c4198e79da9909582d5d64cb821ef1b08d38de7e4e43eaf5",
+  "functional-code-hashes.json": "3756c43b0904c53d9a3ea1fe92496cc9f890cc7b033a3c9577ef53cecd4d8536",
+  "functional-environment.json": "b9a2b4b850b3f26cfa96efc1ac6a9470d678f17197f96a01d9c3bdf5111b148a",
+  "functional-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "functional-run.log": "95684864b79b72dbefdf5485fca00bca3e80719c9e925a2a6fe1a6b0a07825d3",
+  "functional-status.json": "fdda303b7e520f9e0065c65281df5b4bd6e9277edec52f9e5818047598a2a2ea",
+  "gain-factorization-k_to_q-token-nll.npy": "767f2a885532e347a654a86ed0a7f19bd4938c52296dcc74db26876acb3fe6a1",
+  "gain-factorization-kernel-timing.csv": "7ee790a456d4b9dac9d4756b71288f4df165c0c6111fe94657fa6804f91154d1",
+  "gain-factorization-optimized-summary.json": "0e303074384a3933577b2c3c29a042fe00e64cf308eb8f6190db8f7e3db55e7b",
+  "gain-factorization-optimized-timing.csv": "b5c85e1f9bcfb49bf317090470530a44546079c5a79c1df2365c00c36144dc10",
+  "gain-factorization-optimized.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "gain-factorization-protocol.json": "7690e4cf5dec415e414813e5beb3434ae62d51d95e68ad5de48e698980e26cab",
+  "gain-factorization-q_to_k-token-nll.npy": "6ea27edea56593c7269403ce5a0f7eb8673b9d80d1e00806d38bc6a2a681fff8",
+  "gain-factorization-score-metrics.csv": "7cfcc997980cd0566721e529f6bb1bca2f95858da50d5cf533eb31742ab0c230",
+  "gain-factorization-summary.json": "3bfde7dba2abd19e6dc945c05ad924181bc0927e0b88806337c9bbab9db7b359",
+  "gain-factorization.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "install-torch-disk-tmp.log": "85ee085d264707b8fd0031876c3ae9cf4e36151db9cbf53edef979f10d144c13",
+  "install-torch.log": "e297f77b38de146e85e6159025ea1e0ec93622fe26ef46698df180c7c96378b8",
+  "manifest.json": "28efbaf94a5034333339a35f08ead151459a7d20c6c2d2f07a451e7f3bf154de",
+  "model_config.json": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4",
+  "neuron-reuse-activations.npz": "0ce0b7ba4efef0164a68d91595e8eb47a6347f1f364eeda764835cfa09b09448",
+  "neuron-reuse-protocol.json": "cc4fcf06ebd4ee5420933534021006df6e62259b00fd922f6e8fabada89472f7",
+  "neuron-reuse-search-summary.json": "30a5ef914c4ff766e9bde09e3784a229bdbe7a10cea871bef3be327df8047c25",
+  "neuron-reuse-selection.json": "afafc7f8d35c562940f5afe47c5edaf3c5b7ca97744152bb5f9988b9910797a4",
+  "neuron-reuse-summary.json": "f14999d51d262c67f90e2e37918701b6b56107c7d5c47fad6da1cdfeb35d62a2",
+  "neuron-reuse.log": "be387fa429688a6a34131f0a5899a81586a2bde85cf209253c2da977e625168c",
+  "norm-repetition-12-uses-13-token-nll.npy": "fb05b796fd40122aa0a1b05ce92585c980f47a1cb1c771df4fabc720ade4bbd8",
+  "norm-repetition-13-uses-12-token-nll.npy": "5470a87f22f100ed8e43caf90a05831453b4277707d6d7085eecc4bd43ae8cb6",
+  "norm-repetition-function-metrics.csv": "d5f4eed05a9f5c6e0f971331dd9d0b834d6141b4aeb9e3c725a5bee1f4314593",
+  "norm-repetition-input-profiles.npz": "7e3415b8f28d8844658facf2c37415cc0f3a4de8f6d39d3e0c4b01e1b3a4d7e0",
+  "norm-repetition-protocol.json": "ad8d2319572b4a1daa5d3ae227bd344b5f1db7941ee567b43d1f25dbd963f6a0",
+  "norm-repetition-selection.json": "4dae5a62da0889c0ed4de7813a8f720bb94991e468b9d6cdee8c5327d549ac55",
+  "norm-repetition-summary.json": "70b5067d0820b5a5bc7062c71b9f29c5590d9bb12bf538beb961135d1aab8834",
+  "norm-repetition-uniform-bounds.csv": "48e71a9cac9410d110483cef8356e02b2dcdeae1120bcb243ae86903852fb32a",
+  "norm-repetition.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "preflight-first-attempt.log": "1a336568f0ed99ac306ab1d690ff00f325c9c73ec6be84df4ace4374d490c650",
+  "preflight-second-attempt.json": "a6b68d31710e45cbf09ee100b490c781547334f76ecfbb3e09332779e953b30c",
+  "preflight-second-attempt.log": "42a07ef0a3fd8cb1a6475bec6e7958f754a24012fb8269073d8625b1b891594f",
+  "preflight.json": "86113ecbf499f4a84ea4d7fd7893e8c12f2784b6e4678b5cbcbb0e022e295e4b",
+  "preflight.log": "66dd7d1e0b146fc448a1064efbde5d693126d3f3cbb2fc9abdeb4485ead7c951",
+  "reference-check-first-attempt.log": "ed77d6619c19551401938a54fae0ef1aa50846b943ea1ba3321750c9cb985428",
+  "reference-check-second-attempt.log": "5923802faaba183a099d0eb313d5dc2de4825b1d52ec6ce0699a8201f68d99af",
+  "reference-check.json": "e82b0d8634ccae0671269587da960e06cce33e0a0e2ed15db15d946f6399d9ed",
+  "reference-check.log": "e4a1d55d309bff39535ecab272bd2aa6d8fe183656724b7731e4171d03150f11",
+  "retention-policy.json": "e3cdb6707985d2fa1fcb7f088f8f46f9eafb89eea13e37f51c554f57790acd6e",
+  "samples-tokenized.json": "2e56e14a4eefedd702bfe09d3edb2f17f360f53dd420e29fcf0c0ec1affc0184",
+  "selection.json": "c8c86d12fe7db6fb8fffca8078ec0ff5438b97998088c89862db8f9b6152daa3",
+  "source_metadata.json": "d3be2a8560f4270b8af27891d1076cb7cf80bd84c4c19fe7790d5fb06f3a342e",
+  "storage-accounting.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "storage-evidence.json": "c6968f4143910cef9d1359d3cd3342d289b9c791c6e2a6ab598fe235b105e0d9",
+  "supervisor-error.txt": "4cba26d15e963620745ab3d4625f4de967859a9c43993f78ca1ce2e5c06aeb42",
+  "supervisor-first-attempt-error.txt": "e689f94e983b84cfc4e6b372175826d77dcd2d5c9b8d166598bc8607227afbba",
+  "supervisor-first-attempt.json": "93d7185df1810aec2aa396fd8247e188b8a37a821731ef681e66389695bd9a8e",
+  "supervisor-second-attempt.json": "7333a3634f7d169bc78d9eeb5e47d8fbd2970c8401b4d6160b92038ad2a89f55",
+  "supervisor-status.json": "ae078d540c9c683bac468313a85754d8ecf1a88be482a2d5895bbfca0c610404",
+  "supervisor-third-attempt.json": "e86ce8e2f55f75ab3c61ffd7b4a1517e627cebbdb85ee7f966615c5d683722ed",
+  "supervisor.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "tokenizer-download.log": "1d9c8b608f3c79c9d7af93c593543ddd14db3e077e2c108738bccd518df9a79a",
+  "tokenizers-wheel.log": "f0b4f7c1360ff1a00e04fe443e619682c7d7b024df55545d31f54951b3001480",
+  "unit-tests.txt": "fa8ffc7277ea722b9261da462b797710ed4ae3eb8975ad51937c419b3052e3af"
+}

results/cpu-qwen35-2b/checksums-functional.json

@@ -0,0 +1,85 @@
+{
+  "DELIVERY.md": "8f4608add548dc4c6bc3785ccc262a4c695098c043563f43193edac904702cb1",
+  "FUNCTIONAL_REPORT.md": "6bfde7e8843e4c87186f8229bf15ab1db919f08ddd8b40fd04aecdec20c7390d",
+  "attention-head-metrics.csv": "732ada825914f8db49b72083cb129cf1305434c84c88253cc32b499ca1a9a119",
+  "baseline-heldout-token-nll.npy": "303853deb86cb09848b63e2f8b62673db5e730c3b7984c7e65fef3f61c33cda1",
+  "calibration-metrics.csv": "fff8d1e638c6dcfaed88df580de348083fb01621bae7cda7ca2f04ec29636137",
+  "candidate-summary.json": "dc1d738760fa87ef681e33a72caf67cdb68db7413b66f364f80a67b0233fdbdc",
+  "checksums-full-server.json": "152305bd23c849a75a0e0b5b686cadd93e5db017ea9f2733a52aa771353ed5f3",
+  "compute-budget-ledger.json": "33dac0f3e9aaf6a82a01c67c22f9a755dff9be8092710f5d3fa8236ce551b255",
+  "download.log": "9f5b76650a94f15729e949605b50ab172ff0fc3a1d791b35dd2a88d4ace02dd0",
+  "experiment.json": "911ba54b489547373b724bd0e719717d8f836807d8fad8aa221eadb5a525d05a",
+  "external-activation-cache.json": "ceb8ec04ac1ca6fe1a0466520b0c58180741339541b4ace4c5696c359ed25917",
+  "figures/algebraic_gain_identity.pdf": "b9c44ed9f2df66f7b9172e9a1650db31ec3915290fef3d59b77320a0510ee6de",
+  "figures/algebraic_gain_identity.png": "2e637bb3362ac1592959fbe8f84e67ba06b1ea00ec71dfeec80c45a7ce7520b1",
+  "figures/algebraic_gain_identity.svg": "553ac72babfc005f67a91564f8b737b5edd279d62fb013b8d5d87f6884c4f1ac",
+  "figures/candidate_function_errors.pdf": "370f9b888a6cff64d69bb5f52541e71c66695139cc0b1a45ca8115eae44aae33",
+  "figures/candidate_function_errors.png": "a469f33e4f0a8ba81d4cbcaa333ddcf822f6fa2e4f33ea2c0a6d7da3cbd814b9",
+  "figures/candidate_function_errors.svg": "a647cfcf58bd5d96770daed29cf14ffd245aef3a0c463706f3724bf84b800469",
+  "figures/learned_norm_function_errors.pdf": "cba6296e9297d8f14b1f3346b4ab4015845a54bc880d0f153c8d4b586fe16f90",
+  "figures/learned_norm_function_errors.png": "85b1bce927e1c152778b814629a52996bfdb4a0828995492f56e1fb12f68648f",
+  "figures/learned_norm_function_errors.svg": "ce867a8b8aedfd941dde51ae53ad2541f4e407235c2572092dca6446e71d8f8c",
+  "figures/learned_norm_network_substitution.pdf": "6d3440323d276b4a70209f71cd31e016d8eb78d257bdaa41ffc0468d2813a428",
+  "figures/learned_norm_network_substitution.png": "e0dca3ce8082abf3e904aaa27aa581bd1c7ed1ff7a19285dde613dd88b3738ad",
+  "figures/learned_norm_network_substitution.svg": "7753abfa3f85cda50fbc3a00491f54a597cbbb54fdb75f0c0b2fe7a7fcc131c5",
+  "final-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "final-source-hashes.json": "52398645bce397827c36048c417f721a1cd8f76dc38eeaee530b8a5031b09e91",
+  "final-tests.txt": "3eb63c08f7b8de12c4198e79da9909582d5d64cb821ef1b08d38de7e4e43eaf5",
+  "functional-code-hashes.json": "3756c43b0904c53d9a3ea1fe92496cc9f890cc7b033a3c9577ef53cecd4d8536",
+  "functional-environment.json": "b9a2b4b850b3f26cfa96efc1ac6a9470d678f17197f96a01d9c3bdf5111b148a",
+  "functional-report.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "functional-run.log": "95684864b79b72dbefdf5485fca00bca3e80719c9e925a2a6fe1a6b0a07825d3",
+  "functional-status.json": "fdda303b7e520f9e0065c65281df5b4bd6e9277edec52f9e5818047598a2a2ea",
+  "gain-factorization-k_to_q-token-nll.npy": "767f2a885532e347a654a86ed0a7f19bd4938c52296dcc74db26876acb3fe6a1",
+  "gain-factorization-kernel-timing.csv": "7ee790a456d4b9dac9d4756b71288f4df165c0c6111fe94657fa6804f91154d1",
+  "gain-factorization-optimized-summary.json": "0e303074384a3933577b2c3c29a042fe00e64cf308eb8f6190db8f7e3db55e7b",
+  "gain-factorization-optimized-timing.csv": "b5c85e1f9bcfb49bf317090470530a44546079c5a79c1df2365c00c36144dc10",
+  "gain-factorization-optimized.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "gain-factorization-protocol.json": "7690e4cf5dec415e414813e5beb3434ae62d51d95e68ad5de48e698980e26cab",
+  "gain-factorization-q_to_k-token-nll.npy": "6ea27edea56593c7269403ce5a0f7eb8673b9d80d1e00806d38bc6a2a681fff8",
+  "gain-factorization-score-metrics.csv": "7cfcc997980cd0566721e529f6bb1bca2f95858da50d5cf533eb31742ab0c230",
+  "gain-factorization-summary.json": "3bfde7dba2abd19e6dc945c05ad924181bc0927e0b88806337c9bbab9db7b359",
+  "gain-factorization.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "install-torch-disk-tmp.log": "85ee085d264707b8fd0031876c3ae9cf4e36151db9cbf53edef979f10d144c13",
+  "install-torch.log": "e297f77b38de146e85e6159025ea1e0ec93622fe26ef46698df180c7c96378b8",
+  "manifest.json": "28efbaf94a5034333339a35f08ead151459a7d20c6c2d2f07a451e7f3bf154de",
+  "model_config.json": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4",
+  "neuron-reuse-protocol.json": "cc4fcf06ebd4ee5420933534021006df6e62259b00fd922f6e8fabada89472f7",
+  "neuron-reuse-search-summary.json": "30a5ef914c4ff766e9bde09e3784a229bdbe7a10cea871bef3be327df8047c25",
+  "neuron-reuse-selection.json": "afafc7f8d35c562940f5afe47c5edaf3c5b7ca97744152bb5f9988b9910797a4",
+  "neuron-reuse-summary.json": "f14999d51d262c67f90e2e37918701b6b56107c7d5c47fad6da1cdfeb35d62a2",
+  "neuron-reuse.log": "be387fa429688a6a34131f0a5899a81586a2bde85cf209253c2da977e625168c",
+  "norm-repetition-12-uses-13-token-nll.npy": "fb05b796fd40122aa0a1b05ce92585c980f47a1cb1c771df4fabc720ade4bbd8",
+  "norm-repetition-13-uses-12-token-nll.npy": "5470a87f22f100ed8e43caf90a05831453b4277707d6d7085eecc4bd43ae8cb6",
+  "norm-repetition-function-metrics.csv": "d5f4eed05a9f5c6e0f971331dd9d0b834d6141b4aeb9e3c725a5bee1f4314593",
+  "norm-repetition-protocol.json": "ad8d2319572b4a1daa5d3ae227bd344b5f1db7941ee567b43d1f25dbd963f6a0",
+  "norm-repetition-selection.json": "4dae5a62da0889c0ed4de7813a8f720bb94991e468b9d6cdee8c5327d549ac55",
+  "norm-repetition-summary.json": "70b5067d0820b5a5bc7062c71b9f29c5590d9bb12bf538beb961135d1aab8834",
+  "norm-repetition-uniform-bounds.csv": "48e71a9cac9410d110483cef8356e02b2dcdeae1120bcb243ae86903852fb32a",
+  "norm-repetition.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "preflight-first-attempt.log": "1a336568f0ed99ac306ab1d690ff00f325c9c73ec6be84df4ace4374d490c650",
+  "preflight-second-attempt.json": "a6b68d31710e45cbf09ee100b490c781547334f76ecfbb3e09332779e953b30c",
+  "preflight-second-attempt.log": "42a07ef0a3fd8cb1a6475bec6e7958f754a24012fb8269073d8625b1b891594f",
+  "preflight.json": "86113ecbf499f4a84ea4d7fd7893e8c12f2784b6e4678b5cbcbb0e022e295e4b",
+  "preflight.log": "66dd7d1e0b146fc448a1064efbde5d693126d3f3cbb2fc9abdeb4485ead7c951",
+  "reference-check-first-attempt.log": "ed77d6619c19551401938a54fae0ef1aa50846b943ea1ba3321750c9cb985428",
+  "reference-check-second-attempt.log": "5923802faaba183a099d0eb313d5dc2de4825b1d52ec6ce0699a8201f68d99af",
+  "reference-check.json": "e82b0d8634ccae0671269587da960e06cce33e0a0e2ed15db15d946f6399d9ed",
+  "reference-check.log": "e4a1d55d309bff39535ecab272bd2aa6d8fe183656724b7731e4171d03150f11",
+  "retention-policy.json": "e3cdb6707985d2fa1fcb7f088f8f46f9eafb89eea13e37f51c554f57790acd6e",
+  "samples-tokenized.json": "2e56e14a4eefedd702bfe09d3edb2f17f360f53dd420e29fcf0c0ec1affc0184",
+  "selection.json": "c8c86d12fe7db6fb8fffca8078ec0ff5438b97998088c89862db8f9b6152daa3",
+  "source_metadata.json": "d3be2a8560f4270b8af27891d1076cb7cf80bd84c4c19fe7790d5fb06f3a342e",
+  "storage-accounting.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "storage-evidence.json": "c6968f4143910cef9d1359d3cd3342d289b9c791c6e2a6ab598fe235b105e0d9",
+  "supervisor-error.txt": "4cba26d15e963620745ab3d4625f4de967859a9c43993f78ca1ce2e5c06aeb42",
+  "supervisor-first-attempt-error.txt": "e689f94e983b84cfc4e6b372175826d77dcd2d5c9b8d166598bc8607227afbba",
+  "supervisor-first-attempt.json": "93d7185df1810aec2aa396fd8247e188b8a37a821731ef681e66389695bd9a8e",
+  "supervisor-second-attempt.json": "7333a3634f7d169bc78d9eeb5e47d8fbd2970c8401b4d6160b92038ad2a89f55",
+  "supervisor-status.json": "ae078d540c9c683bac468313a85754d8ecf1a88be482a2d5895bbfca0c610404",
+  "supervisor-third-attempt.json": "e86ce8e2f55f75ab3c61ffd7b4a1517e627cebbdb85ee7f966615c5d683722ed",
+  "supervisor.log": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+  "tokenizer-download.log": "1d9c8b608f3c79c9d7af93c593543ddd14db3e077e2c108738bccd518df9a79a",
+  "tokenizers-wheel.log": "f0b4f7c1360ff1a00e04fe443e619682c7d7b024df55545d31f54951b3001480",
+  "unit-tests.txt": "fa8ffc7277ea722b9261da462b797710ed4ae3eb8975ad51937c419b3052e3af"
+}

results/cpu-qwen35-2b/compute-budget-ledger.json

@@ -0,0 +1,54 @@
+{
+  "spent_seconds": 1167.355532555841,
+  "initial_reserve_reason": "Conservative reserve for early reference checks and stopped preflight, not a measured runtime",
+  "calls": [
+    {
+      "module": "functional.reference_check",
+      "seconds": 6.833657197188586
+    },
+    {
+      "module": "functional.preflight",
+      "seconds": 27.313251950312406
+    },
+    {
+      "module": "functional.report",
+      "seconds": 2.1849583038128912
+    },
+    {
+      "module": "functional.reference_check",
+      "seconds": 16.264290761202574
+    },
+    {
+      "module": "functional.preflight",
+      "seconds": 58.00242758495733
+    },
+    {
+      "module": "functional.run",
+      "seconds": 204.33689004788175
+    },
+    {
+      "module": "functional.storage",
+      "seconds": 1.2859071069397032
+    },
+    {
+      "module": "functional.report",
+      "seconds": 10.058376160915941
+    },
+    {
+      "module": "functional.gain_factorization",
+      "seconds": 232.08875080291182
+    },
+    {
+      "module": "functional.benchmark_factorization",
+      "seconds": 4.185962562914938
+    },
+    {
+      "module": "functional.neuron_reuse",
+      "seconds": 55.004304428584874
+    },
+    {
+      "module": "functional.norm_repetition",
+      "seconds": 369.7967556482181
+    }
+  ]
+}

results/cpu-qwen35-2b/download.log

@@ -0,0 +1,73 @@
+Downloading README.md, expected 62814 bytes
+Downloading config.json, expected 2908 bytes
+Downloading model.safetensors-00001-of-00001.safetensors, expected 4548221488 bytes
+Weight download 67108864/4548221488 bytes (1.5%)
+Weight download 134217728/4548221488 bytes (3.0%)
+Weight download 201326592/4548221488 bytes (4.4%)
+Weight download 268435456/4548221488 bytes (5.9%)
+Weight download 335544320/4548221488 bytes (7.4%)
+Weight download 402653184/4548221488 bytes (8.9%)
+Weight download 469762048/4548221488 bytes (10.3%)
+Weight download 536870912/4548221488 bytes (11.8%)
+Weight download 603979776/4548221488 bytes (13.3%)
+Weight download 671088640/4548221488 bytes (14.8%)
+Weight download 738197504/4548221488 bytes (16.2%)
+Weight download 805306368/4548221488 bytes (17.7%)
+Weight download 872415232/4548221488 bytes (19.2%)
+Weight download 939524096/4548221488 bytes (20.7%)
+Weight download 1006632960/4548221488 bytes (22.1%)
+Weight download 1073741824/4548221488 bytes (23.6%)
+Weight download 1140850688/4548221488 bytes (25.1%)
+Weight download 1207959552/4548221488 bytes (26.6%)
+Weight download 1275068416/4548221488 bytes (28.0%)
+Weight download 1342177280/4548221488 bytes (29.5%)
+Weight download 1409286144/4548221488 bytes (31.0%)
+Weight download 1476395008/4548221488 bytes (32.5%)
+Weight download 1543503872/4548221488 bytes (33.9%)
+Weight download 1610612736/4548221488 bytes (35.4%)
+Weight download 1677721600/4548221488 bytes (36.9%)
+Weight download 1744830464/4548221488 bytes (38.4%)
+Weight download 1811939328/4548221488 bytes (39.8%)
+Weight download 1879048192/4548221488 bytes (41.3%)
+Weight download 1946157056/4548221488 bytes (42.8%)
+Weight download 2013265920/4548221488 bytes (44.3%)
+Weight download 2080374784/4548221488 bytes (45.7%)
+Weight download 2147483648/4548221488 bytes (47.2%)
+Weight download 2214592512/4548221488 bytes (48.7%)
+Weight download 2281701376/4548221488 bytes (50.2%)
+Weight download 2348810240/4548221488 bytes (51.6%)
+Weight download 2415919104/4548221488 bytes (53.1%)
+Weight download 2483027968/4548221488 bytes (54.6%)
+Weight download 2550136832/4548221488 bytes (56.1%)
+Weight download 2617245696/4548221488 bytes (57.5%)
+Weight download 2684354560/4548221488 bytes (59.0%)
+Weight download 2751463424/4548221488 bytes (60.5%)
+Weight download 2818572288/4548221488 bytes (62.0%)
+Weight download 2885681152/4548221488 bytes (63.4%)
+Weight download 2952790016/4548221488 bytes (64.9%)
+Weight download 3019898880/4548221488 bytes (66.4%)
+Weight download 3087007744/4548221488 bytes (67.9%)
+Weight download 3154116608/4548221488 bytes (69.3%)
+Weight download 3221225472/4548221488 bytes (70.8%)
+Weight download 3288334336/4548221488 bytes (72.3%)
+Weight download 3355443200/4548221488 bytes (73.8%)
+Weight download 3422552064/4548221488 bytes (75.3%)
+Weight download 3489660928/4548221488 bytes (76.7%)
+Weight download 3556769792/4548221488 bytes (78.2%)
+Weight download 3623878656/4548221488 bytes (79.7%)
+Weight download 3690987520/4548221488 bytes (81.2%)
+Weight download 3758096384/4548221488 bytes (82.6%)
+Weight download 3825205248/4548221488 bytes (84.1%)
+Weight download 3892314112/4548221488 bytes (85.6%)
+Weight download 3959422976/4548221488 bytes (87.1%)
+Weight download 4026531840/4548221488 bytes (88.5%)
+Weight download 4093640704/4548221488 bytes (90.0%)
+Weight download 4160749568/4548221488 bytes (91.5%)
+Weight download 4227858432/4548221488 bytes (93.0%)
+Weight download 4294967296/4548221488 bytes (94.4%)
+Weight download 4346894896/4548221488 bytes (95.6%)
+Weight download 4414003760/4548221488 bytes (97.0%)
+Weight download 4481112624/4548221488 bytes (98.5%)
+Weight download 4548221488/4548221488 bytes (100.0%)
+Downloading model.safetensors.index.json, expected 64460 bytes
+{"revision": "15852e8c16360a2fea060d615a32b45270f8a8fc", "bytes": 4548351670}

results/cpu-qwen35-2b/experiment.json

@@ -0,0 +1,49 @@
+{
+  "revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "attention_pairs": [
+    [
+      3,
+      7
+    ],
+    [
+      11,
+      15
+    ],
+    [
+      19,
+      23
+    ]
+  ],
+  "mlp_pairs": [
+    [
+      5,
+      6
+    ],
+    [
+      20,
+      21
+    ]
+  ],
+  "norm_layers": [
+    3,
+    7,
+    11,
+    15,
+    19,
+    23
+  ],
+  "seconds": 6849,
+  "output_relative_l2_limit": 0.05,
+  "token_p95_limit": 0.1,
+  "attention_js_limit": 0.01,
+  "attention_top5_min": 0.8,
+  "heldout_nll_mean_delta_limit": 0.01,
+  "heldout_nll_sequence_p95_delta_limit": 0.05,
+  "max_intervention_candidates": 1,
+  "directions_per_selected_candidate": 2,
+  "training": false,
+  "search_R": false,
+  "automatic_weight_cleanup": false,
+  "dtype": "original weights decoded to FP32; not a BF16 backend benchmark",
+  "blas_threads": "1"
+}

results/cpu-qwen35-2b/external-activation-cache.json

@@ -0,0 +1,31 @@
+{
+  "host": "8.211.154.134",
+  "files": [
+    {
+      "name": "baseline-activations.npz",
+      "server_path": "/opt/exp-model-parameter-similarity/runs/cpu-qwen35-2b/baseline-activations.npz",
+      "bytes": 38802964,
+      "sha256": "a851b489a8917a63b07cb6615a47fd1399b36e62f62d6140be618c4966d204ef",
+      "retained": true,
+      "reproducible_by": "functional.run / functional.neuron_reuse / functional.norm_repetition"
+    },
+    {
+      "name": "neuron-reuse-activations.npz",
+      "server_path": "/opt/exp-model-parameter-similarity/runs/cpu-qwen35-2b/neuron-reuse-activations.npz",
+      "bytes": 25166826,
+      "sha256": "0ce0b7ba4efef0164a68d91595e8eb47a6347f1f364eeda764835cfa09b09448",
+      "retained": true,
+      "reproducible_by": "functional.run / functional.neuron_reuse / functional.norm_repetition"
+    },
+    {
+      "name": "norm-repetition-input-profiles.npz",
+      "server_path": "/opt/exp-model-parameter-similarity/runs/cpu-qwen35-2b/norm-repetition-input-profiles.npz",
+      "bytes": 4194818,
+      "sha256": "7e3415b8f28d8844658facf2c37415cc0f3a4de8f6d39d3e0c4b01e1b3a4d7e0",
+      "retained": true,
+      "reproducible_by": "functional.run / functional.neuron_reuse / functional.norm_repetition"
+    }
+  ],
+  "not_model_weights": true,
+  "reason": "Large intermediate caches retained on the authorized server; all figure source tables and token NLL arrays are delivered in Git."
+}

results/cpu-qwen35-2b/figures/algebraic_gain_identity.svg

@@ -0,0 +1,581 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="712.424062pt" height="285.518438pt" viewBox="0 0 712.424062 285.518438" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:11:02.311213</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 285.518438 
+L 712.424062 285.518438 
+L 712.424062 -0 
+L 0 -0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 50.466406 247.317656 
+L 353.864062 247.317656 
+L 353.864062 45.880469 
+L 50.466406 45.880469 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="m50faf94306" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m50faf94306" x="64.257209" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="64.257209" y="261.915313" transform="rotate(-0 64.257209 261.915313)">3</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#m50faf94306" x="119.420419" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="119.420419" y="261.915313" transform="rotate(-0 119.420419 261.915313)">7</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#m50faf94306" x="174.583629" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="174.583629" y="261.915313" transform="rotate(-0 174.583629 261.915313)">11</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#m50faf94306" x="229.746839" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="229.746839" y="261.915313" transform="rotate(-0 229.746839 261.915313)">15</text>
+     </g>
+    </g>
+    <g id="xtick_5">
+     <g id="line2d_5">
+      <g>
+       <use xlink:href="#m50faf94306" x="284.91005" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_5">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="284.91005" y="261.915313" transform="rotate(-0 284.91005 261.915313)">19</text>
+     </g>
+    </g>
+    <g id="xtick_6">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#m50faf94306" x="340.07326" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="340.07326" y="261.915313" transform="rotate(-0 340.07326 261.915313)">23</text>
+     </g>
+    </g>
+    <g id="text_7">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="202.165234" y="275.916094" transform="rotate(-0 202.165234 275.916094)">Full-attention layer index</text>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_7">
+      <defs>
+       <path id="m9a2e91a9aa" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="229.265954" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="233.064782" transform="rotate(-0 43.466406 233.064782)">2.00</text>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="205.263431" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="209.06226" transform="rotate(-0 43.466406 209.06226)">2.25</text>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_9">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="181.260909" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="185.059737" transform="rotate(-0 43.466406 185.059737)">2.50</text>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="157.258386" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="161.057214" transform="rotate(-0 43.466406 161.057214)">2.75</text>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_11">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="133.255863" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_12">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="137.054691" transform="rotate(-0 43.466406 137.054691)">3.00</text>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_12">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="109.25334" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_13">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="113.052169" transform="rotate(-0 43.466406 113.052169)">3.25</text>
+     </g>
+    </g>
+    <g id="ytick_7">
+     <g id="line2d_13">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="85.250818" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_14">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="89.049646" transform="rotate(-0 43.466406 89.049646)">3.50</text>
+     </g>
+    </g>
+    <g id="ytick_8">
+     <g id="line2d_14">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="50.466406" y="61.248295" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_15">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="65.047123" transform="rotate(-0 43.466406 65.047123)">3.75</text>
+     </g>
+    </g>
+    <g id="text_16">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="146.599063" transform="rotate(-90 14.798438 146.599063)">Maximum absolute score error</text>
+    </g>
+    <g id="text_17">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="50.466406" y="42.880469" transform="rotate(-0 50.466406 42.880469)">1e−6</text>
+    </g>
+   </g>
+   <g id="line2d_15">
+    <path d="M 64.257209 238.16142 
+L 119.420419 238.16142 
+L 174.583629 238.16142 
+L 229.746839 146.599063 
+L 284.91005 55.036705 
+L 340.07326 55.036705 
+" clip-path="url(#p0542a2a98b)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m9b4cc9d8d2" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #1f77b4"/>
+    </defs>
+    <g clip-path="url(#p0542a2a98b)">
+     <use xlink:href="#m9b4cc9d8d2" x="64.257209" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="119.420419" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="174.583629" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="229.746839" y="146.599063" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="284.91005" y="55.036705" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="340.07326" y="55.036705" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_16">
+    <path d="M 64.257209 238.16142 
+L 119.420419 238.16142 
+L 174.583629 146.599063 
+L 229.746839 238.16142 
+L 284.91005 238.16142 
+L 340.07326 55.036705 
+" clip-path="url(#p0542a2a98b)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m556f16804f" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #ff7f0e"/>
+    </defs>
+    <g clip-path="url(#p0542a2a98b)">
+     <use xlink:href="#m556f16804f" x="64.257209" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="119.420419" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="174.583629" y="146.599063" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="229.746839" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="284.91005" y="238.16142" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="340.07326" y="55.036705" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="patch_3">
+    <path d="M 50.466406 247.317656 
+L 50.466406 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 353.864062 247.317656 
+L 353.864062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 50.466406 247.317656 
+L 353.864062 247.317656 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 50.466406 45.880469 
+L 353.864062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="legend_1">
+    <g id="patch_7">
+     <path d="M 56.066406 76.281719 
+L 107.408906 76.281719 
+Q 109.008906 76.281719 109.008906 74.681719 
+L 109.008906 51.480469 
+Q 109.008906 49.880469 107.408906 49.880469 
+L 56.066406 49.880469 
+Q 54.466406 49.880469 54.466406 51.480469 
+L 54.466406 74.681719 
+Q 54.466406 76.281719 56.066406 76.281719 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_17">
+     <path d="M 57.666406 56.359219 
+L 65.666406 56.359219 
+L 73.666406 56.359219 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m9b4cc9d8d2" x="65.666406" y="56.359219" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_18">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="59.159219" transform="rotate(-0 80.066406 59.159219)">q_to_k</text>
+    </g>
+    <g id="line2d_18">
+     <path d="M 57.666406 68.359844 
+L 65.666406 68.359844 
+L 73.666406 68.359844 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m556f16804f" x="65.666406" y="68.359844" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_19">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="71.159844" transform="rotate(-0 80.066406 71.159844)">k_to_q</text>
+    </g>
+   </g>
+  </g>
+  <g id="axes_2">
+   <g id="patch_8">
+    <path d="M 401.826406 247.317656 
+L 705.224062 247.317656 
+L 705.224062 45.880469 
+L 401.826406 45.880469 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_3">
+    <g id="xtick_7">
+     <g id="line2d_19">
+      <g>
+       <use xlink:href="#m50faf94306" x="415.617209" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_20">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="415.617209" y="261.915313" transform="rotate(-0 415.617209 261.915313)">3</text>
+     </g>
+    </g>
+    <g id="xtick_8">
+     <g id="line2d_20">
+      <g>
+       <use xlink:href="#m50faf94306" x="470.780419" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_21">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="470.780419" y="261.915313" transform="rotate(-0 470.780419 261.915313)">7</text>
+     </g>
+    </g>
+    <g id="xtick_9">
+     <g id="line2d_21">
+      <g>
+       <use xlink:href="#m50faf94306" x="525.943629" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_22">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="525.943629" y="261.915313" transform="rotate(-0 525.943629 261.915313)">11</text>
+     </g>
+    </g>
+    <g id="xtick_10">
+     <g id="line2d_22">
+      <g>
+       <use xlink:href="#m50faf94306" x="581.106839" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_23">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="581.106839" y="261.915313" transform="rotate(-0 581.106839 261.915313)">15</text>
+     </g>
+    </g>
+    <g id="xtick_11">
+     <g id="line2d_23">
+      <g>
+       <use xlink:href="#m50faf94306" x="636.27005" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_24">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="636.27005" y="261.915313" transform="rotate(-0 636.27005 261.915313)">19</text>
+     </g>
+    </g>
+    <g id="xtick_12">
+     <g id="line2d_24">
+      <g>
+       <use xlink:href="#m50faf94306" x="691.43326" y="247.317656" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_25">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="691.43326" y="261.915313" transform="rotate(-0 691.43326 261.915313)">23</text>
+     </g>
+    </g>
+    <g id="text_26">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="553.525234" y="275.916094" transform="rotate(-0 553.525234 275.916094)">Full-attention layer index</text>
+    </g>
+   </g>
+   <g id="matplotlib.axis_4">
+    <g id="ytick_9">
+     <g id="line2d_25">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="242.241146" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_27">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="246.039974" transform="rotate(-0 394.826406 246.039974)">5.0</text>
+     </g>
+    </g>
+    <g id="ytick_10">
+     <g id="line2d_26">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="211.517917" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_28">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="215.316745" transform="rotate(-0 394.826406 215.316745)">5.5</text>
+     </g>
+    </g>
+    <g id="ytick_11">
+     <g id="line2d_27">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="180.794688" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_29">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="184.593516" transform="rotate(-0 394.826406 184.593516)">6.0</text>
+     </g>
+    </g>
+    <g id="ytick_12">
+     <g id="line2d_28">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="150.071459" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_30">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="153.870287" transform="rotate(-0 394.826406 153.870287)">6.5</text>
+     </g>
+    </g>
+    <g id="ytick_13">
+     <g id="line2d_29">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="119.34823" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_31">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="123.147058" transform="rotate(-0 394.826406 123.147058)">7.0</text>
+     </g>
+    </g>
+    <g id="ytick_14">
+     <g id="line2d_30">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="88.625" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_32">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="92.423829" transform="rotate(-0 394.826406 92.423829)">7.5</text>
+     </g>
+    </g>
+    <g id="ytick_15">
+     <g id="line2d_31">
+      <g>
+       <use xlink:href="#m9a2e91a9aa" x="401.826406" y="57.901771" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_33">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="394.826406" y="61.700599" transform="rotate(-0 394.826406 61.700599)">8.0</text>
+     </g>
+    </g>
+    <g id="text_34">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="372.520938" y="146.599063" transform="rotate(-90 372.520938 146.599063)">Maximum absolute probability error</text>
+    </g>
+    <g id="text_35">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="401.826406" y="42.880469" transform="rotate(-0 401.826406 42.880469)">1e−7</text>
+    </g>
+   </g>
+   <g id="line2d_32">
+    <path d="M 415.617209 238.16142 
+L 470.780419 55.036705 
+L 525.943629 146.599063 
+L 581.106839 109.974119 
+L 636.27005 183.224006 
+L 691.43326 219.848949 
+" clip-path="url(#p601cd532af)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <g clip-path="url(#p601cd532af)">
+     <use xlink:href="#m9b4cc9d8d2" x="415.617209" y="238.16142" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="470.780419" y="55.036705" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="525.943629" y="146.599063" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="581.106839" y="109.974119" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="636.27005" y="183.224006" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m9b4cc9d8d2" x="691.43326" y="219.848949" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_33">
+    <path d="M 415.617209 219.848949 
+L 470.780419 183.224006 
+L 525.943629 128.286591 
+L 581.106839 109.974119 
+L 636.27005 183.224006 
+L 691.43326 146.599063 
+" clip-path="url(#p601cd532af)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <g clip-path="url(#p601cd532af)">
+     <use xlink:href="#m556f16804f" x="415.617209" y="219.848949" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="470.780419" y="183.224006" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="525.943629" y="128.286591" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="581.106839" y="109.974119" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="636.27005" y="183.224006" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m556f16804f" x="691.43326" y="146.599063" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="patch_9">
+    <path d="M 401.826406 247.317656 
+L 401.826406 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_10">
+    <path d="M 705.224062 247.317656 
+L 705.224062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_11">
+    <path d="M 401.826406 247.317656 
+L 705.224062 247.317656 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_12">
+    <path d="M 401.826406 45.880469 
+L 705.224062 45.880469 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="legend_2">
+    <g id="patch_13">
+     <path d="M 407.426406 76.281719 
+L 458.768906 76.281719 
+Q 460.368906 76.281719 460.368906 74.681719 
+L 460.368906 51.480469 
+Q 460.368906 49.880469 458.768906 49.880469 
+L 407.426406 49.880469 
+Q 405.826406 49.880469 405.826406 51.480469 
+L 405.826406 74.681719 
+Q 405.826406 76.281719 407.426406 76.281719 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_34">
+     <path d="M 409.026406 56.359219 
+L 417.026406 56.359219 
+L 425.026406 56.359219 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m9b4cc9d8d2" x="417.026406" y="56.359219" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_36">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="431.426406" y="59.159219" transform="rotate(-0 431.426406 59.159219)">q_to_k</text>
+    </g>
+    <g id="line2d_35">
+     <path d="M 409.026406 68.359844 
+L 417.026406 68.359844 
+L 425.026406 68.359844 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m556f16804f" x="417.026406" y="68.359844" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_37">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="431.426406" y="71.159844" transform="rotate(-0 431.426406 71.159844)">k_to_q</text>
+    </g>
+   </g>
+  </g>
+  <g id="text_38">
+   <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="356.024063" y="16.318125" transform="rotate(-0 356.024063 16.318125)">Supplement: algebraic reparameterization, separate from learned-function repetition</text>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="p0542a2a98b">
+   <rect x="50.466406" y="45.880469" width="303.397656" height="201.437187"/>
+  </clipPath>
+  <clipPath id="p601cd532af">
+   <rect x="401.826406" y="45.880469" width="303.397656" height="201.437187"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/figures/candidate_function_errors.svg

@@ -0,0 +1,467 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="712.218437pt" height="280.996571pt" viewBox="0 0 712.218437 280.996571" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:10:51.395327</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 280.996571 
+L 712.218437 280.996571 
+L 712.218437 0 
+L 0 0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 51.700781 195.511975 
+L 705.018437 195.511975 
+L 705.018437 37.922578 
+L 51.700781 37.922578 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="patch_3">
+    <path d="M 81.397038 76335.575007 
+L 125.391493 76335.575007 
+L 125.391493 64.33392 
+L 81.397038 64.33392 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 136.390107 76335.575007 
+L 180.384562 76335.575007 
+L 180.384562 45.085733 
+L 136.390107 45.085733 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 191.383176 76335.575007 
+L 235.377631 76335.575007 
+L 235.377631 45.525425 
+L 191.383176 45.525425 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 246.376244 76335.575007 
+L 290.370699 76335.575007 
+L 290.370699 76.171564 
+L 246.376244 76.171564 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_7">
+    <path d="M 301.369313 76335.575007 
+L 345.363768 76335.575007 
+L 345.363768 76.722901 
+L 301.369313 76.722901 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_8">
+    <path d="M 356.362382 76335.575007 
+L 400.356837 76335.575007 
+L 400.356837 136.725903 
+L 356.362382 136.725903 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_9">
+    <path d="M 411.355451 76335.575007 
+L 455.349906 76335.575007 
+L 455.349906 141.06981 
+L 411.355451 141.06981 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_10">
+    <path d="M 466.348519 76335.575007 
+L 510.342974 76335.575007 
+L 510.342974 140.370183 
+L 466.348519 140.370183 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_11">
+    <path d="M 521.341588 76335.575007 
+L 565.336043 76335.575007 
+L 565.336043 147.505598 
+L 521.341588 147.505598 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_12">
+    <path d="M 576.334657 76335.575007 
+L 620.329112 76335.575007 
+L 620.329112 123.343194 
+L 576.334657 123.343194 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="patch_13">
+    <path d="M 631.327725 76335.575007 
+L 675.32218 76335.575007 
+L 675.32218 109.03712 
+L 631.327725 109.03712 
+z
+" clip-path="url(#pf9e7526e4c)" style="fill: #9e9e9e"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="mc7fdb9fc78" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="103.394266" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(71.267864 238.312018) rotate(-45)">attn_3_7</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="158.387335" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(117.262999 247.309952) rotate(-45)">attn_11_15</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="213.380403" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(172.256067 247.309952) rotate(-45)">attn_19_23</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="268.373472" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(237.264641 237.295) rotate(-45)">mlp_5_6</text>
+     </g>
+    </g>
+    <g id="xtick_5">
+     <g id="line2d_5">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="323.366541" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_5">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(283.259775 246.292933) rotate(-45)">mlp_20_21</text>
+     </g>
+    </g>
+    <g id="xtick_6">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="378.359609" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(338.028559 246.517219) rotate(-45)">qk_norm_3</text>
+     </g>
+    </g>
+    <g id="xtick_7">
+     <g id="line2d_7">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="433.352678" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_7">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(393.021627 246.517219) rotate(-45)">qk_norm_7</text>
+     </g>
+    </g>
+    <g id="xtick_8">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="488.345747" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(443.515729 251.016186) rotate(-45)">qk_norm_11</text>
+     </g>
+    </g>
+    <g id="xtick_9">
+     <g id="line2d_9">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="543.338815" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(498.508798 251.016186) rotate(-45)">qk_norm_15</text>
+     </g>
+    </g>
+    <g id="xtick_10">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="598.331884" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(553.501867 251.016186) rotate(-45)">qk_norm_19</text>
+     </g>
+    </g>
+    <g id="xtick_11">
+     <g id="line2d_11">
+      <g>
+       <use xlink:href="#mc7fdb9fc78" x="653.324953" y="195.511975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(608.494935 251.016186) rotate(-45)">qk_norm_23</text>
+     </g>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_12">
+      <defs>
+       <path id="mdcab3d4c70" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#mdcab3d4c70" x="51.700781" y="165.39636" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_12">
+      <!-- $\mathdefault{10^{-1}}$ -->
+      <g transform="translate(21.200781 170.04636)">
+       <text>
+        <tspan x="0" y="-0.762187" style="font-size: 10px; font-family: 'DejaVu Sans'">1</tspan>
+        <tspan x="6.362305" y="-0.762187" style="font-size: 10px; font-family: 'DejaVu Sans'">0</tspan>
+        <tspan x="12.820312" y="-4.892187" style="font-size: 7px; font-family: 'DejaVu Sans'">−</tspan>
+        <tspan x="18.685547" y="-4.892187" style="font-size: 7px; font-family: 'DejaVu Sans'">1</tspan>
+       </text>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_13">
+      <g>
+       <use xlink:href="#mdcab3d4c70" x="51.700781" y="89.149935" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_13">
+      <!-- $\mathdefault{10^{0}}$ -->
+      <g transform="translate(27.100781 93.849935)">
+       <text>
+        <tspan x="0" y="-0.674688" style="font-size: 10px; font-family: 'DejaVu Sans'">1</tspan>
+        <tspan x="6.362305" y="-0.674688" style="font-size: 10px; font-family: 'DejaVu Sans'">0</tspan>
+        <tspan x="12.820312" y="-4.804688" style="font-size: 7px; font-family: 'DejaVu Sans'">0</tspan>
+       </text>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_14">
+      <defs>
+       <path id="md42a30ea1c" d="M 0 0 
+L -2 0 
+" style="stroke: #000000; stroke-width: 0.6"/>
+      </defs>
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="188.348821" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_15">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="182.311534" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_16">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="177.20708" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_17">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="172.785402" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_7">
+     <g id="line2d_18">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="168.885205" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_8">
+     <g id="line2d_19">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="142.443899" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_9">
+     <g id="line2d_20">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="129.01757" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_10">
+     <g id="line2d_21">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="119.491438" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_11">
+     <g id="line2d_22">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="112.102396" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_12">
+     <g id="line2d_23">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="106.065109" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_13">
+     <g id="line2d_24">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="100.960655" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_14">
+     <g id="line2d_25">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="96.538977" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_15">
+     <g id="line2d_26">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="92.63878" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_16">
+     <g id="line2d_27">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="66.197474" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_17">
+     <g id="line2d_28">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="52.771145" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="ytick_18">
+     <g id="line2d_29">
+      <g>
+       <use xlink:href="#md42a30ea1c" x="51.700781" y="43.245012" style="stroke: #000000; stroke-width: 0.6"/>
+      </g>
+     </g>
+    </g>
+    <g id="text_14">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="116.717277" transform="rotate(-90 14.798438 116.717277)">Worst calibration directional relative L2</text>
+    </g>
+   </g>
+   <g id="line2d_30">
+    <path d="M 51.700781 188.348821 
+L 705.018437 188.348821 
+" clip-path="url(#pf9e7526e4c)" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+   </g>
+   <g id="patch_14">
+    <path d="M 51.700781 195.511975 
+L 51.700781 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_15">
+    <path d="M 705.018437 195.511975 
+L 705.018437 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_16">
+    <path d="M 51.700781 195.511975 
+L 705.018437 195.511975 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_17">
+    <path d="M 51.700781 37.922578 
+L 705.018437 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="text_15">
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(226.773359 17.519297)">Fixed candidates; same-input function comparison</text>
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(254.193359 31.922578)">4 calibration texts, 32 tokens; no training</text>
+   </g>
+   <g id="legend_1">
+    <g id="patch_18">
+     <path d="M 57.300781 273.796571 
+L 280.582031 273.796571 
+Q 282.182031 273.796571 282.182031 272.196571 
+L 282.182031 260.995946 
+Q 282.182031 259.395946 280.582031 259.395946 
+L 57.300781 259.395946 
+Q 55.700781 259.395946 55.700781 260.995946 
+L 55.700781 272.196571 
+Q 55.700781 273.796571 57.300781 273.796571 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_31">
+     <path d="M 58.900781 265.874696 
+L 66.900781 265.874696 
+L 74.900781 265.874696 
+" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+    </g>
+    <g id="text_16">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="81.300781" y="268.674696" transform="rotate(-0 81.300781 268.674696)">Relative L2 screen = 0.05; other gates also apply</text>
+    </g>
+   </g>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="pf9e7526e4c">
+   <rect x="51.700781" y="37.922578" width="653.317656" height="157.589397"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/figures/learned_norm_function_errors.svg

@@ -0,0 +1,388 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="640.424062pt" height="312.347793pt" viewBox="0 0 640.424062 312.347793" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:10:55.472730</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 312.347793 
+L 640.424062 312.347793 
+L 640.424062 0 
+L 0 0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 50.466406 208.18075 
+L 633.224062 208.18075 
+L 633.224062 37.922578 
+L 50.466406 37.922578 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="mb7bbc15d82" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="76.955391" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(41.486676 241.545565) rotate(-30)">cal-zh-1</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="152.638203" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(117.169489 241.545565) rotate(-30)">cal-zh-2</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="228.321016" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(192.068819 241.997909) rotate(-30)">cal-en-1</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="304.003828" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(267.751631 241.997909) rotate(-30)">cal-en-2</text>
+     </g>
+    </g>
+    <g id="xtick_5">
+     <g id="line2d_5">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="379.686641" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_5">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(340.062357 243.944784) rotate(-30)">test-zh-1</text>
+     </g>
+    </g>
+    <g id="xtick_6">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="455.369453" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(415.74517 243.944784) rotate(-30)">test-zh-2</text>
+     </g>
+    </g>
+    <g id="xtick_7">
+     <g id="line2d_7">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="531.052266" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_7">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(490.6445 244.396451) rotate(-30)">test-en-1</text>
+     </g>
+    </g>
+    <g id="xtick_8">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#mb7bbc15d82" x="606.735078" y="208.18075" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(566.327312 244.396451) rotate(-30)">test-en-2</text>
+     </g>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_9">
+      <defs>
+       <path id="ma0713fe695" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="204.137497" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="207.936325" transform="rotate(-0 43.466406 207.936325)">0.04</text>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="182.54738" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="186.346208" transform="rotate(-0 43.466406 186.346208)">0.05</text>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_11">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="160.957263" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="164.756091" transform="rotate(-0 43.466406 164.756091)">0.06</text>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_12">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="139.367146" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_12">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="143.165974" transform="rotate(-0 43.466406 143.165974)">0.07</text>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_13">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="117.777029" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_13">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="121.575857" transform="rotate(-0 43.466406 121.575857)">0.08</text>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_14">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="96.186911" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_14">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="99.98574" transform="rotate(-0 43.466406 99.98574)">0.09</text>
+     </g>
+    </g>
+    <g id="ytick_7">
+     <g id="line2d_15">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="74.596794" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_15">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="78.395623" transform="rotate(-0 43.466406 78.395623)">0.10</text>
+     </g>
+    </g>
+    <g id="ytick_8">
+     <g id="line2d_16">
+      <g>
+       <use xlink:href="#ma0713fe695" x="50.466406" y="53.006677" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_16">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="43.466406" y="56.805505" transform="rotate(-0 43.466406 56.805505)">0.11</text>
+     </g>
+    </g>
+    <g id="text_17">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="123.051664" transform="rotate(-90 14.798438 123.051664)">Worst of two directional relative L2 errors</text>
+    </g>
+   </g>
+   <g id="line2d_17">
+    <path d="M 76.955391 197.451805 
+L 152.638203 192.877368 
+L 228.321016 193.610655 
+L 304.003828 199.099794 
+L 379.686641 196.376861 
+L 455.369453 192.726888 
+L 531.052266 194.835109 
+L 606.735078 198.558148 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m8708773af9" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #1f77b4"/>
+    </defs>
+    <g clip-path="url(#pc5e8d93515)">
+     <use xlink:href="#m8708773af9" x="76.955391" y="197.451805" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="152.638203" y="192.877368" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="228.321016" y="193.610655" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="304.003828" y="199.099794" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="379.686641" y="196.376861" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="455.369453" y="192.726888" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="531.052266" y="194.835109" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#m8708773af9" x="606.735078" y="198.558148" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_18">
+    <path d="M 76.955391 198.934893 
+L 152.638203 194.123521 
+L 228.321016 195.5305 
+L 304.003828 200.441742 
+L 379.686641 197.894474 
+L 455.369453 194.413382 
+L 531.052266 196.725517 
+L 606.735078 199.872446 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m5a845d07a0" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #ff7f0e"/>
+    </defs>
+    <g clip-path="url(#pc5e8d93515)">
+     <use xlink:href="#m5a845d07a0" x="76.955391" y="198.934893" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="152.638203" y="194.123521" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="228.321016" y="195.5305" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="304.003828" y="200.441742" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="379.686641" y="197.894474" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="455.369453" y="194.413382" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="531.052266" y="196.725517" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m5a845d07a0" x="606.735078" y="199.872446" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="line2d_19">
+    <path d="M 50.466406 182.54738 
+L 633.224062 182.54738 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+   </g>
+   <g id="line2d_20">
+    <path d="M 50.466406 45.661586 
+L 633.224062 45.661586 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #6a51a3; stroke-width: 1.5"/>
+   </g>
+   <g id="line2d_21">
+    <path d="M 341.845234 208.18075 
+L 341.845234 37.922578 
+" clip-path="url(#pc5e8d93515)" style="fill: none; stroke: #808080; stroke-width: 0.8; stroke-linecap: square"/>
+   </g>
+   <g id="patch_3">
+    <path d="M 50.466406 208.18075 
+L 50.466406 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 633.224062 208.18075 
+L 633.224062 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 50.466406 208.18075 
+L 633.224063 208.18075 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 50.466406 37.922578 
+L 633.224063 37.922578 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="text_18">
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(126.937422 17.519297)">Learned input RMSNorm at layer indices 12/13: same-input comparison</text>
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif" transform="translate(137.227422 31.922578)">Calibration (left) / independent heldout texts (right), 32 tokens each</text>
+   </g>
+   <g id="legend_1">
+    <g id="patch_7">
+     <path d="M 56.066406 305.147793 
+L 211.338906 305.147793 
+Q 212.938906 305.147793 212.938906 303.547793 
+L 212.938906 256.345293 
+Q 212.938906 254.745293 211.338906 254.745293 
+L 56.066406 254.745293 
+Q 54.466406 254.745293 54.466406 256.345293 
+L 54.466406 303.547793 
+Q 54.466406 305.147793 56.066406 305.147793 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_22">
+     <path d="M 57.666406 261.224043 
+L 65.666406 261.224043 
+L 73.666406 261.224043 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m8708773af9" x="65.666406" y="261.224043" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_19">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="264.024043" transform="rotate(-0 80.066406 264.024043)">Input captured at layer 12</text>
+    </g>
+    <g id="line2d_23">
+     <path d="M 57.666406 273.224668 
+L 65.666406 273.224668 
+L 73.666406 273.224668 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m5a845d07a0" x="65.666406" y="273.224668" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_20">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="276.024668" transform="rotate(-0 80.066406 276.024668)">Input captured at layer 13</text>
+    </g>
+    <g id="line2d_24">
+     <path d="M 57.666406 285.225293 
+L 65.666406 285.225293 
+L 73.666406 285.225293 
+" style="fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #b2182b; stroke-width: 1.5"/>
+    </g>
+    <g id="text_21">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="288.025293" transform="rotate(-0 80.066406 288.025293)">Heldout empirical screen = 0.05</text>
+    </g>
+    <g id="line2d_25">
+     <path d="M 57.666406 297.225918 
+L 65.666406 297.225918 
+L 73.666406 297.225918 
+" style="fill: none; stroke-dasharray: 1.5,2.475; stroke-dashoffset: 0; stroke: #6a51a3; stroke-width: 1.5"/>
+    </g>
+    <g id="text_22">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="80.066406" y="300.025918" transform="rotate(-0 80.066406 300.025918)">Proven uniform all-input bound</text>
+    </g>
+   </g>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="pc5e8d93515">
+   <rect x="50.466406" y="37.922578" width="582.757656" height="170.258172"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/figures/learned_norm_network_substitution.svg

@@ -0,0 +1,286 @@
+<?xml version="1.0" encoding="utf-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
+  "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
+<svg xmlns:xlink="http://www.w3.org/1999/xlink" width="567.92875pt" height="308.816598pt" viewBox="0 0 567.92875 308.816598" xmlns="http://www.w3.org/2000/svg" version="1.1">
+ <metadata>
+  <rdf:RDF xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:cc="http://creativecommons.org/ns#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
+   <cc:Work>
+    <dc:type rdf:resource="http://purl.org/dc/dcmitype/StillImage"/>
+    <dc:date>2026-11-09T14:10:58.596152</dc:date>
+    <dc:format>image/svg+xml</dc:format>
+    <dc:creator>
+     <cc:Agent>
+      <dc:title>Matplotlib v3.11.2, https://matplotlib.org/</dc:title>
+     </cc:Agent>
+    </dc:creator>
+   </cc:Work>
+  </rdf:RDF>
+ </metadata>
+ <defs>
+  <style type="text/css">*{stroke-linejoin: round; stroke-linecap: butt}</style>
+ </defs>
+ <g id="figure_1">
+  <g id="patch_1">
+   <path d="M 0 308.816598 
+L 567.92875 308.816598 
+L 567.92875 0 
+L 0 0 
+z
+" style="fill: #ffffff"/>
+  </g>
+  <g id="axes_1">
+   <g id="patch_2">
+    <path d="M 71.571094 233.247975 
+L 560.72875 233.247975 
+L 560.72875 22.318125 
+L 71.571094 22.318125 
+z
+" style="fill: #ffffff"/>
+   </g>
+   <g id="matplotlib.axis_1">
+    <g id="xtick_1">
+     <g id="line2d_1">
+      <defs>
+       <path id="m4de31d702a" d="M 0 0 
+L 0 3.5 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m4de31d702a" x="93.805533" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_1">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="93.805533" y="247.846412" transform="rotate(-0 93.805533 247.846412)">zh1</text>
+     </g>
+    </g>
+    <g id="xtick_2">
+     <g id="line2d_2">
+      <g>
+       <use xlink:href="#m4de31d702a" x="242.035125" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_2">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="242.035125" y="247.846412" transform="rotate(-0 242.035125 247.846412)">zh2</text>
+     </g>
+    </g>
+    <g id="xtick_3">
+     <g id="line2d_3">
+      <g>
+       <use xlink:href="#m4de31d702a" x="390.264718" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_3">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="390.264718" y="247.845631" transform="rotate(-0 390.264718 247.845631)">en1</text>
+     </g>
+    </g>
+    <g id="xtick_4">
+     <g id="line2d_4">
+      <g>
+       <use xlink:href="#m4de31d702a" x="538.494311" y="233.247975" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_4">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="538.494311" y="247.845631" transform="rotate(-0 538.494311 247.845631)">en2</text>
+     </g>
+    </g>
+    <g id="text_5">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="316.149922" y="261.847194" transform="rotate(-0 316.149922 261.847194)">Independent heldout text (32 tokens)</text>
+    </g>
+   </g>
+   <g id="matplotlib.axis_2">
+    <g id="ytick_1">
+     <g id="line2d_5">
+      <defs>
+       <path id="m32017c2f5a" d="M 0 0 
+L -3.5 0 
+" style="stroke: #000000; stroke-width: 0.8"/>
+      </defs>
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="231.508072" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_6">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="235.3069" transform="rotate(-0 64.571094 235.3069)">−0.0015</text>
+     </g>
+    </g>
+    <g id="ytick_2">
+     <g id="line2d_6">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="192.395623" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_7">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="196.194451" transform="rotate(-0 64.571094 196.194451)">−0.0010</text>
+     </g>
+    </g>
+    <g id="ytick_3">
+     <g id="line2d_7">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="153.283173" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_8">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="157.082001" transform="rotate(-0 64.571094 157.082001)">−0.0005</text>
+     </g>
+    </g>
+    <g id="ytick_4">
+     <g id="line2d_8">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="114.170724" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_9">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="117.969552" transform="rotate(-0 64.571094 117.969552)">0.0000</text>
+     </g>
+    </g>
+    <g id="ytick_5">
+     <g id="line2d_9">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="75.058274" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_10">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="78.857102" transform="rotate(-0 64.571094 78.857102)">0.0005</text>
+     </g>
+    </g>
+    <g id="ytick_6">
+     <g id="line2d_10">
+      <g>
+       <use xlink:href="#m32017c2f5a" x="71.571094" y="35.945825" style="stroke: #000000; stroke-width: 0.8"/>
+      </g>
+     </g>
+     <g id="text_11">
+      <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: end" x="64.571094" y="39.744653" transform="rotate(-0 64.571094 39.744653)">0.0010</text>
+     </g>
+    </g>
+    <g id="text_12">
+     <text style="font-size: 10px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="14.798438" y="127.78305" transform="rotate(-90 14.798438 127.78305)">Mean token NLL change (nat/token)</text>
+    </g>
+   </g>
+   <g id="line2d_11">
+    <path d="M 93.805533 31.905845 
+L 242.035125 223.660255 
+L 390.264718 124.077707 
+L 538.494311 66.84947 
+" clip-path="url(#p78d521230c)" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="ma59ebf376b" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #1f77b4"/>
+    </defs>
+    <g clip-path="url(#p78d521230c)">
+     <use xlink:href="#ma59ebf376b" x="93.805533" y="31.905845" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#ma59ebf376b" x="242.035125" y="223.660255" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#ma59ebf376b" x="390.264718" y="124.077707" style="fill: #1f77b4; stroke: #1f77b4"/>
+     <use xlink:href="#ma59ebf376b" x="538.494311" y="66.84947" style="fill: #1f77b4; stroke: #1f77b4"/>
+    </g>
+   </g>
+   <g id="line2d_12">
+    <path d="M 93.805533 45.225019 
+L 242.035125 215.36524 
+L 390.264718 134.743944 
+L 538.494311 175.235695 
+" clip-path="url(#p78d521230c)" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+    <defs>
+     <path id="m2ce5b34917" d="M 0 3 
+C 0.795609 3 1.55874 2.683901 2.12132 2.12132 
+C 2.683901 1.55874 3 0.795609 3 0 
+C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 
+C 1.55874 -2.683901 0.795609 -3 0 -3 
+C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 
+C -2.683901 -1.55874 -3 -0.795609 -3 0 
+C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 
+C -1.55874 2.683901 -0.795609 3 0 3 
+z
+" style="stroke: #ff7f0e"/>
+    </defs>
+    <g clip-path="url(#p78d521230c)">
+     <use xlink:href="#m2ce5b34917" x="93.805533" y="45.225019" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m2ce5b34917" x="242.035125" y="215.36524" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m2ce5b34917" x="390.264718" y="134.743944" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     <use xlink:href="#m2ce5b34917" x="538.494311" y="175.235695" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+    </g>
+   </g>
+   <g id="line2d_13">
+    <path d="M 71.571094 114.170724 
+L 560.72875 114.170724 
+" clip-path="url(#p78d521230c)" style="fill: none; stroke: #808080; stroke-width: 0.8; stroke-linecap: square"/>
+   </g>
+   <g id="patch_3">
+    <path d="M 71.571094 233.247975 
+L 71.571094 22.318125 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_4">
+    <path d="M 560.72875 233.247975 
+L 560.72875 22.318125 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_5">
+    <path d="M 71.571094 233.247975 
+L 560.72875 233.247975 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="patch_6">
+    <path d="M 71.571094 22.318125 
+L 560.72875 22.318125 
+" style="fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square"/>
+   </g>
+   <g id="text_13">
+    <text style="font-size: 12px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: middle" x="316.149922" y="16.318125" transform="rotate(-0 316.149922 16.318125)">Bidirectional single-site substitution of existing learned RMSNorm modules</text>
+   </g>
+   <g id="legend_1">
+    <g id="patch_7">
+     <path d="M 77.171094 301.616598 
+L 257.259844 301.616598 
+Q 258.859844 301.616598 258.859844 300.016598 
+L 258.859844 276.815348 
+Q 258.859844 275.215348 257.259844 275.215348 
+L 77.171094 275.215348 
+Q 75.571094 275.215348 75.571094 276.815348 
+L 75.571094 300.016598 
+Q 75.571094 301.616598 77.171094 301.616598 
+z
+" style="fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter"/>
+    </g>
+    <g id="line2d_14">
+     <path d="M 78.771094 281.694098 
+L 86.771094 281.694098 
+L 94.771094 281.694098 
+" style="fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#ma59ebf376b" x="86.771094" y="281.694098" style="fill: #1f77b4; stroke: #1f77b4"/>
+     </g>
+    </g>
+    <g id="text_14">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="101.171094" y="284.494098" transform="rotate(-0 101.171094 284.494098)">Layer 12 uses 13; mean=3.2565e-05</text>
+    </g>
+    <g id="line2d_15">
+     <path d="M 78.771094 293.694723 
+L 86.771094 293.694723 
+L 94.771094 293.694723 
+" style="fill: none; stroke: #ff7f0e; stroke-width: 1.5; stroke-linecap: square"/>
+     <g>
+      <use xlink:href="#m2ce5b34917" x="86.771094" y="293.694723" style="fill: #ff7f0e; stroke: #ff7f0e"/>
+     </g>
+    </g>
+    <g id="text_15">
+     <text style="font-size: 8px; font-family: 'DejaVu Sans', 'Bitstream Vera Sans', 'Computer Modern Sans Serif', 'Lucida Grande', 'Verdana', 'Geneva', 'Lucid', 'Arial', 'Helvetica', 'Avant Garde', sans-serif; text-anchor: start" x="101.171094" y="296.494723" transform="rotate(-0 101.171094 296.494723)">Layer 13 uses 12; mean=-0.00036397</text>
+    </g>
+   </g>
+  </g>
+ </g>
+ <defs>
+  <clipPath id="p78d521230c">
+   <rect x="71.571094" y="22.318125" width="489.157656" height="210.92985"/>
+  </clipPath>
+ </defs>
+</svg>

results/cpu-qwen35-2b/final-source-hashes.json

@@ -0,0 +1,18 @@
+{
+  "functional/__init__.py": "e572d3a9040dfb440b92e969df495d2a9e0edbb1bfe1318e5e66841cc6144067",
+  "functional/audit.py": "5e10c858fe574f4c9a51cdb9c61a94932d9e4bb6dac82082a94c2ef7d57a693b",
+  "functional/benchmark_factorization.py": "135759dbe1af48b86e327cf23a1ef656d000dfef83dea2dbf48d2514356ea729",
+  "functional/gain_factorization.py": "33f3620027c50bce913ad07ed12f42fad83d69a811652174c803467c295e77c9",
+  "functional/model.py": "48a2a97d0445dc6e08ac7f27a935637a7553417d0982eb94122d6cb02bbb499c",
+  "functional/neuron_reuse.py": "d408bc14242c7950b846b6d91870d88374cdb8ece7fc111a2ccdc25e725ca19c",
+  "functional/norm_repetition.py": "aac0c44904c2b059510a754f2f0b1247ffb4b2a1715ecee6281b5ed2b43612b3",
+  "functional/preflight.py": "87c13d21512e8cbb82bc8a8aeb614a3be06bddbb8489c52eb2bf316480b0f20c",
+  "functional/reference.json": "af6bf3040369580de3dff4a6733576a75c61bcbd7e94201d309cf28e9b75e3d6",
+  "functional/reference_check.py": "422cd70c4f1276590402b10261aff0fc744a656387eb260b54bac303353c277c",
+  "functional/report.py": "ca74122de373e80429fab0f8c2e2d0dba3f8f7c6ce046510e5bf28fa7b8d827d",
+  "functional/run.py": "89d3abe098c240ec29c3f6684a94e24b951c372e1393c8db5a48f79ef141aa48",
+  "functional/storage.py": "8387fac8c192fe050e6d275dff6fe55a405332ee18c09f3e3b5d4d537a3db59e",
+  "functional/supervise.py": "44c01fd9ba989ed68ef58313e490602a49f5da26aeb918ee0b6545aff766144e",
+  "pilot/tensors.py": "2e620849501b6bc98a59cbf9c39495326b0c85530e18e6166aa9edb68ac2423e",
+  "tests/test_functional.py": "82c8585f6a9f4e6323cd978621639163298a369bdf298dfa5103c020c3e71f38"
+}

results/cpu-qwen35-2b/final-tests.txt

@@ -0,0 +1,23 @@
+test_causal_convolution_and_norm_boundary (test_functional.FunctionalTests.test_causal_convolution_and_norm_boundary) ... ok
+test_causality_and_identical_function_metrics (test_functional.FunctionalTests.test_causality_and_identical_function_metrics) ... ok
+test_neuron_function_metrics_and_targeted_replacement (test_functional.FunctionalTests.test_neuron_function_metrics_and_targeted_replacement) ... ok
+test_nonrotary_gain_identity_and_rotary_negative_control (test_functional.FunctionalTests.test_nonrotary_gain_identity_and_rotary_negative_control) ... ok
+test_streamed_projection_embedding_and_output_head (test_functional.FunctionalTests.test_streamed_projection_embedding_and_output_head) ... ok
+test_uniform_rmsnorm_function_bound_is_tight (test_functional.FunctionalTests.test_uniform_rmsnorm_function_bound_is_tight) ... ok
+test_3d_reader_and_exact_block (test_pilot.PilotTests.test_3d_reader_and_exact_block) ... ok
+test_bfloat16 (test_pilot.PilotTests.test_bfloat16) ... ok
+test_directional_replacement_error_and_scaling (test_pilot.PilotTests.test_directional_replacement_error_and_scaling) ... ok
+test_end_to_end_synthetic_groups_and_savings (test_pilot.PilotTests.test_end_to_end_synthetic_groups_and_savings) ... Inventory: 4 tensors, 33792 bytes
+tensor: sampling 4 objects
+block: sampling 22 objects
+block: verified 100/109
+{"model": "synthetic", "revision": "test", "tensor_count": 4, "parameter_count": 8448, "model_weight_bytes": 33792, "ndim_counts": {"2": 2, "3": 2}, "elapsed_seconds": 0.1379160056822002, "analysis_budget_seconds": 60, "peak_rss_kib": 53592, "python": "3.14.4", "numpy": "2.5.4", "seed": 20261109, "limitations": ["No inference or functional equivalence measured", "No exhaustive pair search", "Block search uses per-role/shape/scale reservoirs of 256", "Different block scales overlap; savings must not be added", "Block savings count only verified disjoint sampled blocks, no extrapolation", "Spectral probes use aligned bounded subvolumes, not whole tensors", "Shared encoding is theoretical, not a runnable compressed checkpoint", "No cross-role grouping, neuron permutation alignment, or training"], "completed": true}
+ok
+test_no_transitive_chaining (test_pilot.PilotTests.test_no_transitive_chaining) ... ok
+test_ranged_download_preserves_prefix (test_pilot.PilotTests.test_ranged_download_preserves_prefix) ... Weight download 36/36 bytes (100.0%)
+ok
+
+----------------------------------------------------------------------
+Ran 12 tests in 40.946s
+
+OK

results/cpu-qwen35-2b/functional-code-hashes.json

@@ -0,0 +1,12 @@
+{
+  "functional/__init__.py": "e572d3a9040dfb440b92e969df495d2a9e0edbb1bfe1318e5e66841cc6144067",
+  "functional/model.py": "48a2a97d0445dc6e08ac7f27a935637a7553417d0982eb94122d6cb02bbb499c",
+  "functional/preflight.py": "87c13d21512e8cbb82bc8a8aeb614a3be06bddbb8489c52eb2bf316480b0f20c",
+  "functional/reference.json": "af6bf3040369580de3dff4a6733576a75c61bcbd7e94201d309cf28e9b75e3d6",
+  "functional/reference_check.py": "422cd70c4f1276590402b10261aff0fc744a656387eb260b54bac303353c277c",
+  "functional/report.py": "e52201fa910a062b8521b4b1b2ea7ac80d276dca06fa8a322b9d3cb926ce100d",
+  "functional/run.py": "89d3abe098c240ec29c3f6684a94e24b951c372e1393c8db5a48f79ef141aa48",
+  "functional/samples.json": "bccb6ed515a8703c6ab2fc29b5227e1d11823b2ac94c95495ecf24a0340ba103",
+  "functional/storage.py": "8387fac8c192fe050e6d275dff6fe55a405332ee18c09f3e3b5d4d537a3db59e",
+  "functional/supervise.py": "44c01fd9ba989ed68ef58313e490602a49f5da26aeb918ee0b6545aff766144e"
+}

results/cpu-qwen35-2b/functional-environment.json

@@ -0,0 +1,34 @@
+{
+  "python": "3.14.4",
+  "platform": "Linux-7.0.0-31-generic-x86_64-with-glibc2.43",
+  "numpy": "2.5.4",
+  "packages": [
+    "contourpy==1.4.0",
+    "cycler==0.12.1",
+    "filelock==4.2.4",
+    "fonttools==4.68.0",
+    "fsspec==2026.9.0",
+    "Jinja2==3.1.6",
+    "kiwisolver==1.5.1",
+    "MarkupSafe==3.0.3",
+    "matplotlib==3.11.2",
+    "mpmath==1.3.0",
+    "networkx==3.7",
+    "numpy==2.5.4",
+    "packaging==26.3",
+    "pillow==13.0.0",
+    "pyparsing==3.3.3",
+    "python-dateutil==4.0.0",
+    "setuptools==78.1.0",
+    "sympy==1.14.0",
+    "tokenizers==0.23.3",
+    "torch==2.15.0+cpu",
+    "typing_extensions==4.16.0",
+    "tzdata==2026.6"
+  ],
+  "weights_retained": true,
+  "no_automatic_cleanup": true,
+  "ledger_seconds_including_initial_reserve": 1167.355532555841,
+  "final_tests_report_audit_additional_budget_upper_reserve_seconds": 600,
+  "historical_error_logs": "Early reference adapter errors and preflight pressure stops; latest summaries/tests are authoritative."
+}

results/cpu-qwen35-2b/functional-run.log

@@ -0,0 +1,24 @@
+{"phase": "baseline", "layer": 0, "elapsed": 6.704378366470337}
+{"phase": "baseline", "layer": 1, "elapsed": 12.428538799285889}
+{"phase": "baseline", "layer": 2, "elapsed": 18.311316967010498}
+{"phase": "baseline", "layer": 3, "elapsed": 22.585159063339233}
+{"phase": "baseline", "layer": 4, "elapsed": 27.968208074569702}
+{"phase": "baseline", "layer": 5, "elapsed": 34.463871240615845}
+{"phase": "baseline", "layer": 6, "elapsed": 41.34764099121094}
+{"phase": "baseline", "layer": 7, "elapsed": 44.77713179588318}
+{"phase": "baseline", "layer": 8, "elapsed": 50.20741152763367}
+{"phase": "baseline", "layer": 9, "elapsed": 55.92143273353577}
+{"phase": "baseline", "layer": 10, "elapsed": 62.15613389015198}
+{"phase": "baseline", "layer": 11, "elapsed": 65.01726865768433}
+{"phase": "baseline", "layer": 12, "elapsed": 70.20210671424866}
+{"phase": "baseline", "layer": 13, "elapsed": 76.30333161354065}
+{"phase": "baseline", "layer": 14, "elapsed": 81.29059743881226}
+{"phase": "baseline", "layer": 15, "elapsed": 85.60254788398743}
+{"phase": "baseline", "layer": 16, "elapsed": 91.78613877296448}
+{"phase": "baseline", "layer": 17, "elapsed": 98.67645645141602}
+{"phase": "baseline", "layer": 18, "elapsed": 106.65264892578125}
+{"phase": "baseline", "layer": 19, "elapsed": 109.9351601600647}
+{"phase": "baseline", "layer": 20, "elapsed": 116.18031859397888}
+{"phase": "baseline", "layer": 21, "elapsed": 122.5530002117157}
+{"phase": "baseline", "layer": 22, "elapsed": 128.36239457130432}
+{"phase": "baseline", "layer": 23, "elapsed": 131.58471012115479}

results/cpu-qwen35-2b/functional-status.json

@@ -0,0 +1,11 @@
+{
+  "state": "completed",
+  "started_unix": 1794202338.6853228,
+  "phase": "done",
+  "last_completed_layer": 23,
+  "elapsed_seconds": 196.65513348579407,
+  "selected_candidate": [],
+  "positive_bidirectional_quality": false,
+  "peak_rss_kib": 162372,
+  "claim_limit": "Small text-only heldout pilot; no global interchangeability proof, no measured GPU speedup; component storage savings require explicit layout accounting."
+}

results/cpu-qwen35-2b/gain-factorization-kernel-timing.csv

@@ -0,0 +1,21 @@
+repeat,implementation,seconds_per_call
+0,baseline,0.0036114158341661096
+0,factored,0.003925090050324797
+1,factored,0.0032154972897842525
+1,baseline,0.0025859603192657232
+2,baseline,0.0023861153982579707
+2,factored,0.0025824901415035127
+3,factored,0.001997539587318897
+3,baseline,0.0016429506475105881
+4,baseline,0.0017620180500671268
+4,factored,0.0027307189535349607
+5,factored,0.002951206616126001
+5,baseline,0.0020800257567316293
+6,baseline,0.0018509919987991451
+6,factored,0.0024777360958978534
+7,factored,0.0025671867886558177
+7,baseline,0.002200991357676685
+8,baseline,0.0020182608393952252
+8,factored,0.002254301542416215
+9,factored,0.0025048630544915796
+9,baseline,0.002296067262068391

results/cpu-qwen35-2b/gain-factorization-optimized-summary.json

@@ -0,0 +1,102 @@
+{
+  "state": "completed",
+  "bitwise_checks": [
+    {
+      "layer": 3,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 3,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 7,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 7,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 11,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 11,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 15,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 15,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 19,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 19,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 23,
+      "direction": "q_to_k",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    },
+    {
+      "layer": 23,
+      "direction": "k_to_q",
+      "query_elements": 524288,
+      "key_elements": 131072,
+      "bitwise_equal": true
+    }
+  ],
+  "medians_seconds": {
+    "baseline_legacy": 0.0005341526120901108,
+    "baseline_inplace": 0.00034534828271716835,
+    "factored_inplace": 0.00048380128573626284
+  },
+  "paired_baseline_over_factored_median": 0.6901853298227001,
+  "paired_ratio_p10": 0.38049978760565983,
+  "paired_ratio_p90": 1.0876508522302268,
+  "fraction_blocks_factored_faster": 0.13333333333333333,
+  "descriptive_local_gain": false,
+  "claim_limit": "Hot-array CPU normalization microbenchmark against equally allocation-optimized baseline; no full-model speedup or independent statistical significance claim.",
+  "elapsed_seconds": 4.185962562914938,
+  "peak_rss_kib": 62168
+}

results/cpu-qwen35-2b/gain-factorization-optimized-timing.csv

@@ -0,0 +1,91 @@
+repeat,implementation,seconds_per_call,calls_per_block
+0,baseline_legacy,0.0005406431574374438,50
+0,baseline_inplace,0.00030974450521171093,50
+0,factored_inplace,0.0004578591044992208,50
+1,baseline_inplace,0.0003033110871911049,50
+1,factored_inplace,0.0005383838433772326,50
+1,baseline_legacy,0.0005124531593173742,50
+2,factored_inplace,0.0004439983796328306,50
+2,baseline_legacy,0.0005330004263669252,50
+2,baseline_inplace,0.00034275135956704614,50
+3,baseline_legacy,0.00048629873432219027,50
+3,baseline_inplace,0.00030624629929661753,50
+3,factored_inplace,0.0004589695204049349,50
+4,baseline_inplace,0.0003086038026958704,50
+4,factored_inplace,0.0005195329245179891,50
+4,baseline_legacy,0.000474175363779068,50
+5,factored_inplace,0.00045118503272533416,50
+5,baseline_legacy,0.00048273258842527865,50
+5,baseline_inplace,0.0003844233974814415,50
+6,baseline_legacy,0.0006650591176003218,50
+6,baseline_inplace,0.0007703853026032448,50
+6,factored_inplace,0.0004403704963624477,50
+7,baseline_inplace,0.00038892081938683986,50
+7,factored_inplace,0.0004549186024814844,50
+7,baseline_legacy,0.0005137955769896508,50
+8,factored_inplace,0.0004692125599831343,50
+8,baseline_legacy,0.0005263624526560307,50
+8,baseline_inplace,0.000349312499165535,50
+9,baseline_legacy,0.0004857080150395632,50
+9,baseline_inplace,0.0012390289548784494,50
+9,factored_inplace,0.0018109613563865423,50
+10,baseline_inplace,0.0013814480416476727,50
+10,factored_inplace,0.0013840428367257118,50
+10,baseline_legacy,0.001071744617074728,50
+11,factored_inplace,0.0016566905751824379,50
+11,baseline_legacy,0.001358420280739665,50
+11,baseline_inplace,0.0003716108575463295,50
+12,baseline_legacy,0.0008325341437011957,50
+12,baseline_inplace,0.0003479452058672905,50
+12,factored_inplace,0.0004391701612621546,50
+13,baseline_inplace,0.0005428373627364636,50
+13,factored_inplace,0.0015458692144602537,50
+13,baseline_legacy,0.0013308324199169874,50
+14,factored_inplace,0.0014238305203616618,50
+14,baseline_legacy,0.0006564708799123764,50
+14,baseline_inplace,0.0003204820398241281,50
+15,baseline_legacy,0.0005279563181102276,50
+15,baseline_inplace,0.00030857514590024946,50
+15,factored_inplace,0.0005265675205737352,50
+16,baseline_inplace,0.000458676153793931,50
+16,factored_inplace,0.00045228853821754455,50
+16,baseline_legacy,0.0004912188835442066,50
+17,factored_inplace,0.0004910450987517834,50
+17,baseline_legacy,0.0005007587838917971,50
+17,baseline_inplace,0.0003247600421309471,50
+18,baseline_legacy,0.0004860841576009989,50
+18,baseline_inplace,0.00032584095373749733,50
+18,factored_inplace,0.0004711645469069481,50
+19,baseline_inplace,0.0003332006186246872,50
+19,factored_inplace,0.0004859814327210188,50
+19,baseline_legacy,0.0006259976234287023,50
+20,factored_inplace,0.0005134599283337593,50
+20,baseline_legacy,0.0006658115796744823,50
+20,baseline_inplace,0.0003131803590804338,50
+21,baseline_legacy,0.00048669124022126197,50
+21,baseline_inplace,0.0003487722761929035,50
+21,factored_inplace,0.00048093565739691256,50
+22,baseline_inplace,0.000310970526188612,50
+22,factored_inplace,0.0004317485820502043,50
+22,baseline_legacy,0.0005281844362616539,50
+23,factored_inplace,0.0004816211387515068,50
+23,baseline_legacy,0.0005441690050065518,50
+23,baseline_inplace,0.0012436784617602826,50
+24,baseline_legacy,0.0013620165828615426,50
+24,baseline_inplace,0.000990196019411087,50
+24,factored_inplace,0.0010849225427955389,50
+25,baseline_inplace,0.0019172614254057408,50
+25,factored_inplace,0.0009820526745170355,50
+25,baseline_legacy,0.0009186737984418869,50
+26,factored_inplace,0.0013663648813962937,50
+26,baseline_legacy,0.0009482460748404264,50
+26,baseline_inplace,0.0005478111188858747,50
+27,baseline_legacy,0.000548997139558196,50
+27,baseline_inplace,0.0003215675055980682,50
+27,factored_inplace,0.0004668480064719915,50
+28,baseline_inplace,0.0003313558641821146,50
+28,factored_inplace,0.00047064563259482386,50
+28,baseline_legacy,0.0005353047978132963,50
+29,factored_inplace,0.0006785523053258657,50
+29,baseline_legacy,0.000497497720643878,50
+29,baseline_inplace,0.000260401563718915,50

results/cpu-qwen35-2b/gain-factorization-protocol.json

@@ -0,0 +1,23 @@
+{
+  "fixed_intervention_layer": 3,
+  "directions": [
+    "q_to_k",
+    "k_to_q"
+  ],
+  "diagnostic_layers": [
+    3,
+    7,
+    11,
+    15,
+    19,
+    23
+  ],
+  "no_selection": true,
+  "no_training": true,
+  "precision": "FP32 product; no BF16 requantization",
+  "quality_gate": {
+    "mean_nll_delta": 0.01,
+    "sequence_p95_nll_delta": 0.05
+  },
+  "registered_unix": 1794202505.9768724
+}

results/cpu-qwen35-2b/gain-factorization-score-metrics.csv

@@ -0,0 +1,97 @@
+layer,direction,sample_id,split,score_max_abs_error,score_relative_l2,probability_max_abs_error,first_key_mass_a,first_key_mass_b,js_without_first_key,without_first_key_valid_rows,js_mean,js_p95,js_without_first_query,top5_overlap
+3,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.101167139945906e-08,3.5762786865234375e-07,0.12333933264017105,0.12333933264017105,6.144742492608657e-09,248,6.2515415066854985e-09,3.3901077500786414e-08,6.453204193235251e-09,1.0
+3,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.091678723372976e-08,5.066394805908203e-07,0.1346932351589203,0.1346932053565979,6.3971117292283e-09,248,5.0883031121884414e-09,3.329163433818394e-08,5.252442036862703e-09,1.0
+3,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.108396212348145e-08,3.725290298461914e-07,0.1436830461025238,0.1436830461025238,6.142506059347852e-09,248,5.038649941724316e-09,2.9201892459695955e-08,5.2011870366186486e-09,1.0
+3,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.1701194575127235e-08,3.8743019104003906e-07,0.1500767469406128,0.1500767469406128,6.898510651609513e-09,248,5.6701336959008586e-09,3.627561184771366e-08,5.853041162851014e-09,1.0
+3,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.220060152896206e-08,4.172325134277344e-07,0.09449940919876099,0.09449941664934158,4.331650149680399e-09,248,5.2700910302405646e-09,3.2252330584015e-08,5.440094152930897e-09,1.0
+3,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.0557365349437256e-08,3.5762786865234375e-07,0.12233515083789825,0.12233515828847885,4.755000837519674e-09,248,5.405037306616123e-09,2.982928037909005e-08,5.579393391741405e-09,1.0
+3,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.036456447011736e-08,4.76837158203125e-07,0.16284234821796417,0.16284234821796417,3.5884786253603806e-09,248,6.1204539214543274e-09,3.24787201577692e-08,6.317887990547888e-09,1.0
+3,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.110061520200993e-08,4.76837158203125e-07,0.1765710860490799,0.1765711009502411,4.401169650947168e-09,248,4.359010929988472e-09,3.009398241715644e-08,4.4996242287709265e-09,1.0
+3,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.1102714936484615e-08,3.2782554626464844e-07,0.12333933264017105,0.12333932518959045,5.54685852804937e-09,248,6.077549130623083e-09,3.364394274285587e-08,6.273598973649541e-09,1.0
+3,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.047857377998922e-08,5.066394805908203e-07,0.1346932351589203,0.1346932202577591,5.5039839352843956e-09,248,5.080743381569164e-09,3.255715697036976e-08,5.244638501267218e-09,1.0
+3,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.134420550367216e-08,4.172325134277344e-07,0.1436830461025238,0.1436830461025238,5.426067595237782e-09,248,6.617717041024207e-09,3.062589826186013e-08,6.831191612377552e-09,1.0
+3,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.226853937237364e-08,5.364418029785156e-07,0.1500767469406128,0.1500767469406128,7.92923859904704e-09,248,3.863607211940234e-09,2.7573278771342302e-08,3.988239516417025e-09,1.0
+3,k_to_q,test-zh-1,heldout,1.9073486328125e-06,5.123835134005697e-08,4.76837158203125e-07,0.09449940919876099,0.09449940919876099,3.631874356813114e-09,248,5.632322164217385e-09,2.992773318055697e-08,5.814010162197292e-09,1.0
+3,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.0224548540677525e-08,4.172325134277344e-07,0.12233515083789825,0.12233515083789825,5.022106286389771e-09,248,4.692121358118584e-09,3.066226383907633e-08,4.843480283511781e-09,1.0
+3,k_to_q,test-en-1,heldout,1.9073486328125e-06,4.9758794253491996e-08,4.76837158203125e-07,0.16284234821796417,0.16284234821796417,3.619864186177324e-09,248,6.1619935820544924e-09,3.4145102745242184e-08,6.360767468294171e-09,1.0
+3,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.142298277550361e-08,2.980232238769531e-07,0.1765710860490799,0.1765710860490799,4.500570138787907e-09,248,5.100083466658134e-09,3.277366289466954e-08,5.264602087606818e-09,1.0
+7,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.222536694121921e-08,4.172325134277344e-07,0.17881479859352112,0.17881479859352112,5.7563616095990255e-09,248,5.532780456007913e-09,3.966695061308201e-08,5.711257244911394e-09,1.0
+7,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.170127993939104e-08,4.470348358154297e-07,0.1558690071105957,0.1558690071105957,4.651088403306858e-09,248,5.351785681284582e-09,4.158310318302938e-08,5.524424029346164e-09,1.0
+7,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.378773497110193e-08,8.046627044677734e-07,0.16970808804035187,0.16970807313919067,7.066096152641421e-09,248,4.739160619493532e-09,3.499418710362079e-08,4.892036553627577e-09,1.0
+7,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.317526550862134e-08,4.76837158203125e-07,0.1774885654449463,0.1774885654449463,2.2907555852214045e-09,248,4.155392474558539e-09,3.55765088499993e-08,4.289437249838102e-09,1.0
+7,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.211860231803155e-08,5.364418029785156e-07,0.13551732897758484,0.13551732897758484,3.869626841179752e-09,248,4.760383198743057e-09,3.7595043522742344e-08,4.91394391843869e-09,1.0
+7,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.3270193367964134e-08,3.8743019104003906e-07,0.16214892268180847,0.16214892268180847,5.834590144360163e-09,248,3.3417555389547715e-09,3.232060308278051e-08,3.4495541978429856e-09,1.0
+7,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.1701003812639285e-08,5.364418029785156e-07,0.16952991485595703,0.16952991485595703,4.620759330720148e-09,248,3.3314184744170916e-09,3.060024056367183e-08,3.4388836223087083e-09,1.0
+7,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.3793891417825e-08,5.364418029785156e-07,0.20624876022338867,0.20624877512454987,5.72924729880242e-09,248,5.260036406440349e-09,3.098259782063906e-08,5.429714899918281e-09,1.0
+7,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.2622550638818123e-08,5.066394805908203e-07,0.17881479859352112,0.17881478369235992,5.618802756401919e-09,248,3.0388158656080577e-09,3.318642782801362e-08,3.1368421193889162e-09,1.0
+7,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.2178797749660394e-08,4.76837158203125e-07,0.1558690071105957,0.1558690220117569,5.6034279438677e-09,248,6.106143590756119e-09,3.9555089870191296e-08,6.3031158070714355e-09,1.0
+7,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.331118520036339e-08,5.364418029785156e-07,0.16970808804035187,0.16970807313919067,4.860980507004342e-09,248,3.6745022580220166e-09,3.220852917706907e-08,3.7930347751569116e-09,1.0
+7,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.3509503240043504e-08,4.172325134277344e-07,0.1774885654449463,0.1774885505437851,2.684577671274724e-09,248,3.914189861120576e-09,3.4250152936010636e-08,4.040454193443566e-09,1.0
+7,k_to_q,test-zh-1,heldout,1.9073486328125e-06,5.186216628978779e-08,5.364418029785156e-07,0.13551732897758484,0.13551732897758484,7.021717873811895e-09,248,3.1829396895943773e-09,3.381542512670421e-08,3.285615113313156e-09,1.0
+7,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.435688576791881e-08,5.960464477539062e-07,0.16214892268180847,0.16214892268180847,4.47947234860635e-09,248,5.434995564712608e-09,3.8530473034370516e-08,5.610317987958524e-09,1.0
+7,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.182678341307826e-08,4.76837158203125e-07,0.16952991485595703,0.16952991485595703,5.902526023504606e-09,248,3.811826410071717e-09,3.153426675339688e-08,3.934788495030261e-09,1.0
+7,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.389036154327309e-08,5.364418029785156e-07,0.20624876022338867,0.20624877512454987,4.393694741366971e-09,248,6.306869693162298e-09,3.442805152076289e-08,6.510317174246438e-09,1.0
+11,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.3373433612649885e-08,4.76837158203125e-07,0.24492809176445007,0.24492809176445007,5.236138189701478e-09,248,4.897512173585028e-09,3.2079633172088506e-08,5.055496465899978e-09,1.0
+11,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.3680472349671996e-08,4.76837158203125e-07,0.18575024604797363,0.18575023114681244,4.706864675796396e-09,248,6.692440379651998e-09,3.841551432515189e-08,6.908325467236409e-09,1.0
+11,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.413658613357613e-08,6.556510925292969e-07,0.24272800981998444,0.24272799491882324,5.221439280944651e-09,248,5.445071948884106e-09,3.742365706216333e-08,5.620719445431632e-09,1.0
+11,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.431145274674744e-08,6.258487701416016e-07,0.2525215446949005,0.2525215148925781,4.965174049686993e-09,248,3.2684353001855015e-09,3.038027784896258e-08,3.373868739942054e-09,1.0
+11,q_to_k,test-zh-1,heldout,1.9073486328125e-06,4.975615695445315e-08,4.76837158203125e-07,0.20683012902736664,0.20683012902736664,4.743543779994752e-09,248,5.727621932294369e-09,3.784743896062537e-08,5.912383915784858e-09,1.0
+11,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.341885392276808e-08,5.066394805908203e-07,0.2263796031475067,0.2263796031475067,5.8260534174792156e-09,248,4.675535514309104e-09,3.7589622081668495e-08,4.826359312204431e-09,1.0
+11,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.3474540038830716e-08,4.76837158203125e-07,0.242807075381279,0.242807075381279,5.4214881473058085e-09,248,3.967941974991618e-09,2.985759550711009e-08,4.095940031589862e-09,1.0
+11,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.447992218148785e-08,5.960464477539062e-07,0.2930097281932831,0.2930097281932831,6.2322125238267745e-09,248,3.5252292196474855e-09,3.577358498318972e-08,3.6389462554353713e-09,1.0
+11,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.28470006223089e-08,4.470348358154297e-07,0.24492809176445007,0.24492809176445007,5.811663150723234e-09,248,5.133418579106319e-09,3.507095414079231e-08,5.29901278412126e-09,1.0
+11,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.340567432956712e-08,5.066394805908203e-07,0.18575024604797363,0.18575024604797363,2.0245674026142524e-09,248,5.099674460495862e-09,3.472874965382289e-08,5.26418020285746e-09,1.0
+11,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.450061260733951e-08,5.066394805908203e-07,0.24272800981998444,0.24272799491882324,6.1068083923032646e-09,248,6.209086134134623e-09,3.4522884106991114e-08,6.409379249561198e-09,1.0
+11,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.419026691337401e-08,5.960464477539062e-07,0.2525215446949005,0.2525215446949005,6.355903803267893e-09,248,4.629203242956237e-09,3.0078894042162574e-08,4.778532236571209e-09,1.0
+11,k_to_q,test-zh-1,heldout,2.86102294921875e-06,5.101806004082587e-08,4.76837158203125e-07,0.20683012902736664,0.20683011412620544,4.4362411522058665e-09,248,5.5794813214049555e-09,3.09307068846465e-08,5.759464460908248e-09,1.0
+11,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.354370300835508e-08,5.364418029785156e-07,0.2263796031475067,0.22637958824634552,5.91522208992501e-09,248,5.174931594353893e-09,3.6706762074345534e-08,5.341864728336532e-09,1.0
+11,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.454591675148741e-08,5.066394805908203e-07,0.242807075381279,0.24280709028244019,4.439097978092832e-09,248,5.318464335601902e-09,3.0657005822831707e-08,5.490027543686438e-09,1.0
+11,k_to_q,test-en-2,heldout,2.86102294921875e-06,5.424732981759204e-08,6.854534149169922e-07,0.2930097281932831,0.2930097281932831,6.705069388601714e-09,248,4.751804283387173e-09,3.399427228600871e-08,4.90508833550507e-09,1.0
+15,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,5.3514833863035024e-08,4.470348358154297e-07,0.33434635400772095,0.33434635400772095,4.285159338479616e-09,248,4.447109791527737e-09,3.414147897728981e-08,4.590564817164022e-09,1.0
+15,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,5.218853708885142e-08,4.76837158203125e-07,0.30227771401405334,0.30227774381637573,5.229982669163746e-09,248,7.101956356336814e-09,4.079232240883357e-08,7.3310517656466345e-09,1.0
+15,q_to_k,cal-en-1,calibration,1.9073486328125e-06,5.4103478102894914e-08,4.76837158203125e-07,0.3320315480232239,0.3320315480232239,5.162816396619974e-09,248,4.775797535216952e-09,3.1169314240742096e-08,4.929855634827618e-09,1.0
+15,q_to_k,cal-en-2,calibration,1.9073486328125e-06,5.31591652511587e-08,3.5762786865234375e-07,0.35803988575935364,0.35803985595703125,5.665685698375e-09,248,3.281495741802587e-09,3.316486996141066e-08,3.3873504001746824e-09,1.0
+15,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.2584229204196615e-08,5.960464477539062e-07,0.3134230375289917,0.3134230375289917,4.611414805566483e-09,248,3.2052387410885785e-09,3.137078863346687e-08,3.3086335893273144e-09,1.0
+15,q_to_k,test-zh-2,heldout,2.86102294921875e-06,5.334021097494736e-08,7.152557373046875e-07,0.31523996591567993,0.31523996591567993,6.134295738036144e-09,248,3.099508649739846e-09,3.3302633539733506e-08,3.1994928928469335e-09,1.0
+15,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.277330399580438e-08,5.662441253662109e-07,0.35721129179000854,0.35721129179000854,3.875603393765914e-09,248,6.00105520831562e-09,3.1643388354041235e-08,6.194637691692151e-09,1.0
+15,q_to_k,test-en-2,heldout,1.9073486328125e-06,5.4245038413836726e-08,5.364418029785156e-07,0.3785627782344818,0.3785627782344818,5.588637108644434e-09,248,5.273080638801275e-09,3.732484543661485e-08,5.443180128850145e-09,1.0
+15,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.370409043564381e-08,4.76837158203125e-07,0.33434635400772095,0.33434635400772095,5.413625991934623e-09,248,7.632287690739759e-09,3.8797516310751234e-08,7.878490748680633e-09,1.0
+15,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.2645265789812776e-08,5.066394805908203e-07,0.30227771401405334,0.30227774381637573,6.007307984390309e-09,248,4.376814466411361e-09,3.4955100147726625e-08,4.518001972542152e-09,1.0
+15,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.3713584045477537e-08,5.364418029785156e-07,0.3320315480232239,0.3320315480232239,3.789843550094929e-09,248,4.10175182707917e-09,3.3631373241860274e-08,4.2340664307971565e-09,1.0
+15,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.329913616712736e-08,4.470348358154297e-07,0.35803988575935364,0.35803985595703125,5.377542411366676e-09,248,3.882600019267102e-09,3.18341726313065e-08,4.007845166853485e-09,1.0
+15,k_to_q,test-zh-1,heldout,1.9073486328125e-06,5.1740198395570186e-08,5.960464477539062e-07,0.3134230375289917,0.3134230077266693,5.392958968286621e-09,248,3.644287094317633e-09,2.9920162347707446e-08,3.761844613592302e-09,1.0
+15,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.17587827155604e-08,7.152557373046875e-07,0.31523996591567993,0.31523996591567993,6.041951827739922e-09,248,4.992775970436014e-09,3.2279736217333266e-08,5.1538333600831265e-09,1.0
+15,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.262549006299803e-08,6.258487701416016e-07,0.35721129179000854,0.35721129179000854,4.4030894486013494e-09,248,4.485939619769397e-09,3.2921569470545364e-08,4.630647421066669e-09,1.0
+15,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.460989035027149e-08,5.662441253662109e-07,0.3785627782344818,0.3785627782344818,6.211682723744616e-09,248,3.997459696591932e-09,3.3428371182253613e-08,4.1264098804560945e-09,1.0
+19,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,4.990160271767093e-08,5.364418029785156e-07,0.08087001740932465,0.08087002485990524,4.9786765821124845e-09,248,4.393741370734006e-09,3.151729544015325e-08,4.535475106592912e-09,1.0
+19,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,4.9742994871294574e-08,4.76837158203125e-07,0.08035354316234589,0.08035353571176529,5.756638277176762e-09,248,4.049901303204706e-09,3.086056210577226e-08,4.180543466958397e-09,1.0
+19,q_to_k,cal-en-1,calibration,3.814697265625e-06,5.087667850358471e-08,5.066394805908203e-07,0.24834147095680237,0.24834147095680237,4.231536010479431e-09,248,4.469023373587788e-09,2.7867795182601185e-08,4.613185389246155e-09,1.0
+19,q_to_k,cal-en-2,calibration,2.86102294921875e-06,5.025801246712497e-08,4.470348358154297e-07,0.169009268283844,0.169009268283844,5.525796265004601e-09,248,4.058557934172313e-09,2.8106194704946574e-08,4.1894789859497905e-09,1.0
+19,q_to_k,test-zh-1,heldout,1.9073486328125e-06,5.0221761295632774e-08,4.172325134277344e-07,0.06396828591823578,0.06396827846765518,3.353163524622005e-09,248,4.251817564693283e-09,2.829117207170384e-08,4.3889731848878455e-09,1.0
+19,q_to_k,test-zh-2,heldout,1.9073486328125e-06,5.0605528455235685e-08,5.960464477539062e-07,0.07303687185049057,0.07303687930107117,5.027091631859548e-09,248,3.928162239930089e-09,3.122080727280263e-08,4.054877322801076e-09,1.0
+19,q_to_k,test-en-1,heldout,1.9073486328125e-06,5.211114332289351e-08,5.364418029785156e-07,0.2881055176258087,0.2881055176258087,3.5609850623785633e-09,248,4.370760198213475e-09,3.161588324473996e-08,4.511752305091932e-09,1.0
+19,q_to_k,test-en-2,heldout,3.814697265625e-06,5.1614248682850084e-08,4.172325134277344e-07,0.08680859953165054,0.08680859953165054,4.462207936484219e-09,248,4.6642192330637045e-09,3.021447270157296e-08,4.814677989628535e-09,1.0
+19,k_to_q,cal-zh-1,calibration,1.9073486328125e-06,5.113471077024349e-08,3.8743019104003906e-07,0.08087001740932465,0.08087001740932465,4.6697485878155476e-09,248,5.351755927307522e-09,3.250153568501446e-08,5.524393387190685e-09,1.0
+19,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,5.029181744672548e-08,5.662441253662109e-07,0.08035354316234589,0.08035354316234589,4.907285688915408e-09,248,5.877964781575429e-09,3.508033685761802e-08,6.06757666332669e-09,1.0
+19,k_to_q,cal-en-1,calibration,1.9073486328125e-06,5.1180455196819575e-08,5.960464477539062e-07,0.24834147095680237,0.24834147095680237,5.958978199771536e-09,248,3.994703678955602e-09,2.9186557171101413e-08,4.123565044977795e-09,1.0
+19,k_to_q,cal-en-2,calibration,1.9073486328125e-06,5.087171697055175e-08,5.662441253662109e-07,0.169009268283844,0.1690092831850052,6.885077397100758e-09,248,4.357643579311343e-09,2.7700329141566726e-08,4.498212913262023e-09,1.0
+19,k_to_q,test-zh-1,heldout,1.9073486328125e-06,4.9891522075043136e-08,4.172325134277344e-07,0.06396828591823578,0.06396828591823578,3.918689817083987e-09,248,6.348101155850827e-09,3.146694638189729e-08,6.552878684118468e-09,1.0
+19,k_to_q,test-zh-2,heldout,1.9073486328125e-06,5.0542783963068e-08,5.066394805908203e-07,0.07303687185049057,0.07303687185049057,6.0575562343956335e-09,248,3.8445029382216944e-09,2.87921402275515e-08,3.968519290964423e-09,1.0
+19,k_to_q,test-en-1,heldout,1.9073486328125e-06,5.202611404804706e-08,5.364418029785156e-07,0.2881055176258087,0.28810548782348633,4.291980548742913e-09,248,5.839524419570807e-09,2.908876339802191e-08,6.027895960158958e-09,1.0
+19,k_to_q,test-en-2,heldout,1.9073486328125e-06,5.31186969928777e-08,4.172325134277344e-07,0.08680859953165054,0.08680859208106995,5.296885152716868e-09,248,6.1070810630781125e-09,2.844184265882177e-08,6.304083477459699e-09,1.0
+23,q_to_k,cal-zh-1,calibration,1.9073486328125e-06,4.932807759644251e-08,4.470348358154297e-07,0.06724601984024048,0.06724601984024048,4.144036225284253e-09,248,4.558249777630863e-09,2.929247777672117e-08,4.705289935458268e-09,1.0
+23,q_to_k,cal-zh-2,calibration,1.9073486328125e-06,4.9029117626786305e-08,4.76837158203125e-07,0.06764671951532364,0.06764672696590424,3.972651985151288e-09,248,4.91240381705893e-09,3.200177900453127e-08,5.070868613898938e-09,1.0
+23,q_to_k,cal-en-1,calibration,3.814697265625e-06,4.95385074720159e-08,3.8743019104003906e-07,0.08747006952762604,0.08747006952762604,3.265131720553427e-09,248,3.707365081595526e-09,3.0631660763447144e-08,3.826957417629728e-09,1.0
+23,q_to_k,cal-en-2,calibration,1.9073486328125e-06,4.922304127933099e-08,4.76837158203125e-07,0.08903923630714417,0.08903922885656357,5.270068825780072e-09,248,3.997369546482332e-09,2.7743276120872906e-08,4.126317065811236e-09,1.0
+23,q_to_k,test-zh-1,heldout,1.9073486328125e-06,4.8364042774388367e-08,3.5762786865234375e-07,0.05931711941957474,0.05931711569428444,3.816367222242434e-09,248,3.653502167466627e-09,2.7387025980374347e-08,3.771357004467291e-09,1.0
+23,q_to_k,test-zh-2,heldout,3.814697265625e-06,4.96448251794115e-08,5.364418029785156e-07,0.08312603086233139,0.08312603831291199,5.321914020584018e-09,248,3.6995391194949434e-09,2.9038851323548442e-08,3.818878990813346e-09,1.0
+23,q_to_k,test-en-1,heldout,3.814697265625e-06,5.034978202226261e-08,4.023313522338867e-07,0.1146349161863327,0.1146349087357521,4.471692793828197e-09,248,3.4011016225576896e-09,2.928134534840865e-08,3.5108145279849623e-09,1.0
+23,q_to_k,test-en-2,heldout,3.814697265625e-06,4.994535705946019e-08,4.470348358154297e-07,0.08886055648326874,0.08886055648326874,4.2449421755463845e-09,248,3.4883309574240684e-09,2.892927319919636e-08,3.600857834129556e-09,1.0
+23,k_to_q,cal-zh-1,calibration,2.86102294921875e-06,4.8831644467980875e-08,4.76837158203125e-07,0.06724601984024048,0.06724601984024048,3.977453033598977e-09,248,4.496496952555162e-09,2.8416268449404924e-08,4.64154537027639e-09,1.0
+23,k_to_q,cal-zh-2,calibration,1.9073486328125e-06,4.9029117626786305e-08,4.470348358154297e-07,0.06764671951532364,0.06764672696590424,4.682830567759311e-09,248,2.651090902361375e-09,2.4917778063127116e-08,2.736610049680621e-09,1.0
+23,k_to_q,cal-en-1,calibration,3.814697265625e-06,5.024324385126141e-08,3.8743019104003906e-07,0.08747006952762604,0.08747006952762604,3.7081571147012937e-09,248,4.977271039763309e-09,3.31379155227296e-08,5.13782838496013e-09,1.0
+23,k_to_q,cal-en-2,calibration,1.9073486328125e-06,4.8518648048109764e-08,2.980232238769531e-07,0.08903923630714417,0.08903922885656357,3.85932619195728e-09,248,4.874880499272649e-09,2.9354460195918364e-08,5.0321347089266055e-09,1.0
+23,k_to_q,test-zh-1,heldout,3.814697265625e-06,4.978168384729527e-08,4.172325134277344e-07,0.05931711941957474,0.05931711941957474,4.285771737499999e-09,248,4.133625886026948e-09,2.540235399806079e-08,4.266968556265738e-09,1.0
+23,k_to_q,test-zh-2,heldout,1.9073486328125e-06,4.943963198590656e-08,6.556510925292969e-07,0.08312603086233139,0.08312603086233139,3.7222060988995054e-09,248,4.004627740528122e-09,2.6530234009669584e-08,4.133809294870616e-09,1.0
+23,k_to_q,test-en-1,heldout,3.814697265625e-06,4.9439166765060464e-08,4.023313522338867e-07,0.1146349161863327,0.1146349161863327,6.534782048817078e-09,248,4.612270565473864e-09,2.6872147174117345e-08,4.7610533293607205e-09,1.0
+23,k_to_q,test-en-2,heldout,1.9073486328125e-06,4.8735765106561524e-08,4.172325134277344e-07,0.08886055648326874,0.08886055648326874,4.1334708988927105e-09,248,4.0138719015203606e-09,2.8323970724386527e-08,4.1433514397226645e-09,1.0

results/cpu-qwen35-2b/gain-factorization-summary.json

@@ -0,0 +1,46 @@
+{
+  "state": "completed",
+  "records": [],
+  "interventions": [
+    {
+      "layer": 3,
+      "direction": "q_to_k",
+      "sequence_nll_deltas": [
+        3.114725206643243e-07,
+        -5.23134924712565e-07,
+        1.7465536439331686e-07,
+        1.733758607791458e-07
+      ],
+      "mean_nll_delta": 3.40922052810555e-08,
+      "sequence_p95_delta": 2.9094994722367315e-07,
+      "passed": true
+    },
+    {
+      "layer": 3,
+      "direction": "k_to_q",
+      "sequence_nll_deltas": [
+        4.0256782884995346e-07,
+        -8.45824208006718e-07,
+        2.1304516684139685e-07,
+        -3.860568877911419e-07
+      ],
+      "mean_nll_delta": -1.540670250266274e-07,
+      "sequence_p95_delta": 3.741394295486699e-07,
+      "passed": true
+    }
+  ],
+  "maximum_score_abs_error": 3.814697265625e-06,
+  "maximum_probability_abs_error": 8.046627044677734e-07,
+  "bidirectional_heldout_pass": true,
+  "elementwise_gain_multiplies_saved_per_token_layer": {
+    "q_to_k": 1536,
+    "k_to_q": 384
+  },
+  "original_bf16_payload_bytes": 1024,
+  "factored_lossless_mixed_payload_bytes": 1024,
+  "fp32_cached_gain_payload_before": 2048,
+  "fp32_cached_gain_payload_after": 1280,
+  "claim_limit": "Exact real-arithmetic identity only on non-RoPE coordinates; empirical FP32/NLL checks on tiny text pilot. Not evidence that two existing full layers interchange, not model-wide speedup.",
+  "elapsed_seconds": 232.08875080291182,
+  "peak_rss_kib": 83704
+}

results/cpu-qwen35-2b/install-torch-disk-tmp.log

@@ -0,0 +1,39 @@
+Looking in indexes: https://download.pytorch.org/whl/cpu
+Collecting torch
+  Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (35 kB)
+Collecting filelock (from torch)
+  Downloading filelock-4.2.4-py3-none-any.whl.metadata (2.0 kB)
+Collecting typing-extensions>=4.13.0 (from torch)
+  Downloading typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
+Collecting setuptools>=77.0.3 (from torch)
+  Downloading https://download.pytorch.org/whl/setuptools-78.1.0-py3-none-any.whl.metadata (6.6 kB)
+Collecting sympy>=1.13.3 (from torch)
+  Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
+Collecting networkx>=2.5.1 (from torch)
+  Downloading networkx-3.7-py3-none-any.whl.metadata (6.7 kB)
+Collecting jinja2 (from torch)
+  Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
+Collecting fsspec>=0.8.5 (from torch)
+  Downloading fsspec-2026.9.0-py3-none-any.whl.metadata (10 kB)
+Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
+  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
+Collecting MarkupSafe>=2.0 (from jinja2->torch)
+  Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
+Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl (209.5 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 209.5/209.5 MB 235.8 kB/s eta 0:00:00
+Downloading fsspec-2026.9.0-py3-none-any.whl (221 kB)
+Downloading networkx-3.7-py3-none-any.whl (2.1 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 143.1 kB/s eta 0:00:00
+Downloading https://download.pytorch.org/whl/setuptools-78.1.0-py3-none-any.whl (1.3 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 368.1 kB/s eta 0:00:00
+Downloading sympy-1.14.0-py3-none-any.whl (6.3 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.3/6.3 MB 27.0 kB/s eta 0:00:00
+Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 19.7 kB/s eta 0:00:00
+Downloading typing_extensions-4.16.0-py3-none-any.whl (45 kB)
+Downloading filelock-4.2.4-py3-none-any.whl (162 kB)
+Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl (134 kB)
+Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (23 kB)
+Installing collected packages: mpmath, typing-extensions, sympy, setuptools, networkx, MarkupSafe, fsspec, filelock, jinja2, torch
+
+Successfully installed MarkupSafe-3.0.3 filelock-4.2.4 fsspec-2026.9.0 jinja2-3.1.6 mpmath-1.3.0 networkx-3.7 setuptools-78.1.0 sympy-1.14.0 torch-2.15.0+cpu typing-extensions-4.16.0

results/cpu-qwen35-2b/install-torch.log

@@ -0,0 +1,25 @@
+Looking in indexes: https://download.pytorch.org/whl/cpu
+Collecting torch
+  Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (35 kB)
+Collecting filelock (from torch)
+  Downloading filelock-4.2.4-py3-none-any.whl.metadata (2.0 kB)
+Collecting typing-extensions>=4.13.0 (from torch)
+  Downloading typing_extensions-4.16.0-py3-none-any.whl.metadata (3.3 kB)
+Collecting setuptools>=77.0.3 (from torch)
+  Downloading https://download.pytorch.org/whl/setuptools-78.1.0-py3-none-any.whl.metadata (6.6 kB)
+Collecting sympy>=1.13.3 (from torch)
+  Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
+Collecting networkx>=2.5.1 (from torch)
+  Downloading networkx-3.7-py3-none-any.whl.metadata (6.7 kB)
+Collecting jinja2 (from torch)
+  Downloading https://download.pytorch.org/whl/jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
+Collecting fsspec>=0.8.5 (from torch)
+  Downloading fsspec-2026.9.0-py3-none-any.whl.metadata (10 kB)
+Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch)
+  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
+Collecting MarkupSafe>=2.0 (from jinja2->torch)
+  Downloading https://download.pytorch.org/whl/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.metadata (2.7 kB)
+Downloading https://download-r2.pytorch.org/whl/cpu/torch-2.15.0%2Bcpu-cp314-cp314-manylinux_2_28_x86_64.whl (209.5 MB)
+   ━━━━━━━━━━━━━━━━━━━━━━━━━━━             145.8/209.5 MB 447.0 kB/s eta 0:02:23
+ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device
+

results/cpu-qwen35-2b/manifest.json

@@ -0,0 +1,34 @@
+{
+  "repo": "Qwen/Qwen3.5-2B",
+  "revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "expected_bytes": 4548351670,
+  "started_unix": 1794169446.8545816,
+  "files": [
+    {
+      "name": "README.md",
+      "bytes": 62814,
+      "sha256": "c0e83a849c776e6fa843d011f023132d942a0f0140d903205bf1c363adad2275",
+      "upstream_sha256": null
+    },
+    {
+      "name": "config.json",
+      "bytes": 2908,
+      "sha256": "ed1c1723241f23f7f4e23430759cbd7dcfb4103cbdfe052bfe7626b57c2615b4",
+      "upstream_sha256": null
+    },
+    {
+      "name": "model.safetensors-00001-of-00001.safetensors",
+      "bytes": 4548221488,
+      "sha256": "aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1",
+      "upstream_sha256": "aa33250c4fc64891ddfaba3a314fd9542ea371843c387178b425fbcc5ed680b1"
+    },
+    {
+      "name": "model.safetensors.index.json",
+      "bytes": 64460,
+      "sha256": "aca8afed9da75b0f050b408d270766fd77627f1af401e240f61c3b47d0db02f9",
+      "upstream_sha256": null
+    }
+  ],
+  "license": "apache-2.0",
+  "completed_unix": 1794173525.0766253
+}

results/cpu-qwen35-2b/model_config.json

@@ -0,0 +1,96 @@
+{
+    "architectures": [
+        "Qwen3_5ForConditionalGeneration"
+    ],
+    "image_token_id": 248056,
+    "model_type": "qwen3_5",
+    "text_config": {
+        "attention_bias": false,
+        "attention_dropout": 0.0,
+        "attn_output_gate": true,
+        "dtype": "bfloat16",
+        "eos_token_id": 248044,
+        "full_attention_interval": 4,
+        "head_dim": 256,
+        "hidden_act": "silu",
+        "hidden_size": 2048,
+        "initializer_range": 0.02,
+        "intermediate_size": 6144,
+        "layer_types": [
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention"
+        ],
+        "linear_conv_kernel_dim": 4,
+        "linear_key_head_dim": 128,
+        "linear_num_key_heads": 16,
+        "linear_num_value_heads": 16,
+        "linear_value_head_dim": 128,
+        "max_position_embeddings": 262144,
+        "mlp_only_layers": [],
+        "model_type": "qwen3_5_text",
+        "mtp_num_hidden_layers": 1,
+        "mtp_use_dedicated_embeddings": false,
+        "num_attention_heads": 8,
+        "num_hidden_layers": 24,
+        "num_key_value_heads": 2,
+        "rms_norm_eps": 1e-06,
+        "tie_word_embeddings": true,
+        "use_cache": true,
+        "vocab_size": 248320,
+        "mamba_ssm_dtype": "float32",
+        "rope_parameters": {
+            "mrope_interleaved": true,
+            "mrope_section": [
+                11,
+                11,
+                10
+            ],
+            "rope_type": "default",
+            "rope_theta": 10000000,
+            "partial_rotary_factor": 0.25
+        }
+    },
+    "tie_word_embeddings": true,
+    "transformers_version": "4.57.0.dev0",
+    "video_token_id": 248057,
+    "vision_config": {
+        "deepstack_visual_indexes": [],
+        "depth": 24,
+        "hidden_act": "gelu_pytorch_tanh",
+        "hidden_size": 1024,
+        "in_channels": 3,
+        "initializer_range": 0.02,
+        "intermediate_size": 4096,
+        "model_type": "qwen3_5",
+        "num_heads": 16,
+        "num_position_embeddings": 2304,
+        "out_hidden_size": 2048,
+        "patch_size": 16,
+        "spatial_merge_size": 2,
+        "temporal_patch_size": 2
+    },
+    "vision_end_token_id": 248054,
+    "vision_start_token_id": 248053
+}

results/cpu-qwen35-2b/neuron-reuse-protocol.json

@@ -0,0 +1,29 @@
+{
+  "layers": [
+    5,
+    6,
+    20,
+    21
+  ],
+  "unit": "post_silu_gate_times_up_scalar_function; keep each output column",
+  "thresholds": [
+    0.05,
+    0.1,
+    0.2
+  ],
+  "maximum_directional_relative_l2": 0.2,
+  "cosine_min": 0.98,
+  "pearson_min": 0.95,
+  "rms_floor": "at least within-layer median calibration neuron RMS",
+  "std_over_rms_min": 0.25,
+  "effective_energy_tokens_min": 16,
+  "max_selected_pairs": 1,
+  "selection": "smallest worst directional calibration error; then layer/id",
+  "heldout_per_text_error_max": 0.35,
+  "heldout_quality_mean_nll_delta_max": 0.01,
+  "heldout_quality_sequence_p95_delta_max": 0.05,
+  "training": false,
+  "scale_fitting": false,
+  "cost_required": false,
+  "registered_unix": 1794203413.872667
+}

results/cpu-qwen35-2b/neuron-reuse-search-summary.json

@@ -0,0 +1,54 @@
+[
+  {
+    "layer": 5,
+    "neurons": 6144,
+    "eligible_neurons": 2111,
+    "eligible_unique_pairs_checked": 2227105,
+    "rms_floor": 0.05843841928142779,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  },
+  {
+    "layer": 6,
+    "neurons": 6144,
+    "eligible_neurons": 2090,
+    "eligible_unique_pairs_checked": 2183005,
+    "rms_floor": 0.05875614556680463,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  },
+  {
+    "layer": 20,
+    "neurons": 6144,
+    "eligible_neurons": 1699,
+    "eligible_unique_pairs_checked": 1442451,
+    "rms_floor": 0.1179593664401982,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  },
+  {
+    "layer": 21,
+    "neurons": 6144,
+    "eligible_neurons": 1788,
+    "eligible_unique_pairs_checked": 1597578,
+    "rms_floor": 0.12231686838397056,
+    "calibration_pair_counts": {
+      "0.05": 0,
+      "0.1": 0,
+      "0.2": 0
+    },
+    "best": null
+  }
+]

results/cpu-qwen35-2b/neuron-reuse-selection.json

@@ -0,0 +1,4 @@
+{
+  "selected": null,
+  "selection_uses_heldout": false
+}

results/cpu-qwen35-2b/neuron-reuse-summary.json

@@ -0,0 +1,10 @@
+{
+  "state": "completed",
+  "selected": null,
+  "function_pass": false,
+  "interventions": [],
+  "empirical_repeated_function_positive": false,
+  "claim_limit": "Distinct learned scalar MLP subfunctions on the specified short-text distribution; not whole-layer duplication, global equivalence or broad task-capability proof. No performance requirement.",
+  "elapsed_seconds": 55.004304428584874,
+  "peak_rss_kib": 185036
+}

results/cpu-qwen35-2b/neuron-reuse.log

@@ -0,0 +1,4 @@
+{"layer": 5, "neurons": 6144, "eligible_neurons": 2111, "eligible_unique_pairs_checked": 2227105, "rms_floor": 0.05843841928142779, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}
+{"layer": 6, "neurons": 6144, "eligible_neurons": 2090, "eligible_unique_pairs_checked": 2183005, "rms_floor": 0.05875614556680463, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}
+{"layer": 20, "neurons": 6144, "eligible_neurons": 1699, "eligible_unique_pairs_checked": 1442451, "rms_floor": 0.1179593664401982, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}
+{"layer": 21, "neurons": 6144, "eligible_neurons": 1788, "eligible_unique_pairs_checked": 1597578, "rms_floor": 0.12231686838397056, "calibration_pair_counts": {"0.05": 0, "0.1": 0, "0.2": 0}, "best": null}

results/cpu-qwen35-2b/norm-repetition-function-metrics.csv

@@ -0,0 +1,17 @@
+source_layer,role,a,b,sample_id,split,relative_l2,reverse_relative_l2,token_relative_p95,reverse_token_relative_p95,cosine,low_norm_tokens
+12,input_layernorm,12,13,cal-zh-1,calibration,0.04185799286154599,0.04309664461005982,0.044446126787094885,0.04595947974911848,0.9995232826957284,0
+12,input_layernorm,12,13,cal-zh-2,calibration,0.04383456526622353,0.04521540878040512,0.046012856287805166,0.047545525578214155,0.9994900129478096,0
+12,input_layernorm,12,13,cal-en-1,calibration,0.04347907732513803,0.04487576882582582,0.04643377233703883,0.048238404047826186,0.9995243164548383,0
+12,input_layernorm,12,13,cal-en-2,calibration,0.041111186492761374,0.04233333726141896,0.04398539072853077,0.04542340159464054,0.9995589314599114,0
+12,input_layernorm,12,13,test-zh-1,heldout,0.04230986484298778,0.04359453156137845,0.04521637327744173,0.046674126177753086,0.9995251413848774,0
+12,input_layernorm,12,13,test-zh-2,heldout,0.043902033219287144,0.04528510757179746,0.047435978938916296,0.04912927000592789,0.9994870299724158,0
+12,input_layernorm,12,13,test-en-1,heldout,0.042965663401383455,0.04430863234604914,0.04650584597139243,0.04815192582322126,0.9995218127753321,0
+12,input_layernorm,12,13,test-en-2,heldout,0.04135010433212149,0.042584214188789195,0.046368214514353176,0.047927790990014825,0.9995520355780412,0
+13,input_layernorm,12,13,cal-zh-1,calibration,0.041182645479044225,0.04240971565525536,0.04352880565374195,0.045001128308403464,0.9995577794005558,0
+13,input_layernorm,12,13,cal-zh-2,calibration,0.04327063750976668,0.04463822187523808,0.045272158865488125,0.04679408388648026,0.999518385770098,0
+13,input_layernorm,12,13,cal-en-1,calibration,0.04262779877016355,0.04398654464567335,0.045497400019515064,0.04723836815505396,0.9995547796247127,0
+13,input_layernorm,12,13,cal-en-2,calibration,0.040518961963038086,0.04171178072648754,0.043328686448901595,0.04470198445787224,0.9995758632252388,0
+13,input_layernorm,12,13,test-zh-1,heldout,0.04162196134899919,0.04289161132734105,0.04428805531751076,0.045755187013951454,0.9995588682914622,0
+13,input_layernorm,12,13,test-zh-2,heldout,0.043144621015003826,0.04450396583648517,0.045958426245692637,0.04754094135863426,0.9995211225735321,0
+13,input_layernorm,12,13,test-en-1,heldout,0.042127619483222106,0.04343304280377689,0.045197718466321105,0.04683576056261672,0.999550812480402,0
+13,input_layernorm,12,13,test-en-2,heldout,0.04076045660321344,0.04197546447097981,0.045561271843477975,0.04710887060271581,0.9995759438544739,0

results/cpu-qwen35-2b/norm-repetition-protocol.json

@@ -0,0 +1,44 @@
+{
+  "roles": [
+    "input_layernorm",
+    "post_attention_layernorm"
+  ],
+  "layers": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23
+  ],
+  "same_role_only": true,
+  "same_eps": 1e-06,
+  "uniform_worst_error_limit": 0.2,
+  "empirical_function_worst_relative_error_limit": 0.05,
+  "selection": "smallest maximum two-direction uniform bound, then role/layer; no input data used",
+  "minimum_gain_standard_deviation": 0.001,
+  "max_selected_pairs": 1,
+  "heldout_nll_mean_delta_limit": 0.01,
+  "heldout_nll_sequence_p95_delta_limit": 0.05,
+  "training": false,
+  "cost_required": false,
+  "registered_unix": 1794203690.521922
+}

results/cpu-qwen35-2b/norm-repetition-selection.json

@@ -0,0 +1,23 @@
+{
+  "selected": [
+    {
+      "role": "input_layernorm",
+      "a": 12,
+      "b": 13,
+      "a_to_b_uniform_bound": 0.10185185185185185,
+      "b_to_a_uniform_bound": 0.1134020618556701,
+      "worst_uniform_bound": 0.1134020618556701,
+      "eligible_nonconstant_gains": true,
+      "gain_std_a": 0.1900545060634613,
+      "gain_std_b": 0.16846242547035217,
+      "gain_min_a": 0.5546875,
+      "gain_min_b": 0.5703125
+    }
+  ],
+  "uniform_bound_counts": {
+    "0.05": 0,
+    "0.1": 0,
+    "0.2": 18
+  },
+  "selection_reads_no_input_data": true
+}

results/cpu-qwen35-2b/norm-repetition-summary.json

@@ -0,0 +1,59 @@
+{
+  "state": "completed",
+  "selected": {
+    "role": "input_layernorm",
+    "a": 12,
+    "b": 13,
+    "a_to_b_uniform_bound": 0.10185185185185185,
+    "b_to_a_uniform_bound": 0.1134020618556701,
+    "worst_uniform_bound": 0.1134020618556701,
+    "eligible_nonconstant_gains": true,
+    "gain_std_a": 0.1900545060634613,
+    "gain_std_b": 0.16846242547035217,
+    "gain_min_a": 0.5546875,
+    "gain_min_b": 0.5703125
+  },
+  "interventions": [
+    {
+      "target": 12,
+      "donor": 13,
+      "sequence_nll_deltas": [
+        0.0010516456964495546,
+        -0.0013996762182379996,
+        -0.00012664743940504763,
+        0.0006049385086996928
+      ],
+      "mean_nll_delta": 3.256513687655005e-05,
+      "sequence_p95_nll_delta": 0.0009846396182870752,
+      "passed_quality": true
+    },
+    {
+      "target": 13,
+      "donor": 12,
+      "sequence_nll_deltas": [
+        0.0008813779918703801,
+        -0.0012936356267054805,
+        -0.0002630009203641315,
+        -0.0007806334344149235
+      ],
+      "mean_nll_delta": -0.00036397299740353883,
+      "sequence_p95_nll_delta": 0.000709721155035203,
+      "passed_quality": true
+    }
+  ],
+  "uniform_bound_counts": {
+    "0.05": 0,
+    "0.1": 0,
+    "0.2": 18
+  },
+  "uniform_function_similarity_proven": true,
+  "heldout_function_pass": true,
+  "worst_heldout_relative_l2": 0.04528510757179746,
+  "worst_heldout_cosine": 0.9994870299724158,
+  "untouched_full_forward_bitwise_matches_baseline": true,
+  "learned_function_repetition_positive": true,
+  "bidirectional_network_quality_pass": true,
+  "claim_limit": "Existing learned RMSNorm module functions with identical eps and shape; uniform real-arithmetic relative bound and tiny real-text probes. Does not establish attention/MLP-body duplication or broad task capability.",
+  "elapsed_seconds": 369.7967556482181,
+  "peak_rss_kib": 71596
+}

results/cpu-qwen35-2b/norm-repetition-uniform-bounds.csv

@@ -0,0 +1,553 @@
+role,a,b,a_to_b_uniform_bound,b_to_a_uniform_bound,worst_uniform_bound,eligible_nonconstant_gains,gain_std_a,gain_std_b,gain_min_a,gain_min_b
+input_layernorm,0,1,0.6032863849765259,1.5207100591715976,1.5207100591715976,True,0.11805251985788345,0.0930413156747818,0.794921875,0.607421875
+input_layernorm,0,2,0.5692488262910798,1.3215258855585832,1.3215258855585832,True,0.11805251985788345,0.10473614931106567,0.794921875,0.6484375
+input_layernorm,0,3,0.5469483568075117,1.2072538860103628,1.2072538860103628,True,0.11805251985788345,0.07621725648641586,0.794921875,0.7265625
+input_layernorm,0,4,0.5751173708920188,1.3535911602209945,1.3535911602209945,True,0.11805251985788345,0.10847259312868118,0.794921875,0.662109375
+input_layernorm,0,5,0.6396713615023474,1.775244299674267,1.775244299674267,True,0.11805251985788345,0.13199757039546967,0.794921875,0.5234375
+input_layernorm,0,6,0.6643192488262911,1.979020979020979,1.979020979020979,True,0.11805251985788345,0.1498883068561554,0.794921875,0.47265625
+input_layernorm,0,7,0.6830985915492958,2.1555555555555554,2.1555555555555554,True,0.11805251985788345,0.095920130610466,0.794921875,0.46484375
+input_layernorm,0,8,0.6913145539906104,2.2395437262357416,2.2395437262357416,True,0.11805251985788345,0.20054922997951508,0.794921875,0.41796875
+input_layernorm,0,9,0.7136150234741784,2.4918032786885247,2.4918032786885247,True,0.11805251985788345,0.22649425268173218,0.794921875,0.42578125
+input_layernorm,0,10,0.7230046948356808,2.610169491525424,2.610169491525424,True,0.11805251985788345,0.2204512506723404,0.794921875,0.3828125
+input_layernorm,0,11,0.7347417840375586,2.769911504424779,2.769911504424779,True,0.11805251985788345,0.16603894531726837,0.794921875,0.34375
+input_layernorm,0,12,0.6338028169014085,1.7307692307692308,1.7307692307692308,True,0.11805251985788345,0.1900545060634613,0.794921875,0.5546875
+input_layernorm,0,13,0.6349765258215962,1.7395498392282958,1.7395498392282958,True,0.11805251985788345,0.16846242547035217,0.794921875,0.5703125
+input_layernorm,0,14,0.5516431924882629,1.2303664921465969,1.2303664921465969,True,0.11805251985788345,0.11825777590274811,0.794921875,0.6875
+input_layernorm,0,15,0.49018199233716475,0.9614842649131048,0.9614842649131048,True,0.11805251985788345,0.06029079854488373,0.794921875,0.779296875
+input_layernorm,0,16,0.5117370892018779,1.0480769230769231,1.0480769230769231,True,0.11805251985788345,0.09195870161056519,0.794921875,0.7763671875
+input_layernorm,0,17,0.6232876712328768,0.7185185185185186,0.7185185185185186,True,0.11805251985788345,0.06483674794435501,0.794921875,1.0012359619140625
+input_layernorm,0,18,0.8402948402948403,0.6637450199203188,0.8402948402948403,True,0.11805251985788345,0.06785038113594055,0.794921875,1.1640625
+input_layernorm,0,19,1.0393120393120394,1.1248445660283513,1.1248445660283513,True,0.11805251985788345,0.19409698247909546,0.794921875,0.884765625
+input_layernorm,0,20,1.113022113022113,0.9869156655168312,1.113022113022113,True,0.11805251985788345,0.08625888079404831,0.794921875,1.0262451171875
+input_layernorm,0,21,1.1867321867321867,0.9545986426398315,1.1867321867321867,True,0.11805251985788345,0.07095508277416229,0.794921875,1.043212890625
+input_layernorm,0,22,0.9656019656019657,0.9514018691588785,0.9656019656019657,True,0.11805251985788345,0.07214703410863876,0.794921875,1.044921875
+input_layernorm,0,23,0.941031941031941,0.9568884723523898,0.9568884723523898,True,0.11805251985788345,0.15369786322116852,0.794921875,0.98577880859375
+input_layernorm,1,2,0.16765578635014836,0.17593150850114556,0.17593150850114556,True,0.0930413156747818,0.10473614931106567,0.607421875,0.6484375
+input_layernorm,1,3,0.28125,0.34967320261437906,0.34967320261437906,True,0.0930413156747818,0.07621725648641586,0.607421875,0.7265625
+input_layernorm,1,4,0.1678622668579627,0.15668366741786266,0.1678622668579627,True,0.0930413156747818,0.10847259312868118,0.607421875,0.662109375
+input_layernorm,1,5,0.23018147086914995,0.2990074441687345,0.2990074441687345,True,0.0930413156747818,0.13199757039546967,0.607421875,0.5234375
+input_layernorm,1,6,0.34367141659681477,0.5236270753512133,0.5236270753512133,True,0.0930413156747818,0.1498883068561554,0.607421875,0.47265625
+input_layernorm,1,7,0.3996670702179177,0.6657423745903706,0.6657423745903706,True,0.0930413156747818,0.095920130610466,0.607421875,0.46484375
+input_layernorm,1,8,0.4585726004922067,0.8469696969696969,0.8469696969696969,True,0.0930413156747818,0.20054922997951508,0.607421875,0.41796875
+input_layernorm,1,9,0.5138306789606035,1.056896551724138,1.056896551724138,True,0.0930413156747818,0.22649425268173218,0.607421875,0.42578125
+input_layernorm,1,10,0.4786253143336127,0.9180064308681672,0.9180064308681672,True,0.0930413156747818,0.2204512506723404,0.607421875,0.3828125
+input_layernorm,1,11,0.49209302325581394,0.9688644688644689,0.9688644688644689,True,0.0930413156747818,0.16603894531726837,0.607421875,0.34375
+input_layernorm,1,12,0.3838141025641026,0.6228868660598179,0.6228868660598179,True,0.0930413156747818,0.1900545060634613,0.607421875,0.5546875
+input_layernorm,1,13,0.39622641509433965,0.65625,0.65625,True,0.0930413156747818,0.16846242547035217,0.607421875,0.5703125
+input_layernorm,1,14,0.27009646302250806,0.28465608465608466,0.28465608465608466,True,0.0930413156747818,0.11825777590274811,0.607421875,0.6875
+input_layernorm,1,15,0.9292604501607717,0.5223386651958081,0.9292604501607717,True,0.0930413156747818,0.06029079854488373,0.607421875,0.779296875
+input_layernorm,1,16,0.36012861736334406,0.2647754137115839,0.36012861736334406,True,0.0930413156747818,0.09195870161056519,0.607421875,0.7763671875
+input_layernorm,1,17,0.7958860759493671,0.4431718061674009,0.7958860759493671,True,0.0930413156747818,0.06483674794435501,0.607421875,1.0012359619140625
+input_layernorm,1,18,1.1044303797468353,0.524812030075188,1.1044303797468353,True,0.0930413156747818,0.06785038113594055,0.607421875,1.1640625
+input_layernorm,1,19,1.3858520900321543,0.5808625336927223,1.3858520900321543,True,0.0930413156747818,0.19409698247909546,0.607421875,0.884765625
+input_layernorm,1,20,1.206896551724138,0.546875,1.206896551724138,True,0.0930413156747818,0.08625888079404831,0.607421875,1.0262451171875
+input_layernorm,1,21,1.630225080385852,0.6198044009779952,1.630225080385852,True,0.0930413156747818,0.07095508277416229,0.607421875,1.043212890625
+input_layernorm,1,22,1.4043887147335423,0.5840938722294654,1.4043887147335423,True,0.0930413156747818,0.07214703410863876,0.607421875,1.044921875
+input_layernorm,1,23,1.9453376205787782,0.6604803493449781,1.9453376205787782,True,0.0930413156747818,0.15369786322116852,0.607421875,0.98577880859375
+input_layernorm,2,3,0.2867132867132867,0.4019607843137255,0.4019607843137255,True,0.10473614931106567,0.07621725648641586,0.6484375,0.7265625
+input_layernorm,2,4,0.10583941605839416,0.11836734693877551,0.11836734693877551,True,0.10473614931106567,0.10847259312868118,0.6484375,0.662109375
+input_layernorm,2,5,0.23737099402498643,0.31125356125356124,0.31125356125356124,True,0.10473614931106567,0.13199757039546967,0.6484375,0.5234375
+input_layernorm,2,6,0.31070496083550914,0.45075757575757575,0.45075757575757575,True,0.10473614931106567,0.1498883068561554,0.6484375,0.47265625
+input_layernorm,2,7,0.42205710955710957,0.7302747668263171,0.7302747668263171,True,0.10473614931106567,0.095920130610466,0.6484375,0.46484375
+input_layernorm,2,8,0.4018264840182648,0.6717557251908397,0.6717557251908397,True,0.10473614931106567,0.20054922997951508,0.6484375,0.41796875
+input_layernorm,2,9,0.4722187382415653,0.8947243346007605,0.8947243346007605,True,0.10473614931106567,0.22649425268173218,0.6484375,0.42578125
+input_layernorm,2,10,0.46013404206147446,0.8523116438356164,0.8523116438356164,True,0.10473614931106567,0.2204512506723404,0.6484375,0.3828125
+input_layernorm,2,11,0.5159817351598174,1.0660377358490567,1.0660377358490567,True,0.10473614931106567,0.16603894531726837,0.6484375,0.34375
+input_layernorm,2,12,0.30617848970251715,0.4412928759894459,0.4412928759894459,True,0.10473614931106567,0.1900545060634613,0.6484375,0.5546875
+input_layernorm,2,13,0.2900036175087423,0.4084578804347826,0.4084578804347826,True,0.10473614931106567,0.16846242547035217,0.6484375,0.5703125
+input_layernorm,2,14,0.19501226492232215,0.24225495175215844,0.24225495175215844,True,0.10473614931106567,0.11825777590274811,0.6484375,0.6875
+input_layernorm,2,15,0.7957142857142857,0.5678391959798995,0.7957142857142857,True,0.10473614931106567,0.06029079854488373,0.6484375,0.779296875
+input_layernorm,2,16,0.2740963855421687,0.21513002364066194,0.2740963855421687,True,0.10473614931106567,0.09195870161056519,0.6484375,0.7763671875
+input_layernorm,2,17,0.7093373493975904,0.4149779735682819,0.7093373493975904,True,0.10473614931106567,0.06483674794435501,0.6484375,1.0012359619140625
+input_layernorm,2,18,1.0030120481927711,0.5007518796992482,1.0030120481927711,True,0.10473614931106567,0.06785038113594055,0.6484375,1.1640625
+input_layernorm,2,19,1.2680722891566265,0.5590969455511288,1.2680722891566265,True,0.10473614931106567,0.19409698247909546,0.6484375,0.884765625
+input_layernorm,2,20,1.0873493975903614,0.5209235209235209,1.0873493975903614,True,0.10473614931106567,0.08625888079404831,0.6484375,1.0262451171875
+input_layernorm,2,21,1.463855421686747,0.5941320293398533,1.463855421686747,True,0.10473614931106567,0.07095508277416229,0.6484375,1.043212890625
+input_layernorm,2,22,1.249266862170088,0.5554106910039114,1.249266862170088,True,0.10473614931106567,0.07214703410863876,0.6484375,1.044921875
+input_layernorm,2,23,1.7448680351906158,0.6356837606837606,1.7448680351906158,True,0.10473614931106567,0.15369786322116852,0.6484375,0.98577880859375
+input_layernorm,3,4,0.40522875816993464,0.28837209302325584,0.40522875816993464,True,0.07621725648641586,0.10847259312868118,0.7265625,0.662109375
+input_layernorm,3,5,0.3660130718954248,0.39925373134328357,0.39925373134328357,True,0.07621725648641586,0.13199757039546967,0.7265625,0.5234375
+input_layernorm,3,6,0.35609756097560974,0.553030303030303,0.553030303030303,True,0.07621725648641586,0.1498883068561554,0.7265625,0.47265625
+input_layernorm,3,7,0.39575530586766544,0.6549586776859504,0.6549586776859504,True,0.07621725648641586,0.095920130610466,0.7265625,0.46484375
+input_layernorm,3,8,0.42473118279569894,0.7383177570093458,0.7383177570093458,True,0.07621725648641586,0.20054922997951508,0.7265625,0.41796875
+input_layernorm,3,9,0.49180741114192084,0.9677579365079365,0.9677579365079365,True,0.07621725648641586,0.22649425268173218,0.7265625,0.42578125
+input_layernorm,3,10,0.47804878048780486,0.9158878504672897,0.9158878504672897,True,0.07621725648641586,0.2204512506723404,0.7265625,0.3828125
+input_layernorm,3,11,0.545338441890166,1.199438202247191,1.199438202247191,True,0.07621725648641586,0.16603894531726837,0.7265625,0.34375
+input_layernorm,3,12,0.40460669274228594,0.5290237467018469,0.5290237467018469,True,0.07621725648641586,0.1900545060634613,0.7265625,0.5546875
+input_layernorm,3,13,0.3559322033898305,0.5441576086956522,0.5441576086956522,True,0.07621725648641586,0.16846242547035217,0.7265625,0.5703125
+input_layernorm,3,14,0.28804587560652845,0.22363013698630138,0.28804587560652845,True,0.07621725648641586,0.11825777590274811,0.7265625,0.6875
+input_layernorm,3,15,0.5853333333333334,0.36921783010933557,0.5853333333333334,True,0.07621725648641586,0.06029079854488373,0.7265625,0.779296875
+input_layernorm,3,16,0.43790849673202614,0.30454545454545456,0.43790849673202614,True,0.07621725648641586,0.09195870161056519,0.7265625,0.7763671875
+input_layernorm,3,17,0.5490196078431373,0.35443037974683544,0.5490196078431373,True,0.07621725648641586,0.06483674794435501,0.7265625,1.0012359619140625
+input_layernorm,3,18,0.7876344086021505,0.4406015037593985,0.7876344086021505,True,0.07621725648641586,0.06785038113594055,0.7265625,1.1640625
+input_layernorm,3,19,1.0241935483870968,0.5059760956175299,1.0241935483870968,True,0.07621725648641586,0.19409698247909546,0.7265625,0.884765625
+input_layernorm,3,20,0.9153567110036276,0.4779040404040404,0.9153567110036276,True,0.07621725648641586,0.08625888079404831,0.7265625,1.0262451171875
+input_layernorm,3,21,1.1706666666666667,0.5393120393120393,1.1706666666666667,True,0.07621725648641586,0.07095508277416229,0.7265625,1.043212890625
+input_layernorm,3,22,1.0453333333333332,0.5110821382007823,1.0453333333333332,True,0.07621725648641586,0.07214703410863876,0.7265625,1.044921875
+input_layernorm,3,23,1.496,0.5993589743589743,1.496,True,0.07621725648641586,0.15369786322116852,0.7265625,0.98577880859375
+input_layernorm,4,5,0.21176470588235294,0.26865671641791045,0.26865671641791045,True,0.10847259312868118,0.13199757039546967,0.662109375,0.5234375
+input_layernorm,4,6,0.2985507246376812,0.4256198347107438,0.4256198347107438,True,0.10847259312868118,0.1498883068561554,0.662109375,0.47265625
+input_layernorm,4,7,0.42340116279069767,0.7343080413410638,0.7343080413410638,True,0.10847259312868118,0.095920130610466,0.662109375,0.46484375
+input_layernorm,4,8,0.37971014492753624,0.6121495327102804,0.6121495327102804,True,0.10847259312868118,0.20054922997951508,0.662109375,0.41796875
+input_layernorm,4,9,0.4610917537746806,0.8556034482758621,0.8556034482758621,True,0.10847259312868118,0.22649425268173218,0.662109375,0.42578125
+input_layernorm,4,10,0.4318840579710145,0.7602040816326531,0.7602040816326531,True,0.10847259312868118,0.2204512506723404,0.662109375,0.3828125
+input_layernorm,4,11,0.5041140976412507,1.0165929203539823,1.0165929203539823,True,0.10847259312868118,0.16603894531726837,0.662109375,0.34375
+input_layernorm,4,12,0.31369924141008476,0.45708712613784136,0.45708712613784136,True,0.10847259312868118,0.1900545060634613,0.662109375,0.5546875
+input_layernorm,4,13,0.3016249555212905,0.4318953804347826,0.4318953804347826,True,0.10847259312868118,0.16846242547035217,0.662109375,0.5703125
+input_layernorm,4,14,0.177762982689747,0.21619433198380567,0.21619433198380567,True,0.10847259312868118,0.11825777590274811,0.662109375,0.6875
+input_layernorm,4,15,0.799410029498525,0.5714938328003655,0.799410029498525,True,0.10847259312868118,0.06029079854488373,0.662109375,0.779296875
+input_layernorm,4,16,0.2727919383521043,0.21432563338301044,0.2727919383521043,True,0.10847259312868118,0.09195870161056519,0.662109375,0.7763671875
+input_layernorm,4,17,0.644927536231884,0.3920704845814978,0.644927536231884,True,0.10847259312868118,0.06483674794435501,0.662109375,1.0012359619140625
+input_layernorm,4,18,0.927536231884058,0.48120300751879697,0.927536231884058,True,0.10847259312868118,0.06785038113594055,0.662109375,1.1640625
+input_layernorm,4,19,1.191740412979351,0.5437415881561238,1.191740412979351,True,0.10847259312868118,0.19409698247909546,0.662109375,0.884765625
+input_layernorm,4,20,1.0705882352941176,0.5170454545454546,1.0705882352941176,True,0.10847259312868118,0.08625888079404831,0.662109375,1.0262451171875
+input_layernorm,4,21,1.4129793510324484,0.5855745721271394,1.4129793510324484,True,0.10847259312868118,0.07095508277416229,0.662109375,1.043212890625
+input_layernorm,4,22,1.2558823529411764,0.5567144719687093,1.2558823529411764,True,0.10847259312868118,0.07214703410863876,0.662109375,1.044921875
+input_layernorm,4,23,1.7529411764705882,0.6367521367521367,1.7529411764705882,True,0.10847259312868118,0.15369786322116852,0.662109375,0.98577880859375
+input_layernorm,5,6,0.21281039684381528,0.27034198113207547,0.27034198113207547,True,0.13199757039546967,0.1498883068561554,0.5234375,0.47265625
+input_layernorm,5,7,0.40684808612440193,0.6859087471641039,0.6859087471641039,True,0.13199757039546967,0.095920130610466,0.5234375,0.46484375
+input_layernorm,5,8,0.3513911921872121,0.5417613636363636,0.5417613636363636,True,0.13199757039546967,0.20054922997951508,0.5234375,0.41796875
+input_layernorm,5,9,0.41179234202490017,0.7000798722044729,0.7000798722044729,True,0.13199757039546967,0.22649425268173218,0.5234375,0.42578125
+input_layernorm,5,10,0.35843218153687467,0.5586816720257235,0.5586816720257235,True,0.13199757039546967,0.2204512506723404,0.5234375,0.3828125
+input_layernorm,5,11,0.4428827751196172,0.7949543746645196,0.7949543746645196,True,0.13199757039546967,0.16603894531726837,0.5234375,0.34375
+input_layernorm,5,12,0.2774254169603007,0.3839401820546164,0.3839401820546164,True,0.13199757039546967,0.1900545060634613,0.5234375,0.5546875
+input_layernorm,5,13,0.2767029052269517,0.3825577445652174,0.3825577445652174,True,0.13199757039546967,0.16846242547035217,0.5234375,0.5703125
+input_layernorm,5,14,0.4141791044776119,0.2928759894459103,0.4141791044776119,True,0.13199757039546967,0.11825777590274811,0.5234375,0.6875
+input_layernorm,5,15,1.2182835820895523,0.5492010092514719,1.2182835820895523,True,0.13199757039546967,0.06029079854488373,0.5234375,0.779296875
+input_layernorm,5,16,0.5316051136363636,0.34709019244145606,0.5316051136363636,True,0.13199757039546967,0.09195870161056519,0.5234375,0.7763671875
+input_layernorm,5,17,1.0124113475177305,0.5030837004405286,1.0124113475177305,True,0.13199757039546967,0.06483674794435501,0.5234375,1.0012359619140625
+input_layernorm,5,18,1.3581560283687943,0.5759398496240602,1.3581560283687943,True,0.13199757039546967,0.06785038113594055,0.5234375,1.1640625
+input_layernorm,5,19,1.7798507462686568,0.640268456375839,1.7798507462686568,True,0.13199757039546967,0.19409698247909546,0.5234375,0.884765625
+input_layernorm,5,20,1.626865671641791,0.6193181818181818,1.626865671641791,True,0.13199757039546967,0.08625888079404831,0.5234375,1.0262451171875
+input_layernorm,5,21,2.037313432835821,0.6707616707616708,2.037313432835821,True,0.13199757039546967,0.07095508277416229,0.5234375,1.043212890625
+input_layernorm,5,22,1.8619402985074627,0.6505867014341591,1.8619402985074627,True,0.13199757039546967,0.07214703410863876,0.5234375,1.044921875
+input_layernorm,5,23,2.4925373134328357,0.7136752136752137,2.4925373134328357,True,0.13199757039546967,0.15369786322116852,0.5234375,0.98577880859375
+input_layernorm,6,7,0.37231012658227847,0.5931434333249307,0.5931434333249307,True,0.1498883068561554,0.095920130610466,0.47265625,0.46484375
+input_layernorm,6,8,0.1901840490797546,0.23484848484848486,0.23484848484848486,True,0.1498883068561554,0.20054922997951508,0.47265625,0.41796875
+input_layernorm,6,9,0.26697892271662765,0.36421725239616615,0.36421725239616615,True,0.1498883068561554,0.22649425268173218,0.47265625,0.42578125
+input_layernorm,6,10,0.2579185520361991,0.3475609756097561,0.3475609756097561,True,0.1498883068561554,0.2204512506723404,0.47265625,0.3828125
+input_layernorm,6,11,0.41044303797468357,0.6961889425657541,0.6961889425657541,True,0.1498883068561554,0.16603894531726837,0.47265625,0.34375
+input_layernorm,6,12,0.2694422623723488,0.21225247524752475,0.2694422623723488,True,0.1498883068561554,0.1900545060634613,0.47265625,0.5546875
+input_layernorm,6,13,0.2518703241895262,0.22759714512291832,0.2518703241895262,True,0.1498883068561554,0.16846242547035217,0.47265625,0.5703125
+input_layernorm,6,14,0.5578512396694215,0.35809018567639256,0.5578512396694215,True,0.1498883068561554,0.11825777590274811,0.47265625,0.6875
+input_layernorm,6,15,1.3971774193548387,0.5828427249789739,1.3971774193548387,True,0.1498883068561554,0.06029079854488373,0.47265625,0.779296875
+input_layernorm,6,16,0.7217261904761905,0.4191875540190147,0.7217261904761905,True,0.1498883068561554,0.09195870161056519,0.47265625,0.7763671875
+input_layernorm,6,17,1.3450413223140496,0.573568281938326,1.3450413223140496,True,0.1498883068561554,0.06483674794435501,0.47265625,1.0012359619140625
+input_layernorm,6,18,1.7479338842975207,0.6360902255639098,1.7479338842975207,True,0.1498883068561554,0.06785038113594055,0.47265625,1.1640625
+input_layernorm,6,19,2.1115702479338845,0.6786188579017264,2.1115702479338845,True,0.1498883068561554,0.19409698247909546,0.47265625,0.884765625
+input_layernorm,6,20,1.8636363636363635,0.6507936507936508,1.8636363636363635,True,0.1498883068561554,0.08625888079404831,0.47265625,1.0262451171875
+input_layernorm,6,21,2.282258064516129,0.6953316953316954,2.282258064516129,True,0.1498883068561554,0.07095508277416229,0.47265625,1.043212890625
+input_layernorm,6,22,2.092741935483871,0.6766623207301173,2.092741935483871,True,0.1498883068561554,0.07214703410863876,0.47265625,1.044921875
+input_layernorm,6,23,2.774193548387097,0.7350427350427351,2.774193548387097,True,0.1498883068561554,0.15369786322116852,0.47265625,0.98577880859375
+input_layernorm,7,8,0.564910511721704,0.6390151515151515,0.6390151515151515,True,0.095920130610466,0.20054922997951508,0.46484375,0.41796875
+input_layernorm,7,9,0.4822283841693975,0.8075657894736842,0.8075657894736842,True,0.095920130610466,0.22649425268173218,0.46484375,0.42578125
+input_layernorm,7,10,0.4459289135366776,0.6383139067524116,0.6383139067524116,True,0.095920130610466,0.2204512506723404,0.46484375,0.3828125
+input_layernorm,7,11,0.3941018766756032,0.6504424778761062,0.6504424778761062,True,0.095920130610466,0.16603894531726837,0.46484375,0.34375
+input_layernorm,7,12,0.5676959619952494,0.449868073878628,0.5676959619952494,True,0.095920130610466,0.1900545060634613,0.46484375,0.5546875
+input_layernorm,7,13,0.5866983372921615,0.46976902173913043,0.5866983372921615,True,0.095920130610466,0.16846242547035217,0.46484375,0.5703125
+input_layernorm,7,14,0.630188679245283,0.38657407407407407,0.630188679245283,True,0.095920130610466,0.11825777590274811,0.46484375,0.6875
+input_layernorm,7,15,1.4814049586776858,0.5970024979184013,1.4814049586776858,True,0.095920130610466,0.06029079854488373,0.46484375,0.779296875
+input_layernorm,7,16,0.9477434679334917,0.4865853658536585,0.9477434679334917,True,0.095920130610466,0.09195870161056519,0.46484375,0.7763671875
+input_layernorm,7,17,1.2394957983193278,0.5534709193245778,1.2394957983193278,True,0.095920130610466,0.06483674794435501,0.46484375,1.0012359619140625
+input_layernorm,7,18,1.5630081300813008,0.6098334655035687,1.5630081300813008,True,0.095920130610466,0.06785038113594055,0.46484375,1.1640625
+input_layernorm,7,19,2.19327731092437,0.6868421052631579,2.19327731092437,True,0.095920130610466,0.19409698247909546,0.46484375,0.884765625
+input_layernorm,7,20,1.9201680672268908,0.6575539568345323,1.9201680672268908,True,0.095920130610466,0.08625888079404831,0.46484375,1.0262451171875
+input_layernorm,7,21,2.3445378151260505,0.7010050251256281,2.3445378151260505,True,0.095920130610466,0.07095508277416229,0.46484375,1.043212890625
+input_layernorm,7,22,2.1178861788617884,0.6792698826597132,2.1178861788617884,True,0.095920130610466,0.07214703410863876,0.46484375,1.044921875
+input_layernorm,7,23,2.8048780487804876,0.7371794871794872,2.8048780487804876,True,0.095920130610466,0.15369786322116852,0.46484375,0.98577880859375
+input_layernorm,8,9,0.2168141592920354,0.1781818181818182,0.2168141592920354,True,0.20054922997951508,0.22649425268173218,0.41796875,0.42578125
+input_layernorm,8,10,0.19393939393939394,0.1813186813186813,0.19393939393939394,True,0.20054922997951508,0.2204512506723404,0.41796875,0.3828125
+input_layernorm,8,11,0.39980670103092786,0.666129898013956,0.666129898013956,True,0.20054922997951508,0.16603894531726837,0.41796875,0.34375
+input_layernorm,8,12,0.3672566371681416,0.2686084142394822,0.3672566371681416,True,0.20054922997951508,0.1900545060634613,0.41796875,0.5546875
+input_layernorm,8,13,0.3644859813084112,0.2671232876712329,0.3644859813084112,True,0.20054922997951508,0.16846242547035217,0.41796875,0.5703125
+input_layernorm,8,14,0.7616822429906542,0.4323607427055703,0.7616822429906542,True,0.20054922997951508,0.11825777590274811,0.41796875,0.6875
+input_layernorm,8,15,1.677927927927928,0.6265769554247267,1.677927927927928,True,0.20054922997951508,0.06029079854488373,0.41796875,0.779296875
+input_layernorm,8,16,0.9478114478114478,0.48660328435609335,0.9478114478114478,True,0.20054922997951508,0.09195870161056519,0.41796875,0.7763671875
+input_layernorm,8,17,1.6518691588785046,0.6229074889867842,1.6518691588785046,True,0.20054922997951508,0.06483674794435501,0.41796875,1.0012359619140625
+input_layernorm,8,18,2.107476635514019,0.6781954887218045,2.107476635514019,True,0.20054922997951508,0.06785038113594055,0.41796875,1.1640625
+input_layernorm,8,19,2.5186915887850465,0.7158034528552457,2.5186915887850465,True,0.20054922997951508,0.19409698247909546,0.41796875,0.884765625
+input_layernorm,8,20,2.282442748091603,0.6953488372093023,2.282442748091603,True,0.20054922997951508,0.08625888079404831,0.41796875,1.0262451171875
+input_layernorm,8,21,2.6666666666666665,0.7272727272727273,2.6666666666666665,True,0.20054922997951508,0.07095508277416229,0.41796875,1.043212890625
+input_layernorm,8,22,2.467289719626168,0.7115902964959568,2.467289719626168,True,0.20054922997951508,0.07214703410863876,0.41796875,1.044921875
+input_layernorm,8,23,3.2162162162162162,0.7628205128205128,3.2162162162162162,True,0.20054922997951508,0.15369786322116852,0.41796875,0.98577880859375
+input_layernorm,9,10,0.31333333333333335,0.3008849557522124,0.31333333333333335,True,0.22649425268173218,0.2204512506723404,0.42578125,0.3828125
+input_layernorm,9,11,0.5083333333333333,0.5780998389694042,0.5780998389694042,True,0.22649425268173218,0.16603894531726837,0.42578125,0.34375
+input_layernorm,9,12,0.452991452991453,0.31176470588235294,0.452991452991453,True,0.22649425268173218,0.1900545060634613,0.42578125,0.5546875
+input_layernorm,9,13,0.4921259842519685,0.32981530343007914,0.4921259842519685,True,0.22649425268173218,0.16846242547035217,0.42578125,0.5703125
+input_layernorm,9,14,0.8134920634920635,0.4485776805251641,0.8134920634920635,True,0.22649425268173218,0.11825777590274811,0.42578125,0.6875
+input_layernorm,9,15,1.7727272727272727,0.639344262295082,1.7727272727272727,True,0.22649425268173218,0.06029079854488373,0.42578125,0.779296875
+input_layernorm,9,16,1.2080152671755726,0.547104580812446,1.2080152671755726,True,0.22649425268173218,0.09195870161056519,0.42578125,0.7763671875
+input_layernorm,9,17,1.7058823529411764,0.6304347826086957,1.7058823529411764,True,0.22649425268173218,0.06483674794435501,0.42578125,1.0012359619140625
+input_layernorm,9,18,2.1470588235294117,0.6822429906542056,2.1470588235294117,True,0.22649425268173218,0.06785038113594055,0.42578125,1.1640625
+input_layernorm,9,19,2.4873949579831933,0.7132530120481928,2.4873949579831933,True,0.22649425268173218,0.19409698247909546,0.42578125,0.884765625
+input_layernorm,9,20,2.6134453781512605,0.7232558139534884,2.6134453781512605,True,0.22649425268173218,0.08625888079404831,0.42578125,1.0262451171875
+input_layernorm,9,21,2.7394957983193278,0.7325842696629213,2.7394957983193278,True,0.22649425268173218,0.07095508277416229,0.42578125,1.043212890625
+input_layernorm,9,22,2.403669724770642,0.706199460916442,2.403669724770642,True,0.22649425268173218,0.07214703410863876,0.42578125,1.044921875
+input_layernorm,9,23,2.990990990990991,0.7494356659142212,2.990990990990991,True,0.22649425268173218,0.15369786322116852,0.42578125,0.98577880859375
+input_layernorm,10,11,0.350418410041841,0.5394524959742351,0.5394524959742351,True,0.2204512506723404,0.16603894531726837,0.3828125,0.34375
+input_layernorm,10,12,0.4489795918367347,0.30985915492957744,0.4489795918367347,True,0.2204512506723404,0.1900545060634613,0.3828125,0.5546875
+input_layernorm,10,13,0.4897959183673469,0.3287671232876712,0.4897959183673469,True,0.2204512506723404,0.16846242547035217,0.3828125,0.5703125
+input_layernorm,10,14,0.923469387755102,0.48010610079575594,0.923469387755102,True,0.2204512506723404,0.11825777590274811,0.3828125,0.6875
+input_layernorm,10,15,1.8762755102040816,0.6523281596452328,1.8762755102040816,True,0.2204512506723404,0.06029079854488373,0.3828125,0.779296875
+input_layernorm,10,16,1.1346863468634687,0.5315471045808124,1.1346863468634687,True,0.2204512506723404,0.09195870161056519,0.3828125,0.7763671875
+input_layernorm,10,17,1.8954081632653061,0.654625550660793,1.8954081632653061,True,0.2204512506723404,0.06483674794435501,0.3828125,1.0012359619140625
+input_layernorm,10,18,2.392857142857143,0.7052631578947368,2.392857142857143,True,0.2204512506723404,0.06785038113594055,0.3828125,1.1640625
+input_layernorm,10,19,2.8418367346938775,0.7397078353253652,2.8418367346938775,True,0.2204512506723404,0.19409698247909546,0.3828125,0.884765625
+input_layernorm,10,20,2.5357142857142856,0.7171717171717171,2.5357142857142856,True,0.2204512506723404,0.08625888079404831,0.3828125,1.0262451171875
+input_layernorm,10,21,2.9285714285714284,0.7454545454545455,2.9285714285714284,True,0.2204512506723404,0.07095508277416229,0.3828125,1.043212890625
+input_layernorm,10,22,2.7857142857142856,0.7358490566037735,2.7857142857142856,True,0.2204512506723404,0.07214703410863876,0.3828125,1.044921875
+input_layernorm,10,23,3.3333333333333335,0.7692307692307693,3.3333333333333335,True,0.2204512506723404,0.15369786322116852,0.3828125,0.98577880859375
+input_layernorm,11,12,0.7078651685393258,0.4144736842105263,0.7078651685393258,True,0.16603894531726837,0.1900545060634613,0.34375,0.5546875
+input_layernorm,11,13,0.7368421052631579,0.42424242424242425,0.7368421052631579,True,0.16603894531726837,0.16846242547035217,0.34375,0.5703125
+input_layernorm,11,14,1.1420454545454546,0.53315649867374,1.1420454545454546,True,0.16603894531726837,0.11825777590274811,0.34375,0.6875
+input_layernorm,11,15,2.3516483516483517,0.7016393442622951,2.3516483516483517,True,0.16603894531726837,0.06029079854488373,0.34375,0.779296875
+input_layernorm,11,16,1.414745145631068,0.5858776228169368,1.414745145631068,True,0.16603894531726837,0.09195870161056519,0.34375,0.7763671875
+input_layernorm,11,17,2.2244318181818183,0.6898678414096916,2.2244318181818183,True,0.16603894531726837,0.06483674794435501,0.34375,1.0012359619140625
+input_layernorm,11,18,2.778409090909091,0.7353383458646616,2.778409090909091,True,0.16603894531726837,0.06785038113594055,0.34375,1.1640625
+input_layernorm,11,19,3.278409090909091,0.7662682602921647,3.278409090909091,True,0.16603894531726837,0.19409698247909546,0.34375,0.884765625
+input_layernorm,11,20,3.056603773584906,0.7534883720930232,3.056603773584906,True,0.16603894531726837,0.08625888079404831,0.34375,1.0262451171875
+input_layernorm,11,21,3.4831460674157304,0.7769423558897243,3.4831460674157304,True,0.16603894531726837,0.07095508277416229,0.34375,1.043212890625
+input_layernorm,11,22,3.215909090909091,0.7628032345013477,3.215909090909091,True,0.16603894531726837,0.07214703410863876,0.34375,1.044921875
+input_layernorm,11,23,4.142857142857143,0.8055555555555556,4.142857142857143,True,0.16603894531726837,0.15369786322116852,0.34375,0.98577880859375
+input_layernorm,12,13,0.10185185185185185,0.1134020618556701,0.1134020618556701,True,0.1900545060634613,0.16846242547035217,0.5546875,0.5703125
+input_layernorm,12,14,0.3644343302990897,0.26709554443650224,0.3644343302990897,True,0.1900545060634613,0.11825777590274811,0.5546875,0.6875
+input_layernorm,12,15,1.0359589041095891,0.5088309503784693,1.0359589041095891,True,0.1900545060634613,0.06029079854488373,0.5546875,0.779296875
+input_layernorm,12,16,0.6295774647887324,0.3863439930855661,0.6295774647887324,True,0.1900545060634613,0.09195870161056519,0.5546875,0.7763671875
+input_layernorm,12,17,0.9982394366197183,0.4995594713656388,0.9982394366197183,True,0.1900545060634613,0.06483674794435501,0.5546875,1.0012359619140625
+input_layernorm,12,18,1.341549295774648,0.5729323308270676,1.341549295774648,True,0.1900545060634613,0.06785038113594055,0.5546875,1.1640625
+input_layernorm,12,19,1.6514084507042253,0.6228419654714475,1.6514084507042253,True,0.1900545060634613,0.19409698247909546,0.5546875,0.884765625
+input_layernorm,12,20,1.5294117647058822,0.6046511627906976,1.5294117647058822,True,0.1900545060634613,0.08625888079404831,0.5546875,1.0262451171875
+input_layernorm,12,21,1.7876712328767124,0.6412776412776413,1.7876712328767124,True,0.1900545060634613,0.07095508277416229,0.5546875,1.043212890625
+input_layernorm,12,22,1.6267123287671232,0.6192959582790091,1.6267123287671232,True,0.1900545060634613,0.07214703410863876,0.5546875,1.044921875
+input_layernorm,12,23,2.2054794520547945,0.688034188034188,2.2054794520547945,True,0.1900545060634613,0.15369786322116852,0.5546875,0.98577880859375
+input_layernorm,13,14,0.3580163043478261,0.263631815907954,0.3580163043478261,True,0.16846242547035217,0.11825777590274811,0.5703125,0.6875
+input_layernorm,13,15,1.0890410958904109,0.521311475409836,1.0890410958904109,True,0.16846242547035217,0.06029079854488373,0.5703125,0.779296875
+input_layernorm,13,16,0.6413043478260869,0.39072847682119205,0.6413043478260869,True,0.16846242547035217,0.09195870161056519,0.5703125,0.7763671875
+input_layernorm,13,17,0.9434931506849316,0.48546255506607927,0.9434931506849316,True,0.16846242547035217,0.06483674794435501,0.5703125,1.0012359619140625
+input_layernorm,13,18,1.2773972602739727,0.5609022556390978,1.2773972602739727,True,0.16846242547035217,0.06785038113594055,0.5703125,1.1640625
+input_layernorm,13,19,1.5787671232876712,0.6122177954847278,1.5787671232876712,True,0.16846242547035217,0.19409698247909546,0.5703125,0.884765625
+input_layernorm,13,20,1.457142857142857,0.5930232558139535,1.457142857142857,True,0.16846242547035217,0.08625888079404831,0.5703125,1.0262451171875
+input_layernorm,13,21,1.759322033898305,0.6375921375921376,1.759322033898305,True,0.16846242547035217,0.07095508277416229,0.5703125,1.043212890625
+input_layernorm,13,22,1.6,0.6153846153846154,1.6,True,0.16846242547035217,0.07214703410863876,0.5703125,1.044921875
+input_layernorm,13,23,2.1728813559322036,0.6848290598290598,2.1728813559322036,True,0.16846242547035217,0.15369786322116852,0.5703125,0.98577880859375
+input_layernorm,14,15,0.6661931818181818,0.667953667953668,0.667953667953668,True,0.11825777590274811,0.06029079854488373,0.6875,0.779296875
+input_layernorm,14,16,0.4027630180658874,0.2871212121212121,0.4027630180658874,True,0.11825777590274811,0.09195870161056519,0.6875,0.7763671875
+input_layernorm,14,17,0.640807651434644,0.3905440414507772,0.640807651434644,True,0.11825777590274811,0.06483674794435501,0.6875,1.0012359619140625
+input_layernorm,14,18,0.8575982996811902,0.46167048054919907,0.8575982996811902,True,0.11825777590274811,0.06785038113594055,0.6875,1.1640625
+input_layernorm,14,19,1.1590909090909092,0.5774647887323944,1.1590909090909092,True,0.11825777590274811,0.19409698247909546,0.6875,0.884765625
+input_layernorm,14,20,1.107843137254902,0.5255813953488372,1.107843137254902,True,0.11825777590274811,0.08625888079404831,0.6875,1.0262451171875
+input_layernorm,14,21,1.2613636363636365,0.5577889447236181,1.2613636363636365,True,0.11825777590274811,0.07095508277416229,0.6875,1.043212890625
+input_layernorm,14,22,1.0710227272727273,0.5171467764060357,1.0710227272727273,True,0.11825777590274811,0.07214703410863876,0.6875,1.044921875
+input_layernorm,14,23,1.5056818181818181,0.6009070294784581,1.5056818181818181,True,0.11825777590274811,0.15369786322116852,0.6875,0.98577880859375
+input_layernorm,15,16,0.8797573083287369,0.4993757802746567,0.8797573083287369,True,0.06029079854488373,0.09195870161056519,0.779296875,0.7763671875
+input_layernorm,15,17,0.732297852900868,0.42273206751054854,0.732297852900868,True,0.06029079854488373,0.06483674794435501,0.779296875,1.0012359619140625
+input_layernorm,15,18,0.8297872340425532,0.45348837209302323,0.8297872340425532,True,0.06029079854488373,0.06785038113594055,0.779296875,1.1640625
+input_layernorm,15,19,1.0300751879699248,0.5074074074074074,1.0300751879699248,True,0.06029079854488373,0.19409698247909546,0.779296875,0.884765625
+input_layernorm,15,20,1.0451127819548873,0.5110294117647058,1.0451127819548873,True,0.06029079854488373,0.08625888079404831,0.779296875,1.0262451171875
+input_layernorm,15,21,1.150375939849624,0.534965034965035,1.150375939849624,True,0.06029079854488373,0.07095508277416229,0.779296875,1.043212890625
+input_layernorm,15,22,1.005012531328321,0.50125,1.005012531328321,True,0.06029079854488373,0.07214703410863876,0.779296875,1.044921875
+input_layernorm,15,23,1.100250626566416,0.5238663484486874,1.100250626566416,True,0.06029079854488373,0.15369786322116852,0.779296875,0.98577880859375
+input_layernorm,16,17,0.3909313725490196,0.28105726872246695,0.3909313725490196,True,0.09195870161056519,0.06483674794435501,0.7763671875,1.0012359619140625
+input_layernorm,16,18,0.6299019607843137,0.38646616541353385,0.6299019607843137,True,0.09195870161056519,0.06785038113594055,0.7763671875,1.1640625
+input_layernorm,16,19,0.8952618453865336,0.7777777777777778,0.8952618453865336,True,0.09195870161056519,0.19409698247909546,0.7763671875,0.884765625
+input_layernorm,16,20,0.7578027465667915,0.43110795454545453,0.7578027465667915,True,0.09195870161056519,0.08625888079404831,0.7763671875,1.0262451171875
+input_layernorm,16,21,1.0324594257178528,0.507985257985258,1.0324594257178528,True,0.09195870161056519,0.07095508277416229,0.7763671875,1.043212890625
+input_layernorm,16,22,0.9151061173533084,0.47783572359843546,0.9151061173533084,True,0.09195870161056519,0.07214703410863876,0.7763671875,1.044921875
+input_layernorm,16,23,1.3370786516853932,0.5721153846153846,1.3370786516853932,True,0.09195870161056519,0.15369786322116852,0.7763671875,0.98577880859375
+input_layernorm,17,18,0.20625,0.17098445595854922,0.20625,True,0.06483674794435501,0.06785038113594055,1.0012359619140625,1.1640625
+input_layernorm,17,19,0.4493804959080726,0.6191966614501826,0.6191966614501826,True,0.06483674794435501,0.19409698247909546,1.0012359619140625,0.884765625
+input_layernorm,17,20,0.35731807031888796,0.2632530120481928,0.35731807031888796,True,0.06483674794435501,0.08625888079404831,1.0012359619140625,1.0262451171875
+input_layernorm,17,21,0.5803900259614804,0.367244804400978,0.5803900259614804,True,0.06483674794435501,0.07095508277416229,1.0012359619140625,1.043212890625
+input_layernorm,17,22,0.4475112054729889,0.30915906127770537,0.4475112054729889,True,0.06483674794435501,0.07214703410863876,1.0012359619140625,1.044921875
+input_layernorm,17,23,0.7790511605224256,0.45824255628177196,0.7790511605224256,True,0.06483674794435501,0.15369786322116852,1.0012359619140625,0.98577880859375
+input_layernorm,18,19,0.40243142144638405,0.6734480959833072,0.6734480959833072,True,0.06785038113594055,0.19409698247909546,1.1640625,0.884765625
+input_layernorm,18,20,0.17917304747320062,0.19578313253012047,0.19578313253012047,True,0.06785038113594055,0.08625888079404831,1.1640625,1.0262451171875
+input_layernorm,18,21,0.3624161073825503,0.2660098522167488,0.3624161073825503,True,0.06785038113594055,0.07095508277416229,1.1640625,1.043212890625
+input_layernorm,18,22,0.22378199834847234,0.27217125382262997,0.27217125382262997,True,0.06785038113594055,0.07214703410863876,1.1640625,1.044921875
+input_layernorm,18,23,0.4929343308395677,0.5599088838268793,0.5599088838268793,True,0.06785038113594055,0.15369786322116852,1.1640625,0.98577880859375
+input_layernorm,19,20,0.702803274621682,0.4127331002331002,0.702803274621682,True,0.19409698247909546,0.08625888079404831,0.884765625,1.0262451171875
+input_layernorm,19,21,0.8282009724473258,0.4530141843971631,0.8282009724473258,True,0.19409698247909546,0.07095508277416229,0.884765625,1.043212890625
+input_layernorm,19,22,0.6790923824959482,0.40444015444015446,0.6790923824959482,True,0.19409698247909546,0.07214703410863876,0.884765625,1.044921875
+input_layernorm,19,23,0.34451054624121147,0.5234624145785877,0.5234624145785877,True,0.19409698247909546,0.15369786322116852,0.884765625,0.98577880859375
+input_layernorm,20,21,0.2359208523592085,0.19088669950738915,0.2359208523592085,True,0.08625888079404831,0.07095508277416229,1.0262451171875,1.043212890625
+input_layernorm,20,22,0.18092909535452323,0.2208955223880597,0.2208955223880597,True,0.08625888079404831,0.07214703410863876,1.0262451171875,1.044921875
+input_layernorm,20,23,0.3733133433283358,0.5809627347949508,0.5809627347949508,True,0.08625888079404831,0.15369786322116852,1.0262451171875,0.98577880859375
+input_layernorm,21,22,0.14822546972860126,0.17401960784313725,0.17401960784313725,True,0.07095508277416229,0.07214703410863876,1.043212890625,1.044921875
+input_layernorm,21,23,0.3882978723404255,0.6347826086956522,0.6347826086956522,True,0.07095508277416229,0.15369786322116852,1.043212890625,0.98577880859375
+input_layernorm,22,23,0.333976833976834,0.5014492753623189,0.5014492753623189,True,0.07214703410863876,0.15369786322116852,1.044921875,0.98577880859375
+post_attention_layernorm,0,1,129.5,0.9923371647509579,129.5,True,0.108433797955513,0.09410307556390762,0.00390625,0.45703125
+post_attention_layernorm,0,2,154.0,0.9935483870967742,154.0,True,0.108433797955513,0.10385182499885559,0.00390625,0.533203125
+post_attention_layernorm,0,3,165.5,0.993993993993994,165.5,True,0.108433797955513,0.11546041816473007,0.00390625,0.48828125
+post_attention_layernorm,0,4,181.5,0.9945205479452055,181.5,True,0.108433797955513,0.11874310672283173,0.00390625,0.4765625
+post_attention_layernorm,0,5,194.25,0.9948783610755442,194.25,True,0.108433797955513,0.14567376673221588,0.00390625,0.421875
+post_attention_layernorm,0,6,221.0,0.9954954954954955,221.0,True,0.108433797955513,0.13559450209140778,0.00390625,0.43359375
+post_attention_layernorm,0,7,188.5,0.9947229551451188,188.5,True,0.108433797955513,0.17147497832775116,0.00390625,0.375
+post_attention_layernorm,0,8,187.0,0.9946808510638298,187.0,True,0.108433797955513,0.18255221843719482,0.00390625,0.33984375
+post_attention_layernorm,0,9,185.0,0.9946236559139785,185.0,True,0.108433797955513,0.17869991064071655,0.00390625,0.3515625
+post_attention_layernorm,0,10,215.75,0.9953863898500577,215.75,True,0.108433797955513,0.16611140966415405,0.00390625,0.36328125
+post_attention_layernorm,0,11,195.75,0.9949174078780177,195.75,True,0.108433797955513,0.1528426557779312,0.00390625,0.41796875
+post_attention_layernorm,0,12,188.0,0.9947089947089947,188.0,True,0.108433797955513,0.12497646361589432,0.00390625,0.5234375
+post_attention_layernorm,0,13,199.25,0.9950062421972534,199.25,True,0.108433797955513,0.09632012248039246,0.00390625,0.619140625
+post_attention_layernorm,0,14,228.875,0.9956498096791735,228.875,True,0.108433797955513,0.053455036133527756,0.00390625,0.87646484375
+post_attention_layernorm,0,15,197.25,0.9949558638083228,197.25,True,0.108433797955513,0.06801187992095947,0.00390625,0.736328125
+post_attention_layernorm,0,16,189.5,0.994750656167979,189.5,True,0.108433797955513,0.08680811524391174,0.00390625,0.72265625
+post_attention_layernorm,0,17,181.5,0.9945205479452055,181.5,True,0.108433797955513,0.09242258220911026,0.00390625,0.712890625
+post_attention_layernorm,0,18,224.875,0.9955727725511898,224.875,True,0.108433797955513,0.09676424413919449,0.00390625,0.8095703125
+post_attention_layernorm,0,19,233.5,0.9957356076759062,233.5,True,0.108433797955513,0.07613449543714523,0.00390625,0.81640625
+post_attention_layernorm,0,20,209.5,0.995249406175772,209.5,True,0.108433797955513,0.07808392494916916,0.00390625,0.822265625
+post_attention_layernorm,0,21,220.0,0.995475113122172,220.0,True,0.108433797955513,0.06570179015398026,0.00390625,0.86328125
+post_attention_layernorm,0,22,248.5625,0.995992987728525,248.5625,True,0.108433797955513,0.05012959614396095,0.00390625,0.9713134765625
+post_attention_layernorm,0,23,258.484375,0.9961462034082014,258.484375,True,0.108433797955513,0.06521295011043549,0.00390625,1.01361083984375
+post_attention_layernorm,1,2,0.42512908777969016,0.29830917874396135,0.42512908777969016,True,0.09410307556390762,0.10385182499885559,0.45703125,0.533203125
+post_attention_layernorm,1,3,0.27586206896551724,0.26858457997698504,0.27586206896551724,True,0.09410307556390762,0.11546041816473007,0.45703125,0.48828125
+post_attention_layernorm,1,4,0.39846743295019155,0.28493150684931506,0.39846743295019155,True,0.09410307556390762,0.11874310672283173,0.45703125,0.4765625
+post_attention_layernorm,1,5,0.49616858237547895,0.33162612035851474,0.49616858237547895,True,0.09410307556390762,0.14567376673221588,0.45703125,0.421875
+post_attention_layernorm,1,6,0.7011494252873564,0.41216216216216217,0.7011494252873564,True,0.09410307556390762,0.13559450209140778,0.45703125,0.43359375
+post_attention_layernorm,1,7,0.4521072796934866,0.45161290322580644,0.4521072796934866,True,0.09410307556390762,0.17147497832775116,0.45703125,0.375
+post_attention_layernorm,1,8,0.5296052631578947,0.594488188976378,0.594488188976378,True,0.09410307556390762,0.18255221843719482,0.45703125,0.33984375
+post_attention_layernorm,1,9,0.42528735632183906,0.5133079847908745,0.5133079847908745,True,0.09410307556390762,0.17869991064071655,0.45703125,0.3515625
+post_attention_layernorm,1,10,0.6609195402298851,0.5073260073260073,0.6609195402298851,True,0.09410307556390762,0.16611140966415405,0.45703125,0.36328125
+post_attention_layernorm,1,11,0.5076628352490421,0.3785807291666667,0.5076628352490421,True,0.09410307556390762,0.1528426557779312,0.45703125,0.41796875
+post_attention_layernorm,1,12,0.7397203947368421,0.4274732620320856,0.7397203947368421,True,0.09410307556390762,0.12497646361589432,0.45703125,0.5234375
+post_attention_layernorm,1,13,0.8725328947368421,0.46596398770311814,0.8725328947368421,True,0.09410307556390762,0.09632012248039246,0.45703125,0.619140625
+post_attention_layernorm,1,14,1.08739406779661,0.5209337731540218,1.08739406779661,True,0.09410307556390762,0.053455036133527756,0.45703125,0.87646484375
+post_attention_layernorm,1,15,1.7415254237288136,0.6352395672333848,1.7415254237288136,True,0.09410307556390762,0.06801187992095947,0.45703125,0.736328125
+post_attention_layernorm,1,16,0.9901315789473685,0.4975206611570248,0.9901315789473685,True,0.09410307556390762,0.08680811524391174,0.45703125,0.72265625
+post_attention_layernorm,1,17,1.009733606557377,0.5024216161101198,1.009733606557377,True,0.09410307556390762,0.09242258220911026,0.45703125,0.712890625
+post_attention_layernorm,1,18,1.194078947368421,0.5442278860569715,1.194078947368421,True,0.09410307556390762,0.09676424413919449,0.45703125,0.8095703125
+post_attention_layernorm,1,19,1.5277777777777777,0.6043956043956044,1.5277777777777777,True,0.09410307556390762,0.07613449543714523,0.45703125,0.81640625
+post_attention_layernorm,1,20,1.3644067796610169,0.5770609318996416,1.3644067796610169,True,0.09410307556390762,0.07808392494916916,0.45703125,0.822265625
+post_attention_layernorm,1,21,1.7166666666666666,0.6319018404907976,1.7166666666666666,True,0.09410307556390762,0.06570179015398026,0.45703125,0.86328125
+post_attention_layernorm,1,22,2.377049180327869,0.7038834951456311,2.377049180327869,True,0.09410307556390762,0.05012959614396095,0.45703125,0.9713134765625
+post_attention_layernorm,1,23,2.6324786324786325,0.7247058823529412,2.6324786324786325,True,0.09410307556390762,0.06521295011043549,0.45703125,1.01361083984375
+post_attention_layernorm,2,3,0.3058252427184466,0.4405594405594406,0.4405594405594406,True,0.10385182499885559,0.11546041816473007,0.533203125,0.48828125
+post_attention_layernorm,2,4,0.3094660194174757,0.44815465729349735,0.44815465729349735,True,0.10385182499885559,0.11874310672283173,0.533203125,0.4765625
+post_attention_layernorm,2,5,0.3109903381642512,0.45135845749342685,0.45135845749342685,True,0.10385182499885559,0.14567376673221588,0.533203125,0.421875
+post_attention_layernorm,2,6,0.432258064516129,0.5354659248956884,0.5354659248956884,True,0.10385182499885559,0.13559450209140778,0.533203125,0.43359375
+post_attention_layernorm,2,7,0.3689613526570048,0.584688995215311,0.584688995215311,True,0.10385182499885559,0.17147497832775116,0.533203125,0.375
+post_attention_layernorm,2,8,0.42462600690448793,0.738,0.738,True,0.10385182499885559,0.18255221843719482,0.533203125,0.33984375
+post_attention_layernorm,2,9,0.42002301495972383,0.7242063492063492,0.7242063492063492,True,0.10385182499885559,0.17869991064071655,0.533203125,0.3515625
+post_attention_layernorm,2,10,0.414368739953011,0.7075591216216216,0.7075591216216216,True,0.10385182499885559,0.16611140966415405,0.533203125,0.36328125
+post_attention_layernorm,2,11,0.37333937198067635,0.5957600578173934,0.5957600578173934,True,0.10385182499885559,0.1528426557779312,0.533203125,0.41796875
+post_attention_layernorm,2,12,0.46098066298342544,0.5644780349551252,0.5644780349551252,True,0.10385182499885559,0.12497646361589432,0.533203125,0.5234375
+post_attention_layernorm,2,13,0.5725138121546961,0.52416014726185,0.5725138121546961,True,0.10385182499885559,0.09632012248039246,0.533203125,0.619140625
+post_attention_layernorm,2,14,0.8044871794871795,0.4500659920809503,0.8044871794871795,True,0.10385182499885559,0.053455036133527756,0.533203125,0.87646484375
+post_attention_layernorm,2,15,1.36996336996337,0.5780525502318392,1.36996336996337,True,0.10385182499885559,0.06801187992095947,0.533203125,0.736328125
+post_attention_layernorm,2,16,0.6712707182320442,0.47909888895709285,0.6712707182320442,True,0.10385182499885559,0.08680811524391174,0.533203125,0.72265625
+post_attention_layernorm,2,17,0.7513489208633094,0.4290115532734275,0.7513489208633094,True,0.10385182499885559,0.09242258220911026,0.533203125,0.712890625
+post_attention_layernorm,2,18,0.8425414364640884,0.4572713643178411,0.8425414364640884,True,0.10385182499885559,0.09676424413919449,0.533203125,0.8095703125
+post_attention_layernorm,2,19,1.1666666666666667,0.5384615384615384,1.1666666666666667,True,0.10385182499885559,0.07613449543714523,0.533203125,0.81640625
+post_attention_layernorm,2,20,1.043956043956044,0.510752688172043,1.043956043956044,True,0.10385182499885559,0.07808392494916916,0.533203125,0.822265625
+post_attention_layernorm,2,21,1.381294964028777,0.5800604229607251,1.381294964028777,True,0.10385182499885559,0.06570179015398026,0.533203125,0.86328125
+post_attention_layernorm,2,22,1.9640287769784173,0.662621359223301,1.9640287769784173,True,0.10385182499885559,0.05012959614396095,0.533203125,0.9713134765625
+post_attention_layernorm,2,23,2.1135531135531136,0.6788235294117647,2.1135531135531136,True,0.10385182499885559,0.06521295011043549,0.533203125,1.01361083984375
+post_attention_layernorm,3,4,0.1832089552238806,0.22430333485609869,0.22430333485609869,True,0.11546041816473007,0.11874310672283173,0.48828125,0.4765625
+post_attention_layernorm,3,5,0.1939149560117302,0.2405638926784902,0.2405638926784902,True,0.11546041816473007,0.14567376673221588,0.48828125,0.421875
+post_attention_layernorm,3,6,0.3333333333333333,0.25,0.3333333333333333,True,0.11546041816473007,0.13559450209140778,0.48828125,0.43359375
+post_attention_layernorm,3,7,0.2893772893772894,0.4072164948453608,0.4072164948453608,True,0.11546041816473007,0.17147497832775116,0.48828125,0.375
+post_attention_layernorm,3,8,0.34798534798534797,0.5337078651685393,0.5337078651685393,True,0.11546041816473007,0.18255221843719482,0.48828125,0.33984375
+post_attention_layernorm,3,9,0.34065934065934067,0.5166666666666667,0.5166666666666667,True,0.11546041816473007,0.17869991064071655,0.48828125,0.3515625
+post_attention_layernorm,3,10,0.3333333333333333,0.5,0.5,True,0.11546041816473007,0.16611140966415405,0.48828125,0.36328125
+post_attention_layernorm,3,11,0.2514124293785311,0.28756009615384615,0.28756009615384615,True,0.11546041816473007,0.1528426557779312,0.48828125,0.41796875
+post_attention_layernorm,3,12,0.43326558265582654,0.3787001287001287,0.43326558265582654,True,0.11546041816473007,0.12497646361589432,0.48828125,0.5234375
+post_attention_layernorm,3,13,0.5426829268292683,0.35177865612648224,0.5426829268292683,True,0.11546041816473007,0.09632012248039246,0.48828125,0.619140625
+post_attention_layernorm,3,14,0.902027027027027,0.47424511545293074,0.902027027027027,True,0.11546041816473007,0.053455036133527756,0.48828125,0.87646484375
+post_attention_layernorm,3,15,1.56,0.609375,1.56,True,0.11546041816473007,0.06801187992095947,0.48828125,0.736328125
+post_attention_layernorm,3,16,0.8092643051771117,0.44728915662650603,0.8092643051771117,True,0.11546041816473007,0.08680811524391174,0.48828125,0.72265625
+post_attention_layernorm,3,17,0.8933397683397684,0.47183278103492227,0.8933397683397684,True,0.11546041816473007,0.09242258220911026,0.48828125,0.712890625
+post_attention_layernorm,3,18,0.9215,0.47957325006505336,0.9215,True,0.11546041816473007,0.09676424413919449,0.48828125,0.8095703125
+post_attention_layernorm,3,19,1.366,0.577345731191885,1.366,True,0.11546041816473007,0.07613449543714523,0.48828125,0.81640625
+post_attention_layernorm,3,20,1.169,0.5389580451821115,1.169,True,0.11546041816473007,0.07808392494916916,0.48828125,0.822265625
+post_attention_layernorm,3,21,1.5396825396825398,0.60625,1.5396825396825398,True,0.11546041816473007,0.06570179015398026,0.48828125,0.86328125
+post_attention_layernorm,3,22,2.2698412698412698,0.6941747572815534,2.2698412698412698,True,0.11546041816473007,0.05012959614396095,0.48828125,0.9713134765625
+post_attention_layernorm,3,23,2.4,0.7058823529411765,2.4,True,0.11546041816473007,0.06521295011043549,0.48828125,1.01361083984375
+post_attention_layernorm,4,5,0.16475095785440613,0.19724770642201836,0.19724770642201836,True,0.11874310672283173,0.14567376673221588,0.4765625,0.421875
+post_attention_layernorm,4,6,0.2548342541436464,0.2030820033021464,0.2548342541436464,True,0.11874310672283173,0.13559450209140778,0.4765625,0.43359375
+post_attention_layernorm,4,7,0.2785923753665689,0.3861788617886179,0.3861788617886179,True,0.11874310672283173,0.17147497832775116,0.4765625,0.375
+post_attention_layernorm,4,8,0.3253731343283582,0.4823008849557522,0.4823008849557522,True,0.11874310672283173,0.18255221843719482,0.4765625,0.33984375
+post_attention_layernorm,4,9,0.3128491620111732,0.45528455284552843,0.45528455284552843,True,0.11874310672283173,0.17869991064071655,0.4765625,0.3515625
+post_attention_layernorm,4,10,0.30434782608695654,0.4375,0.4375,True,0.11874310672283173,0.16611140966415405,0.4765625,0.36328125
+post_attention_layernorm,4,11,0.2160188457008245,0.27554086538461536,0.27554086538461536,True,0.11874310672283173,0.1528426557779312,0.4765625,0.41796875
+post_attention_layernorm,4,12,0.3737012987012987,0.3658301158301158,0.3737012987012987,True,0.11874310672283173,0.12497646361589432,0.4765625,0.5234375
+post_attention_layernorm,4,13,0.4785714285714286,0.32367149758454106,0.4785714285714286,True,0.11874310672283173,0.09632012248039246,0.4765625,0.619140625
+post_attention_layernorm,4,14,0.9615384615384616,0.49019607843137253,0.9615384615384616,True,0.11874310672283173,0.053455036133527756,0.4765625,0.87646484375
+post_attention_layernorm,4,15,1.6598360655737705,0.6240369799691834,1.6598360655737705,True,0.11874310672283173,0.06801187992095947,0.4765625,0.736328125
+post_attention_layernorm,4,16,0.7994579945799458,0.4442771084337349,0.7994579945799458,True,0.11874310672283173,0.08680811524391174,0.4765625,0.72265625
+post_attention_layernorm,4,17,0.9569672131147541,0.48900523560209425,0.9569672131147541,True,0.11874310672283173,0.09242258220911026,0.4765625,0.712890625
+post_attention_layernorm,4,18,1.0363729508196722,0.5089308176100629,1.0363729508196722,True,0.11874310672283173,0.09676424413919449,0.4765625,0.8095703125
+post_attention_layernorm,4,19,1.4246031746031746,0.5875613747954174,1.4246031746031746,True,0.11874310672283173,0.07613449543714523,0.4765625,0.81640625
+post_attention_layernorm,4,20,1.2930327868852458,0.5638963360142984,1.2930327868852458,True,0.11874310672283173,0.07808392494916916,0.4765625,0.822265625
+post_attention_layernorm,4,21,1.5806451612903225,0.6125,1.5806451612903225,True,0.11874310672283173,0.06570179015398026,0.4765625,0.86328125
+post_attention_layernorm,4,22,2.3225806451612905,0.6990291262135923,2.3225806451612905,True,0.11874310672283173,0.05012959614396095,0.4765625,0.9713134765625
+post_attention_layernorm,4,23,2.373015873015873,0.7035294117647058,2.373015873015873,True,0.11874310672283173,0.06521295011043549,0.4765625,1.01361083984375
+post_attention_layernorm,5,6,0.32515337423312884,0.24537037037037038,0.32515337423312884,True,0.14567376673221588,0.13559450209140778,0.421875,0.43359375
+post_attention_layernorm,5,7,0.1608832807570978,0.19172932330827067,0.19172932330827067,True,0.14567376673221588,0.17147497832775116,0.421875,0.375
+post_attention_layernorm,5,8,0.2413793103448276,0.3181818181818182,0.3181818181818182,True,0.14567376673221588,0.18255221843719482,0.421875,0.33984375
+post_attention_layernorm,5,9,0.24968314321926488,0.3327702702702703,0.3327702702702703,True,0.14567376673221588,0.17869991064071655,0.421875,0.3515625
+post_attention_layernorm,5,10,0.2541436464088398,0.30067567567567566,0.30067567567567566,True,0.14567376673221588,0.16611140966415405,0.421875,0.36328125
+post_attention_layernorm,5,11,0.214,0.23786808894230768,0.23786808894230768,True,0.14567376673221588,0.1528426557779312,0.421875,0.41796875
+post_attention_layernorm,5,12,0.4103333333333333,0.3254906692406692,0.4103333333333333,True,0.14567376673221588,0.12497646361589432,0.421875,0.5234375
+post_attention_layernorm,5,13,0.555045871559633,0.35693215339233036,0.555045871559633,True,0.14567376673221588,0.09632012248039246,0.421875,0.619140625
+post_attention_layernorm,5,14,1.275092936802974,0.5604575163398693,1.275092936802974,True,0.14567376673221588,0.053455036133527756,0.421875,0.87646484375
+post_attention_layernorm,5,15,1.962962962962963,0.6625,1.962962962962963,True,0.14567376673221588,0.06801187992095947,0.421875,0.736328125
+post_attention_layernorm,5,16,1.063197026022305,0.5153153153153153,1.063197026022305,True,0.14567376673221588,0.08680811524391174,0.421875,0.72265625
+post_attention_layernorm,5,17,1.2333715596330275,0.5522464698331194,1.2333715596330275,True,0.14567376673221588,0.09242258220911026,0.421875,0.712890625
+post_attention_layernorm,5,18,1.2573327850877194,0.5569993017395792,1.2573327850877194,True,0.14567376673221588,0.09676424413919449,0.421875,0.8095703125
+post_attention_layernorm,5,19,1.738532110091743,0.6348408710217756,1.738532110091743,True,0.14567376673221588,0.07613449543714523,0.421875,0.81640625
+post_attention_layernorm,5,20,1.5136363636363637,0.6021699819168174,1.5136363636363637,True,0.14567376673221588,0.07808392494916916,0.421875,0.822265625
+post_attention_layernorm,5,21,2.036697247706422,0.6706948640483383,2.036697247706422,True,0.14567376673221588,0.06570179015398026,0.421875,0.86328125
+post_attention_layernorm,5,22,2.779816513761468,0.7354368932038835,2.779816513761468,True,0.14567376673221588,0.05012959614396095,0.421875,0.9713134765625
+post_attention_layernorm,5,23,2.935185185185185,0.7458823529411764,2.935185185185185,True,0.14567376673221588,0.06521295011043549,0.421875,1.01361083984375
+post_attention_layernorm,6,7,0.3519163763066202,0.543010752688172,0.543010752688172,True,0.13559450209140778,0.17147497832775116,0.43359375,0.375
+post_attention_layernorm,6,8,0.4122905027932961,0.7015209125475285,0.7015209125475285,True,0.13559450209140778,0.18255221843719482,0.43359375,0.33984375
+post_attention_layernorm,6,9,0.35195530726256985,0.5431034482758621,0.5431034482758621,True,0.13559450209140778,0.17869991064071655,0.43359375,0.3515625
+post_attention_layernorm,6,10,0.3743016759776536,0.5982142857142857,0.5982142857142857,True,0.13559450209140778,0.16611140966415405,0.43359375,0.36328125
+post_attention_layernorm,6,11,0.2961672473867596,0.4207920792079208,0.4207920792079208,True,0.13559450209140778,0.1528426557779312,0.43359375,0.41796875
+post_attention_layernorm,6,12,0.3181818181818182,0.29812834224598933,0.3181818181818182,True,0.13559450209140778,0.12497646361589432,0.43359375,0.5234375
+post_attention_layernorm,6,13,0.5526315789473685,0.3559322033898305,0.5526315789473685,True,0.13559450209140778,0.09632012248039246,0.43359375,0.619140625
+post_attention_layernorm,6,14,1.1714801444043321,0.5394846217788861,1.1714801444043321,True,0.13559450209140778,0.053455036133527756,0.43359375,0.87646484375
+post_attention_layernorm,6,15,1.927927927927928,0.6584615384615384,1.927927927927928,True,0.13559450209140778,0.06801187992095947,0.43359375,0.736328125
+post_attention_layernorm,6,16,0.9859913793103449,0.4964731416169289,0.9859913793103449,True,0.13559450209140778,0.08680811524391174,0.43359375,0.72265625
+post_attention_layernorm,6,17,1.2088963963963963,0.5472852408870762,1.2088963963963963,True,0.13559450209140778,0.09242258220911026,0.43359375,0.712890625
+post_attention_layernorm,6,18,1.2401463963963963,0.5536006032424281,1.2401463963963963,True,0.13559450209140778,0.09676424413919449,0.43359375,0.8095703125
+post_attention_layernorm,6,19,1.6644144144144144,0.6246830092983939,1.6644144144144144,True,0.13559450209140778,0.07613449543714523,0.43359375,0.81640625
+post_attention_layernorm,6,20,1.5202702702702702,0.6032171581769437,1.5202702702702702,True,0.13559450209140778,0.07808392494916916,0.43359375,0.822265625
+post_attention_layernorm,6,21,1.8782608695652174,0.6525679758308157,1.8782608695652174,True,0.13559450209140778,0.06570179015398026,0.43359375,0.86328125
+post_attention_layernorm,6,22,2.6140350877192984,0.7233009708737864,2.6140350877192984,True,0.13559450209140778,0.05012959614396095,0.43359375,0.9713134765625
+post_attention_layernorm,6,23,2.828828828828829,0.7388235294117647,2.828828828828829,True,0.13559450209140778,0.06521295011043549,0.43359375,1.01361083984375
+post_attention_layernorm,7,8,0.19383825417201542,0.16236559139784945,0.19383825417201542,True,0.17147497832775116,0.18255221843719482,0.375,0.33984375
+post_attention_layernorm,7,9,0.12010443864229765,0.13649851632047477,0.13649851632047477,True,0.17147497832775116,0.17869991064071655,0.375,0.3515625
+post_attention_layernorm,7,10,0.15468549422336328,0.1339633129516398,0.15468549422336328,True,0.17147497832775116,0.16611140966415405,0.375,0.36328125
+post_attention_layernorm,7,11,0.23232323232323232,0.1885245901639344,0.23232323232323232,True,0.17147497832775116,0.1528426557779312,0.375,0.41796875
+post_attention_layernorm,7,12,0.5047169811320755,0.335423197492163,0.5047169811320755,True,0.17147497832775116,0.12497646361589432,0.375,0.5234375
+post_attention_layernorm,7,13,0.7424242424242424,0.4260869565217391,0.7424242424242424,True,0.17147497832775116,0.09632012248039246,0.375,0.619140625
+post_attention_layernorm,7,14,1.5932203389830508,0.6143790849673203,1.5932203389830508,True,0.17147497832775116,0.053455036133527756,0.375,0.87646484375
+post_attention_layernorm,7,15,2.3645833333333335,0.7027863777089783,2.3645833333333335,True,0.17147497832775116,0.06801187992095947,0.375,0.736328125
+post_attention_layernorm,7,16,1.3799283154121864,0.5798192771084337,1.3799283154121864,True,0.17147497832775116,0.08680811524391174,0.375,0.72265625
+post_attention_layernorm,7,17,1.5019132653061225,0.6003058883507519,1.5019132653061225,True,0.17147497832775116,0.09242258220911026,0.375,0.712890625
+post_attention_layernorm,7,18,1.579954954954955,0.6123963334788302,1.579954954954955,True,0.17147497832775116,0.09676424413919449,0.375,0.8095703125
+post_attention_layernorm,7,19,2.085858585858586,0.6759410801963993,2.085858585858586,True,0.17147497832775116,0.07613449543714523,0.375,0.81640625
+post_attention_layernorm,7,20,1.884020618556701,0.6532618409294012,1.884020618556701,True,0.17147497832775116,0.07808392494916916,0.375,0.822265625
+post_attention_layernorm,7,21,2.377551020408163,0.7039274924471299,2.377551020408163,True,0.17147497832775116,0.06570179015398026,0.375,0.86328125
+post_attention_layernorm,7,22,3.2916666666666665,0.7669902912621359,3.2916666666666665,True,0.17147497832775116,0.05012959614396095,0.375,0.9713134765625
+post_attention_layernorm,7,23,3.381443298969072,0.7717647058823529,3.381443298969072,True,0.17147497832775116,0.06521295011043549,0.375,1.01361083984375
+post_attention_layernorm,8,9,0.1510791366906475,0.13125,0.1510791366906475,True,0.18255221843719482,0.17869991064071655,0.33984375,0.3515625
+post_attention_layernorm,8,10,0.17464424320827943,0.1486784140969163,0.17464424320827943,True,0.18255221843719482,0.16611140966415405,0.33984375,0.36328125
+post_attention_layernorm,8,11,0.28421052631578947,0.22131147540983606,0.28421052631578947,True,0.18255221843719482,0.1528426557779312,0.33984375,0.41796875
+post_attention_layernorm,8,12,0.6377551020408163,0.3894080996884735,0.6377551020408163,True,0.18255221843719482,0.12497646361589432,0.33984375,0.5234375
+post_attention_layernorm,8,13,0.8956043956043956,0.47246376811594204,0.8956043956043956,True,0.18255221843719482,0.09632012248039246,0.33984375,0.619140625
+post_attention_layernorm,8,14,1.7567567567567568,0.6372549019607843,1.7567567567567568,True,0.18255221843719482,0.053455036133527756,0.33984375,0.87646484375
+post_attention_layernorm,8,15,2.7011494252873565,0.7298136645962733,2.7011494252873565,True,0.18255221843719482,0.06801187992095947,0.33984375,0.736328125
+post_attention_layernorm,8,16,1.6141732283464567,0.6174698795180723,1.6141732283464567,True,0.18255221843719482,0.08680811524391174,0.33984375,0.72265625
+post_attention_layernorm,8,17,1.7862215909090908,0.6410910017843487,1.7862215909090908,True,0.18255221843719482,0.09242258220911026,0.33984375,0.712890625
+post_attention_layernorm,8,18,1.8256392045454546,0.6460977755435465,1.8256392045454546,True,0.18255221843719482,0.09676424413919449,0.33984375,0.8095703125
+post_attention_layernorm,8,19,2.357142857142857,0.7021276595744681,2.357142857142857,True,0.18255221843719482,0.07613449543714523,0.33984375,0.81640625
+post_attention_layernorm,8,20,2.143258426966292,0.6818588025022342,2.143258426966292,True,0.18255221843719482,0.07808392494916916,0.33984375,0.822265625
+post_attention_layernorm,8,21,2.67816091954023,0.728125,2.67816091954023,True,0.18255221843719482,0.06570179015398026,0.33984375,0.86328125
+post_attention_layernorm,8,22,3.735632183908046,0.7888349514563107,3.735632183908046,True,0.18255221843719482,0.05012959614396095,0.33984375,0.9713134765625
+post_attention_layernorm,8,23,3.7222222222222223,0.788235294117647,3.7222222222222223,True,0.18255221843719482,0.06521295011043549,0.33984375,1.01361083984375
+post_attention_layernorm,9,10,0.16532258064516128,0.14186851211072665,0.16532258064516128,True,0.17869991064071655,0.16611140966415405,0.3515625,0.36328125
+post_attention_layernorm,9,11,0.2708333333333333,0.21311475409836064,0.2708333333333333,True,0.17869991064071655,0.1528426557779312,0.3515625,0.41796875
+post_attention_layernorm,9,12,0.5735294117647058,0.3644859813084112,0.5735294117647058,True,0.17869991064071655,0.12497646361589432,0.3515625,0.5234375
+post_attention_layernorm,9,13,0.8166666666666667,0.44954128440366975,0.8166666666666667,True,0.17869991064071655,0.09632012248039246,0.3515625,0.619140625
+post_attention_layernorm,9,14,1.6645299145299146,0.6246992782678428,1.6645299145299146,True,0.17869991064071655,0.053455036133527756,0.3515625,0.87646484375
+post_attention_layernorm,9,15,2.6055555555555556,0.7226502311248074,2.6055555555555556,True,0.17869991064071655,0.06801187992095947,0.3515625,0.736328125
+post_attention_layernorm,9,16,1.4316239316239316,0.5887521968365553,1.4316239316239316,True,0.17869991064071655,0.08680811524391174,0.3515625,0.72265625
+post_attention_layernorm,9,17,1.6527777777777777,0.6230366492146597,1.6527777777777777,True,0.17869991064071655,0.09242258220911026,0.3515625,0.712890625
+post_attention_layernorm,9,18,1.7604166666666667,0.6377358490566037,1.7604166666666667,True,0.17869991064071655,0.09676424413919449,0.3515625,0.8095703125
+post_attention_layernorm,9,19,2.1822916666666665,0.6857610474631751,2.1822916666666665,True,0.17869991064071655,0.07613449543714523,0.3515625,0.81640625
+post_attention_layernorm,9,20,2.1083333333333334,0.67828418230563,2.1083333333333334,True,0.17869991064071655,0.07808392494916916,0.3515625,0.822265625
+post_attention_layernorm,9,21,2.4916666666666667,0.7136038186157518,2.4916666666666667,True,0.17869991064071655,0.06570179015398026,0.3515625,0.86328125
+post_attention_layernorm,9,22,3.382978723404255,0.7718446601941747,3.382978723404255,True,0.17869991064071655,0.05012959614396095,0.3515625,0.9713134765625
+post_attention_layernorm,9,23,3.473684210526316,0.7764705882352941,3.473684210526316,True,0.17869991064071655,0.06521295011043549,0.3515625,1.01361083984375
+post_attention_layernorm,10,11,0.23232323232323232,0.1885245901639344,0.23232323232323232,True,0.16611140966415405,0.1528426557779312,0.36328125,0.41796875
+post_attention_layernorm,10,12,0.510752688172043,0.33807829181494664,0.510752688172043,True,0.16611140966415405,0.12497646361589432,0.36328125,0.5234375
+post_attention_layernorm,10,13,0.796875,0.4434782608695652,0.796875,True,0.16611140966415405,0.09632012248039246,0.36328125,0.619140625
+post_attention_layernorm,10,14,1.7079646017699115,0.630718954248366,1.7079646017699115,True,0.16611140966415405,0.053455036133527756,0.36328125,0.87646484375
+post_attention_layernorm,10,15,2.4946236559139785,0.7138461538461538,2.4946236559139785,True,0.16611140966415405,0.06801187992095947,0.36328125,0.736328125
+post_attention_layernorm,10,16,1.4557522123893805,0.5927927927927928,1.4557522123893805,True,0.16611140966415405,0.08680811524391174,0.36328125,0.72265625
+post_attention_layernorm,10,17,1.6364247311827957,0.6206984450675503,1.6364247311827957,True,0.16611140966415405,0.09242258220911026,0.36328125,0.712890625
+post_attention_layernorm,10,18,1.67372311827957,0.6259896946085208,1.67372311827957,True,0.16611140966415405,0.09676424413919449,0.36328125,0.8095703125
+post_attention_layernorm,10,19,2.146276595744681,0.6821639898562976,2.146276595744681,True,0.16611140966415405,0.07613449543714523,0.36328125,0.81640625
+post_attention_layernorm,10,20,1.976063829787234,0.6639857015192135,1.976063829787234,True,0.16611140966415405,0.07808392494916916,0.36328125,0.822265625
+post_attention_layernorm,10,21,2.4123711340206184,0.7069486404833837,2.4123711340206184,True,0.16611140966415405,0.06570179015398026,0.36328125,0.86328125
+post_attention_layernorm,10,22,3.247422680412371,0.7645631067961165,3.247422680412371,True,0.16611140966415405,0.05012959614396095,0.36328125,0.9713134765625
+post_attention_layernorm,10,23,3.521276595744681,0.7788235294117647,3.521276595744681,True,0.16611140966415405,0.06521295011043549,0.36328125,1.01361083984375
+post_attention_layernorm,11,12,0.3375,0.2523364485981308,0.3375,True,0.1528426557779312,0.12497646361589432,0.41796875,0.5234375
+post_attention_layernorm,11,13,0.5131578947368421,0.3391304347826087,0.5131578947368421,True,0.1528426557779312,0.09632012248039246,0.41796875,0.619140625
+post_attention_layernorm,11,14,1.2612781954887218,0.5577722360764755,1.2612781954887218,True,0.1528426557779312,0.053455036133527756,0.41796875,0.87646484375
+post_attention_layernorm,11,15,1.9906542056074767,0.665625,1.9906542056074767,True,0.1528426557779312,0.06801187992095947,0.41796875,0.736328125
+post_attention_layernorm,11,16,1.1914191419141915,0.5436746987951807,1.1914191419141915,True,0.1528426557779312,0.08680811524391174,0.41796875,0.72265625
+post_attention_layernorm,11,17,1.249426605504587,0.5554422635737956,1.249426605504587,True,0.1528426557779312,0.09242258220911026,0.41796875,0.712890625
+post_attention_layernorm,11,18,1.28125,0.5616438356164384,1.28125,True,0.1528426557779312,0.09676424413919449,0.41796875,0.8095703125
+post_attention_layernorm,11,19,1.764018691588785,0.6382079459002535,1.764018691588785,True,0.1528426557779312,0.07613449543714523,0.41796875,0.81640625
+post_attention_layernorm,11,20,1.5344036697247707,0.6054298642533936,1.5344036697247707,True,0.1528426557779312,0.07808392494916916,0.41796875,0.822265625
+post_attention_layernorm,11,21,1.9369158878504673,0.6595067621320605,1.9369158878504673,True,0.1528426557779312,0.06570179015398026,0.41796875,0.86328125
+post_attention_layernorm,11,22,2.7454545454545456,0.7330097087378641,2.7454545454545456,True,0.1528426557779312,0.05012959614396095,0.41796875,0.9713134765625
+post_attention_layernorm,11,23,2.97196261682243,0.7482352941176471,2.97196261682243,True,0.1528426557779312,0.06521295011043549,0.41796875,1.01361083984375
+post_attention_layernorm,12,13,0.26492537313432835,0.20943952802359883,0.26492537313432835,True,0.12497646361589432,0.09632012248039246,0.5234375,0.619140625
+post_attention_layernorm,12,14,0.9213836477987422,0.4795417348608838,0.9213836477987422,True,0.12497646361589432,0.053455036133527756,0.5234375,0.87646484375
+post_attention_layernorm,12,15,1.3861940298507462,0.5809225957779516,1.3861940298507462,True,0.12497646361589432,0.06801187992095947,0.5234375,0.736328125
+post_attention_layernorm,12,16,0.9358600583090378,0.48343373493975905,0.9358600583090378,True,0.12497646361589432,0.08680811524391174,0.5234375,0.72265625
+post_attention_layernorm,12,17,0.9096209912536443,0.4763358778625954,0.9096209912536443,True,0.12497646361589432,0.09242258220911026,0.5234375,0.712890625
+post_attention_layernorm,12,18,0.9358600583090378,0.48343373493975905,0.9358600583090378,True,0.12497646361589432,0.09676424413919449,0.5234375,0.8095703125
+post_attention_layernorm,12,19,1.2276119402985075,0.5510887772194305,1.2276119402985075,True,0.12497646361589432,0.07613449543714523,0.5234375,0.81640625
+post_attention_layernorm,12,20,1.056985294117647,0.5138516532618409,1.056985294117647,True,0.12497646361589432,0.07808392494916916,0.5234375,0.822265625
+post_attention_layernorm,12,21,1.4701492537313432,0.595166163141994,1.4701492537313432,True,0.12497646361589432,0.06570179015398026,0.5234375,0.86328125
+post_attention_layernorm,12,22,2.0294117647058822,0.6699029126213593,2.0294117647058822,True,0.12497646361589432,0.05012959614396095,0.5234375,0.9713134765625
+post_attention_layernorm,12,23,2.1481481481481484,0.6823529411764706,2.1481481481481484,True,0.12497646361589432,0.06521295011043549,0.5234375,1.01361083984375
+post_attention_layernorm,13,14,0.6276595744680851,0.38562091503267976,0.6276595744680851,True,0.09632012248039246,0.053455036133527756,0.619140625,0.87646484375
+post_attention_layernorm,13,15,1.0504731861198737,0.5123076923076924,1.0504731861198737,True,0.09632012248039246,0.06801187992095947,0.619140625,0.736328125
+post_attention_layernorm,13,16,0.5697399527186762,0.3629518072289157,0.5697399527186762,True,0.09632012248039246,0.08680811524391174,0.619140625,0.72265625
+post_attention_layernorm,13,17,0.5619930475086906,0.35979228486646886,0.5619930475086906,True,0.09632012248039246,0.09242258220911026,0.619140625,0.712890625
+post_attention_layernorm,13,18,0.657010428736964,0.3965034965034965,0.657010428736964,True,0.09632012248039246,0.09676424413919449,0.619140625,0.8095703125
+post_attention_layernorm,13,19,0.8659305993690851,0.4640743871513102,0.8659305993690851,True,0.09632012248039246,0.07613449543714523,0.619140625,0.81640625
+post_attention_layernorm,13,20,0.7811594202898551,0.4385679414157852,0.7811594202898551,True,0.09632012248039246,0.07808392494916916,0.619140625,0.822265625
+post_attention_layernorm,13,21,0.9826498422712934,0.49562450278440734,0.9826498422712934,True,0.09632012248039246,0.06570179015398026,0.619140625,0.86328125
+post_attention_layernorm,13,22,1.5432098765432098,0.6067961165048543,1.5432098765432098,True,0.09632012248039246,0.05012959614396095,0.619140625,0.9713134765625
+post_attention_layernorm,13,23,1.6813880126182965,0.6270588235294118,1.6813880126182965,True,0.09632012248039246,0.06521295011043549,0.619140625,1.01361083984375
+post_attention_layernorm,14,15,0.4135667396061269,0.29256965944272445,0.4135667396061269,True,0.053455036133527756,0.06801187992095947,0.87646484375,0.736328125
+post_attention_layernorm,14,16,0.2551505546751189,0.2418918918918919,0.2551505546751189,True,0.053455036133527756,0.08680811524391174,0.87646484375,0.72265625
+post_attention_layernorm,14,17,0.28498372849837283,0.2595890410958904,0.28498372849837283,True,0.053455036133527756,0.09242258220911026,0.87646484375,0.712890625
+post_attention_layernorm,14,18,0.39717282261741904,0.28426892950391647,0.39717282261741904,True,0.053455036133527756,0.09676424413919449,0.87646484375,0.8095703125
+post_attention_layernorm,14,19,0.3989968080255358,0.28520208604954367,0.3989968080255358,True,0.053455036133527756,0.07613449543714523,0.87646484375,0.81640625
+post_attention_layernorm,14,20,0.4737802097583219,0.3214727722772277,0.4737802097583219,True,0.053455036133527756,0.07808392494916916,0.87646484375,0.822265625
+post_attention_layernorm,14,21,0.47515257192676547,0.32210401891252954,0.47515257192676547,True,0.053455036133527756,0.06570179015398026,0.87646484375,0.86328125
+post_attention_layernorm,14,22,0.7402323125659979,0.4253640776699029,0.7402323125659979,True,0.053455036133527756,0.05012959614396095,0.87646484375,0.9713134765625
+post_attention_layernorm,14,23,0.8220793140407289,0.4511764705882353,0.8220793140407289,True,0.053455036133527756,0.06521295011043549,0.87646484375,1.01361083984375
+post_attention_layernorm,15,16,0.3656832298136646,0.576499388004896,0.576499388004896,True,0.06801187992095947,0.08680811524391174,0.736328125,0.72265625
+post_attention_layernorm,15,17,0.29876160990712075,0.4260485651214128,0.4260485651214128,True,0.06801187992095947,0.09242258220911026,0.736328125,0.712890625
+post_attention_layernorm,15,18,0.30124223602484473,0.4311111111111111,0.4311111111111111,True,0.06801187992095947,0.09676424413919449,0.736328125,0.8095703125
+post_attention_layernorm,15,19,0.2840549102428722,0.22121710526315788,0.2840549102428722,True,0.06801187992095947,0.07613449543714523,0.736328125,0.81640625
+post_attention_layernorm,15,20,0.33996683250414594,0.2537128712871287,0.33996683250414594,True,0.06801187992095947,0.07808392494916916,0.736328125,0.822265625
+post_attention_layernorm,15,21,0.4206008583690987,0.29607250755287007,0.4206008583690987,True,0.06801187992095947,0.06570179015398026,0.736328125,0.86328125
+post_attention_layernorm,15,22,0.45278969957081544,0.31166912850812406,0.45278969957081544,True,0.06801187992095947,0.05012959614396095,0.736328125,0.9713134765625
+post_attention_layernorm,15,23,0.815378923352038,0.4491508152173913,0.815378923352038,True,0.06801187992095947,0.06521295011043549,0.736328125,1.01361083984375
+post_attention_layernorm,16,17,0.17602657004830918,0.1496790757381258,0.17602657004830918,True,0.08680811524391174,0.09242258220911026,0.72265625,0.712890625
+post_attention_layernorm,16,18,0.2154215976331361,0.17724022516354784,0.2154215976331361,True,0.08680811524391174,0.09676424413919449,0.72265625,0.8095703125
+post_attention_layernorm,16,19,0.4420289855072464,0.3065326633165829,0.4420289855072464,True,0.08680811524391174,0.07613449543714523,0.72265625,0.81640625
+post_attention_layernorm,16,20,0.33108935128518974,0.24873563218390804,0.33108935128518974,True,0.08680811524391174,0.07808392494916916,0.72265625,0.822265625
+post_attention_layernorm,16,21,0.5990338164251208,0.37462235649546827,0.5990338164251208,True,0.08680811524391174,0.06570179015398026,0.72265625,0.86328125
+post_attention_layernorm,16,22,1.0171358629130967,0.504247572815534,1.0171358629130967,True,0.08680811524391174,0.05012959614396095,0.72265625,0.9713134765625
+post_attention_layernorm,16,23,1.0506634499396863,0.5123529411764706,1.0506634499396863,True,0.08680811524391174,0.06521295011043549,0.72265625,1.01361083984375
+post_attention_layernorm,17,18,0.23767123287671232,0.19203099059214168,0.23767123287671232,True,0.09242258220911026,0.09676424413919449,0.712890625,0.8095703125
+post_attention_layernorm,17,19,0.29360306178239476,0.22696534234995774,0.29360306178239476,True,0.09242258220911026,0.07613449543714523,0.712890625,0.81640625
+post_attention_layernorm,17,20,0.20969441517386722,0.17334494773519163,0.20969441517386722,True,0.09242258220911026,0.07808392494916916,0.712890625,0.822265625
+post_attention_layernorm,17,21,0.40350877192982454,0.2875,0.40350877192982454,True,0.09242258220911026,0.06570179015398026,0.712890625,0.86328125
+post_attention_layernorm,17,22,0.8070175438596491,0.44660194174757284,0.8070175438596491,True,0.09242258220911026,0.05012959614396095,0.712890625,0.9713134765625
+post_attention_layernorm,17,23,0.8589393110989612,0.46205882352941174,0.8589393110989612,True,0.09242258220911026,0.06521295011043549,0.712890625,1.01361083984375
+post_attention_layernorm,18,19,0.27958115183246074,0.21849427168576105,0.27958115183246074,True,0.09676424413919449,0.07613449543714523,0.8095703125,0.81640625
+post_attention_layernorm,18,20,0.20833333333333334,0.1724137931034483,0.20833333333333334,True,0.09676424413919449,0.07808392494916916,0.8095703125,0.822265625
+post_attention_layernorm,18,21,0.4222222222222222,0.296875,0.4222222222222222,True,0.09676424413919449,0.06570179015398026,0.8095703125,0.86328125
+post_attention_layernorm,18,22,0.8311111111111111,0.4538834951456311,0.8311111111111111,True,0.09676424413919449,0.05012959614396095,0.8095703125,0.9713134765625
+post_attention_layernorm,18,23,0.8,0.4444444444444444,0.8,True,0.09676424413919449,0.06521295011043549,0.8095703125,1.01361083984375
+post_attention_layernorm,19,20,0.151033386327504,0.1441947565543071,0.151033386327504,True,0.07613449543714523,0.07808392494916916,0.81640625,0.822265625
+post_attention_layernorm,19,21,0.1678832116788321,0.14375,0.1678832116788321,True,0.07613449543714523,0.06570179015398026,0.81640625,0.86328125
+post_attention_layernorm,19,22,0.5036496350364964,0.33495145631067963,0.5036496350364964,True,0.07613449543714523,0.05012959614396095,0.81640625,0.9713134765625
+post_attention_layernorm,19,23,0.48267526188557613,0.34782608695652173,0.48267526188557613,True,0.07613449543714523,0.06521295011043549,0.81640625,1.01361083984375
+post_attention_layernorm,20,21,0.2091324200913242,0.1729607250755287,0.2091324200913242,True,0.07808392494916916,0.06570179015398026,0.822265625,0.86328125
+post_attention_layernorm,20,22,0.5154022988505748,0.3401092233009709,0.5154022988505748,True,0.07808392494916916,0.05012959614396095,0.822265625,0.9713134765625
+post_attention_layernorm,20,23,0.5675426463808206,0.4219858156028369,0.5675426463808206,True,0.07808392494916916,0.06521295011043549,0.822265625,1.01361083984375
+post_attention_layernorm,21,22,0.2875,0.22330097087378642,0.2875,True,0.06570179015398026,0.05012959614396095,0.86328125,0.9713134765625
+post_attention_layernorm,21,23,0.3960546282245827,0.4148936170212766,0.4148936170212766,True,0.06570179015398026,0.06521295011043549,0.86328125,1.01361083984375
+post_attention_layernorm,22,23,0.3124108416547789,0.34263295553618134,0.34263295553618134,True,0.05012959614396095,0.06521295011043549,0.9713134765625,1.01361083984375

results/cpu-qwen35-2b/preflight-first-attempt.log

@@ -0,0 +1,29 @@
+{"preflight_layer": 0, "seconds": 2.0485799312591553}
+{"preflight_layer": 1, "seconds": 4.110389947891235}
+{"preflight_layer": 2, "seconds": 5.677527189254761}
+{"preflight_layer": 3, "seconds": 6.754502058029175}
+{"preflight_layer": 4, "seconds": 8.340956926345825}
+{"preflight_layer": 5, "seconds": 10.697716236114502}
+{"preflight_layer": 6, "seconds": 12.576658964157104}
+{"preflight_layer": 7, "seconds": 14.14732575416565}
+Traceback (most recent call last):
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/opt/exp-model-parameter-similarity/functional/preflight.py", line 27, in <module>
+    if __name__=='__main__':main()
+                            ~~~~^^
+  File "/opt/exp-model-parameter-similarity/functional/preflight.py", line 18, in main
+    h,_=lm.forward(ids[:1,:8],progress=progress)
+        ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/opt/exp-model-parameter-similarity/functional/model.py", line 215, in forward
+    h = h + self.mlp(n,donor)
+            ~~~~~~~~^^^^^^^^^
+  File "/opt/exp-model-parameter-similarity/functional/model.py", line 149, in mlp
+    gate = silu(self.linear(h, p+'gate_proj'))
+                ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
+  File "/opt/exp-model-parameter-similarity/functional/model.py", line 127, in linear
+    self.guard()
+    ~~~~~~~~~~^^
+  File "/opt/exp-model-parameter-similarity/functional/model.py", line 108, in guard
+    raise MemoryError('Sustained host swap IO above 4 MiB/s in three observation windows')
+MemoryError: Sustained host swap IO above 4 MiB/s in three observation windows

results/cpu-qwen35-2b/preflight-second-attempt.json

@@ -0,0 +1,61 @@
+{
+  "passed": false,
+  "sample_id": "cal-zh-1",
+  "tokens": 8,
+  "error_type": "MemoryError",
+  "error": "Sustained host swap pressure plus constrained memory: available=1071348 KiB, process_swap=23640 KiB",
+  "elapsed_seconds": 15.565188646316528,
+  "peak_rss_kib": 162324,
+  "resource_snapshots": [
+    {
+      "layer": 0,
+      "seconds": 3.669800043106079,
+      "rss_kib": 117576,
+      "process_swap_kib": 0,
+      "available_kib": 975920
+    },
+    {
+      "layer": 1,
+      "seconds": 6.548107862472534,
+      "rss_kib": 117440,
+      "process_swap_kib": 0,
+      "available_kib": 984012
+    },
+    {
+      "layer": 2,
+      "seconds": 9.072005033493042,
+      "rss_kib": 115956,
+      "process_swap_kib": 0,
+      "available_kib": 925720
+    },
+    {
+      "layer": 3,
+      "seconds": 10.377433776855469,
+      "rss_kib": 114984,
+      "process_swap_kib": 0,
+      "available_kib": 958016
+    },
+    {
+      "layer": 4,
+      "seconds": 11.7694993019104,
+      "rss_kib": 108236,
+      "process_swap_kib": 0,
+      "available_kib": 1055012
+    },
+    {
+      "layer": 5,
+      "seconds": 13.289605379104614,
+      "rss_kib": 108244,
+      "process_swap_kib": 0,
+      "available_kib": 1056400
+    },
+    {
+      "layer": 6,
+      "seconds": 15.41482949256897,
+      "rss_kib": 84512,
+      "process_swap_kib": 23736,
+      "available_kib": 1072308
+    }
+  ],
+  "limitation": "Real-weight capacity check, not an independent correctness oracle or quality benchmark"
+}

results/cpu-qwen35-2b/preflight-second-attempt.log

@@ -0,0 +1,22 @@
+{"layer": 0, "seconds": 3.669800043106079, "rss_kib": 117576, "process_swap_kib": 0, "available_kib": 975920}
+{"layer": 1, "seconds": 6.548107862472534, "rss_kib": 117440, "process_swap_kib": 0, "available_kib": 984012}
+{"layer": 2, "seconds": 9.072005033493042, "rss_kib": 115956, "process_swap_kib": 0, "available_kib": 925720}
+{"layer": 3, "seconds": 10.377433776855469, "rss_kib": 114984, "process_swap_kib": 0, "available_kib": 958016}
+{"layer": 4, "seconds": 11.7694993019104, "rss_kib": 108236, "process_swap_kib": 0, "available_kib": 1055012}
+{"layer": 5, "seconds": 13.289605379104614, "rss_kib": 108244, "process_swap_kib": 0, "available_kib": 1056400}
+{"layer": 6, "seconds": 15.41482949256897, "rss_kib": 84512, "process_swap_kib": 23736, "available_kib": 1072308}
+Traceback (most recent call last):
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/opt/exp-model-parameter-similarity/functional/preflight.py", line 39, in <module>
+    if __name__=='__main__':main()
+                            ~~~~^^
+  File "/opt/exp-model-parameter-similarity/functional/preflight.py", line 26, in main
+    h,_=lm.forward(ids[:1,:8],progress=progress)
+        ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/opt/exp-model-parameter-similarity/functional/model.py", line 201, in forward
+    self.guard()
+    ~~~~~~~~~~^^
+  File "/opt/exp-model-parameter-similarity/functional/model.py", line 109, in guard
+    raise MemoryError(f'Sustained host swap pressure plus constrained memory: available={available} KiB, process_swap={own_swap_kib} KiB')
+MemoryError: Sustained host swap pressure plus constrained memory: available=1071348 KiB, process_swap=23640 KiB

results/cpu-qwen35-2b/preflight.json

@@ -0,0 +1,190 @@
+{
+  "passed": true,
+  "sample_id": "cal-zh-1",
+  "tokens": 8,
+  "mean_nll": 4.089094978395148,
+  "token_nll": [
+    [
+      0.7776033786968188,
+      12.836286295347621,
+      6.470462449543003,
+      4.530625316962006,
+      3.1924382665907807,
+      0.703601432631153,
+      0.11264770899465404
+    ]
+  ],
+  "elapsed_seconds": 47.741777658462524,
+  "peak_rss_kib": 162276,
+  "resource_snapshots": [
+    {
+      "layer": 0,
+      "seconds": 1.4413530826568604,
+      "rss_kib": 48904,
+      "process_swap_kib": 0,
+      "available_kib": 930888
+    },
+    {
+      "layer": 1,
+      "seconds": 3.073808431625366,
+      "rss_kib": 48180,
+      "process_swap_kib": 0,
+      "available_kib": 983076
+    },
+    {
+      "layer": 2,
+      "seconds": 4.584789514541626,
+      "rss_kib": 48192,
+      "process_swap_kib": 0,
+      "available_kib": 963772
+    },
+    {
+      "layer": 3,
+      "seconds": 5.736491918563843,
+      "rss_kib": 46648,
+      "process_swap_kib": 0,
+      "available_kib": 966496
+    },
+    {
+      "layer": 4,
+      "seconds": 7.517967462539673,
+      "rss_kib": 45724,
+      "process_swap_kib": 0,
+      "available_kib": 986008
+    },
+    {
+      "layer": 5,
+      "seconds": 11.781850576400757,
+      "rss_kib": 45332,
+      "process_swap_kib": 0,
+      "available_kib": 935860
+    },
+    {
+      "layer": 6,
+      "seconds": 14.041518926620483,
+      "rss_kib": 45340,
+      "process_swap_kib": 0,
+      "available_kib": 1002640
+    },
+    {
+      "layer": 7,
+      "seconds": 15.087216138839722,
+      "rss_kib": 35548,
+      "process_swap_kib": 9892,
+      "available_kib": 1026692
+    },
+    {
+      "layer": 8,
+      "seconds": 16.508682250976562,
+      "rss_kib": 32500,
+      "process_swap_kib": 11864,
+      "available_kib": 1039852
+    },
+    {
+      "layer": 9,
+      "seconds": 17.623984813690186,
+      "rss_kib": 26508,
+      "process_swap_kib": 11848,
+      "available_kib": 997296
+    },
+    {
+      "layer": 10,
+      "seconds": 19.696174144744873,
+      "rss_kib": 26372,
+      "process_swap_kib": 11824,
+      "available_kib": 1018064
+    },
+    {
+      "layer": 11,
+      "seconds": 20.69566321372986,
+      "rss_kib": 26528,
+      "process_swap_kib": 11796,
+      "available_kib": 1054760
+    },
+    {
+      "layer": 12,
+      "seconds": 22.038830041885376,
+      "rss_kib": 24104,
+      "process_swap_kib": 13480,
+      "available_kib": 1062360
+    },
+    {
+      "layer": 13,
+      "seconds": 23.356847524642944,
+      "rss_kib": 24124,
+      "process_swap_kib": 13468,
+      "available_kib": 1062328
+    },
+    {
+      "layer": 14,
+      "seconds": 24.617902040481567,
+      "rss_kib": 24144,
+      "process_swap_kib": 13456,
+      "available_kib": 1039196
+    },
+    {
+      "layer": 15,
+      "seconds": 25.96587586402893,
+      "rss_kib": 24068,
+      "process_swap_kib": 13448,
+      "available_kib": 1032608
+    },
+    {
+      "layer": 16,
+      "seconds": 27.733217239379883,
+      "rss_kib": 24056,
+      "process_swap_kib": 13432,
+      "available_kib": 1015000
+    },
+    {
+      "layer": 17,
+      "seconds": 29.303704977035522,
+      "rss_kib": 20852,
+      "process_swap_kib": 16632,
+      "available_kib": 1041560
+    },
+    {
+      "layer": 18,
+      "seconds": 30.38013005256653,
+      "rss_kib": 20876,
+      "process_swap_kib": 16616,
+      "available_kib": 1061004
+    },
+    {
+      "layer": 19,
+      "seconds": 31.271567583084106,
+      "rss_kib": 20896,
+      "process_swap_kib": 16596,
+      "available_kib": 1059348
+    },
+    {
+      "layer": 20,
+      "seconds": 32.31741189956665,
+      "rss_kib": 20936,
+      "process_swap_kib": 16564,
+      "available_kib": 1078812
+    },
+    {
+      "layer": 21,
+      "seconds": 33.8273024559021,
+      "rss_kib": 20396,
+      "process_swap_kib": 17044,
+      "available_kib": 1088472
+    },
+    {
+      "layer": 22,
+      "seconds": 35.43067383766174,
+      "rss_kib": 20436,
+      "process_swap_kib": 17012,
+      "available_kib": 1109268
+    },
+    {
+      "layer": 23,
+      "seconds": 36.55965542793274,
+      "rss_kib": 20448,
+      "process_swap_kib": 17000,
+      "available_kib": 1109588
+    }
+  ],
+  "limitation": "Real-weight capacity check, not an independent correctness oracle or quality benchmark"
+}

results/cpu-qwen35-2b/preflight.log

@@ -0,0 +1,24 @@
+{"layer": 0, "seconds": 1.4413530826568604, "rss_kib": 48904, "process_swap_kib": 0, "available_kib": 930888}
+{"layer": 1, "seconds": 3.073808431625366, "rss_kib": 48180, "process_swap_kib": 0, "available_kib": 983076}
+{"layer": 2, "seconds": 4.584789514541626, "rss_kib": 48192, "process_swap_kib": 0, "available_kib": 963772}
+{"layer": 3, "seconds": 5.736491918563843, "rss_kib": 46648, "process_swap_kib": 0, "available_kib": 966496}
+{"layer": 4, "seconds": 7.517967462539673, "rss_kib": 45724, "process_swap_kib": 0, "available_kib": 986008}
+{"layer": 5, "seconds": 11.781850576400757, "rss_kib": 45332, "process_swap_kib": 0, "available_kib": 935860}
+{"layer": 6, "seconds": 14.041518926620483, "rss_kib": 45340, "process_swap_kib": 0, "available_kib": 1002640}
+{"layer": 7, "seconds": 15.087216138839722, "rss_kib": 35548, "process_swap_kib": 9892, "available_kib": 1026692}
+{"layer": 8, "seconds": 16.508682250976562, "rss_kib": 32500, "process_swap_kib": 11864, "available_kib": 1039852}
+{"layer": 9, "seconds": 17.623984813690186, "rss_kib": 26508, "process_swap_kib": 11848, "available_kib": 997296}
+{"layer": 10, "seconds": 19.696174144744873, "rss_kib": 26372, "process_swap_kib": 11824, "available_kib": 1018064}
+{"layer": 11, "seconds": 20.69566321372986, "rss_kib": 26528, "process_swap_kib": 11796, "available_kib": 1054760}
+{"layer": 12, "seconds": 22.038830041885376, "rss_kib": 24104, "process_swap_kib": 13480, "available_kib": 1062360}
+{"layer": 13, "seconds": 23.356847524642944, "rss_kib": 24124, "process_swap_kib": 13468, "available_kib": 1062328}
+{"layer": 14, "seconds": 24.617902040481567, "rss_kib": 24144, "process_swap_kib": 13456, "available_kib": 1039196}
+{"layer": 15, "seconds": 25.96587586402893, "rss_kib": 24068, "process_swap_kib": 13448, "available_kib": 1032608}
+{"layer": 16, "seconds": 27.733217239379883, "rss_kib": 24056, "process_swap_kib": 13432, "available_kib": 1015000}
+{"layer": 17, "seconds": 29.303704977035522, "rss_kib": 20852, "process_swap_kib": 16632, "available_kib": 1041560}
+{"layer": 18, "seconds": 30.38013005256653, "rss_kib": 20876, "process_swap_kib": 16616, "available_kib": 1061004}
+{"layer": 19, "seconds": 31.271567583084106, "rss_kib": 20896, "process_swap_kib": 16596, "available_kib": 1059348}
+{"layer": 20, "seconds": 32.31741189956665, "rss_kib": 20936, "process_swap_kib": 16564, "available_kib": 1078812}
+{"layer": 21, "seconds": 33.8273024559021, "rss_kib": 20396, "process_swap_kib": 17044, "available_kib": 1088472}
+{"layer": 22, "seconds": 35.43067383766174, "rss_kib": 20436, "process_swap_kib": 17012, "available_kib": 1109268}
+{"layer": 23, "seconds": 36.55965542793274, "rss_kib": 20448, "process_swap_kib": 17000, "available_kib": 1109588}

results/cpu-qwen35-2b/reference-check-first-attempt.log

@@ -0,0 +1,15 @@
+Traceback (most recent call last):
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/opt/exp-model-parameter-similarity/functional/reference_check.py", line 142, in <module>
+    if __name__=='__main__':main()
+                            ~~~~^^
+  File "/opt/exp-model-parameter-similarity/functional/reference_check.py", line 87, in main
+    rotary=ns['Qwen3_5TextRotaryEmbedding'](config)
+  File "pinned_transformers_reference", line 146, in __init__
+  File "pinned_transformers_reference", line 165, in compute_default_rope_parameters
+  File "/opt/exp-model-parameter-similarity/venv/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1976, in __getattr__
+    raise AttributeError(
+        f"'{type(self).__name__}' object has no attribute '{name}'"
+    )
+AttributeError: 'Qwen3_5TextRotaryEmbedding' object has no attribute 'rope_parameters'. Did you mean: '_parameters'?

results/cpu-qwen35-2b/reference-check-second-attempt.log

@@ -0,0 +1,15 @@
+Traceback (most recent call last):
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/opt/exp-model-parameter-similarity/functional/reference_check.py", line 142, in <module>
+    if __name__=='__main__':main()
+                            ~~~~^^
+  File "/opt/exp-model-parameter-similarity/functional/reference_check.py", line 102, in main
+    tout=layer.linear_attn(tn)
+  File "/opt/exp-model-parameter-similarity/venv/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1787, in _wrapped_call_impl
+    return self._call_impl(*args, **kwargs)
+           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
+  File "/opt/exp-model-parameter-similarity/venv/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1798, in _call_impl
+    return forward_call(*args, **kwargs)
+  File "pinned_transformers_reference", line 542, in forward
+NameError: name 'apply_mask_to_padding_states' is not defined

results/cpu-qwen35-2b/reference-check.json

@@ -0,0 +1,288 @@
+{
+  "passed": true,
+  "reference_commit": "fd3cf0140de03f7619ac95b5cfed05b711ff9e22",
+  "source_sha256": "4463b8a108f678c166755101d95f188179f54143305953d78147de0389b80ebc",
+  "torch": "2.15.0+cpu",
+  "numpy": "2.5.4",
+  "implementation_sha256": "48a2a97d0445dc6e08ac7f27a935637a7553417d0982eb94122d6cb02bbb499c",
+  "tests": [
+    {
+      "test": "layer0_rmsnorm",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 2.384185791015625e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer0_mixer",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 1.862645149230957e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer0_mlp",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 2.9802322387695312e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer0_complete",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 2.384185791015625e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer1_rmsnorm",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 2.384185791015625e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "attention_probabilities",
+      "shape": [
+        2,
+        2,
+        7,
+        7
+      ],
+      "max_abs_error": 1.1920928955078125e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer1_mixer",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 5.21540641784668e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer1_mlp",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 4.470348358154297e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "layer1_complete",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 2.384185791015625e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_recurrent_gated_delta_rule_length1_output",
+      "shape": [
+        2,
+        1,
+        2,
+        5
+      ],
+      "max_abs_error": 4.470348358154297e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_recurrent_gated_delta_rule_length1_state",
+      "shape": [
+        2,
+        2,
+        4,
+        5
+      ],
+      "max_abs_error": 5.960464477539063e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_chunk_gated_delta_rule_length1_output",
+      "shape": [
+        2,
+        1,
+        2,
+        5
+      ],
+      "max_abs_error": 2.9802322387695312e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_chunk_gated_delta_rule_length1_state",
+      "shape": [
+        2,
+        2,
+        4,
+        5
+      ],
+      "max_abs_error": 5.960464477539063e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_recurrent_gated_delta_rule_length7_output",
+      "shape": [
+        2,
+        7,
+        2,
+        5
+      ],
+      "max_abs_error": 5.960464477539063e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_recurrent_gated_delta_rule_length7_state",
+      "shape": [
+        2,
+        2,
+        4,
+        5
+      ],
+      "max_abs_error": 5.960464477539063e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_chunk_gated_delta_rule_length7_output",
+      "shape": [
+        2,
+        7,
+        2,
+        5
+      ],
+      "max_abs_error": 7.450580596923828e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_chunk_gated_delta_rule_length7_state",
+      "shape": [
+        2,
+        2,
+        4,
+        5
+      ],
+      "max_abs_error": 1.7881393432617188e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_recurrent_gated_delta_rule_length33_output",
+      "shape": [
+        2,
+        33,
+        2,
+        5
+      ],
+      "max_abs_error": 1.1920928955078125e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_recurrent_gated_delta_rule_length33_state",
+      "shape": [
+        2,
+        2,
+        4,
+        5
+      ],
+      "max_abs_error": 2.9802322387695312e-08,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_chunk_gated_delta_rule_length33_output",
+      "shape": [
+        2,
+        33,
+        2,
+        5
+      ],
+      "max_abs_error": 5.364418029785156e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "torch_chunk_gated_delta_rule_length33_state",
+      "shape": [
+        2,
+        2,
+        4,
+        5
+      ],
+      "max_abs_error": 3.5762786865234375e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    },
+    {
+      "test": "complete_two_layer_text_forward",
+      "shape": [
+        2,
+        7,
+        16
+      ],
+      "max_abs_error": 3.5762786865234375e-07,
+      "atol": 2e-05,
+      "rtol": 2e-05,
+      "passed": true
+    }
+  ],
+  "limitations": "FP32 CPU semantics on tiny synthetic configurations; not full-checkpoint BF16 kernel equivalence"
+}

results/cpu-qwen35-2b/reference-check.log

@@ -0,0 +1 @@
+{"passed": true, "checks": 22, "maximum_absolute_discrepancy": 5.364418029785156e-07}

results/cpu-qwen35-2b/retention-policy.json

@@ -0,0 +1,5 @@
+{
+  "weights": "retain_until_explicit_user_delete_request",
+  "automatic_cleanup": false,
+  "instruction_date": "2026-11-09"
+}

results/cpu-qwen35-2b/samples-tokenized.json

@@ -0,0 +1,342 @@
+{
+  "revision": "15852e8c16360a2fea060d615a32b45270f8a8fc",
+  "tokenizer_sha256": "5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42",
+  "samples": [
+    {
+      "id": "cal-zh-1",
+      "split": "calibration",
+      "language": "zh",
+      "text": "\u6e05\u6668\uff0c\u7814\u7a76\u5458\u6253\u5f00\u5b9e\u9a8c\u8bb0\u5f55\u672c\uff0c\u68c0\u67e5\u6628\u5929\u4fdd\u5b58\u7684\u6570\u636e\u3002\u4ed6\u5148\u786e\u8ba4\u6587\u4ef6\u5b8c\u6574\uff0c\u518d\u6bd4\u8f83\u4e24\u7ec4\u89c2\u6d4b\u7ed3\u679c\u3002\u4e3a\u4e86\u907f\u514d\u5076\u7136\u8bef\u5dee\uff0c\u4ed6\u4e3a\u6bcf\u4e00\u6b65\u7559\u4e0b\u65f6\u95f4\u548c\u6761\u4ef6\uff0c\u5e76\u5c06\u5c1a\u672a\u9a8c\u8bc1\u7684\u63a8\u6d4b\u5355\u72ec\u8bb0\u5f55\u3002",
+      "text_sha256": "ac4ed82b8c201d572f5085c76c15cff31d700106aa46b18146da8ca14db299a1",
+      "full_token_count": 43,
+      "token_ids": [
+        111164,
+        3709,
+        112124,
+        103704,
+        98631,
+        98151,
+        95879,
+        3709,
+        97552,
+        103892,
+        101754,
+        104027,
+        1710,
+        95948,
+        96218,
+        101632,
+        97380,
+        99492,
+        3709,
+        96390,
+        97016,
+        120408,
+        112769,
+        97989,
+        1710,
+        120190,
+        112614,
+        111162,
+        3709,
+        125067,
+        121735,
+        108648
+      ]
+    },
+    {
+      "id": "cal-zh-2",
+      "split": "calibration",
+      "language": "zh",
+      "text": "\u4e00\u8f86\u5217\u8f66\u4ece\u5c71\u95f4\u9a76\u5411\u6d77\u8fb9\uff0c\u7ecf\u8fc7\u6751\u5e84\u3001\u6865\u6881\u548c\u5f00\u9614\u7684\u7530\u91ce\u3002\u65c5\u5ba2\u671b\u7740\u7a97\u5916\u53d8\u5316\u7684\u666f\u8272\uff0c\u8ba1\u5212\u62b5\u8fbe\u540e\u5148\u6b65\u884c\u5230\u65c5\u9986\uff0c\u518d\u53bb\u5e02\u573a\u8d2d\u4e70\u665a\u9910\u9700\u8981\u7684\u98df\u6750\u3002\u5929\u6c14\u9884\u62a5\u8bf4\u508d\u665a\u53ef\u80fd\u4e0b\u96e8\u3002",
+      "text_sha256": "442a5d62ba8c4a3ffda0e97b81c1f2fbdc31da045baa8ce44564e4ce5526f429",
+      "full_token_count": 45,
+      "token_ids": [
+        110708,
+        110768,
+        96200,
+        96144,
+        96037,
+        97523,
+        96127,
+        120243,
+        3709,
+        99442,
+        111028,
+        5205,
+        104619,
+        95793,
+        112965,
+        95726,
+        116239,
+        1710,
+        109080,
+        104553,
+        113733,
+        114462,
+        110868,
+        3709,
+        97787,
+        110677,
+        95946,
+        96218,
+        104608,
+        95839,
+        118309,
+        3709
+      ]
+    },
+    {
+      "id": "cal-en-1",
+      "split": "calibration",
+      "language": "en",
+      "text": "A small library opens every morning beside the river. The librarian checks returned books, repairs damaged covers, and places new arrivals on a wooden table. Visitors often stop to read before continuing their journey across the old bridge.",
+      "text_sha256": "c917f3d62e6ecaff19a324e6c0aa1436e33d7d89b436c758458349a4e0e2fe38",
+      "full_token_count": 43,
+      "token_ids": [
+        32,
+        2526,
+        6522,
+        15417,
+        1396,
+        6353,
+        28428,
+        279,
+        14367,
+        13,
+        561,
+        90956,
+        11988,
+        5743,
+        6265,
+        11,
+        29201,
+        19510,
+        14103,
+        11,
+        321,
+        7247,
+        491,
+        66710,
+        383,
+        264,
+        21664,
+        1898,
+        13,
+        66511,
+        3428,
+        2842
+      ]
+    },
+    {
+      "id": "cal-en-2",
+      "split": "calibration",
+      "language": "en",
+      "text": "To compare two measurements, first record their units and the conditions under which they were collected. Compute the difference, inspect unusual observations, and repeat the procedure with independent examples. A short report should separate observations from explanations and unresolved questions.",
+      "text_sha256": "ccb9be13ae9a881a9364211b20f31b6e11c75df482a02359625e7021b3869837",
+      "full_token_count": 46,
+      "token_ids": [
+        1206,
+        9150,
+        1330,
+        20934,
+        11,
+        1118,
+        3150,
+        836,
+        7896,
+        321,
+        279,
+        4530,
+        1172,
+        864,
+        781,
+        998,
+        14129,
+        13,
+        21902,
+        279,
+        6463,
+        11,
+        23331,
+        17957,
+        22980,
+        11,
+        321,
+        12771,
+        279,
+        10018,
+        440,
+        9209
+      ]
+    },
+    {
+      "id": "test-zh-1",
+      "split": "heldout",
+      "language": "zh",
+      "text": "\u5348\u540e\uff0c\u5de5\u7a0b\u5e08\u68c0\u67e5\u6e29\u5ba4\u4e2d\u7684\u4f20\u611f\u5668\u3002\u6e29\u5ea6\u8bb0\u5f55\u51fa\u73b0\u4e00\u4e2a\u5f02\u5e38\u503c\uff0c\u5979\u6ca1\u6709\u7acb\u523b\u5220\u9664\uff0c\u800c\u662f\u67e5\u770b\u8bbe\u5907\u72b6\u6001\u5e76\u8865\u505a\u4e00\u6b21\u6d4b\u91cf\u3002\u786e\u8ba4\u539f\u56e0\u4ee5\u540e\uff0c\u5979\u8c03\u6574\u4e86\u91c7\u6837\u8ba1\u5212\uff0c\u5728\u62a5\u544a\u4e2d\u8bf4\u660e\u8fd9\u6b21\u53d8\u5316\u3002",
+      "text_sha256": "5f30e958b262b83dd050d5d695b24cd160c966dc4151225c4e4bcb4be1ed7c96",
+      "full_token_count": 44,
+      "token_ids": [
+        118360,
+        3709,
+        98941,
+        97552,
+        106098,
+        96875,
+        108862,
+        1710,
+        99472,
+        98151,
+        97150,
+        96125,
+        100016,
+        96326,
+        3709,
+        133014,
+        103924,
+        99553,
+        3709,
+        101594,
+        97239,
+        96958,
+        98822,
+        96172,
+        96770,
+        134943,
+        108562,
+        1710,
+        101632,
+        97494,
+        99925,
+        3709
+      ]
+    },
+    {
+      "id": "test-zh-2",
+      "split": "heldout",
+      "language": "zh",
+      "text": "\u5468\u672b\uff0c\u5bb6\u4eba\u6cbf\u7740\u6e56\u8fb9\u7684\u5c0f\u8def\u6563\u6b65\u3002\u4ed6\u4eec\u5e26\u4e86\u6c34\u548c\u7b80\u5355\u7684\u5348\u9910\uff0c\u6253\u7b97\u5728\u6811\u6797\u65c1\u4f11\u606f\u3002\u9014\u4e2d\u9047\u5230\u4e00\u5904\u6b63\u5728\u7ef4\u4fee\u7684\u6865\uff0c\u4e8e\u662f\u53c2\u8003\u8def\u6807\u9009\u62e9\u53e6\u4e00\u6761\u9053\u8def\uff0c\u7ec8\u4e8e\u5728\u65e5\u843d\u524d\u56de\u5230\u8f66\u7ad9\u3002",
+      "text_sha256": "309dc66b4bb36ee28a0efe3f2ac91bced12fb3727f83fe4f556be583888d2f69",
+      "full_token_count": 44,
+      "token_ids": [
+        104195,
+        3709,
+        99993,
+        110530,
+        131667,
+        97626,
+        96112,
+        112869,
+        1710,
+        96697,
+        125485,
+        114701,
+        101829,
+        110637,
+        3709,
+        108758,
+        95772,
+        113762,
+        97760,
+        100020,
+        1710,
+        104992,
+        99475,
+        112430,
+        99443,
+        99523,
+        95726,
+        97461,
+        3709,
+        100556,
+        97910,
+        96112
+      ]
+    },
+    {
+      "id": "test-en-1",
+      "split": "heldout",
+      "language": "en",
+      "text": "The gardener planted several rows of beans behind the house. Each evening she checked the soil and wrote down how much water the plants received. After a week of warm weather, the first green shoots appeared near the fence beside the apple tree.",
+      "text_sha256": "9f25e796901243c848d93ec8c3dd74a13b50e8207388ff41d24cdfba3869db62",
+      "full_token_count": 48,
+      "token_ids": [
+        760,
+        21208,
+        772,
+        37070,
+        3679,
+        6761,
+        314,
+        26034,
+        4661,
+        279,
+        3626,
+        13,
+        8618,
+        11129,
+        1292,
+        9770,
+        279,
+        16665,
+        321,
+        5947,
+        1441,
+        1204,
+        1691,
+        2919,
+        279,
+        10462,
+        3816,
+        13,
+        4486,
+        264,
+        1936,
+        314
+      ]
+    },
+    {
+      "id": "test-en-2",
+      "split": "heldout",
+      "language": "en",
+      "text": "When a program reads a table, it should check column names and handle missing entries explicitly. The developer tested a small example, compared the output with a hand calculation, and saved the input alongside the result so another person could reproduce the check.",
+      "text_sha256": "6c47a00b9b81ae364d9cedb79f7ba46fbb104fe90ce35b6e742e45c6b106ae9a",
+      "full_token_count": 48,
+      "token_ids": [
+        4350,
+        264,
+        1957,
+        15339,
+        264,
+        1898,
+        11,
+        424,
+        1220,
+        1716,
+        3145,
+        4874,
+        321,
+        3579,
+        7167,
+        10379,
+        20335,
+        13,
+        561,
+        15291,
+        12152,
+        264,
+        2526,
+        3010,
+        11,
+        7463,
+        279,
+        2468,
+        440,
+        264,
+        1375,
+        21261
+      ]
+    }
+  ]
+}

results/cpu-qwen35-2b/selection.json

@@ -0,0 +1,4 @@
+{
+  "selected": [],
+  "rule": "calibration pass; descending component bytes, ascending maximum symmetric error; maximum one candidate"
+}