Record bounded Qwen3.5-2B pilot and stratified diagnostics

docs/SOP.md

@@ -1,6 +1,8 @@
 # Parameter similarity SOP — 2B pilot draft
 
-Status: implementation draft; findings and revisions are added after the first real run.
+Status: v0.1, executed for the bounded Qwen3.5-2B pilot. See [findings](PILOT_FINDINGS.md).
+Changes proposed from this experience require a small equivalence/performance check before
+the next model; no second model is started automatically.
 
 ## Scope and decision
 
@@ -62,7 +64,9 @@ Weight error is not task-accuracy loss, and parameter sharing does not imply few
 - Save exact-pair tables every 100 pairs; stage deadlines prevent whole-tensor work
   from consuming the complete block budget.
 - On timeout or failure, retain source weights until partial results are recoverable
-  and report coverage/failure truthfully. Never call an incomplete run complete.
+  and report coverage/failure truthfully. An internal planned cutoff may produce a complete
+  bounded report with partial candidate coverage; always state verified/selected counts.
+  An external kill or missing required report is an incomplete run, not a completed experiment.
 - Empty or negligible low-error sharing is a valid negative result. Do not add scales,
   increase error thresholds, align permutations, train or expand to other models automatically.
 - No runtime capability/performance claim without separately authorized evaluation.
@@ -70,10 +74,12 @@ Weight error is not task-accuracy loss, and parameter sharing does not imply few
 ## Commands (remote, from project root)
 
 ```sh
-OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest discover -s tests -v
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 timeout 300 venv/bin/python -B -m unittest discover -s tests -v
 venv/bin/python -B -m pilot.download --root runs/qwen35-2b
 OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 timeout 7200 venv/bin/python -B -m pilot.analyze --root runs/qwen35-2b --seconds 6600
 OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m pilot.plot --root runs/qwen35-2b
+OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 timeout 300 venv/bin/python -B -m unittest discover -s tests -v > runs/qwen35-2b/tests.log 2>&1
+venv/bin/python -B -m pilot.audit --root runs/qwen35-2b --analysis-code-commit 6121016
 venv/bin/python -B -m pilot.validate --root runs/qwen35-2b
 # Copy archived results, verify checksums at the destination, then:
 venv/bin/python -B -m pilot.validate --root runs/qwen35-2b --cleanup-weights
@@ -81,6 +87,23 @@ venv/bin/python -B -m pilot.validate --root runs/qwen35-2b --cleanup-weights
 
 ## Follow-up decisions after the pilot
 
+- The pilot verified 2,000 tensor pairs and 14,778/20,000 selected block pairs before cutoff.
+  It took about 111 minutes; the final six-test suite including revised figure checks passed in about 77 seconds.
+- Hatch missing heatmap cells and label identity values as constructed. Separate layer-distance
+  curves by dimension/scale; retain warnings about within-stratum module/selection bias.
+  Normalize control histograms, display sample counts, and verify their areas equal one.
+  Keep these diagnostics distinct from task-equivalence or statistical-significance claims.
+- Avoid concurrent plotting tests and heavy weight IO on the shared host. Record IO wait,
+  swap, CPU and RSS; a low process RSS is not proof of low total resource cost.
+- Distinguish tensor/group counts from byte contributions: 1D tensors were about half the
+  inventory count but only 0.0197% of weight bytes in this checkpoint.
+- Keep fair candidate selection separate from IO-local execution order. Benchmark batching
+  by source file/tensor without changing the selected pairs or metrics.
+- The sampled raw blocks total about 244.5 MiB. Test an explicitly bounded raw BF16/F32
+  cache against the uncached implementation before adopting it; include fingerprint and
+  temporary conversion memory in the budget. This was not enabled in the current run.
+- Report both pre-metadata and net savings. Tiny blocks can have groups yet lose all savings
+  to mapping overhead. Any alternative sparse encoding needs its own explicit accounting.
 - Separate low-byte normalization groups from matrix/expert compression opportunities.
 - Decide whether approximate-neighbor recall and block coverage justify another bounded pass.
 - If meaningful candidates exist, request a small calibration-forward experiment to measure