69 lines
4.2 KiB
Markdown
69 lines
4.2 KiB
Markdown
# Bounded CPU functional pilot
|
|
|
|
This is the execution implementation of the CPU-first plan, not the completed static
|
|
parameter pilot. Frozen Qwen3.5-2B weights are decoded to FP32 a projection block at a
|
|
time. CPU only, no autograd/optimization, no rotation fitting, no model-hub Python code.
|
|
The user explicitly requires retaining this checkpoint until they request deletion.
|
|
|
|
## Fixed scope
|
|
|
|
- Batch of eight distinct experiment-authored plain texts, first 32 tokens each:
|
|
four calibration and four heldout, two Chinese/two English in each split.
|
|
- Full attention pairs: (3,7), (11,15), (19,23).
|
|
- MLP pairs: (5,6), (20,21).
|
|
- Smaller-granularity Q/K RMSNorm pairs inside layers 3,7,11,15,19,23.
|
|
Added before observing real weights/functions for this pilot; these are small storage
|
|
opportunities, not a claim of substantial compute savings.
|
|
- At most one calibration-selected candidate, two separate substitution directions.
|
|
Selection prefers component size, then worst symmetric output error. No heldout-driven retry.
|
|
- Same-input relative L2 <=0.05 in both directions and token P95 <=0.1 in both directions;
|
|
attention additionally requires mean JS <=0.01 and top-5 overlap >=0.8.
|
|
- Heldout mean NLL increase <=0.01 nat/token and per-sequence P95 increase <=0.05.
|
|
These are pilot decision criteria, not mathematical capability guarantees.
|
|
|
|
## Source and checks
|
|
|
|
`reference.json` pins the official Transformers Qwen3.5 source commit, URL and hash.
|
|
`reference_check.py` extracts only named mathematical definitions from that exact file,
|
|
removes optional fused-kernel decorators, and compares the NumPy implementation against
|
|
Torch operators, full toy decoder layers and a two-layer text forward. Chunked delta and
|
|
recurrent delta implementations are both checked, including sequence padding boundaries.
|
|
The official source is Apache-2.0, copyright Qwen Team and HuggingFace; the reference file
|
|
is fetched for testing and is not bundled here. FP32 equivalence is checked, not equality
|
|
to every deployed BF16 kernel.
|
|
|
|
`tests/test_functional.py` independently checks streamed projection/embedding/NLL,
|
|
causality, causal convolution, normalization conventions and basic metric identities.
|
|
These tests use tiny synthetic arrays. Real model evidence is produced only by `run.py`.
|
|
|
|
## Remote execution
|
|
|
|
Use the previously authorized CPU host and existing NumPy/Matplotlib environment. Install
|
|
only binary wheels: tokenizers==0.23.3 from HTTPS PyPI; torch==2.15.0+cpu from the official
|
|
CPU wheel index for independent reference checks. Use an experiment-owned on-disk TMPDIR:
|
|
the shared host's /tmp tmpfs was too small during the first wheel download. Do not change
|
|
system swap or shared temporary files.
|
|
|
|
```sh
|
|
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m unittest tests.test_functional -v
|
|
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m functional.reference_check --source runs/cpu-qwen35-2b/reference.py --output runs/cpu-qwen35-2b/reference-check.json
|
|
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 venv/bin/python -B -m functional.supervise --root runs/cpu-qwen35-2b
|
|
```
|
|
|
|
The supervisor waits at most one hour for binary dependencies and four hours for the
|
|
already started pinned, verified checkpoint download. It does not install, retry downloads,
|
|
search extra candidates, mutate Git or delete weights. After reference checks, it runs one
|
|
8-token real text capacity check, then the bounded functional pilot. Reference, preflight
|
|
and functional compute together are capped at two hours with a reporting reserve;
|
|
download wait is recorded separately. Projection memory is checked, RSS soft stop is
|
|
512 MiB, available-system-memory floor 256 MiB; sustained host swap IO also stops the run.
|
|
|
|
`preflight.py` is a capacity smoke check, not a semantic oracle. `run.py` records exact
|
|
sample/token IDs, fixed configuration, source hashes, activations, per-text/per-head metrics,
|
|
selection, per-token NLL, intervention results and truthful stop coverage. `report.py` reads
|
|
saved data to produce PNG/PDF/SVG figures without reopening weights.
|
|
|
|
No generation throughput is measured in this short-text pilot. Any parameter-tying storage
|
|
benefit must be accounted for separately, including metadata. Neither same-input similarity
|
|
nor favorable tiny heldout NLL demonstrates broad mutual interchangeability or GPU speedup.
|