84 lines
5.1 KiB
Markdown
84 lines
5.1 KiB
Markdown
# Qwen3.5-2B parameter similarity pilot
|
|
|
|
- Revision: `15852e8c16360a2fea060d615a32b45270f8a8fc`
|
|
- Stored parameters: 2,274,069,824
|
|
- Weight bytes: 4,548,144,832
|
|
- Tensor dimensionalities: {'1': 319, '2': 294, '3': 18, '5': 1}
|
|
- Analysis and grouping/export wall time: 6661.1 seconds
|
|
- Pair-comparison cutoff: 6600 seconds; final grouping/export is additional and remains under the supervisor hard limit.
|
|
- Peak analysis RSS: 201.3 MiB
|
|
|
|
## Verified sharing estimates
|
|
|
|
| Representation | Weight-error limit | Verified sharing groups | Conservative total groups* | Largest group | Net model savings |
|
|
|---|---:|---:|---:|---:|---:|---:|
|
|
| whole | 1% | 0 | 632 | 1 | 0.000000% |
|
|
| whole | 2% | 0 | 632 | 1 | 0.000000% |
|
|
| whole | 5% | 1 | 631 | 2 | 0.000000% |
|
|
| whole | 10% | 11 | 613 | 6 | 0.000719% |
|
|
| whole | 20% | 16 | 572 | 15 | 0.002626% |
|
|
| 2d-128 | 1% | 0 | 138576 | 1 | 0.000000% |
|
|
| 2d-128 | 2% | 0 | 138576 | 1 | 0.000000% |
|
|
| 2d-128 | 5% | 0 | 138576 | 1 | 0.000000% |
|
|
| 2d-128 | 10% | 0 | 138576 | 1 | 0.000000% |
|
|
| 2d-128 | 20% | 1 | 138575 | 2 | 0.000000% |
|
|
| 2d-64 | 1% | 0 | 554304 | 1 | 0.000000% |
|
|
| 2d-64 | 2% | 0 | 554304 | 1 | 0.000000% |
|
|
| 2d-64 | 5% | 0 | 554304 | 1 | 0.000000% |
|
|
| 2d-64 | 10% | 1 | 554303 | 2 | 0.000000% |
|
|
| 2d-64 | 20% | 3 | 554299 | 3 | 0.000000% |
|
|
| 3d-axis0-slice | 1% | 0 | 110592 | 1 | 0.000000% |
|
|
| 3d-axis0-slice | 2% | 0 | 110592 | 1 | 0.000000% |
|
|
| 3d-axis0-slice | 5% | 9 | 110581 | 3 | 0.000000% |
|
|
| 3d-axis0-slice | 10% | 33 | 110542 | 7 | 0.000000% |
|
|
| 3d-axis0-slice | 20% | 45 | 110483 | 12 | 0.000000% |
|
|
| 3d-volume | 1% | 0 | 27648 | 1 | 0.000000% |
|
|
| 3d-volume | 2% | 0 | 27648 | 1 | 0.000000% |
|
|
| 3d-volume | 5% | 0 | 27648 | 1 | 0.000000% |
|
|
| 3d-volume | 10% | 0 | 27648 | 1 | 0.000000% |
|
|
| 3d-volume | 20% | 2 | 27646 | 2 | 0.000000% |
|
|
|
|
*Conservative total groups counts unexamined full blocks as singletons; boundary regions and ineligible tensors stay unchanged and are outside that block count.
|
|
Savings count only verified replacement edges. Unsampled blocks remain unchanged. Different block scales must not be summed.
|
|
Theoretical encoding: one stored representative per group, 16 bytes per object mapping and 64 bytes per shared-group descriptor; identity fallback if larger.
|
|
Exact grouping uses direct replacement only. Scaled and spectral results are diagnostic.
|
|
|
|
## Stratified diagnostics
|
|
|
|
See `analysis/role_summary.csv` for same-role, same-shape statistics, separated by tensor dimensionality and block scale.
|
|
See `analysis/dimensional_bytes.csv` for tensor counts versus weight-byte contributions by dimensionality.
|
|
Heatmap missing pairs are hatched (`heatmap_pairs.csv`); identity entries are not empirical matches.
|
|
Layer-distance medians are split by dimension/scale (`layer_distance_stratified.csv`); role and selection bias remain.
|
|
Controls are split by dimension and density-normalized, with sample counts (`null_histograms.csv`); these are not significance tests.
|
|
These are screened-candidate statistics, not estimates of all possible pairs. Small normalization vectors must not be conflated with large matrices.
|
|
|
|
## Coverage
|
|
|
|
- tensor: `{"eligible_pairs_in_screened_objects": 6159, "candidate_pairs_before_cap": 4724, "selected_pairs": 2000, "objects_screened": 632, "pairs_verified": 2000, "timed_out": false, "screen_sample_elements": 2048, "candidate_neighbors_per_metric": 10}`
|
|
- block: `{"eligible_pairs_in_screened_objects": 1678968, "candidate_pairs_before_cap": 175036, "selected_pairs": 20000, "objects_screened": 13328, "pairs_verified": 14778, "timed_out": true, "screen_sample_elements": 2048, "candidate_neighbors_per_metric": 10}`
|
|
- Complete-block partition totals (per alternative): `{"2d-64": {"blocks": 554304, "bytes": 4540858368, "edge_bytes": 0}, "2d-128": {"blocks": 138576, "bytes": 4540858368, "edge_bytes": 0}, "3d-axis0-slice": {"blocks": 110592, "bytes": 884736, "edge_bytes": 0}, "3d-volume": {"blocks": 27648, "bytes": 884736, "edge_bytes": 0}}`
|
|
|
|
## Interpretation limits
|
|
|
|
- No inference or functional equivalence measured
|
|
- No exhaustive pair search
|
|
- Block search uses per-role/shape/scale reservoirs of 256
|
|
- Different block scales overlap; savings must not be added
|
|
- Block savings count only verified disjoint sampled blocks, no extrapolation
|
|
- Spectral probes use aligned bounded subvolumes, not whole tensors
|
|
- Shared encoding is theoretical, not a runnable compressed checkpoint
|
|
- No cross-role grouping, neuron permutation alignment, or training
|
|
- Candidate histograms are selection-biased and do not represent the distribution of all tensor pairs.
|
|
- 1D normalization weights may cluster strongly but carry few bytes; do not infer material compression from group counts.
|
|
- No task accuracy, perplexity, inference speed or runtime-memory improvement has been measured.
|
|
- Stored parameter counts may omit tied aliases; no duplicate-weight equality search is performed.
|
|
|
|
## Figures
|
|
|
|

|
|

|
|

|
|

|
|

|
|

|