fix(pilot): preserve partial results and stop on provider billing errors
This commit is contained in:
@@ -44,6 +44,9 @@ Labeling must be finished before anyone opens `results.jsonl`.
|
||||
- Metadata is stored beside the output as `results.meta.json`. Model, endpoint, temperature,
|
||||
input hash, frozen-label hash and prompt hashes must match on resume. Usage is cumulative.
|
||||
A file lock prevents concurrent writers. A budget stop returns exit status 2.
|
||||
- Billing/authentication failures stop immediately. Resume retries only missing detector components,
|
||||
keeping both complete items and the successful side of partial items. Rows are replaced atomically.
|
||||
`--audit-only` reconciles saved completion counts and status without any API request.
|
||||
- The nominal successful path uses 130 calls (one source extraction and one judge per item);
|
||||
malformed replies and network retries may increase the actual count.
|
||||
- Scoring requires complete labels and results and no unresolved detector errors. The primary
|
||||
@@ -52,3 +55,11 @@ Labeling must be finished before anyone opens `results.jsonl`.
|
||||
by the judge. No threshold, concept table or detector prompt was tuned for these labels.
|
||||
- Raw passages, labeling HTML and model responses remain in the ignored `data/` directory.
|
||||
Only verdicts, abstract annotation notes, hashes and text-free summaries belong in Git.
|
||||
|
||||
## Current execution
|
||||
|
||||
The 2026-10-28 run was blocked by DashScope `Arrearage`: 42 items are complete, 23 need
|
||||
42 detector components, and 131 of the 400 request reservations have been consumed.
|
||||
The owner requested fixes and a report now, with paid resumption deferred.
|
||||
See `docs/experiments/P0-execution-2026-10-28.md` and `results/execution.json`.
|
||||
No gate decision has been issued. Restore provider billing before resuming the same run.
|
||||
|
||||
Reference in New Issue
Block a user