Document skeleton IR and retain evaluation artifacts
This commit is contained in:
@@ -14,6 +14,8 @@ Three finite skeleton families are included:
|
||||
|
||||
Every generated skeleton contains its facts, rules, query, permitted answer labels, deterministic `gold_answer`, and proof trace. The renderer receives a copy with `gold_answer` and `proof` removed.
|
||||
|
||||
The full field grammar, predicate signatures, generator invariants, and renderer boundary are defined in [the Skeleton IR specification](docs/skeleton-language.md).
|
||||
|
||||
`length` means logical complexity, not natural-language tokens. It is the number of proof-bearing facts or state updates. Presets are `short=2`, `medium=4`, and `long=6`; an integer from 1 through 20 is also accepted.
|
||||
|
||||
## Local use
|
||||
@@ -54,6 +56,8 @@ Outputs are written under the task's `output_dir`:
|
||||
|
||||
The workflow `.gitea/workflows/short-eval.yml` has only `workflow_dispatch`, so it consumes model quota only when someone manually clicks **Run workflow**. It runs three cases with `length: short` and has a 15-minute timeout.
|
||||
|
||||
The run log prints the rendered questions and scores. The same records, original answer-bearing skeletons, and summary are retained in the `short-abstraction-evaluation` Actions artifact for inspection.
|
||||
|
||||
Create these repository Actions settings:
|
||||
|
||||
1. Secret `OPENAI_API_KEY`, or separate secrets `RENDERER_API_KEY` and `SOLVER_API_KEY`.
|
||||
|
||||
Reference in New Issue
Block a user