logic-skeleton-bench/docs/skeleton-language.md

Dynamic World Generator and Intermediate Language

This specification defines the answer-bearing world representation generated before natural-language rendering.

Pipeline boundary

HiddenProgram + seed
        ↓
Initial WorldState
        ↓ stochastic transition sampling
Realized Event Trace
        ↓ deterministic closure and query selection
Query + Gold Proof
        ↓ redact program, hidden state, proof, and answer
Renderer Payload
        ↓
Natural-language history → Solver

Randomness affects which events occur. It never makes the final question probabilistic: after the trace is realized, the evaluator asks whether a proposition follows in that particular world.

Hidden program

HiddenProgram ::= {
  types: TypeName+,
  relations: PredicateName+,
  operators: OperatorName+,
  quantifiers: QuantifierName+,
  structures: StructureName+,
  inference_rules: RuleExpression+,
  transition_rules: TransitionRule+
}

TransitionRule ::= {
  id,
  trigger,
  preconditions: Expression*,
  probability: Number[0,1],
  effects: Expression+
}

The current finite vocabulary includes:

types:
  Entity, Set, Region, Number, Boolean, State

relations:
  EQUAL, NOT_EQUAL, ABOVE, BELOW,
  MEMBER, SUBSET, DISJOINT,
  ADJACENT, CONNECTED, CAUSES, LOCATED_IN

operators:
  NOT, AND, OR, IMPLIES,
  ADD, SUBTRACT, COMPARE,
  UNION, INTERSECTION,
  MOVE, SWAP, SET, COMPOSE

quantifiers:
  ALL, EXISTS

structures:
  CHAIN, BRANCH, MERGE, CYCLE, NEST, CONDITION, SEQUENCE

Not every primitive must affect every query. The point of the shared world is that many logic domains evolve concurrently and become realistic distractors for one proof-bearing target.

Entities and initial state

Entity ::= { id, type }

WorldState ::= {
  locations,
  directed_order_relations,
  memberships,
  subset_relations,
  disjoint_relations,
  connections,
  numeric_values,
  boolean_flags,
  hidden_generation_attributes
}

The default 128-entity allocation includes ordinary entities, sets, regions, and numeric counters. Entity IDs are abstract symbols; the renderer must assign stable names within its chosen scenario.

Some initial facts are public, such as locations and counter values. Each has an I#### audit ID. Latent generation attributes, such as levels used to keep randomly generated order and subset graphs consistent, are stored for audit but removed from the renderer payload.

Event language

WorldEvent ::= {
  id,
  time,
  kind,
  participants,
  arguments
}

Current realized event forms are:

MOVE(entity, origin, destination)
RELATE(ABOVE, higher, lower, region)
RELATE(MEMBER, entity, set)
RELATE(SUBSET, child_set, parent_set)
RELATE(DISJOINT, left_set, right_set)
RELATE(CONNECTED, left_entity, right_entity)
UPDATE_NUMBER(counter, ADD|SUBTRACT, amount, before, after)
SET_STATE(entity, flag, before, after)

Every simulator step selects a candidate transition type. Preconditions are evaluated, then the transition's probability determines whether an event actually happens. Only realized events enter the trace. A fixed seed makes the complete trace reproducible.

The default short configuration runs 1000 candidate steps and currently realizes about 400 events. Thus probability controls world evolution, while the resulting event history is an ordinary deterministic collection of facts and changes.

Current inference semantics

ABOVE(x,y) ∧ ABOVE(y,z) → ABOVE(x,z)
ABOVE(x,y) → BELOW(y,x)

MEMBER(x,A) ∧ SUBSET(A,B) → MEMBER(x,B)
MEMBER(x,A) ∧ DISJOINT(A,B) → ¬MEMBER(x,B)

CONNECTED(x,y) ↔ CONNECTED(y,x)

numeric updates execute in timestamp order
later SET_STATE operations replace the prior value of the same flag
MOVE replaces the current region of its participant

ABOVE and SUBSET events are generated from adjacent hidden levels. This allows long chains while preventing contradictory cycles. The levels are generator machinery, not observable premises.

Query generation

Query ::= {
  predicate,
  arguments,
  time
}

After simulation, the query builder searches the realized graphs rather than inserting a prefabricated puzzle. Current query families are:

• spatial_order: transitive ABOVE proof;
• reverse spatial_order: a NO answer proved by an opposite-direction chain;
• membership_inheritance: membership propagated through nested subsets;
• membership_disjointness: inherited membership in one set plus disjointness from the queried set.

For a requested complexity k, the selected proof contains at least k realized event IDs. The queried proposition itself must not appear as a direct event. The full record stores both the event IDs and normalized proof facts.

Current generation checks record that:

• no event directly states the query;
• proof length meets the requested minimum;
• proof events are a small fraction of the full history;
• simple direct-fact and too-short-proof baselines fail.

These checks are structural. They should be expanded with executable solver baselines as the benchmark grows.

Full skeleton

WorldSkeleton ::= {
  id,
  family,
  complexity,
  entity_count,
  simulation_steps,
  hidden_program,
  entities,
  initial_state,
  events,
  final_state,
  query,
  answer_options,
  gold_answer,
  proof_event_ids,
  proof_facts,
  generator_checks
}

skeletons.jsonl contains this complete record for reproducibility and auditing.

Renderer boundary

The renderer receives only:

family
complexity
entity_types
initial_observations
events
query
answer_options

It does not receive:

world id
hidden program or transition probabilities
latent initial-state attributes
complete final state
gold answer
proof event IDs or proof facts
generator checks

Rendering uses one planning request followed by bounded history chunks. The plan fixes one setting and style, while the evaluator itself creates a deterministic symbol-to-name map and supplies that exact complete map to every chunk. render_chunk_size is the maximum records per chunk and defaults to 60. It is configurable in a task file or through --render-chunk-size.

The renderer must use the supplied stable surface names, preserve all events and their direction and ordering, and produce one chronological fictional history. Every rendered initial fact and event line retains its I#### or V##### audit ID. The evaluator concatenates chunks in source order and rejects a rendering when any expected ID is absent. The renderer may compress formatting, but it may not summarize away distractors, state hidden rules, expose the proof, or directly assert the queried proposition.

Complexity dimensions

The axes are deliberately independent:

length             minimum gold-proof events
entity_count       total typed entities
simulation_steps   candidate evolution steps
event_count        realized events; measured after simulation
render_chunk_size  maximum records in one rendering request; default 60

Named proof presets are short=4, medium=6, and long=8. The task file separately controls world size and simulation duration.

Planned extensions

The IR is intended to grow without changing the visibility boundary. Useful next additions include:

• relation deletion and expiry;
• explicit event causation chains;
• multi-effect transactions and conflict policies;
• union and intersection queries;
• quantified ALL and EXISTS questions;
• temporal queries about intermediate rather than final state;
• counterfactual interventions;
• executable nearest-neighbor, keyword, local-window, and shallow-proof baselines;
• renderer fidelity verification against every source event.