An English-language prototype that combines spaCy's tokenizer and learned dependency parser with explicit graph rules. The graph interface can support other encoders and relation miners as the prototype evolves.

The road flooded because it rained.
  → Cause: it rained. Effect: The road flooded.

Run with Python 3.12 after installing requirements.txt.

python -m syntax_dag --commit 'The road flooded because it rained.'
python -m unittest -v

Omit --commit to recover the original token surfaces and whitespace. The command prints one final newline.

Encoder creates token nodes with original whitespace and head-to-dependent edges. The causal rule adds clause nodes with edges to their member tokens. A token can have incoming edges from both a clause and a syntactic head.
The rule mines a candidate cause → effect edge with provenance from a trailing because clause. The candidate stays separate until committed.
SyntaxDAG.commit creates a new immutable graph revision and records the proposal. A cycle rejects the entire edit. Repeating a commit on the same revision returns that revision.
Decoder uses committed causal edges to select the clause order and labels. Text outside the selected clauses retains its original surface form.

The first rule covers simple affirmative past-tense statements with a trailing finite causal clause. Extraction is heuristic and depends on parser accuracy. Commits are in-memory graph edits. The prototype uses explicit authorization via the CLI flag and has no learned rewrite policy.

Model token IDs

python -m syntax_dag --ids --commit 'The road flooded because it rained.'

JSON includes encoding, input_ids, output_ids, and the output text. These are integer vocabulary IDs from tiktoken's o200k_base BPE encoding. spaCy tokens still serve as syntax nodes. Their positions are independent of BPE positions. A commit preserves the original input IDs, and the decoder tokenizes the full rendered text so its output IDs reflect the new boundaries. The tokenizer downloads its vocabulary on first use if it is absent from cache.