Integrate the syntax DAG and model token IDs #3
Reference in New Issue
Block a user
Delete Branch "codex/dag-tokenizer-integration"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Consolidates the syntax DAG from #1 (already in main) and the model-token IDs from #2 (merged into this branch with original history). Review this PR for the combined delivery. The diff against main contains only the tokenizer addition.
Validation passed on ubuntu24.04-x64-lean4.35: https://gitea.entroia.com/Zayden-Jung/syntax-dag-transformer/actions/runs/750