feat(ai): v5 extreme-alpha prompt — checklist gate + drop sell signal
Why
v4 was firing buy/short on 13% of posts, but only 9% of those had a
≥1% move within the hour. Median move on 'actionable' was 0.298% vs
0.258% on 'hold' — a 1.15× signal-to-noise ratio (random would be 1.0).
The model was confabulating transmission chains to please the user
rather than holding when uncertain.
Separately: 'sell' meant 'close longs / de-risk' in the prompt but
was traded as 'open short' by bot_engine.py, producing systematically
negative results on sell signals (27% win rate vs 57% on real shorts).
What changed
• analysis.py rewritten as v5-extreme-alpha:
- Asymmetric error costs framing (false positive = -$30, FN = $0)
- 7-item checklist that MUST all pass before buy/short
- Only 4 named transmission paths (a/b/c/d); anything else = HOLD
- 5 positive + 5 negative few-shot examples
- UTC hour injected with liquidity context (Asia thin → stricter)
- Adversarial steelman self-check before final output
- confidence < 80 + checklist failure both force-collapse to HOLD
in code, regardless of what the model returns (defense-in-depth)
- 'sell' removed from output schema entirely
• bot_engine.py: stop trading 'sell' signals (treat as hold)
• Case-insensitive normalization on checklist values so model
returning 'None'/'True' (capitalized) doesn't slip through
Expected impact (to validate over next 2-3 weeks of new posts)
• actionable rate: 13% → 2-4%
• signal/hold MFE ratio: 1.15× → 3-5×
• ≥1% hit rate among actionable: 9% → 40-60%
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -70,12 +70,14 @@ async def process_post(post: Post, db: AsyncSession) -> None:
|
||||
"""
|
||||
if not post.relevant:
|
||||
return
|
||||
if post.signal not in ('buy', 'short', 'sell'):
|
||||
# v5: "sell" is no longer emitted by the AI. Old DB rows might still have
|
||||
# it; treat as hold (do not trade) — see analysis.py docstring for why
|
||||
# the previous "sell→short" treatment was a semantic bug.
|
||||
if post.signal not in ('buy', 'short'):
|
||||
logger.info("Post %d skipped: signal=%s is not actionable", post.id, post.signal)
|
||||
return
|
||||
|
||||
asset = post.price_impact_asset or 'BTC'
|
||||
# "sell" is treated as a short signal (bearish directional trade)
|
||||
side = 'long' if post.signal == 'buy' else 'short'
|
||||
|
||||
logger.info("Signal: post=%d asset=%s side=%s confidence=%d", post.id, asset, side, post.ai_confidence)
|
||||
|
||||
Reference in New Issue
Block a user