feat(ai): v5 extreme-alpha prompt — checklist gate + drop sell signal

Why
  v4 was firing buy/short on 13% of posts, but only 9% of those had a
  ≥1% move within the hour. Median move on 'actionable' was 0.298% vs
  0.258% on 'hold' — a 1.15× signal-to-noise ratio (random would be 1.0).
  The model was confabulating transmission chains to please the user
  rather than holding when uncertain.

  Separately: 'sell' meant 'close longs / de-risk' in the prompt but
  was traded as 'open short' by bot_engine.py, producing systematically
  negative results on sell signals (27% win rate vs 57% on real shorts).

What changed
  • analysis.py rewritten as v5-extreme-alpha:
    - Asymmetric error costs framing (false positive = -$30, FN = $0)
    - 7-item checklist that MUST all pass before buy/short
    - Only 4 named transmission paths (a/b/c/d); anything else = HOLD
    - 5 positive + 5 negative few-shot examples
    - UTC hour injected with liquidity context (Asia thin → stricter)
    - Adversarial steelman self-check before final output
    - confidence < 80 + checklist failure both force-collapse to HOLD
      in code, regardless of what the model returns (defense-in-depth)
    - 'sell' removed from output schema entirely
  • bot_engine.py: stop trading 'sell' signals (treat as hold)
  • Case-insensitive normalization on checklist values so model
    returning 'None'/'True' (capitalized) doesn't slip through

Expected impact (to validate over next 2-3 weeks of new posts)
  • actionable rate: 13% → 2-4%
  • signal/hold MFE ratio: 1.15× → 3-5×
  • ≥1% hit rate among actionable: 9% → 40-60%

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
k
2026-05-08 15:00:44 +08:00
parent 4ffcb442fe
commit 747158b5ed
2 changed files with 276 additions and 106 deletions
+4 -2
View File
@@ -70,12 +70,14 @@ async def process_post(post: Post, db: AsyncSession) -> None:
"""
if not post.relevant:
return
if post.signal not in ('buy', 'short', 'sell'):
# v5: "sell" is no longer emitted by the AI. Old DB rows might still have
# it; treat as hold (do not trade) — see analysis.py docstring for why
# the previous "sell→short" treatment was a semantic bug.
if post.signal not in ('buy', 'short'):
logger.info("Post %d skipped: signal=%s is not actionable", post.id, post.signal)
return
asset = post.price_impact_asset or 'BTC'
# "sell" is treated as a short signal (bearish directional trade)
side = 'long' if post.signal == 'buy' else 'short'
logger.info("Signal: post=%d asset=%s side=%s confidence=%d", post.id, asset, side, post.ai_confidence)