Compare commits
13 Commits
06d367f073
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 k
|
1ab553f778 | ||
|
k
|
fa1700d9e8 | ||
|
k
|
57d39d0d4b | ||
|
k
|
78fb63be8e | ||
|
k
|
54884f3e24 | ||
|
k
|
213bb911e3 | ||
|
k
|
041f010a30 | ||
|
k
|
0d88e3e43a | ||
|
k
|
752652f463 | ||
|
k
|
e2c00937b5 | ||
|
k
|
fd2e61dc48 | ||
|
k
|
eb8ee9ea91 | ||
|
k
|
cc1dda832b |
@@ -69,3 +69,30 @@ HL_MAINNET=true
|
||||
# auto-runs create_all() on startup (Alembic bypass — DEV ONLY).
|
||||
# production → both are off; schema strictly Alembic-managed.
|
||||
ENVIRONMENT=production
|
||||
|
||||
# ── X (Twitter) auto-poster ───────────────────────────────────────────────────
|
||||
# Viral "live prediction" tweets: when Trump posts an actionable signal, the bot
|
||||
# tweets the call instantly, then posts a result follow-up X_FOLLOWUP_MINUTES later.
|
||||
# OAuth 1.0a User Context. From developer.x.com → your App → "Keys and tokens".
|
||||
# ⚠️ App permissions MUST be "Read and Write" — then REGENERATE the access token.
|
||||
# ⚠️ Do NOT use Bearer Token / Client ID / Client Secret (those are read/OAuth2).
|
||||
X_API_KEY=
|
||||
X_API_SECRET=
|
||||
X_ACCESS_TOKEN=
|
||||
X_ACCESS_SECRET=
|
||||
# Master switch. false = dry-run (logs tweet text, sends nothing) even with keys set.
|
||||
X_ENABLED=false
|
||||
# Only tweet when ai_confidence >= this (0–100).
|
||||
X_MIN_CONFIDENCE=60
|
||||
# Hard daily tweet cap (initial + follow-ups), resets at UTC midnight.
|
||||
X_DAILY_CAP=40
|
||||
# Minutes after the prediction tweet to post the result follow-up.
|
||||
X_FOLLOWUP_MINUTES=15
|
||||
# Public URL appended to the follow-up tweet (defaults to FRONTEND_URL if empty).
|
||||
X_LINK_URL=
|
||||
|
||||
# ── X (Twitter) READ — KOL tweet ingestion (kol_x.py) ────────────────────────
|
||||
# twitterapi.io managed-scraper key (x-api-key). Reads other people's tweets —
|
||||
# unrelated to the OAuth poster creds above. Empty = X ingestion disabled.
|
||||
# Get a pay-as-you-go key at twitterapi.io (~$0.15 / 1k tweets).
|
||||
TWITTERAPI_IO_KEY=
|
||||
|
||||
@@ -0,0 +1,609 @@
|
||||
# Trump Alpha — Backend
|
||||
|
||||
> AI-powered crypto signal aggregator. Surfaces four uncorrelated signal
|
||||
> streams (Trump Truth Social, Macro Vibes, KOL talks-vs-trades, funding
|
||||
> reversal) and runs an optional execution layer on Hyperliquid perps.
|
||||
> Real money — handle every change to the trading layer like surgery.
|
||||
|
||||
This file is the **first thing an AI agent should read** when entering this
|
||||
repo. It encodes the invariants that aren't visible from any single file.
|
||||
|
||||
---
|
||||
|
||||
## 🛑 Read this BEFORE touching anything trading-related
|
||||
|
||||
This backend manages real Hyperliquid leveraged positions for real users.
|
||||
Bugs cost users money. **Five non-negotiable rules:**
|
||||
|
||||
1. **Two systems, one execution layer.** System 1 (Trump scalp) auto-opens.
|
||||
System 2 (Macro Vibes / reversal) is **MANAGE-ONLY since v2.0** — the
|
||||
user opens manually on Hyperliquid, then `/adopt` hands it to the bot.
|
||||
`process_post()` early-returns for sys2 sources. Re-enabling sys2
|
||||
auto-open would silently reintroduce all the leverage/budget/concurrency
|
||||
race conditions we excised. Don't do it without an ADR.
|
||||
|
||||
2. **`released_at` is the "user took back control" marker.** A trade with
|
||||
`released_at IS NOT NULL` is **OUT OF BOUNDS for the bot**:
|
||||
- `recovery.rehydrate_open_trades` skips it
|
||||
- `reconciler` skips it
|
||||
- `close_and_finalize`'s atomic claim requires `released_at IS NULL`
|
||||
unless `force=True` (only manual_close passes that)
|
||||
- `partial_derisk` and `pyramid_add` early-return idempotent-success
|
||||
- `/positions/open`, `/positions/today`, telegram_digest all filter it
|
||||
If you add ANY new code path that touches BotTrade rows, ask yourself
|
||||
"does this respect released_at?" Almost always yes.
|
||||
|
||||
3. **Effective exit params are FROZEN on the BotTrade row at open time.**
|
||||
See the `eff_*` columns on `BotTrade`. Recovery rebuilds the watchdog
|
||||
from these, NOT from the live Subscription. Without this, restarting
|
||||
the backend silently rewrites every open System-2 reversal's stop loss
|
||||
to the user's Trump scalp setting (1.5%). NEVER read live
|
||||
`Subscription.stop_loss_pct` etc. in the close path.
|
||||
|
||||
4. **HL leverage is what HL says, not what the user requested.** Hyperliquid
|
||||
silently clips the requested leverage to the asset's max (memes capped
|
||||
at 3×). `hyperliquid.open_position()` returns `effective_leverage` —
|
||||
bot_engine and adoption must use THAT value to compute
|
||||
`sys2_protective_stop_pct(lev)` and the derisk ladder. Using the
|
||||
requested value puts the stop OUTSIDE the real liquidation line.
|
||||
|
||||
5. **Per-wallet asyncio locks wrap "check + open" critical sections.**
|
||||
See `_wallet_lock` in `bot_engine` and `_wallet_adopt_lock` in
|
||||
`adoption`. Without them, two concurrent signals can both pass the
|
||||
daily-budget / concurrency / already-open check before either commits.
|
||||
|
||||
---
|
||||
|
||||
## What this product does (90 seconds)
|
||||
|
||||
```
|
||||
Four signal sources → one bot → optional Hyperliquid execution
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 1. Trump Truth Social │── auto-classify (DeepSeek) → "buy"/"short"/"noise"
|
||||
│ (every post, <3s) │ if actionable: Trump scalp auto-open (System 1)
|
||||
└──────────────────────────┘ Tight 1.5% SL, 12h cooldown, ≥30min min-hold
|
||||
Optional: post prediction tweet via x_poster.py
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 2. Macro Vibes │── 8 daily macro indicators (AHR999, F&G, etc.)
|
||||
│ (BTC bottom + funding)│ + 2-of-3 bottom-reversal trigger
|
||||
└──────────────────────────┘ Telegram alert with /adopt CTA — NO auto-open.
|
||||
User opens on HL → /adopt → bot manages with
|
||||
5-rung stop ladder, de-risk, pyramid, peak-trail.
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 3. KOL talks-vs-trades │── Substack/podcast/X ingest + ETH on-chain diff
|
||||
│ (29 feeds, daily) │ Divergence (publicly bullish, secretly selling)
|
||||
└──────────────────────────┘ is the platform's highest-conviction signal.
|
||||
Telegram alert only — never auto-trades.
|
||||
x_analysis.py adds real-time X post scoring.
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 4. Funding extreme │── Hourly BTC perp funding scan
|
||||
└──────────────────────────┘ Alert only (manage-only via /adopt like Macro)
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ Telegram daily digest │── Once-a-day 3-section brief (Macro/KOL/Trump)
|
||||
│ (per-user hour, opt-out)│ to every subscriber. Cron at minute=0 each hour.
|
||||
└──────────────────────────┘
|
||||
```
|
||||
|
||||
Free tier = read everything + Telegram alerts. Pro tier (Hyperliquid wallet
|
||||
linked) = Trump auto-trade + /adopt manage-only flow for sys2.
|
||||
|
||||
---
|
||||
|
||||
## Stack
|
||||
|
||||
- **Python 3.9+** / FastAPI / async SQLAlchemy 2.x / APScheduler
|
||||
- **DB**: SQLite dev, **Postgres prod**. All schema lives in
|
||||
`alembic/versions/NNN_*.py`, ordered. Currently at head **026**.
|
||||
- **AI**: DeepSeek via OpenAI-compatible API (`AI_BASE_URL`, `AI_MODEL`).
|
||||
- Live scoring uses `AI_LIVE_MODEL` (~2s, latency-critical)
|
||||
- Batch / reanalysis uses `AI_MODEL` (quality, ~10s)
|
||||
- **Trading**: Hyperliquid SDK; API-wallet keys are envelope-encrypted with
|
||||
`ENCRYPTION_KEY` (KEK), per-user DEK derivation via `crypto.py`.
|
||||
- **Prices**: Two feeds:
|
||||
- `binance.py` WebSocket → 30 mainstream perps (BTC, ETH, SOL, TRUMP, BNB,
|
||||
DOGE, LINK, AAVE + AVAX/ARB/OP/SUI/APT/INJ/ATOM/XRP/LTC/ADA/MATIC/SHIB/
|
||||
PEPE/WIF/BONK/TAO/JUP/RENDER/FET/TIA/SEI/PENDLE). The WS URL is built FROM
|
||||
`ASSET_MAP` so the two never drift. Any asset NOT in ASSET_MAP loses
|
||||
TP/SL/trailing protection (only max-hold remains) — see the ASSET_MAP
|
||||
docstring. `tp_sl_monitor.register_trade` logs an ERROR if a trade opens on
|
||||
an uncovered asset.
|
||||
- `hl_price_feed.py` polls HL `allMids` every 2s → HYPE, PURR (HL-native assets not on Binance)
|
||||
Both pump `price_store` + `tp_sl_monitor` on every tick.
|
||||
- **Telegram**: long-poll mode (single instance), HTML messages, inline
|
||||
keyboards. `telegram.py` send/edit/answer + `telegram_bot.py` commands.
|
||||
Public channel broadcast (`TELEGRAM_PUBLIC_CHANNEL_ID` env) sends a
|
||||
sanitised `format_public_post` version (no execution details, tier label
|
||||
instead of raw confidence) after every per-user fan-out in `_dispatch`.
|
||||
- **X (Twitter)**: `x_poster.py` — optional viral prediction tweets after each
|
||||
actionable Trump signal. Gated by `x_enabled=False` (off by default). Full
|
||||
no-op if creds missing. OAuth 1.0a hand-signed with stdlib hmac — no extra deps.
|
||||
|
||||
---
|
||||
|
||||
## Module map (where things live)
|
||||
|
||||
```
|
||||
app/
|
||||
├── api/ HTTP routes
|
||||
│ ├── signals.py POST /api/signals/ingest ← scanners write here
|
||||
│ ├── positions.py /positions/open|today|close|grow|adopt|release|hl
|
||||
│ ├── user.py /subscribe|settings|manual-window|auto-trade
|
||||
│ ├── telegram.py /telegram/{preferences,bind,unbind,test}
|
||||
│ ├── macro.py /macro/{snapshot,history}
|
||||
│ ├── kol.py /kol/{posts,digest,wallets,divergence}
|
||||
│ ├── performance.py /performance ← wallet-scoped real-money stats (30d)
|
||||
│ ├── funding_reversal.py /funding/snapshot ← live funding state + 7d history
|
||||
│ ├── funding_signal.py /signal/{status,toggle,history} ← breakout monitor
|
||||
│ └── dev.py Dev-only routes (only mounted in development env)
|
||||
├── services/
|
||||
│ ├── bot_engine.py ★ TRADING CORE — process_post, _execute_for_subscriber,
|
||||
│ │ _broadcast_trade_alert (WS failure notifications),
|
||||
│ │ close_and_finalize, partial_derisk, pyramid_add
|
||||
│ ├── adoption.py ★ /adopt + /release flow (sys2 manage-only)
|
||||
│ ├── tp_sl_monitor.py Per-price-tick close evaluator. on_price_tick is
|
||||
│ │ called from binance.py + hl_price_feed.py once/sec
|
||||
│ ├── hyperliquid.py HL trader (open/close/reduce/leverage)
|
||||
│ ├── recovery.py Startup rehydration of open BotTrades into watchdog
|
||||
│ ├── reconciler.py Every 60s: compare DB ↔ HL state, mark drift
|
||||
│ ├── circuit_breaker.py Per-system (sys1/sys2) CB, daily DD + N-loss streak
|
||||
│ ├── signal_categories.py CRITICAL CONFIG — sys1/sys2 sources, ladders,
|
||||
│ │ leverage clamping, protective stop formulas
|
||||
│ ├── regime_filter.py Sys1 only — recent-move / vol-contraction gates
|
||||
│ ├── analysis.py AI signal scoring (DeepSeek) for Trump posts
|
||||
│ ├── x_analysis.py AI scoring for X (Twitter) KOL posts. Three tiers:
|
||||
│ │ TRADE_SIGNAL / DIRECTIONAL / NOISE. Strict NOISE
|
||||
│ │ default — most X posts should be filtered out.
|
||||
│ │ Consumed by kol_x.py. tickers come out in the
|
||||
│ │ {ticker,action,conviction} shape kol_divergence reads.
|
||||
│ ├── x_poster.py X (Twitter) auto-poster for Trump signals.
|
||||
│ │ Fires a prediction tweet then a follow-up at
|
||||
│ │ +x_followup_minutes with the actual move. Gated by
|
||||
│ │ x_enabled env var (False by default). Full no-op
|
||||
│ │ if creds missing — never blocks signal flow.
|
||||
│ ├── entry_filter.py Cheap text-based pre-filter (skip RT/URL-only)
|
||||
│ ├── telegram.py send_message / edit_message / answer_callback
|
||||
│ ├── telegram_bot.py Long-poll loop + /start /digest /adopt /release ...
|
||||
│ ├── telegram_digest.py Daily 3-section brief (rule-based; no LLM)
|
||||
│ ├── price_store.py In-memory latest price per asset
|
||||
│ ├── price_backfill.py Backfill historical 5min bars from Binance
|
||||
│ ├── hl_price_feed.py Supplemental HL price feed for HL-native assets
|
||||
│ │ (HYPE, PURR). Polls allMids every 2s. Runs
|
||||
│ │ alongside binance.py. Without this, TP/SL
|
||||
│ │ silently stops protecting HL-native trades.
|
||||
│ ├── backtest.py Single-post backtest harness. Fetches 1m Binance
|
||||
│ │ candles for [published_at, +max_hold_h] and
|
||||
│ │ replays current exit rules. Conservative (uses
|
||||
│ │ HIGH/LOW within bar). No fees. Batch runner on top.
|
||||
│ ├── crypto.py HL API-key envelope encryption. enc:v2 =
|
||||
│ │ PBKDF2-salted (H4 fix); enc:v1 read-compat.
|
||||
│ │ scripts/reencrypt_keys.py upgrades stored rows.
|
||||
│ ├── scanner_state.py In-memory toggle + observability for scanners
|
||||
│ ├── macro/
|
||||
│ │ ├── fetchers.py 8 macro indicator HTTP fetchers (each @_none_on_fail)
|
||||
│ │ ├── scoring.py Weighted composite -100..+100
|
||||
│ │ └── poll.py Daily UPSERT into macro_snapshots
|
||||
│ ├── scanners/
|
||||
│ │ ├── btc_bottom_reversal.py 2-of-3 AHR999 + 200WMA + Pi Bottom
|
||||
│ │ ├── funding_reversal.py Hourly funding extreme
|
||||
│ │ └── sma_reclaim.py (archive — not scheduled)
|
||||
│ ├── kol_substack.py RSS ingest for 29 KOL feeds (substack/blog/podcast)
|
||||
│ ├── kol_x.py X (Twitter) ingest via twitterapi.io → x_analysis →
|
||||
│ │ KolPost(source="twitter"). Daily 01:30 UTC. No-op
|
||||
│ │ if twitterapi_io_key unset. Provides the post-side
|
||||
│ │ feed for X-only KOLs (andrewkang, murad).
|
||||
│ ├── kol_onchain.py HL public API + Etherscan diff
|
||||
│ ├── kol_divergence.py Cross-ref talks vs trades within ±7d
|
||||
│ ├── kol_analysis.py AI ticker/direction/conviction extract (Substack).
|
||||
│ │ `_derive_tier()` maps its conviction + talks-vs-
|
||||
│ │ trades score → the SAME trade_signal/directional/
|
||||
│ │ noise tiers x_analysis emits, so non-Twitter posts
|
||||
│ │ get tier set in kol_substack (SIGNAL/VIEW badges +
|
||||
│ │ "Signals only" filter work for blog/substack/pod).
|
||||
│ ├── bottom_indicators.py AHR999 / Pi Cycle / 200WMA math
|
||||
│ ├── funding_signal.py Real-time funding extreme detector
|
||||
│ ├── signed_request.py EIP-191 signature verification (+ replay cache).
|
||||
│ │ signed_read_creds / optional_signed_read_creds:
|
||||
│ │ header-based (X-Sig-Ts/X-Sig-Sig) creds for read
|
||||
│ │ endpoints, query fallback deprecated (C3 fix).
|
||||
│ └── http_client.py Shared pooled httpx.AsyncClient (keep-alive).
|
||||
│ Hot paths (scrapers, telegram send/poll,
|
||||
│ hl_price_feed, binance REST, x_poster) use
|
||||
│ get_client() with per-request timeout instead of
|
||||
│ new-client-per-call. Closed in lifespan shutdown.
|
||||
├── scrapers/
|
||||
│ ├── truth_social.py CNN archive poller (5s interval) — primary.
|
||||
│ │ Conditional GET (ETag/Last-Modified → 304 skips
|
||||
│ │ the 30k-post download), batch dedup (1 IN-query
|
||||
│ │ per poll instead of 50 SELECTs), per-post
|
||||
│ │ commit+dispatch so an actionable post never
|
||||
│ │ waits behind older entries' AI analysis.
|
||||
│ │ dispatch_post() is the shared WS/TG/X/trade
|
||||
│ │ fan-out used by BOTH pollers.
|
||||
│ └── trumpstruth.py trumpstruth.org RSS fallback poller. Same post id
|
||||
│ hash → automatic dedup. Whoever sees first wins.
|
||||
│ Offset by half the interval so the two pollers
|
||||
│ don't hit upstream simultaneously.
|
||||
├── ws/
|
||||
│ └── manager.py WebSocket fan-out for live UI updates.
|
||||
│ Broadcasts trade_alert events (execution_failed /
|
||||
│ insufficient_balance / budget_reached) via
|
||||
│ _broadcast_trade_alert() in bot_engine.py.
|
||||
├── models.py ★ All SQLAlchemy models in one file
|
||||
├── database.py Async engine + session factory
|
||||
├── config.py Pydantic Settings — reads .env
|
||||
└── main.py FastAPI lifespan, scheduler setup, route mount
|
||||
Includes: singleton lock guard (one-leader, multi-
|
||||
worker safe), deep health check /api/health/deep,
|
||||
boot-grace window for price feeds.
|
||||
|
||||
alembic/versions/ Migrations (numbered NNN). Latest = 026
|
||||
026 = composite index (wallet_address, closed_at) on bot_trades
|
||||
scripts/ One-shot ops
|
||||
├── preflight.py Pre-launch readiness gate (env / DB / TG / AI)
|
||||
├── launch_smoke.py End-to-end smoke (14 checks against running API)
|
||||
├── launch_seed.py Pre-launch data prep: drops test sources, trims KOL
|
||||
│ window to last 30d, refetches all upstream sources.
|
||||
│ Run ONCE before flipping traffic to a fresh DB.
|
||||
├── seed_kol_wallets.py Seeds the KOL wallet table with known addresses.
|
||||
│ Idempotent (INSERT OR IGNORE). Run once at first deploy.
|
||||
├── rescore_v5.py Re-score every Post with current AI prompt
|
||||
├── backfill_signals.py Fill in signal for posts missing it
|
||||
├── reencrypt_keys.py Upgrade stored HL keys to enc:v2 (+KEK rotation
|
||||
│ via OLD_ENCRYPTION_KEY). Idempotent; --dry-run.
|
||||
└── verify_sys2_lifecycle.py Manual System-2 lifecycle walk-through
|
||||
|
||||
tests/ pytest, 112 tests, fast (<3s total)
|
||||
├── test_adoption.py Adoption + release flow (snapshot-style, no real HL/AI)
|
||||
├── test_telegram_digest.py Daily digest formatting
|
||||
├── test_kol_tier.py kol_analysis._derive_tier (non-Twitter tier mapping)
|
||||
├── test_kol_x.py X ingest: dedup / mapping / no-op (mocked fetch+AI)
|
||||
├── test_ratelimit.py Rate limit coverage (BUG-02 fix)
|
||||
├── test_bottom_reversal_strategy.py btc_bottom_reversal 2-of-3 logic
|
||||
├── test_macro_fetchers_timing.py Macro fetcher timeout + error handling
|
||||
└── test_production_readiness.py Environment / config sanity checks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## The two trading systems (memorise this)
|
||||
|
||||
```
|
||||
System 1 System 2
|
||||
──────── ────────
|
||||
Source "truth" "btc_bottom_reversal"
|
||||
(+"funding_reversal" alert-only)
|
||||
Trigger Trump posts a thing Daily scanner: 2-of-3 confluence
|
||||
Latency need Seconds (price moves fast) Days/weeks (signal lives a long time)
|
||||
Open path Auto (bot_engine.process_post fires _execute_for_subscriber)
|
||||
MANUAL: user opens on HL UI, then
|
||||
/adopt hands it to the bot
|
||||
Stop loss User-configured + tight sys2_protective_stop_pct(actual_lev)
|
||||
1.5% floor (TRUMP_*) = 85% × (100/lev), capped at 35%
|
||||
Exit model TP / trailing / SL 5-rung stop ladder + downside de-risk
|
||||
+ pyramid + peak-trail. NO TP.
|
||||
Min hold 30 min (suppresses TP) n/a
|
||||
Max hold 6h 18 months (ladder is the real exit)
|
||||
Sizing base × regime multiplier Whatever user opened on HL
|
||||
Concurrency cap n/a 3 positions / wallet (correlated beta)
|
||||
Confidence min 88 (platform) / user 85 (platform)
|
||||
Circuit breaker sys1_* sys2_* (independent)
|
||||
Daily budget Full daily_budget_usd n/a — user controls notional on HL
|
||||
Telegram alert Trump alert format Macro/funding alert + /adopt CTA
|
||||
X tweet Optional prediction tweet n/a
|
||||
```
|
||||
|
||||
**If you're tempted to put sys2 logic in `_execute_for_subscriber`**: stop.
|
||||
`process_post()` early-returns for sys2. The function only runs for sys1 now.
|
||||
The dead sys2 branches inside `_execute_for_subscriber` are kept for diff
|
||||
minimalism — don't extend them.
|
||||
|
||||
---
|
||||
|
||||
## The /adopt flow (System-2 lifecycle in detail)
|
||||
|
||||
```
|
||||
1. Scanner fires
|
||||
└─ POST /api/signals/ingest (source=btc_bottom_reversal, signal=buy)
|
||||
└─ Post row created
|
||||
└─ process_post() early-returns for sys2 (no auto-open)
|
||||
└─ notify_signal() → Telegram fan-out with /adopt CTA appended
|
||||
|
||||
2. User opens BTC long on Hyperliquid manually
|
||||
└─ size / leverage of their choice
|
||||
|
||||
3. User in bot: /adopt
|
||||
└─ adoption.list_hl_positions(wallet) reads HL state
|
||||
└─ Telegram inline keyboard: tap [🟢 BTC long $1500 @72k · 2x]
|
||||
└─ Mode picker: [📈 Standard] or [🚀 Aggressive]
|
||||
└─ adoption.adopt_position(wallet, asset, mode):
|
||||
a. Per-wallet asyncio lock acquired
|
||||
b. Pre-flight: no_subscription / no_hl_key / paper_mode /
|
||||
macro_disabled (Subscription.macro_enabled must be ON — /adopt is the
|
||||
sys2 management entry point, so the Macro Vibes toggle gates it) /
|
||||
circuit_breaker (sys2 CB still gates adopt!) /
|
||||
already_adopted / concurrency_cap (3)
|
||||
c. Re-read HL state inside lock (fresh entry/size/lev)
|
||||
d. Reject if leverage > SYS2_MAX_LEVERAGE (BUG-09 fix)
|
||||
e. Resolve sys2_protective_stop_pct(HL_actual_leverage)
|
||||
f. INSERT BotTrade with eff_* frozen + sys2_mode + hl_order_id="adopted:<ts>"
|
||||
+ trigger_post_id=NULL
|
||||
g. register_trade() with full ladder/de-risk/addon/peak_trail
|
||||
|
||||
4. tp_sl_monitor drives the position
|
||||
└─ Stop ratchet, downside de-risk partial reduces, pyramid add-ons,
|
||||
peak-trail close, max_hold backstop. All staged through the
|
||||
lock-protected partial_derisk / pyramid_add / close_and_finalize.
|
||||
|
||||
5a. User wants out: /release
|
||||
└─ Sets BotTrade.released_at = now; unregister(trade_id) from watchdog
|
||||
└─ HL position UNTOUCHED — bot stops driving, user has manual control
|
||||
|
||||
5b. Bot drives the close (ladder / max-hold)
|
||||
└─ close_and_finalize() atomic claim sets closed_at, computes pnl
|
||||
|
||||
5c. User force-closes via UI: POST /api/positions/{id}/close
|
||||
└─ manual_close calls close_and_finalize(force=True) — bypasses
|
||||
released_at guard. Works on adopted-and-released trades too.
|
||||
|
||||
6. Recovery on restart:
|
||||
└─ recovery.rehydrate_open_trades reads BotTrade WHERE closed_at IS NULL
|
||||
AND released_at IS NULL
|
||||
└─ For each sys2 trade: rebuild ladder from sys2_mode + adopted fallback.
|
||||
Also reschedules _time_stop_check with remaining seconds (elapsed windows
|
||||
fire immediately with delay=0). This is the critical fix — without it
|
||||
adopted trades lose their entire sys2 ladder on restart.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Singleton lock guard (critical infrastructure detail)
|
||||
|
||||
`main.py:lifespan` calls `_acquire_singleton_lock()` at boot using an
|
||||
advisory file lock (`/tmp/trumpsignal-backend.lock`, configurable via
|
||||
`SINGLETON_LOCK_PATH`). Only ONE process — "the leader" — starts the
|
||||
scheduler, scrapers, price feeds, and Telegram poller. Any additional
|
||||
worker (e.g. accidental `--workers 2`) serves HTTP reads only and logs
|
||||
`SINGLETON LOCK NOT ACQUIRED`. The OS releases the lock automatically
|
||||
when the leader exits, so a crashed leader unblocks the next start.
|
||||
|
||||
The `/api/health/deep` endpoint surfaces `"is_leader": false` and adds
|
||||
it to `problems[]` so uptime monitors catch mis-configured multi-worker
|
||||
deployments.
|
||||
|
||||
---
|
||||
|
||||
## Critical invariants checklist (when reviewing any trading change)
|
||||
|
||||
- [ ] Does this code path respect `released_at IS NULL`?
|
||||
- [ ] Does it use `eff_*` (frozen) not live `Subscription.*` for exit math?
|
||||
- [ ] If it opens a new position, does it use HL's actual leverage (not requested)?
|
||||
- [ ] If it touches an open position concurrently, is it wrapped in the
|
||||
per-trade `_lock_for(trade_id)` lock?
|
||||
- [ ] If it opens, is it inside `_wallet_lock(wallet)` so the budget /
|
||||
concurrency check is atomic with the write?
|
||||
- [ ] If it closes, does it use the conditional `UPDATE ... WHERE closed_at
|
||||
IS NULL` atomic claim?
|
||||
- [ ] Does it handle the `already_closed` path from HL gracefully (preserve
|
||||
banked partial PnL)?
|
||||
- [ ] Does it correctly check sys1 vs sys2 CB independently?
|
||||
|
||||
---
|
||||
|
||||
## Common workflows
|
||||
|
||||
### Add a new signal source
|
||||
|
||||
1. Decide: System 1 (auto-trade) or System 2 (alert + /adopt) or alert-only?
|
||||
2. Write a scanner under `app/services/scanners/NEW.py` that posts to
|
||||
`POST /api/signals/ingest` with `{source: "your_new_source", ...}`.
|
||||
3. Schedule it in `app/main.py` (`_scheduler.add_job`).
|
||||
4. Add the source to `signal_categories.SYSTEM_1_SOURCES` or
|
||||
`SYSTEM_2_SOURCES` if it should trade. Leave it out if alert-only.
|
||||
5. Add a Telegram preference column to `TelegramBinding` (migration)
|
||||
+ a mapping entry in `telegram._pref_column_for_source`.
|
||||
6. Add a label to `telegram._source_label` and `_signal_emoji`.
|
||||
7. Add a `/yoursource on|off` command in `telegram_bot.py`.
|
||||
8. If sys2: extend `signal_categories._CATEGORY_EXITS` if it needs a custom
|
||||
exit profile (otherwise default works).
|
||||
9. Add the deep-link path in `telegram.format_post` AND `telegram.format_public_post`.
|
||||
|
||||
### Add a new bot command
|
||||
|
||||
1. `_cmd_x` async function in `telegram_bot.py`.
|
||||
2. Route it in `_handle_message`.
|
||||
3. If it needs inline buttons: build `reply_markup` payload, handle
|
||||
callbacks in `_handle_callback` (route by `callback_data` prefix).
|
||||
4. Update `HELP_TEXT` and remind the user to add it to BotFather
|
||||
`/setcommands` after deploy.
|
||||
|
||||
### Add a column to an existing table
|
||||
|
||||
1. New migration `alembic/versions/NNN_description.py`.
|
||||
- Use `op.batch_alter_table` (sqlite-compatible).
|
||||
- Default values via `server_default=` so backfill is implicit.
|
||||
2. Mirror the field on the SQLAlchemy model in `app/models.py`.
|
||||
3. Apply locally: `DATABASE_URL=<sqlite> alembic upgrade head`.
|
||||
|
||||
### Enable X (Twitter) posting
|
||||
|
||||
1. Create a Twitter developer app with OAuth 1.0a user context permissions.
|
||||
2. Set in `.env`: `X_API_KEY`, `X_API_SECRET`, `X_ACCESS_TOKEN`,
|
||||
`X_ACCESS_SECRET`, `X_ENABLED=true`.
|
||||
3. Optionally tune: `X_DAILY_CAP` (default 40), `X_FOLLOWUP_MINUTES` (default 15).
|
||||
4. Verify with a dry-run: set `X_ENABLED=false` and check logs — the poster
|
||||
logs what it WOULD tweet without sending.
|
||||
|
||||
### Deploy
|
||||
|
||||
```bash
|
||||
# On the server:
|
||||
DATABASE_URL=$PROD_URL alembic upgrade head
|
||||
systemctl restart trumpalpha-backend # or whatever the unit is
|
||||
python scripts/preflight.py # MUST pass before flipping traffic
|
||||
python scripts/launch_smoke.py --base https://api.trumpalpha.io
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
cd backend && python -m venv venv && source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
cp .env.example .env # fill in: ENCRYPTION_KEY, AI_API_KEY, INGEST_API_KEY
|
||||
uvicorn app.main:app --reload # dev only; prod uses --workers 1, no --reload
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m pytest tests/ -q # 112 tests, ~2s
|
||||
python scripts/preflight.py # env + DB + TG + AI auth checks
|
||||
python scripts/launch_smoke.py # 14 end-to-end checks vs running API
|
||||
```
|
||||
|
||||
Adoption + telegram_digest are snapshot-style (no real HL/AI).
|
||||
End-to-end trading is verified manually via the bot.
|
||||
|
||||
---
|
||||
|
||||
## Telegram bot mechanics (since it's a custom integration)
|
||||
|
||||
- **Long-poll mode** via `getUpdates`. Only ONE process can long-poll a
|
||||
given bot token at a time — if you horizontally scale, switch to
|
||||
webhook (not done yet).
|
||||
- Bot must be re-bound via `@BotFather` `/setcommands` whenever new
|
||||
commands are added (the slash-menu users see is separate from what
|
||||
the bot internally handles).
|
||||
- `send_message` returns `False` on failure; per-user binding rows track
|
||||
`total_alerts_sent` / `total_alerts_failed` counters.
|
||||
- **Inline keyboards** = the `reply_markup` payload to `sendMessage`.
|
||||
Callback data is capped at 64 bytes; keep it short (`adopt:mode:BTC:standard`).
|
||||
`_handle_callback` MUST end with `answer_callback` or the button spins
|
||||
forever on the user's client.
|
||||
- Free tier = walletless `/start` (chat_id only). Pro tier = wallet bound
|
||||
via `/start CODE` where CODE comes from Settings UI.
|
||||
|
||||
---
|
||||
|
||||
## Why "Macro Vibes" became manage-only (the ADR)
|
||||
|
||||
V1.0: System 2 auto-opened sys2 trades on user wallets. Carried real
|
||||
execution surface: leverage clipping, daily budget split, concurrency caps,
|
||||
sys2 paper branches, key handling per user. Audit surfaced ~6 bugs.
|
||||
|
||||
V2.0 (current): sys2 manage-only. The strategy is day-K — entry delay of
|
||||
24h doesn't matter. The valuable part is multi-month exit management
|
||||
(5-rung ladder, de-risk, pyramid, peak-trail), which still runs against
|
||||
positions the user adopts.
|
||||
|
||||
**Net effect**: massive reduction in execution risk surface; same alpha
|
||||
(strategy logic unchanged); legal/responsibility shifts from "bot opened
|
||||
this for you" to "you opened it, bot manages your discipline".
|
||||
|
||||
---
|
||||
|
||||
## Things that LOOK like bugs but aren't
|
||||
|
||||
- **`_execute_for_subscriber` has lots of `if sub["_is_system_2"]` branches.**
|
||||
Dead code under v2.0 (process_post early-returns for sys2). Kept for diff
|
||||
minimalism — don't extend or re-enable.
|
||||
- **`kol_x.X_KOLS` handle ≠ X username.** `handle` is the CANONICAL key
|
||||
(e.g. "cryptohayes") that MUST match `KolWallet.handle` so divergence can
|
||||
join post-side ↔ on-chain; `x_username` is the screen name fetched from X.
|
||||
andrewkang/murad wallets stay dark (zero divergence detections) until X
|
||||
ingestion supplies their post side — that's the whole reason kol_x exists.
|
||||
Empty `twitterapi_io_key` → kol_x is a full no-op (no error).
|
||||
- **`funding_reversal` source is in `SYSTEM_2_SOURCES`? No.** It's
|
||||
intentionally NOT in either supported set — it ingests as a Post for
|
||||
audit + sends Telegram alert via the CTA path, but doesn't trigger any
|
||||
auto-trading. Adopt still works (it's asset-based, not signal-based).
|
||||
- **`Subscription.sys2_budget_pct` defaults to 0.7.** Legacy field from the
|
||||
auto-open era. With v2.0 manage-only, it's effectively unused — sys1
|
||||
(Trump) reads full `daily_budget_usd`. Don't read it for new code.
|
||||
- **Adopted trades have `hl_order_id` starting with `"adopted:"`.** Distinct
|
||||
from auto-opened (HL order id integer) and paper (`"paper"` literal).
|
||||
Useful for telemetry filtering — AND it's the canonical sys2 marker the
|
||||
daily-budget query uses, because adopted trades have `trigger_post_id=NULL`
|
||||
so source-based classification fails (M1 fix). The `/trades` serializer also
|
||||
reports `trigger_source="adopted"` from this prefix.
|
||||
- **`/signals/accuracy` is scoped to production sources + buy/short.** It
|
||||
intentionally restricts to `SUPPORTED_TRADING_SOURCES` and the CURRENT
|
||||
buy/short vocabulary — retired/test sources (rsi_reversal, sma_reclaim,
|
||||
breakout, phase1, `test`) and the legacy `sell` signal are excluded so the
|
||||
public accuracy scoreboard reflects what the live bot actually trades.
|
||||
- **macro_enabled vs Telegram alerts are TWO separate switches.** Turning off
|
||||
Macro Vibes (`Subscription.macro_enabled`) gates sys2 *management* — it now
|
||||
blocks `/adopt` (macro_disabled). It does NOT silence Telegram alerts; those
|
||||
are controlled independently by the per-source `TelegramBinding` preference
|
||||
columns. "Alerts on, don't auto-manage" is a deliberately supported combo.
|
||||
- **`telegram.send_message` accepts `int | str` for `chat_id`.** Intentional.
|
||||
Integer = private chat, string = public channel username (e.g. `"@trumpalpha"`).
|
||||
- **`format_public_post` deliberately omits `expected_move_pct`,
|
||||
`invalidation_price`, and `/adopt` CTA.** Execution-sensitive data stays
|
||||
private. The public version shows confidence tier (HIGH/MED/LOW) instead
|
||||
of the raw score.
|
||||
- **`_adopt_locks` in adoption.py** is an `OrderedDict` capped at 512 with
|
||||
LRU eviction — matches the `_WALLET_LOCK_MAX` pattern in `bot_engine`.
|
||||
- **`trumpstruth.py` runs at half-interval offset** — the two CNN + trumpstruth
|
||||
pollers are deliberately staggered so they don't hammer upstream simultaneously.
|
||||
The offset is `truth_social_poll_seconds // 2`, set via APScheduler
|
||||
`next_run_time` at boot.
|
||||
|
||||
---
|
||||
|
||||
## Open known issues (not blocking launch but worth fixing later)
|
||||
|
||||
- **`adopt:choose:BTC` callback may show stale prices** if user takes >60s
|
||||
to tap (HL fees, partial fills can change entry/size). adopt_position
|
||||
re-reads HL at mode-tap time so the FROZEN BotTrade is always fresh,
|
||||
but the picker label could be outdated.
|
||||
|
||||
- ~~**Telegram bot offset on restart**~~ **FIXED 2026-06-01**: startup drain
|
||||
added. Stale `/adopt` replays suppressed.
|
||||
|
||||
- **`/adopt` picker label** can show stale price if user waits >60s to tap
|
||||
(frozen BotTrade is always fresh, but the Telegram picker label may be outdated).
|
||||
|
||||
- ~~**M1 adopted positions miscounted against sys1 budget**~~ **FIXED 2026-06-09**:
|
||||
the daily-budget query in `bot_engine` now treats any trade whose
|
||||
`hl_order_id` starts with `"adopted:"` as sys2, regardless of the NULL
|
||||
`trigger_post_id`. Previously the outerjoin to Post yielded src=NULL →
|
||||
classified as sys1 → every adopted macro position inflated the Trump
|
||||
scalp budget and could prematurely trip `budget_reached`.
|
||||
|
||||
- **Funding-reversal `/adopt` uses the btc_bottom_reversal exit profile**
|
||||
(`ADOPTED_CATEGORY` is fixed). This is BY DESIGN, not a bug: adopt is
|
||||
ASSET-based (the user opens any position on HL and adopts it) — the bot has
|
||||
no reliable link back to which signal motivated it, and the sys2 ladder is
|
||||
direction/horizon-agnostic. Changing this needs a source-tracking mechanism
|
||||
at adopt time (an ADR), not a one-line tweak.
|
||||
|
||||
**Deferred security items — ALL RESOLVED 2026-06-12:**
|
||||
- ~~**C3**~~ FIXED: signed reads now send `X-Sig-Ts` / `X-Sig-Sig` HEADERS
|
||||
(see `signed_read_creds` in signed_request.py). Legacy `?ts=&sig=` query
|
||||
params still accepted (deprecated) for old clients.
|
||||
- ~~**H4**~~ FIXED: keys now encrypt as `enc:v2` (PBKDF2-HMAC-SHA256, per-blob
|
||||
salt, 600k iters). v1 blobs still decrypt; run `scripts/reencrypt_keys.py`
|
||||
ONCE in prod (after DB backup) to upgrade stored rows.
|
||||
- ~~**M5**~~ Was already fixed: unauthenticated `/telegram/{wallet}/status`
|
||||
returns only `configured`/`bound` booleans; full details require a signed read.
|
||||
|
||||
---
|
||||
|
||||
## Repos in this project
|
||||
|
||||
- **This repo** (`/Users/k/Public/trumpsignal/backend`) — Python/FastAPI backend
|
||||
- **Sibling frontend** (`/Users/k/Public/trumpsignal/frontend`) — Next.js 16
|
||||
dashboard at trumpsignal.com. See its own AGENTS.md.
|
||||
|
||||
Both deployed independently. Backend serves the JSON API + Telegram bot.
|
||||
Frontend is a thin SPA over the API + WebSocket.
|
||||
@@ -63,6 +63,7 @@ Four signal sources → one bot → optional Hyperliquid execution
|
||||
│ 1. Trump Truth Social │── auto-classify (DeepSeek) → "buy"/"short"/"noise"
|
||||
│ (every post, <3s) │ if actionable: Trump scalp auto-open (System 1)
|
||||
└──────────────────────────┘ Tight 1.5% SL, 12h cooldown, ≥30min min-hold
|
||||
Optional: post prediction tweet via x_poster.py
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 2. Macro Vibes │── 8 daily macro indicators (AHR999, F&G, etc.)
|
||||
@@ -72,10 +73,11 @@ Four signal sources → one bot → optional Hyperliquid execution
|
||||
5-rung stop ladder, de-risk, pyramid, peak-trail.
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 3. KOL talks-vs-trades │── Substack/podcast ingest + ETH on-chain diff
|
||||
│ (19 KOLs, daily) │ Divergence (publicly bullish, secretly selling)
|
||||
│ 3. KOL talks-vs-trades │── Substack/podcast/X ingest + ETH on-chain diff
|
||||
│ (29 feeds, daily) │ Divergence (publicly bullish, secretly selling)
|
||||
└──────────────────────────┘ is the platform's highest-conviction signal.
|
||||
Telegram alert only — never auto-trades.
|
||||
x_analysis.py adds real-time X post scoring.
|
||||
|
||||
┌──────────────────────────┐
|
||||
│ 4. Funding extreme │── Hourly BTC perp funding scan
|
||||
@@ -102,13 +104,24 @@ linked) = Trump auto-trade + /adopt manage-only flow for sys2.
|
||||
- Batch / reanalysis uses `AI_MODEL` (quality, ~10s)
|
||||
- **Trading**: Hyperliquid SDK; API-wallet keys are envelope-encrypted with
|
||||
`ENCRYPTION_KEY` (KEK), per-user DEK derivation via `crypto.py`.
|
||||
- **Prices**: Binance WS (`binance.py`) feeds `price_store` + powers the
|
||||
`tp_sl_monitor` per-tick evaluator.
|
||||
- **Prices**: Two feeds:
|
||||
- `binance.py` WebSocket → 30 mainstream perps (BTC, ETH, SOL, TRUMP, BNB,
|
||||
DOGE, LINK, AAVE + AVAX/ARB/OP/SUI/APT/INJ/ATOM/XRP/LTC/ADA/MATIC/SHIB/
|
||||
PEPE/WIF/BONK/TAO/JUP/RENDER/FET/TIA/SEI/PENDLE). The WS URL is built FROM
|
||||
`ASSET_MAP` so the two never drift. Any asset NOT in ASSET_MAP loses
|
||||
TP/SL/trailing protection (only max-hold remains) — see the ASSET_MAP
|
||||
docstring. `tp_sl_monitor.register_trade` logs an ERROR if a trade opens on
|
||||
an uncovered asset.
|
||||
- `hl_price_feed.py` polls HL `allMids` every 2s → HYPE, PURR (HL-native assets not on Binance)
|
||||
Both pump `price_store` + `tp_sl_monitor` on every tick.
|
||||
- **Telegram**: long-poll mode (single instance), HTML messages, inline
|
||||
keyboards. `telegram.py` send/edit/answer + `telegram_bot.py` commands.
|
||||
Public channel broadcast (`TELEGRAM_PUBLIC_CHANNEL_ID` env) sends a
|
||||
sanitised `format_public_post` version (no execution details, tier label
|
||||
instead of raw confidence) after every per-user fan-out in `_dispatch`.
|
||||
- **X (Twitter)**: `x_poster.py` — optional viral prediction tweets after each
|
||||
actionable Trump signal. Gated by `x_enabled=False` (off by default). Full
|
||||
no-op if creds missing. OAuth 1.0a hand-signed with stdlib hmac — no extra deps.
|
||||
|
||||
---
|
||||
|
||||
@@ -122,13 +135,18 @@ app/
|
||||
│ ├── user.py /subscribe|settings|manual-window|auto-trade
|
||||
│ ├── telegram.py /telegram/{preferences,bind,unbind,test}
|
||||
│ ├── macro.py /macro/{snapshot,history}
|
||||
│ └── kol.py /kol/{posts,digest,wallets,divergence}
|
||||
│ ├── kol.py /kol/{posts,digest,wallets,divergence}
|
||||
│ ├── performance.py /performance ← wallet-scoped real-money stats (30d)
|
||||
│ ├── funding_reversal.py /funding/snapshot ← live funding state + 7d history
|
||||
│ ├── funding_signal.py /signal/{status,toggle,history} ← breakout monitor
|
||||
│ └── dev.py Dev-only routes (only mounted in development env)
|
||||
├── services/
|
||||
│ ├── bot_engine.py ★ TRADING CORE — process_post, _execute_for_subscriber,
|
||||
│ │ _broadcast_trade_alert (WS failure notifications),
|
||||
│ │ close_and_finalize, partial_derisk, pyramid_add
|
||||
│ ├── adoption.py ★ /adopt + /release flow (sys2 manage-only)
|
||||
│ ├── tp_sl_monitor.py Per-price-tick close evaluator. on_price_tick is
|
||||
│ │ called from binance.py once per second
|
||||
│ │ called from binance.py + hl_price_feed.py once/sec
|
||||
│ ├── hyperliquid.py HL trader (open/close/reduce/leverage)
|
||||
│ ├── recovery.py Startup rehydration of open BotTrades into watchdog
|
||||
│ ├── reconciler.py Every 60s: compare DB ↔ HL state, mark drift
|
||||
@@ -136,14 +154,34 @@ app/
|
||||
│ ├── signal_categories.py CRITICAL CONFIG — sys1/sys2 sources, ladders,
|
||||
│ │ leverage clamping, protective stop formulas
|
||||
│ ├── regime_filter.py Sys1 only — recent-move / vol-contraction gates
|
||||
│ ├── analysis.py AI signal scoring (DeepSeek)
|
||||
│ ├── analysis.py AI signal scoring (DeepSeek) for Trump posts
|
||||
│ ├── x_analysis.py AI scoring for X (Twitter) KOL posts. Three tiers:
|
||||
│ │ TRADE_SIGNAL / DIRECTIONAL / NOISE. Strict NOISE
|
||||
│ │ default — most X posts should be filtered out.
|
||||
│ │ Consumed by kol_x.py. tickers come out in the
|
||||
│ │ {ticker,action,conviction} shape kol_divergence reads.
|
||||
│ ├── x_poster.py X (Twitter) auto-poster for Trump signals.
|
||||
│ │ Fires a prediction tweet then a follow-up at
|
||||
│ │ +x_followup_minutes with the actual move. Gated by
|
||||
│ │ x_enabled env var (False by default). Full no-op
|
||||
│ │ if creds missing — never blocks signal flow.
|
||||
│ ├── entry_filter.py Cheap text-based pre-filter (skip RT/URL-only)
|
||||
│ ├── telegram.py send_message / edit_message / answer_callback
|
||||
│ ├── telegram_bot.py Long-poll loop + /start /digest /adopt /release ...
|
||||
│ ├── telegram_digest.py Daily 3-section brief (rule-based; no LLM)
|
||||
│ ├── price_store.py In-memory latest price per asset
|
||||
│ ├── price_backfill.py Backfill historical 5min bars from Binance
|
||||
│ ├── crypto.py HL API-key envelope encryption
|
||||
│ ├── hl_price_feed.py Supplemental HL price feed for HL-native assets
|
||||
│ │ (HYPE, PURR). Polls allMids every 2s. Runs
|
||||
│ │ alongside binance.py. Without this, TP/SL
|
||||
│ │ silently stops protecting HL-native trades.
|
||||
│ ├── backtest.py Single-post backtest harness. Fetches 1m Binance
|
||||
│ │ candles for [published_at, +max_hold_h] and
|
||||
│ │ replays current exit rules. Conservative (uses
|
||||
│ │ HIGH/LOW within bar). No fees. Batch runner on top.
|
||||
│ ├── crypto.py HL API-key envelope encryption. enc:v2 =
|
||||
│ │ PBKDF2-salted (H4 fix); enc:v1 read-compat.
|
||||
│ │ scripts/reencrypt_keys.py upgrades stored rows.
|
||||
│ ├── scanner_state.py In-memory toggle + observability for scanners
|
||||
│ ├── macro/
|
||||
│ │ ├── fetchers.py 8 macro indicator HTTP fetchers (each @_none_on_fail)
|
||||
@@ -153,31 +191,81 @@ app/
|
||||
│ │ ├── btc_bottom_reversal.py 2-of-3 AHR999 + 200WMA + Pi Bottom
|
||||
│ │ ├── funding_reversal.py Hourly funding extreme
|
||||
│ │ └── sma_reclaim.py (archive — not scheduled)
|
||||
│ ├── kol_substack.py RSS ingest for 19 KOL feeds
|
||||
│ ├── kol_substack.py RSS ingest for 29 KOL feeds (substack/blog/podcast)
|
||||
│ ├── kol_x.py X (Twitter) ingest via twitterapi.io → x_analysis →
|
||||
│ │ KolPost(source="twitter"). Daily 01:30 UTC. No-op
|
||||
│ │ if twitterapi_io_key unset. Provides the post-side
|
||||
│ │ feed for X-only KOLs (andrewkang, murad).
|
||||
│ ├── kol_onchain.py HL public API + Etherscan diff
|
||||
│ ├── kol_divergence.py Cross-ref talks vs trades within ±7d
|
||||
│ ├── kol_analysis.py AI ticker/direction/conviction extract
|
||||
│ ├── kol_analysis.py AI ticker/direction/conviction extract (Substack).
|
||||
│ │ `_derive_tier()` maps its conviction + talks-vs-
|
||||
│ │ trades score → the SAME trade_signal/directional/
|
||||
│ │ noise tiers x_analysis emits, so non-Twitter posts
|
||||
│ │ get tier set in kol_substack (SIGNAL/VIEW badges +
|
||||
│ │ "Signals only" filter work for blog/substack/pod).
|
||||
│ ├── bottom_indicators.py AHR999 / Pi Cycle / 200WMA math
|
||||
│ └── funding_signal.py Real-time funding extreme detector
|
||||
│ ├── funding_signal.py Real-time funding extreme detector
|
||||
│ ├── signed_request.py EIP-191 signature verification (+ replay cache).
|
||||
│ │ signed_read_creds / optional_signed_read_creds:
|
||||
│ │ header-based (X-Sig-Ts/X-Sig-Sig) creds for read
|
||||
│ │ endpoints, query fallback deprecated (C3 fix).
|
||||
│ └── http_client.py Shared pooled httpx.AsyncClient (keep-alive).
|
||||
│ Hot paths (scrapers, telegram send/poll,
|
||||
│ hl_price_feed, binance REST, x_poster) use
|
||||
│ get_client() with per-request timeout instead of
|
||||
│ new-client-per-call. Closed in lifespan shutdown.
|
||||
├── scrapers/
|
||||
│ └── truth_social.py CNN + trumpstruth.org pollers (5s interval)
|
||||
│ ├── truth_social.py CNN archive poller (5s interval) — primary.
|
||||
│ │ Conditional GET (ETag/Last-Modified → 304 skips
|
||||
│ │ the 30k-post download), batch dedup (1 IN-query
|
||||
│ │ per poll instead of 50 SELECTs), per-post
|
||||
│ │ commit+dispatch so an actionable post never
|
||||
│ │ waits behind older entries' AI analysis.
|
||||
│ │ dispatch_post() is the shared WS/TG/X/trade
|
||||
│ │ fan-out used by BOTH pollers.
|
||||
│ └── trumpstruth.py trumpstruth.org RSS fallback poller. Same post id
|
||||
│ hash → automatic dedup. Whoever sees first wins.
|
||||
│ Offset by half the interval so the two pollers
|
||||
│ don't hit upstream simultaneously.
|
||||
├── ws/
|
||||
│ └── manager.py WebSocket fan-out for live UI updates
|
||||
│ └── manager.py WebSocket fan-out for live UI updates.
|
||||
│ Broadcasts trade_alert events (execution_failed /
|
||||
│ insufficient_balance / budget_reached) via
|
||||
│ _broadcast_trade_alert() in bot_engine.py.
|
||||
├── models.py ★ All SQLAlchemy models in one file
|
||||
├── database.py Async engine + session factory
|
||||
├── config.py Pydantic Settings — reads .env
|
||||
└── main.py FastAPI lifespan, scheduler setup, route mount
|
||||
Includes: singleton lock guard (one-leader, multi-
|
||||
worker safe), deep health check /api/health/deep,
|
||||
boot-grace window for price feeds.
|
||||
|
||||
alembic/versions/ Migrations (numbered NNN). Latest = 026
|
||||
026 = composite index (wallet_address, closed_at) on bot_trades
|
||||
scripts/ One-shot ops
|
||||
├── preflight.py Pre-launch readiness gate (env / DB / TG / AI)
|
||||
├── launch_smoke.py End-to-end smoke (14 checks against running API)
|
||||
├── launch_seed.py Pre-launch data prep: drops test sources, trims KOL
|
||||
│ window to last 30d, refetches all upstream sources.
|
||||
│ Run ONCE before flipping traffic to a fresh DB.
|
||||
├── seed_kol_wallets.py Seeds the KOL wallet table with known addresses.
|
||||
│ Idempotent (INSERT OR IGNORE). Run once at first deploy.
|
||||
├── rescore_v5.py Re-score every Post with current AI prompt
|
||||
├── backfill_signals.py Fill in signal for posts missing it
|
||||
├── reencrypt_keys.py Upgrade stored HL keys to enc:v2 (+KEK rotation
|
||||
│ via OLD_ENCRYPTION_KEY). Idempotent; --dry-run.
|
||||
└── verify_sys2_lifecycle.py Manual System-2 lifecycle walk-through
|
||||
|
||||
tests/ pytest, 64 tests, fast (<1s total)
|
||||
tests/ pytest, 112 tests, fast (<3s total)
|
||||
├── test_adoption.py Adoption + release flow (snapshot-style, no real HL/AI)
|
||||
├── test_telegram_digest.py Daily digest formatting
|
||||
├── test_kol_tier.py kol_analysis._derive_tier (non-Twitter tier mapping)
|
||||
├── test_kol_x.py X ingest: dedup / mapping / no-op (mocked fetch+AI)
|
||||
├── test_ratelimit.py Rate limit coverage (BUG-02 fix)
|
||||
├── test_bottom_reversal_strategy.py btc_bottom_reversal 2-of-3 logic
|
||||
├── test_macro_fetchers_timing.py Macro fetcher timeout + error handling
|
||||
└── test_production_readiness.py Environment / config sanity checks
|
||||
```
|
||||
|
||||
---
|
||||
@@ -206,6 +294,7 @@ Confidence min 88 (platform) / user 85 (platform)
|
||||
Circuit breaker sys1_* sys2_* (independent)
|
||||
Daily budget Full daily_budget_usd n/a — user controls notional on HL
|
||||
Telegram alert Trump alert format Macro/funding alert + /adopt CTA
|
||||
X tweet Optional prediction tweet n/a
|
||||
```
|
||||
|
||||
**If you're tempted to put sys2 logic in `_execute_for_subscriber`**: stop.
|
||||
@@ -234,14 +323,16 @@ minimalism — don't extend them.
|
||||
└─ adoption.adopt_position(wallet, asset, mode):
|
||||
a. Per-wallet asyncio lock acquired
|
||||
b. Pre-flight: no_subscription / no_hl_key / paper_mode /
|
||||
macro_disabled (Subscription.macro_enabled must be ON — /adopt is the
|
||||
sys2 management entry point, so the Macro Vibes toggle gates it) /
|
||||
circuit_breaker (sys2 CB still gates adopt!) /
|
||||
already_adopted / concurrency_cap (3)
|
||||
c. Re-read HL state inside lock (fresh entry/size/lev)
|
||||
d. Resolve sys2_protective_stop_pct(HL_actual_leverage)
|
||||
e. INSERT BotTrade with eff_* frozen + sys2_mode + hl_order_id="adopted:<ts>"
|
||||
d. Reject if leverage > SYS2_MAX_LEVERAGE (BUG-09 fix)
|
||||
e. Resolve sys2_protective_stop_pct(HL_actual_leverage)
|
||||
f. INSERT BotTrade with eff_* frozen + sys2_mode + hl_order_id="adopted:<ts>"
|
||||
+ trigger_post_id=NULL
|
||||
f. register_trade() with full ladder/de-risk/addon/peak_trail
|
||||
└─ Telegram confirmation w/ ladder summary
|
||||
g. register_trade() with full ladder/de-risk/addon/peak_trail
|
||||
|
||||
4. tp_sl_monitor drives the position
|
||||
└─ Stop ratchet, downside de-risk partial reduces, pyramid add-ons,
|
||||
@@ -249,17 +340,11 @@ minimalism — don't extend them.
|
||||
lock-protected partial_derisk / pyramid_add / close_and_finalize.
|
||||
|
||||
5a. User wants out: /release
|
||||
└─ release_management(wallet, trade_id):
|
||||
a. Sets BotTrade.released_at = now
|
||||
b. unregister(trade_id) from watchdog
|
||||
└─ Sets BotTrade.released_at = now; unregister(trade_id) from watchdog
|
||||
└─ HL position UNTOUCHED — bot stops driving, user has manual control
|
||||
└─ Across restarts: recovery skips released rows (released_at filter)
|
||||
└─ Reconciler skips them. Released trades don't appear in
|
||||
/positions/open or the digest "your status" line.
|
||||
|
||||
5b. Bot drives the close (ladder / max-hold)
|
||||
└─ close_and_finalize() atomic claim sets closed_at, computes pnl
|
||||
└─ Trade is now CLOSED (closed_at set, exit_price + pnl_usd written)
|
||||
|
||||
5c. User force-closes via UI: POST /api/positions/{id}/close
|
||||
└─ manual_close calls close_and_finalize(force=True) — bypasses
|
||||
@@ -268,13 +353,30 @@ minimalism — don't extend them.
|
||||
6. Recovery on restart:
|
||||
└─ recovery.rehydrate_open_trades reads BotTrade WHERE closed_at IS NULL
|
||||
AND released_at IS NULL
|
||||
└─ For each: rebuild ladder from sys2_mode + (category OR adopted fallback).
|
||||
The fallback for trigger_post_id IS NULL is the critical fix —
|
||||
without it adopted trades lose their entire sys2 ladder on restart.
|
||||
└─ For each sys2 trade: rebuild ladder from sys2_mode + adopted fallback.
|
||||
Also reschedules _time_stop_check with remaining seconds (elapsed windows
|
||||
fire immediately with delay=0). This is the critical fix — without it
|
||||
adopted trades lose their entire sys2 ladder on restart.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Singleton lock guard (critical infrastructure detail)
|
||||
|
||||
`main.py:lifespan` calls `_acquire_singleton_lock()` at boot using an
|
||||
advisory file lock (`/tmp/trumpsignal-backend.lock`, configurable via
|
||||
`SINGLETON_LOCK_PATH`). Only ONE process — "the leader" — starts the
|
||||
scheduler, scrapers, price feeds, and Telegram poller. Any additional
|
||||
worker (e.g. accidental `--workers 2`) serves HTTP reads only and logs
|
||||
`SINGLETON LOCK NOT ACQUIRED`. The OS releases the lock automatically
|
||||
when the leader exits, so a crashed leader unblocks the next start.
|
||||
|
||||
The `/api/health/deep` endpoint surfaces `"is_leader": false` and adds
|
||||
it to `problems[]` so uptime monitors catch mis-configured multi-worker
|
||||
deployments.
|
||||
|
||||
---
|
||||
|
||||
## Critical invariants checklist (when reviewing any trading change)
|
||||
|
||||
- [ ] Does this code path respect `released_at IS NULL`?
|
||||
@@ -308,8 +410,7 @@ minimalism — don't extend them.
|
||||
7. Add a `/yoursource on|off` command in `telegram_bot.py`.
|
||||
8. If sys2: extend `signal_categories._CATEGORY_EXITS` if it needs a custom
|
||||
exit profile (otherwise default works).
|
||||
9. Add the deep-link path in `telegram.format_post` AND `telegram.format_public_post`
|
||||
(both the per-user private alert and the public channel use the same path map).
|
||||
9. Add the deep-link path in `telegram.format_post` AND `telegram.format_public_post`.
|
||||
|
||||
### Add a new bot command
|
||||
|
||||
@@ -328,6 +429,15 @@ minimalism — don't extend them.
|
||||
2. Mirror the field on the SQLAlchemy model in `app/models.py`.
|
||||
3. Apply locally: `DATABASE_URL=<sqlite> alembic upgrade head`.
|
||||
|
||||
### Enable X (Twitter) posting
|
||||
|
||||
1. Create a Twitter developer app with OAuth 1.0a user context permissions.
|
||||
2. Set in `.env`: `X_API_KEY`, `X_API_SECRET`, `X_ACCESS_TOKEN`,
|
||||
`X_ACCESS_SECRET`, `X_ENABLED=true`.
|
||||
3. Optionally tune: `X_DAILY_CAP` (default 40), `X_FOLLOWUP_MINUTES` (default 15).
|
||||
4. Verify with a dry-run: set `X_ENABLED=false` and check logs — the poster
|
||||
logs what it WOULD tweet without sending.
|
||||
|
||||
### Deploy
|
||||
|
||||
```bash
|
||||
@@ -340,16 +450,27 @@ python scripts/launch_smoke.py --base https://api.trumpalpha.io
|
||||
|
||||
---
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
cd backend && python -m venv venv && source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
cp .env.example .env # fill in: ENCRYPTION_KEY, AI_API_KEY, INGEST_API_KEY
|
||||
uvicorn app.main:app --reload # dev only; prod uses --workers 1, no --reload
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
python -m pytest tests/ -q # full suite, ~0.5s
|
||||
python -m pytest tests/ -q # 112 tests, ~2s
|
||||
python scripts/preflight.py # env + DB + TG + AI auth checks
|
||||
python scripts/launch_smoke.py # 14 end-to-end checks vs running API
|
||||
```
|
||||
|
||||
64 tests. Adoption + telegram_digest are snapshot-style (no real HL/AI).
|
||||
Adoption + telegram_digest are snapshot-style (no real HL/AI).
|
||||
End-to-end trading is verified manually via the bot.
|
||||
|
||||
---
|
||||
@@ -395,6 +516,12 @@ this for you" to "you opened it, bot manages your discipline".
|
||||
- **`_execute_for_subscriber` has lots of `if sub["_is_system_2"]` branches.**
|
||||
Dead code under v2.0 (process_post early-returns for sys2). Kept for diff
|
||||
minimalism — don't extend or re-enable.
|
||||
- **`kol_x.X_KOLS` handle ≠ X username.** `handle` is the CANONICAL key
|
||||
(e.g. "cryptohayes") that MUST match `KolWallet.handle` so divergence can
|
||||
join post-side ↔ on-chain; `x_username` is the screen name fetched from X.
|
||||
andrewkang/murad wallets stay dark (zero divergence detections) until X
|
||||
ingestion supplies their post side — that's the whole reason kol_x exists.
|
||||
Empty `twitterapi_io_key` → kol_x is a full no-op (no error).
|
||||
- **`funding_reversal` source is in `SYSTEM_2_SOURCES`? No.** It's
|
||||
intentionally NOT in either supported set — it ingests as a Post for
|
||||
audit + sends Telegram alert via the CTA path, but doesn't trigger any
|
||||
@@ -404,122 +531,78 @@ this for you" to "you opened it, bot manages your discipline".
|
||||
(Trump) reads full `daily_budget_usd`. Don't read it for new code.
|
||||
- **Adopted trades have `hl_order_id` starting with `"adopted:"`.** Distinct
|
||||
from auto-opened (HL order id integer) and paper (`"paper"` literal).
|
||||
Useful for telemetry filtering.
|
||||
Useful for telemetry filtering — AND it's the canonical sys2 marker the
|
||||
daily-budget query uses, because adopted trades have `trigger_post_id=NULL`
|
||||
so source-based classification fails (M1 fix). The `/trades` serializer also
|
||||
reports `trigger_source="adopted"` from this prefix.
|
||||
- **`/signals/accuracy` is scoped to production sources + buy/short.** It
|
||||
intentionally restricts to `SUPPORTED_TRADING_SOURCES` and the CURRENT
|
||||
buy/short vocabulary — retired/test sources (rsi_reversal, sma_reclaim,
|
||||
breakout, phase1, `test`) and the legacy `sell` signal are excluded so the
|
||||
public accuracy scoreboard reflects what the live bot actually trades.
|
||||
- **macro_enabled vs Telegram alerts are TWO separate switches.** Turning off
|
||||
Macro Vibes (`Subscription.macro_enabled`) gates sys2 *management* — it now
|
||||
blocks `/adopt` (macro_disabled). It does NOT silence Telegram alerts; those
|
||||
are controlled independently by the per-source `TelegramBinding` preference
|
||||
columns. "Alerts on, don't auto-manage" is a deliberately supported combo.
|
||||
- **`telegram.send_message` accepts `int | str` for `chat_id`.** Intentional.
|
||||
Integer = private chat, string = public channel username (e.g. `"@trumpalpha"`).
|
||||
The public channel uses the string form; per-user alerts still pass integers.
|
||||
- **`format_public_post` deliberately omits `expected_move_pct`,
|
||||
`invalidation_price`, and `/adopt` CTA.** Execution-sensitive data stays
|
||||
private (per-user). The public version shows confidence tier (HIGH/MED/LOW)
|
||||
instead of the raw score for readability.
|
||||
- **`_adopt_locks` in adoption.py looks like it should have a cap like
|
||||
`_wallet_open_locks` (512).** It doesn't yet — see Known Issues below.
|
||||
private. The public version shows confidence tier (HIGH/MED/LOW) instead
|
||||
of the raw score.
|
||||
- **`_adopt_locks` in adoption.py** is an `OrderedDict` capped at 512 with
|
||||
LRU eviction — matches the `_WALLET_LOCK_MAX` pattern in `bot_engine`.
|
||||
- **`trumpstruth.py` runs at half-interval offset** — the two CNN + trumpstruth
|
||||
pollers are deliberately staggered so they don't hammer upstream simultaneously.
|
||||
The offset is `truth_social_poll_seconds // 2`, set via APScheduler
|
||||
`next_run_time` at boot.
|
||||
|
||||
---
|
||||
|
||||
## Open known issues (not blocking launch but worth fixing later)
|
||||
|
||||
- ~~**`_time_stop_check` tasks not rehydrated on restart** (BUG-01, FIXED 2026-05-29):~~
|
||||
`recovery.py` now imports `_time_stop_check` + `_background_tasks` from
|
||||
`bot_engine` and `get_exit_profile` from `signal_categories`. After each
|
||||
`register_trade()` call for a sys2 trade, it checks `exit_profile.time_stop_hours`,
|
||||
computes remaining seconds, and reschedules the task. Elapsed windows (backend
|
||||
was down longer than the time-stop period) fire immediately with `delay_seconds=0`.
|
||||
|
||||
- ~~**Rate limit bypass via proxy x-forwarded-for** (BUG-02, HIGH, FIXED 2026-05-29):~~
|
||||
Two-part fix. (1) Frontend proxy `app/api/proxy/[...path]/route.ts` now
|
||||
relays the real client IP via `x-forwarded-for` / `x-real-ip`. (2) Backend
|
||||
had the matching gap: `slowapi`'s default `get_remote_address` reads
|
||||
`request.client.host` (the proxy IP, since uvicorn runs without
|
||||
`--proxy-headers`), so the relayed header was ignored and all users still
|
||||
shared one bucket. Now a shared `app/ratelimit.py` exposes `client_ip_key`
|
||||
(reads `x-forwarded-for[0]` → `x-real-ip` → peer) used by ONE shared
|
||||
`limiter` across `main.py`, `posts.py`, `prices.py`. Also registered
|
||||
`SlowAPIMiddleware` so `default_limits` (60/min) actually applies to every
|
||||
route — previously only the 2 decorated read endpoints were limited and all
|
||||
signed-mutation routes had no limit at all. Covered by `tests/test_ratelimit.py`.
|
||||
|
||||
- ~~**`close_and_finalize` double-failure leaves DB/HL state inconsistent**
|
||||
(BUG-03, MITIGATED 2026-05-29):~~ Full two-phase-commit is out of scope.
|
||||
Mitigation: `reconciler._reconcile_wallet` now runs a "ghost position" pass —
|
||||
queries DB-closed trades from the last 2h and cross-checks against HL open
|
||||
positions. Mismatches are logged at ERROR level and broadcast via WS
|
||||
(`reconcile_drift.ghost_positions`). Manual close required on HL UI.
|
||||
|
||||
- ~~**`_adopt_locks` has no capacity cap** (BUG-04, FIXED 2026-05-29):~~
|
||||
`adoption._adopt_locks` is now an `OrderedDict` capped at `_ADOPT_LOCK_MAX=512`
|
||||
with LRU eviction — matches the `_WALLET_LOCK_MAX` pattern in `bot_engine`.
|
||||
|
||||
- ~~**Reconciler runs wallets sequentially** (BUG-05, FIXED 2026-05-29):~~
|
||||
`reconcile_all_once` now fans out with `asyncio.gather` + `asyncio.Semaphore(10)`.
|
||||
Worst-case tail latency is `ceil(N/10) × 30s` instead of `N × 30s`.
|
||||
|
||||
- ~~**`/stop` also silently disables daily digest** (BUG-07, FIXED 2026-05-29):~~
|
||||
`send_daily_digest` no longer filters on `alerts_enabled` — digest and
|
||||
real-time alerts are independent. `/stop` reply updated to say "Real-time
|
||||
alerts paused … send `/digest off` to stop the daily brief separately".
|
||||
|
||||
- ~~**`binance.py` ASSET_MAP only had BTC + ETH** (BUG-08, CRITICAL, FIXED 2026-05-29):~~
|
||||
Trump's AI can set `target_asset` to SOL/TRUMP/BNB/DOGE/LINK/AAVE.
|
||||
`ASSET_MAP` now covers all those; `BINANCE_WS_URL` is derived from it
|
||||
automatically so the stream list and routing table can never diverge.
|
||||
**Still missing**: HYPE (HL-native, not on Binance) — trades on HYPE fall
|
||||
back to max-hold only until a supplemental HL price feed is added.
|
||||
|
||||
- ~~**`_close_locks` leaked one entry per concurrent-close loser** (BUG-11, FIXED 2026-05-29):~~
|
||||
`close_and_finalize` now pops `_close_locks[trade_id]` in the
|
||||
`rowcount == 0` early-return path, not just on the success paths.
|
||||
|
||||
- ~~**`signed_request._seen` O(n) purge at 5000 entries** (BUG-12, FIXED 2026-05-29):~~
|
||||
Threshold reduced to `_SEEN_PURGE_THRESHOLD = 1000` and the purge now
|
||||
builds an `expired` list in one pass rather than calling `dict.pop` in a loop.
|
||||
|
||||
- ~~**`_get_max_leverage` makes a fresh HL `meta()` call on every trade open**
|
||||
(BUG-10, FIXED 2026-05-29):~~ `_MAX_LEV_CACHE` added (same 300s TTL as
|
||||
`_SZ_DECIMALS_CACHE`). Cache miss populates ALL coins from the single
|
||||
`meta()` response, so concurrent opens pay one API call, not N.
|
||||
|
||||
- ~~**Trump daily budget split**~~ (FIXED 2026-05-29): System-2 is manage-only
|
||||
so the `sys2_pct` reservation no longer makes sense for auto-opens.
|
||||
`daily_cap` for System-1 (Trump) is now `total_cap × 1.0` instead of
|
||||
`total_cap × (1 - sys2_pct)`. The split logic remains intact so it would
|
||||
work correctly if sys2 auto-open is ever re-enabled via ADR.
|
||||
|
||||
- ~~**HL high-leverage adoption**~~ (BUG-09, FIXED 2026-05-29): `adopt_position`
|
||||
now rejects positions where `leverage > SYS2_MAX_LEVERAGE` with error code
|
||||
`leverage_too_high`. Previously the protective stop was computed for 10×
|
||||
but applied to a 25× position — the stop was OUTSIDE the liquidation band.
|
||||
|
||||
- **`adopt:choose:BTC` callback may show stale prices** if user takes >60s
|
||||
to tap (HL fees, partial fills can change entry/size). adopt_position
|
||||
re-reads HL at mode-tap time so the FROZEN BotTrade is always fresh,
|
||||
but the picker label could be outdated.
|
||||
|
||||
- **Telegram bot offset on restart**: `getUpdates` may replay last 24h of
|
||||
messages on backend restart. Stale `/adopt` could re-fire. User can
|
||||
/release to recover.
|
||||
- ~~**Telegram bot offset on restart**~~ **FIXED 2026-06-01**: startup drain
|
||||
added. Stale `/adopt` replays suppressed.
|
||||
|
||||
- ~~**`pyramid_add` double-add on DB commit failure** (BUG-13, FIXED 2026-05-29):~~
|
||||
Same pattern as BUG-03 / partial_derisk. HL `open_position` succeeds but the
|
||||
subsequent `db.commit()` of `addon_steps_done` fails → retrigger sees same
|
||||
`step_idx` and double-adds. Fix: pre-claim `addon_steps_done = step_idx + 1`
|
||||
with a conditional UPDATE (WHERE `addon_steps_done == step_idx`) BEFORE calling
|
||||
HL. If `rowcount == 0`, return early. Second UPDATE writes `entry_price` and
|
||||
`size_usd` after fill confirmation.
|
||||
- **`/adopt` picker label** can show stale price if user waits >60s to tap
|
||||
(frozen BotTrade is always fresh, but the Telegram picker label may be outdated).
|
||||
|
||||
- ~~**`price_impact_monitor` measured wrong asset** (BUG-14, FIXED 2026-05-29):~~
|
||||
`truth_social.py` passed `analysis["asset"]` (BTC/ETH sentiment proxy) to
|
||||
`register_post` instead of `target_asset` (SOL/TRUMP/etc. — the perp we actually
|
||||
trade). Impact % was measuring BTC/ETH not the traded coin. Fix: introduced
|
||||
`tracked_asset = analysis.get("target_asset") or asset` and used it for
|
||||
`price_at_post`, `price_impact_asset`, and `register_post(asset=…)`.
|
||||
- ~~**M1 adopted positions miscounted against sys1 budget**~~ **FIXED 2026-06-09**:
|
||||
the daily-budget query in `bot_engine` now treats any trade whose
|
||||
`hl_order_id` starts with `"adopted:"` as sys2, regardless of the NULL
|
||||
`trigger_post_id`. Previously the outerjoin to Post yielded src=NULL →
|
||||
classified as sys1 → every adopted macro position inflated the Trump
|
||||
scalp budget and could prematurely trip `budget_reached`.
|
||||
|
||||
- **Funding-reversal `/adopt` uses the btc_bottom_reversal exit profile**
|
||||
(`ADOPTED_CATEGORY` is fixed). This is BY DESIGN, not a bug: adopt is
|
||||
ASSET-based (the user opens any position on HL and adopts it) — the bot has
|
||||
no reliable link back to which signal motivated it, and the sys2 ladder is
|
||||
direction/horizon-agnostic. Changing this needs a source-tracking mechanism
|
||||
at adopt time (an ADR), not a one-line tweak.
|
||||
|
||||
**Deferred security items — ALL RESOLVED 2026-06-12:**
|
||||
- ~~**C3**~~ FIXED: signed reads now send `X-Sig-Ts` / `X-Sig-Sig` HEADERS
|
||||
(see `signed_read_creds` in signed_request.py). Legacy `?ts=&sig=` query
|
||||
params still accepted (deprecated) for old clients.
|
||||
- ~~**H4**~~ FIXED: keys now encrypt as `enc:v2` (PBKDF2-HMAC-SHA256, per-blob
|
||||
salt, 600k iters). v1 blobs still decrypt; run `scripts/reencrypt_keys.py`
|
||||
ONCE in prod (after DB backup) to upgrade stored rows.
|
||||
- ~~**M5**~~ Was already fixed: unauthenticated `/telegram/{wallet}/status`
|
||||
returns only `configured`/`bound` booleans; full details require a signed read.
|
||||
|
||||
---
|
||||
|
||||
## Repos in this project
|
||||
|
||||
- **This repo** (`/Users/k/Public/Claude/backend`) — Python/FastAPI backend
|
||||
- **Sibling frontend** (`/Users/k/Public/Claude/trumpsignal`) — Next.js 16
|
||||
- **This repo** (`/Users/k/Public/trumpsignal/backend`) — Python/FastAPI backend
|
||||
- **Sibling frontend** (`/Users/k/Public/trumpsignal/frontend`) — Next.js 16
|
||||
dashboard at trumpsignal.com. See its own CLAUDE.md.
|
||||
|
||||
Both deployed independently. Backend serves the JSON API + Telegram bot.
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Add tier / post_type / talks_vs_trades_flag / sentiment to kol_posts
|
||||
|
||||
Revision ID: 027
|
||||
Revises: 026
|
||||
Create Date: 2026-06-05
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision = '027'
|
||||
down_revision = '026'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table('kol_posts') as batch:
|
||||
batch.add_column(sa.Column('tier', sa.String(16), nullable=True))
|
||||
batch.add_column(sa.Column('post_type', sa.String(16), nullable=True))
|
||||
batch.add_column(sa.Column('talks_vs_trades_flag', sa.Boolean,
|
||||
nullable=True, server_default='0'))
|
||||
batch.add_column(sa.Column('sentiment', sa.String(16), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table('kol_posts') as batch:
|
||||
batch.drop_column('sentiment')
|
||||
batch.drop_column('talks_vs_trades_flag')
|
||||
batch.drop_column('post_type')
|
||||
batch.drop_column('tier')
|
||||
@@ -2,12 +2,14 @@
|
||||
API endpoints for the breakout signal monitor.
|
||||
|
||||
GET /api/signal/status — current state (enabled, recent signals)
|
||||
POST /api/signal/toggle — flip the on/off switch
|
||||
POST /api/signal/toggle — flip the on/off switch (requires X-Ingest-Key)
|
||||
GET /api/signal/history — last N signals (fired regardless of enabled state)
|
||||
"""
|
||||
|
||||
from fastapi import APIRouter
|
||||
from fastapi import APIRouter, Header, HTTPException
|
||||
from typing import Optional
|
||||
|
||||
from app.config import settings
|
||||
from app.services.funding_signal import (
|
||||
set_enabled,
|
||||
is_enabled,
|
||||
@@ -18,17 +20,29 @@ from app.services.funding_signal import (
|
||||
router = APIRouter(prefix="/signal", tags=["signal"])
|
||||
|
||||
|
||||
def _require_ingest_key(x_ingest_key: Optional[str]) -> None:
|
||||
"""Fail-closed operator-only guard (same pattern as signals.py)."""
|
||||
expected = settings.ingest_api_key
|
||||
if not expected:
|
||||
raise HTTPException(503, "toggle disabled (INGEST_API_KEY not configured)")
|
||||
if not x_ingest_key:
|
||||
raise HTTPException(401, "missing X-Ingest-Key header")
|
||||
if x_ingest_key != expected:
|
||||
raise HTTPException(401, "invalid X-Ingest-Key")
|
||||
|
||||
|
||||
@router.get("/status")
|
||||
async def status():
|
||||
return get_status()
|
||||
|
||||
|
||||
@router.post("/toggle")
|
||||
async def toggle(enabled: bool):
|
||||
async def toggle(enabled: bool, x_ingest_key: Optional[str] = Header(default=None)):
|
||||
"""
|
||||
Body: ?enabled=true or ?enabled=false
|
||||
Example: POST /api/signal/toggle?enabled=true
|
||||
Operator-only toggle — requires X-Ingest-Key header (same secret as signal ingest).
|
||||
Example: POST /api/signal/toggle?enabled=true -H 'X-Ingest-Key: …'
|
||||
"""
|
||||
_require_ingest_key(x_ingest_key)
|
||||
set_enabled(enabled)
|
||||
return {"enabled": is_enabled()}
|
||||
|
||||
|
||||
+52
-10
@@ -13,7 +13,7 @@ from datetime import datetime, timedelta, timezone
|
||||
from typing import Any, List, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException, Query
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.config import settings
|
||||
@@ -59,6 +59,11 @@ def _summary_dto(post: KolPost) -> dict:
|
||||
"tickers": _parse_tickers(post.tickers_json),
|
||||
"analyzed_at": iso_utc(post.analyzed_at),
|
||||
"analysis_model": post.analysis_model,
|
||||
# Extended x_analysis fields (migration 027)
|
||||
"tier": post.tier,
|
||||
"post_type": post.post_type,
|
||||
"talks_vs_trades_flag": post.talks_vs_trades_flag or False,
|
||||
"sentiment": post.sentiment,
|
||||
}
|
||||
|
||||
|
||||
@@ -71,19 +76,47 @@ def _detail_dto(post: KolPost) -> dict:
|
||||
@router.get("/kol/posts")
|
||||
async def list_kol_posts(
|
||||
handle: Optional[str] = Query(default=None, description="filter by kol_handle"),
|
||||
source: Optional[str] = Query(default=None, description="substack | twitter"),
|
||||
source: Optional[str] = Query(default=None, description="substack | blog | podcast"),
|
||||
signals_only: bool = Query(default=False,
|
||||
description="exclude noise posts (tier='noise')"),
|
||||
ticker: Optional[str] = Query(default=None, description="filter by ticker symbol e.g. BTC"),
|
||||
days: Optional[int] = Query(default=None, ge=1, le=365, description="restrict to last N days"),
|
||||
limit: int = Query(default=50, ge=1, le=200),
|
||||
page: int = Query(default=1, ge=1),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> dict[str, Any]:
|
||||
stmt = select(KolPost)
|
||||
base = select(KolPost)
|
||||
if handle:
|
||||
stmt = stmt.where(KolPost.kol_handle == handle)
|
||||
base = base.where(KolPost.kol_handle == handle)
|
||||
if source:
|
||||
stmt = stmt.where(KolPost.source == source)
|
||||
stmt = stmt.order_by(KolPost.published_at.desc()).offset((page - 1) * limit).limit(limit)
|
||||
rows = (await db.execute(stmt)).scalars().all()
|
||||
return {"items": [_summary_dto(p) for p in rows], "page": page, "limit": limit}
|
||||
base = base.where(KolPost.source == source)
|
||||
if signals_only:
|
||||
base = base.where(
|
||||
(KolPost.tier.is_(None)) | (KolPost.tier != "noise")
|
||||
)
|
||||
if ticker:
|
||||
# tickers_json stores [{"ticker":"BTC",...}] — match the key-value pair
|
||||
base = base.where(KolPost.tickers_json.like(f'%"ticker": "{ticker.upper()}"%'))
|
||||
if days:
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=days)
|
||||
base = base.where(KolPost.published_at >= cutoff)
|
||||
|
||||
total: int = (await db.execute(
|
||||
select(func.count()).select_from(base.subquery())
|
||||
)).scalar_one()
|
||||
|
||||
rows = (await db.execute(
|
||||
base.order_by(KolPost.published_at.desc())
|
||||
.offset((page - 1) * limit)
|
||||
.limit(limit)
|
||||
)).scalars().all()
|
||||
|
||||
return {
|
||||
"items": [_summary_dto(p) for p in rows],
|
||||
"page": page,
|
||||
"limit": limit,
|
||||
"total": total,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/kol/posts/{post_id}")
|
||||
@@ -365,14 +398,23 @@ async def list_divergence(
|
||||
signal_type=alignment → KOL's words matched their on-chain action (reinforced signal).
|
||||
"""
|
||||
since = datetime.now(timezone.utc).replace(tzinfo=None) - timedelta(days=days)
|
||||
stmt = select(KolDivergence).where(KolDivergence.created_at >= since)
|
||||
# Use post_at (when the KOL post was published) rather than created_at
|
||||
# (when the divergence row was written to DB). created_at can lag by
|
||||
# days or weeks when bulk scans are run, causing old event-pairs to
|
||||
# appear as "recent" divergences long after they occurred.
|
||||
stmt = select(KolDivergence).where(KolDivergence.post_at >= since)
|
||||
if handle:
|
||||
stmt = stmt.where(KolDivergence.handle == handle)
|
||||
if ticker:
|
||||
stmt = stmt.where(KolDivergence.ticker == ticker.upper())
|
||||
if signal_type:
|
||||
stmt = stmt.where(KolDivergence.signal_type == signal_type)
|
||||
stmt = stmt.order_by(KolDivergence.created_at.desc()).limit(200)
|
||||
# Order by post_at (when the KOL actually published), consistent with the
|
||||
# post_at window filter above. Ordering by created_at (DB write time) put
|
||||
# a backfilled older post (post_at=05-18, created_at=05-28) ahead of a
|
||||
# newer one (post_at=05-23), so the "latest divergence" read wrong.
|
||||
# Tie-break on created_at so same-day posts have a stable order.
|
||||
stmt = stmt.order_by(KolDivergence.post_at.desc(), KolDivergence.created_at.desc()).limit(200)
|
||||
rows = (await db.execute(stmt)).scalars().all()
|
||||
return {
|
||||
"window_days": days,
|
||||
|
||||
+40
-12
@@ -8,7 +8,8 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.database import get_db
|
||||
from app.models import BotTrade
|
||||
from app.schemas import BotPerformance
|
||||
from app.services.signed_request import verify_signed_request_any
|
||||
from app.services.signed_request import (
|
||||
SignedReadCreds, signed_read_creds, verify_signed_request_any)
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -21,16 +22,21 @@ ACTION_VIEW_USER = "view_user"
|
||||
@router.get("/performance", response_model=BotPerformance)
|
||||
async def get_performance(
|
||||
wallet: str = Query(..., description="Wallet address (lower-cased internally)"),
|
||||
ts: int = Query(..., description="Signed timestamp (ms)"),
|
||||
sig: str = Query(..., description="EIP-191 signature"),
|
||||
creds: SignedReadCreds = Depends(signed_read_creds),
|
||||
include_paper: bool = Query(
|
||||
False,
|
||||
description="Include paper (simulated) trades. Default false — this "
|
||||
"endpoint reports REAL-money performance, so paper fills "
|
||||
"(hl_order_id='paper') are excluded unless explicitly asked.",
|
||||
),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
wallet = wallet.lower().strip()
|
||||
verify_signed_request_any(
|
||||
actions=[ACTION_VIEW_PERFORMANCE, ACTION_VIEW_USER],
|
||||
wallet=wallet,
|
||||
timestamp_ms=ts,
|
||||
signature=sig,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True,
|
||||
)
|
||||
@@ -43,13 +49,20 @@ async def get_performance(
|
||||
# realization order. A trade opened before the window but closed inside it
|
||||
# correctly counts; one opened inside but still open does not (closed_at IS
|
||||
# NOT NULL already excludes it).
|
||||
result = await db.execute(
|
||||
#
|
||||
# MONEY-SAFETY: by default exclude paper trades (hl_order_id == "paper").
|
||||
# Mixing simulated and real P&L into one "performance" number is misleading
|
||||
# — the dashboard tile that consumes this shows it as real performance.
|
||||
stmt = (
|
||||
select(BotTrade)
|
||||
.where(BotTrade.wallet_address == wallet)
|
||||
.where(BotTrade.closed_at.is_not(None))
|
||||
.where(BotTrade.closed_at >= since)
|
||||
.order_by(BotTrade.closed_at.asc())
|
||||
)
|
||||
if not include_paper:
|
||||
stmt = stmt.where(BotTrade.hl_order_id != "paper")
|
||||
stmt = stmt.order_by(BotTrade.closed_at.asc())
|
||||
result = await db.execute(stmt)
|
||||
trades = result.scalars().all()
|
||||
|
||||
total_trades = len(trades)
|
||||
@@ -63,14 +76,29 @@ async def get_performance(
|
||||
max_drawdown_pct=0.0,
|
||||
)
|
||||
|
||||
winning = sum(1 for t in trades if (t.pnl_usd or 0) > 0)
|
||||
win_rate = winning / total_trades
|
||||
# Only include trades with a known PnL in financial statistics.
|
||||
# Trades with pnl_usd=NULL were externally closed or unsettled — treating
|
||||
# them as 0 silently inflates trade count and distorts win rate / net PnL.
|
||||
settled = [t for t in trades if t.pnl_usd is not None]
|
||||
if not settled:
|
||||
return BotPerformance(
|
||||
period_days=PERIOD_DAYS,
|
||||
total_trades=total_trades,
|
||||
win_rate=0.0,
|
||||
net_pnl_usd=0.0,
|
||||
avg_hold_seconds=0.0,
|
||||
max_drawdown_pct=0.0,
|
||||
)
|
||||
|
||||
pnl_values = [(t.pnl_usd or 0.0) for t in trades]
|
||||
winning = sum(1 for t in settled if t.pnl_usd > 0) # type: ignore[operator]
|
||||
win_rate = winning / len(settled)
|
||||
|
||||
pnl_values = [t.pnl_usd for t in settled] # type: ignore[misc]
|
||||
net_pnl = sum(pnl_values)
|
||||
|
||||
hold_values = [(t.hold_seconds or 0) for t in trades]
|
||||
avg_hold = sum(hold_values) / len(hold_values)
|
||||
# For hold time use all trades (we always have opened_at + closed_at when closed)
|
||||
hold_values = [t.hold_seconds for t in trades if t.hold_seconds is not None]
|
||||
avg_hold = sum(hold_values) / len(hold_values) if hold_values else 0.0
|
||||
|
||||
# Max drawdown: running peak → trough of cumulative PnL
|
||||
cumulative = 0.0
|
||||
|
||||
+53
-30
@@ -35,7 +35,9 @@ from app.database import get_db
|
||||
from app.models import BotTrade, Subscription, iso_utc
|
||||
from app.services.crypto import decrypt_api_key
|
||||
from app.services.price_store import price_store
|
||||
from app.services.signed_request import verify_signed_request, verify_signed_request_any
|
||||
from app.services.signed_request import (
|
||||
SignedReadCreds, signed_read_creds,
|
||||
verify_signed_request, verify_signed_request_any)
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -144,8 +146,7 @@ def _enrich(trade: BotTrade) -> OpenPosition:
|
||||
@router.get("/positions/open", response_model=OpenPositionsResponse)
|
||||
async def get_open_positions(
|
||||
wallet: str = Query(..., description="Wallet address (lower-cased internally)"),
|
||||
ts: int = Query(..., description="Signed timestamp (ms)"),
|
||||
sig: str = Query(..., description="EIP-191 signature"),
|
||||
creds: SignedReadCreds = Depends(signed_read_creds),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""Live open positions for the wallet, with mark-to-market PnL."""
|
||||
@@ -153,8 +154,8 @@ async def get_open_positions(
|
||||
verify_signed_request_any(
|
||||
actions=[ACTION_VIEW_POSITIONS, ACTION_VIEW_USER],
|
||||
wallet=wallet,
|
||||
timestamp_ms=ts,
|
||||
signature=sig,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True,
|
||||
)
|
||||
@@ -177,8 +178,7 @@ async def get_open_positions(
|
||||
@router.get("/positions/today", response_model=TodayStatsResponse)
|
||||
async def get_today_stats(
|
||||
wallet: str = Query(...),
|
||||
ts: int = Query(..., description="Signed timestamp (ms)"),
|
||||
sig: str = Query(..., description="EIP-191 signature"),
|
||||
creds: SignedReadCreds = Depends(signed_read_creds),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""Today's realized P&L (since UTC midnight) + open count.
|
||||
@@ -190,8 +190,8 @@ async def get_today_stats(
|
||||
verify_signed_request_any(
|
||||
actions=[ACTION_VIEW_POSITIONS, ACTION_VIEW_USER],
|
||||
wallet=wallet,
|
||||
timestamp_ms=ts,
|
||||
signature=sig,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True,
|
||||
)
|
||||
@@ -199,11 +199,15 @@ async def get_today_stats(
|
||||
hour=0, minute=0, second=0, microsecond=0, tzinfo=None
|
||||
)
|
||||
|
||||
# Exclude released trades — these were released back to user control and
|
||||
# closed by the user on HL directly, not by the bot. Including them would
|
||||
# inflate the bot's "today" P&L with trades the bot didn't manage.
|
||||
closed_rows = await db.execute(
|
||||
select(BotTrade).where(
|
||||
BotTrade.wallet_address == wallet,
|
||||
BotTrade.closed_at >= midnight,
|
||||
BotTrade.pnl_usd.is_not(None),
|
||||
BotTrade.released_at.is_(None),
|
||||
)
|
||||
)
|
||||
closed = closed_rows.scalars().all()
|
||||
@@ -280,6 +284,15 @@ async def manual_close(
|
||||
if not wallet or not isinstance(timestamp, int) or not isinstance(signature, str):
|
||||
raise HTTPException(422, "wallet, timestamp, signature required")
|
||||
|
||||
# Verify signature first — prevents trade_id enumeration via 403/409 before auth.
|
||||
verify_signed_request(
|
||||
action=ACTION_CLOSE_TRADE,
|
||||
wallet=wallet,
|
||||
timestamp_ms=timestamp,
|
||||
signature=signature,
|
||||
body={"trade_id": trade_id},
|
||||
)
|
||||
|
||||
# Load the trade. Must be open AND owned by the signing wallet.
|
||||
trade = (await db.execute(select(BotTrade).where(BotTrade.id == trade_id))).scalar_one_or_none()
|
||||
if trade is None:
|
||||
@@ -289,14 +302,6 @@ async def manual_close(
|
||||
if trade.closed_at is not None:
|
||||
raise HTTPException(409, f"trade {trade_id} is already closed")
|
||||
|
||||
verify_signed_request(
|
||||
action=ACTION_CLOSE_TRADE,
|
||||
wallet=wallet,
|
||||
timestamp_ms=timestamp,
|
||||
signature=signature,
|
||||
body={"trade_id": trade_id},
|
||||
)
|
||||
|
||||
# Find the API key from the subscription.
|
||||
sub = (await db.execute(
|
||||
select(Subscription).where(Subscription.wallet_address == wallet)
|
||||
@@ -330,7 +335,26 @@ async def manual_close(
|
||||
force=True,
|
||||
)
|
||||
|
||||
closed = (await db.execute(select(BotTrade).where(BotTrade.id == trade_id))).scalar_one()
|
||||
# B45: close_and_finalize uses its own AsyncSessionLocal session, so the
|
||||
# route's `db` session has a stale identity-map cache for this trade row.
|
||||
# `populate_existing=True` forces SQLAlchemy to overwrite the cached
|
||||
# instance with the freshly-committed values (exit_price, pnl_usd, etc.)
|
||||
# rather than returning the pre-close snapshot from the identity map.
|
||||
closed = (await db.execute(
|
||||
select(BotTrade).where(BotTrade.id == trade_id).execution_options(populate_existing=True)
|
||||
)).scalar_one()
|
||||
|
||||
# B46: close_and_finalize returns silently on certain failures (no price
|
||||
# for paper close, HL returns no fill, etc.) without raising an exception.
|
||||
# Detect the failure by checking whether closed_at was actually written.
|
||||
if closed.closed_at is None:
|
||||
raise HTTPException(
|
||||
500,
|
||||
"Close command issued but the position could not be closed "
|
||||
"(no price feed, HL fill failure, or another caller closed it first). "
|
||||
"Refresh open positions — if it still shows, retry or close manually on HL."
|
||||
)
|
||||
|
||||
return CloseTradeResponse(
|
||||
status="ok",
|
||||
trade_id=trade_id,
|
||||
@@ -373,14 +397,6 @@ async def set_trade_grow(
|
||||
if body_tid != trade_id:
|
||||
raise HTTPException(400, "trade_id mismatch (path vs signed body)")
|
||||
|
||||
trade = (await db.execute(select(BotTrade).where(BotTrade.id == trade_id))).scalar_one_or_none()
|
||||
if trade is None:
|
||||
raise HTTPException(404, f"trade {trade_id} not found")
|
||||
if trade.wallet_address.lower() != wallet:
|
||||
raise HTTPException(403, "trade belongs to a different wallet")
|
||||
if trade.closed_at is not None:
|
||||
raise HTTPException(409, f"trade {trade_id} is already closed")
|
||||
|
||||
verify_signed_request(
|
||||
action=ACTION_SET_GROW,
|
||||
wallet=wallet,
|
||||
@@ -389,6 +405,14 @@ async def set_trade_grow(
|
||||
body={"trade_id": trade_id, "enabled": enabled},
|
||||
)
|
||||
|
||||
trade = (await db.execute(select(BotTrade).where(BotTrade.id == trade_id))).scalar_one_or_none()
|
||||
if trade is None:
|
||||
raise HTTPException(404, f"trade {trade_id} not found")
|
||||
if trade.wallet_address.lower() != wallet:
|
||||
raise HTTPException(403, "trade belongs to a different wallet")
|
||||
if trade.closed_at is not None:
|
||||
raise HTTPException(409, f"trade {trade_id} is already closed")
|
||||
|
||||
trade.grow_mode = enabled
|
||||
await db.commit()
|
||||
|
||||
@@ -429,8 +453,7 @@ class HLPositionsResponse(BaseModel):
|
||||
@router.get("/positions/hl/{wallet}", response_model=HLPositionsResponse)
|
||||
async def list_hl_positions_endpoint(
|
||||
wallet: str,
|
||||
ts: int = Query(..., description="Signed timestamp (ms)"),
|
||||
sig: str = Query(..., description="EIP-191 signature"),
|
||||
creds: SignedReadCreds = Depends(signed_read_creds),
|
||||
):
|
||||
"""Read the wallet's CURRENT Hyperliquid open positions, annotated with
|
||||
'already adopted' flag. Used by the Adopt picker on the frontend.
|
||||
@@ -445,8 +468,8 @@ async def list_hl_positions_endpoint(
|
||||
verify_signed_request_any(
|
||||
actions=[ACTION_VIEW_POSITIONS, ACTION_VIEW_USER],
|
||||
wallet=wallet,
|
||||
timestamp_ms=ts,
|
||||
signature=sig,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True,
|
||||
)
|
||||
|
||||
+167
-4
@@ -5,16 +5,29 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request
|
||||
from fastapi.responses import Response
|
||||
|
||||
from app.ratelimit import limiter
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import case, func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.models import Post, iso_utc
|
||||
from app.schemas import PriceImpact, TrumpPost
|
||||
from app.schemas import PostFilterCounts, PostListResponse, PriceImpact, SourceCount, TrumpPost
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_ARCHIVE_EXCLUDED_SOURCES = (
|
||||
"truth",
|
||||
"btc_bottom_reversal",
|
||||
"funding_reversal",
|
||||
"kol_divergence",
|
||||
)
|
||||
|
||||
|
||||
_AI_SCORED_EXPR = (
|
||||
(func.coalesce(Post.ai_confidence, 0) > 0) |
|
||||
Post.ai_reasoning.is_not(None)
|
||||
)
|
||||
|
||||
|
||||
def _direction_correct(signal: Optional[str], pct: Optional[float]) -> Optional[bool]:
|
||||
if pct is None or signal is None:
|
||||
@@ -98,11 +111,161 @@ async def get_posts(
|
||||
return [_post_to_schema(p) for p in posts]
|
||||
|
||||
|
||||
@router.get("/posts-paged", response_model=PostListResponse)
|
||||
@limiter.limit("60/minute")
|
||||
async def get_posts_page(
|
||||
request: Request,
|
||||
limit: int = Query(default=20, ge=1, le=500),
|
||||
page: int = Query(default=1, ge=1),
|
||||
source: Optional[str] = Query(
|
||||
default=None,
|
||||
description="Filter to a single source (e.g. 'truth', 'btc_bottom_reversal').",
|
||||
),
|
||||
source_in: Optional[str] = Query(
|
||||
default=None,
|
||||
description="Comma-separated allowlist of sources.",
|
||||
),
|
||||
source_not_in: Optional[str] = Query(
|
||||
default=None,
|
||||
description="Comma-separated denylist of sources.",
|
||||
),
|
||||
archive_only: bool = Query(
|
||||
default=False,
|
||||
description="When true, return only archived/retired sources (exclude live modules).",
|
||||
),
|
||||
sentiment: Optional[str] = Query(
|
||||
default=None,
|
||||
pattern="^(bullish|bearish|neutral)$",
|
||||
description="Optional sentiment filter.",
|
||||
),
|
||||
signal: Optional[str] = Query(
|
||||
default=None,
|
||||
pattern="^(buy|short|actionable)$",
|
||||
description="Optional signal filter. 'actionable' = buy or short.",
|
||||
),
|
||||
ai_scored_only: bool = Query(
|
||||
default=False,
|
||||
description="When true, exclude off-topic rows that were skipped before AI scoring.",
|
||||
),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
response: Response = None,
|
||||
):
|
||||
offset = (page - 1) * limit
|
||||
stmt = select(Post)
|
||||
count_stmt = select(func.count()).select_from(Post)
|
||||
counts_stmt = select(
|
||||
func.count().label("all_count"),
|
||||
func.sum(case((Post.signal.in_(("buy", "short")), 1), else_=0)).label("actionable_count"),
|
||||
func.sum(case((Post.signal == "buy", 1), else_=0)).label("buy_count"),
|
||||
func.sum(case((Post.signal == "short", 1), else_=0)).label("short_count"),
|
||||
func.sum(case((_AI_SCORED_EXPR, 0), else_=1)).label("off_topic_count"),
|
||||
).select_from(Post)
|
||||
source_counts_stmt = select(
|
||||
Post.source.label("source"),
|
||||
func.count(Post.id).label("count"),
|
||||
func.max(Post.published_at).label("latest"),
|
||||
).select_from(Post)
|
||||
|
||||
included_sources = [s.strip() for s in (source_in or "").split(",") if s.strip()]
|
||||
excluded_sources = [s.strip() for s in (source_not_in or "").split(",") if s.strip()]
|
||||
if archive_only:
|
||||
excluded_sources = list(dict.fromkeys([*excluded_sources, *_ARCHIVE_EXCLUDED_SOURCES]))
|
||||
|
||||
if source:
|
||||
stmt = stmt.where(Post.source == source)
|
||||
count_stmt = count_stmt.where(Post.source == source)
|
||||
counts_stmt = counts_stmt.where(Post.source == source)
|
||||
source_counts_stmt = source_counts_stmt.where(Post.source == source)
|
||||
elif included_sources:
|
||||
stmt = stmt.where(Post.source.in_(included_sources))
|
||||
count_stmt = count_stmt.where(Post.source.in_(included_sources))
|
||||
counts_stmt = counts_stmt.where(Post.source.in_(included_sources))
|
||||
# NOTE: source_counts is the chip/source breakdown the UI renders the
|
||||
# filter bar from. It must reflect every source available in the
|
||||
# current view scope — NOT just the one the user has selected. So
|
||||
# `source_in` (the chip-selection narrowing) is deliberately NOT
|
||||
# applied here; only the exclusion filters (archive_only /
|
||||
# source_not_in) below scope it. Applying it would collapse the chip
|
||||
# bar to the single selected source with no way back to "all".
|
||||
|
||||
if excluded_sources:
|
||||
stmt = stmt.where(~Post.source.in_(excluded_sources))
|
||||
count_stmt = count_stmt.where(~Post.source.in_(excluded_sources))
|
||||
counts_stmt = counts_stmt.where(~Post.source.in_(excluded_sources))
|
||||
source_counts_stmt = source_counts_stmt.where(~Post.source.in_(excluded_sources))
|
||||
|
||||
if sentiment:
|
||||
stmt = stmt.where(Post.sentiment == sentiment)
|
||||
count_stmt = count_stmt.where(Post.sentiment == sentiment)
|
||||
counts_stmt = counts_stmt.where(Post.sentiment == sentiment)
|
||||
source_counts_stmt = source_counts_stmt.where(Post.sentiment == sentiment)
|
||||
if ai_scored_only:
|
||||
stmt = stmt.where(_AI_SCORED_EXPR)
|
||||
count_stmt = count_stmt.where(_AI_SCORED_EXPR)
|
||||
source_counts_stmt = source_counts_stmt.where(_AI_SCORED_EXPR)
|
||||
|
||||
if signal == "actionable":
|
||||
stmt = stmt.where(Post.signal.in_(("buy", "short")))
|
||||
count_stmt = count_stmt.where(Post.signal.in_(("buy", "short")))
|
||||
elif signal:
|
||||
stmt = stmt.where(Post.signal == signal)
|
||||
count_stmt = count_stmt.where(Post.signal == signal)
|
||||
|
||||
stmt = stmt.order_by(Post.published_at.desc()).offset(offset).limit(limit)
|
||||
result = await db.execute(stmt)
|
||||
total_result = await db.execute(count_stmt)
|
||||
counts_result = await db.execute(counts_stmt)
|
||||
source_counts_result = await db.execute(
|
||||
source_counts_stmt.group_by(Post.source).order_by(func.count(Post.id).desc(), Post.source.asc())
|
||||
)
|
||||
posts = result.scalars().all()
|
||||
total = int(total_result.scalar_one() or 0)
|
||||
counts_row = counts_result.one()
|
||||
off_topic = int(counts_row.off_topic_count or 0)
|
||||
all_count = int(counts_row.all_count or 0)
|
||||
if response is not None:
|
||||
response.headers["Cache-Control"] = "public, max-age=30, stale-while-revalidate=60"
|
||||
return PostListResponse(
|
||||
items=[_post_to_schema(p) for p in posts],
|
||||
total=total,
|
||||
page=page,
|
||||
limit=limit,
|
||||
counts=PostFilterCounts(
|
||||
all=max(0, all_count - off_topic) if ai_scored_only else all_count,
|
||||
actionable=int(counts_row.actionable_count or 0),
|
||||
buy=int(counts_row.buy_count or 0),
|
||||
short=int(counts_row.short_count or 0),
|
||||
off_topic=off_topic,
|
||||
),
|
||||
source_counts=[
|
||||
SourceCount(
|
||||
source=row.source,
|
||||
count=int(row.count or 0),
|
||||
latest=iso_utc(row.latest),
|
||||
)
|
||||
for row in source_counts_result.all()
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
@router.get("/signals/accuracy")
|
||||
async def signal_accuracy(db: AsyncSession = Depends(get_db)):
|
||||
"""Aggregate accuracy of directional signals (buy/sell/short) against realised price moves."""
|
||||
"""Aggregate accuracy of directional signals against realised price moves.
|
||||
|
||||
Scoped to the CURRENT signal taxonomy the live bot actually trades:
|
||||
* only buy/short (the retired "sell" vocabulary is excluded — the bot
|
||||
emits buy/short now, and 42 legacy truth/sell rows would otherwise
|
||||
pollute the public scoreboard),
|
||||
* only production sources (SUPPORTED_TRADING_SOURCES) — retired/test
|
||||
ingest sources like rsi_reversal, sma_reclaim, breakout, phase1 and
|
||||
`test` must not appear in the public accuracy stats.
|
||||
"""
|
||||
from app.services.signal_categories import SUPPORTED_TRADING_SOURCES
|
||||
result = await db.execute(
|
||||
select(Post).where(Post.signal.in_(["buy", "sell", "short"]))
|
||||
select(Post).where(
|
||||
Post.signal.in_(["buy", "short"]),
|
||||
func.lower(Post.source).in_(SUPPORTED_TRADING_SOURCES),
|
||||
)
|
||||
)
|
||||
posts = result.scalars().all()
|
||||
|
||||
|
||||
+2
-3
@@ -1,7 +1,6 @@
|
||||
import logging
|
||||
from typing import List
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
from fastapi.responses import Response
|
||||
|
||||
@@ -33,8 +32,8 @@ async def fetch_binance_candles(asset: str, tf: str, limit: int) -> List[Candle]
|
||||
symbol = SYMBOL_MAP[asset]
|
||||
interval = BINANCE_INTERVAL[tf]
|
||||
url = f"{settings.binance_rest_url}/api/v3/klines?symbol={symbol}&interval={interval}&limit={limit}"
|
||||
async with httpx.AsyncClient(timeout=15) as client:
|
||||
resp = await client.get(url)
|
||||
from app.services.http_client import get_client
|
||||
resp = await get_client().get(url, timeout=15)
|
||||
resp.raise_for_status()
|
||||
rows = resp.json()
|
||||
return [
|
||||
|
||||
@@ -31,6 +31,7 @@ from typing import Optional
|
||||
from fastapi import APIRouter, Depends, Header, HTTPException
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.config import settings
|
||||
@@ -174,8 +175,22 @@ async def ingest_signal(
|
||||
prefilter_reason="external_signal", # bypasses entry-filter audit
|
||||
)
|
||||
db.add(post)
|
||||
try:
|
||||
await db.commit()
|
||||
await db.refresh(post)
|
||||
except IntegrityError:
|
||||
# Concurrent request beat us to the INSERT — fetch the existing row.
|
||||
await db.rollback()
|
||||
existing2 = await db.execute(select(Post).where(Post.external_id == ext_id_hashed))
|
||||
prior2 = existing2.scalar_one_or_none()
|
||||
if prior2:
|
||||
return SignalIngestResponse(
|
||||
status="duplicate",
|
||||
post_id=prior2.id,
|
||||
dedup_against=prior2.id,
|
||||
note=f"concurrent ingest: external_id {body.external_id!r} already exists",
|
||||
)
|
||||
raise # unexpected — re-raise if we still can't find the row
|
||||
|
||||
logger.info(
|
||||
"Ingested signal: source=%s id=%s → post_id=%d, %s/%s conf=%d category=%s",
|
||||
@@ -219,6 +234,14 @@ async def ingest_signal(
|
||||
except Exception as exc:
|
||||
logger.warning("Telegram notify failed for signal %d: %s", post.id, exc)
|
||||
|
||||
# Auto-post a viral "live prediction" tweet (fire-and-forget; no-op unless
|
||||
# X is configured + enabled). Internally gated to source=truth buy/short.
|
||||
try:
|
||||
from app.services.x_poster import notify_x_signal
|
||||
notify_x_signal(post)
|
||||
except Exception as exc:
|
||||
logger.warning("X notify failed for signal %d: %s", post.id, exc)
|
||||
|
||||
return SignalIngestResponse(status="accepted", post_id=post.id)
|
||||
|
||||
|
||||
|
||||
@@ -63,6 +63,18 @@ async def subscribe(request: Request, db: AsyncSession = Depends(get_db)):
|
||||
# Re-subscribing is allowed to change paper mode (e.g. user wants to
|
||||
# promote from paper to live). Otherwise leave existing flag alone.
|
||||
if paper_mode != sub.paper_mode:
|
||||
# SAFETY: promoting paper → live raises the risk level from
|
||||
# "simulated" to "real money". Force Auto-Trade OFF on that
|
||||
# transition so a switch the user flipped while it was harmless
|
||||
# (paper) can NEVER carry over and silently start opening real
|
||||
# positions the moment they add an API key. They must re-enable
|
||||
# Auto-Trade explicitly while live — an intentional, signed action.
|
||||
if sub.paper_mode and not paper_mode and sub.auto_trade:
|
||||
sub.auto_trade = False
|
||||
logger.info(
|
||||
"Subscription %s promoted paper→live — Auto-Trade forced "
|
||||
"OFF (must be re-enabled explicitly while live)", wallet,
|
||||
)
|
||||
sub.paper_mode = paper_mode
|
||||
|
||||
await db.commit()
|
||||
|
||||
+69
-8
@@ -25,7 +25,8 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.config import settings
|
||||
from app.database import get_db
|
||||
from app.models import TelegramBinding, Subscription
|
||||
from app.services.signed_request import verify_signed_request
|
||||
from app.services.signed_request import (
|
||||
SignedReadCreds, optional_signed_read_creds, verify_signed_request)
|
||||
from app.services.telegram import send_test_message
|
||||
from app.services.telegram_bot import issue_binding_code, unbind_wallet
|
||||
|
||||
@@ -87,14 +88,21 @@ class InitResponse(BaseModel):
|
||||
expires_in_seconds: int
|
||||
|
||||
|
||||
# ── Endpoints ────────────────────────────────────────────────────────────
|
||||
# ── Helpers ──────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/{wallet}/status", response_model=StatusResponse)
|
||||
async def status(wallet: str, db: AsyncSession = Depends(get_db)) -> StatusResponse:
|
||||
"""Public-by-wallet read. Returns whether server is configured AND
|
||||
whether this wallet has bound a Telegram chat."""
|
||||
wallet = wallet.lower().strip()
|
||||
async def _build_status(
|
||||
wallet: str,
|
||||
db: AsyncSession,
|
||||
authenticated: bool = False,
|
||||
) -> StatusResponse:
|
||||
"""Shared status-building logic used by both the GET endpoint and
|
||||
update_preferences so both always return a consistent StatusResponse.
|
||||
|
||||
B37: previously update_preferences called `await status(wallet, db)`,
|
||||
which passed db as the `timestamp` positional arg and left the DI-managed
|
||||
`db` param as a raw Depends() wrapper — crashing on `.execute()`.
|
||||
"""
|
||||
configured = bool(settings.telegram_bot_token and settings.telegram_bot_username)
|
||||
b = (await db.execute(
|
||||
select(TelegramBinding).where(TelegramBinding.wallet_address == wallet)
|
||||
@@ -104,6 +112,14 @@ async def status(wallet: str, db: AsyncSession = Depends(get_db)) -> StatusRespo
|
||||
return StatusResponse(configured=configured,
|
||||
bot_username=settings.telegram_bot_username or None,
|
||||
bound=False)
|
||||
|
||||
if not authenticated:
|
||||
return StatusResponse(
|
||||
configured=configured,
|
||||
bot_username=settings.telegram_bot_username or None,
|
||||
bound=True,
|
||||
)
|
||||
|
||||
return StatusResponse(
|
||||
configured=configured,
|
||||
bot_username=settings.telegram_bot_username or None,
|
||||
@@ -123,6 +139,48 @@ async def status(wallet: str, db: AsyncSession = Depends(get_db)) -> StatusRespo
|
||||
)
|
||||
|
||||
|
||||
# ── Endpoints ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/{wallet}/status", response_model=StatusResponse)
|
||||
async def status(
|
||||
wallet: str,
|
||||
creds: Optional[SignedReadCreds] = Depends(optional_signed_read_creds),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> StatusResponse:
|
||||
"""Wallet Telegram status.
|
||||
|
||||
Unauthenticated: returns configured + bound (boolean only).
|
||||
Authenticated (timestamp + signature): returns full binding details.
|
||||
This prevents third parties from de-anonymising wallets via tg_username/chat_id.
|
||||
"""
|
||||
wallet = wallet.lower().strip()
|
||||
|
||||
# Verify ownership if credentials provided (best-effort — never block on failure).
|
||||
# Accept either "view_telegram_status" (dedicated action) or "view_user"
|
||||
# (broad read action that the frontend already caches for other endpoints).
|
||||
# Accepting view_user avoids requiring a separate wallet signature just to
|
||||
# see Telegram status — the frontend reuses its cached view_user envelope.
|
||||
authenticated = False
|
||||
if creds is not None:
|
||||
for _action in ("view_telegram_status", "view_user"):
|
||||
try:
|
||||
verify_signed_request(
|
||||
action=_action,
|
||||
wallet=wallet,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True,
|
||||
)
|
||||
authenticated = True
|
||||
break
|
||||
except Exception:
|
||||
pass # Try next action; fall through to redacted response if all fail
|
||||
|
||||
return await _build_status(wallet, db, authenticated=authenticated)
|
||||
|
||||
|
||||
@router.post("/{wallet}/init", response_model=InitResponse)
|
||||
async def init_binding(
|
||||
wallet: str, body: SignedEnvelope,
|
||||
@@ -192,7 +250,10 @@ async def update_preferences(
|
||||
await db.commit()
|
||||
await db.refresh(b)
|
||||
|
||||
return await status(wallet, db)
|
||||
# Return full authenticated status — the caller already proved ownership
|
||||
# via the signed request verified above (B37: was `await status(wallet, db)`
|
||||
# which passed db as the timestamp arg, crashing inside status()).
|
||||
return await _build_status(wallet, db, authenticated=True)
|
||||
|
||||
|
||||
@router.post("/{wallet}/unbind")
|
||||
|
||||
+25
-18
@@ -9,7 +9,8 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.database import get_db
|
||||
from app.models import BotTrade, iso_utc
|
||||
from app.schemas import BotTrade as BotTradeSchema
|
||||
from app.services.signed_request import verify_signed_request_any
|
||||
from app.services.signed_request import (
|
||||
SignedReadCreds, signed_read_creds, verify_signed_request_any)
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -19,27 +20,34 @@ ACTION_VIEW_USER = "view_user"
|
||||
|
||||
|
||||
def _trade_to_schema(trade: BotTrade) -> BotTradeSchema:
|
||||
# Join against trigger_post to surface the source tag. When a trade was
|
||||
# opened by an ingested signal (VCP scanner, user's module, etc.) the
|
||||
# source reveals WHICH module produced it — critical for "is module X
|
||||
# actually making money?" attribution analysis.
|
||||
trigger_source = None
|
||||
trigger_post_text = None
|
||||
if trade.trigger_post is not None:
|
||||
trigger_source = trade.trigger_post.source
|
||||
# Paper trades are tagged via hl_order_id at open time; that's the only
|
||||
# stable signal we have to distinguish them in aggregate views.
|
||||
trigger_post_text = trade.trigger_post.text
|
||||
elif trade.hl_order_id and str(trade.hl_order_id).startswith("adopted:"):
|
||||
# Adopted (sys2 manage-only) trades have trigger_post_id=NULL by
|
||||
# construction, so trigger_source would otherwise fall through to
|
||||
# "Unknown" in the UI. The hl_order_id prefix is the reliable marker
|
||||
# (see adoption.py / CLAUDE.md) — surface it as a proper attribution.
|
||||
trigger_source = "adopted"
|
||||
is_paper = (trade.hl_order_id == "paper")
|
||||
# Preserve None for nullable fields — do NOT coerce to 0 / "".
|
||||
# The frontend uses null to distinguish "unknown/not yet settled" from
|
||||
# a genuine zero (e.g. a break-even trade, a 0-second hold).
|
||||
# `or 0` was silently masking externally-closed trades and unsettled PnL.
|
||||
return BotTradeSchema(
|
||||
id=trade.id,
|
||||
asset=trade.asset,
|
||||
side=trade.side,
|
||||
entry_price=trade.entry_price,
|
||||
exit_price=trade.exit_price or 0.0,
|
||||
pnl_usd=trade.pnl_usd or 0.0,
|
||||
hold_seconds=trade.hold_seconds or 0,
|
||||
trigger_post_id=trade.trigger_post_id or 0,
|
||||
exit_price=trade.exit_price, # None = position still open or extern-closed
|
||||
pnl_usd=trade.pnl_usd, # None = unsettled / extern-closed
|
||||
hold_seconds=trade.hold_seconds, # None = not yet computed
|
||||
trigger_post_id=trade.trigger_post_id, # None = adopted/manual, no trigger post
|
||||
opened_at=iso_utc(trade.opened_at) or "",
|
||||
closed_at=iso_utc(trade.closed_at) or "",
|
||||
closed_at=iso_utc(trade.closed_at), # None = still open (shouldn't happen here)
|
||||
trigger_post_text=trigger_post_text,
|
||||
trigger_source=trigger_source,
|
||||
is_paper=is_paper,
|
||||
)
|
||||
@@ -48,9 +56,8 @@ def _trade_to_schema(trade: BotTrade) -> BotTradeSchema:
|
||||
@router.get("/trades", response_model=List[BotTradeSchema])
|
||||
async def get_trades(
|
||||
wallet: str = Query(..., description="Wallet address (lower-cased internally)"),
|
||||
ts: int = Query(..., description="Signed timestamp (ms)"),
|
||||
sig: str = Query(..., description="EIP-191 signature"),
|
||||
limit: int = Query(default=20, ge=1, le=100),
|
||||
creds: SignedReadCreds = Depends(signed_read_creds),
|
||||
limit: int = Query(default=20, ge=1, le=500), # raised from 100: Analytics needs 500 for full history
|
||||
page: int = Query(default=1, ge=1),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
@@ -58,8 +65,8 @@ async def get_trades(
|
||||
verify_signed_request_any(
|
||||
actions=[ACTION_VIEW_TRADES, ACTION_VIEW_USER],
|
||||
wallet=wallet,
|
||||
timestamp_ms=ts,
|
||||
signature=sig,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True,
|
||||
)
|
||||
@@ -71,7 +78,7 @@ async def get_trades(
|
||||
.options(joinedload(BotTrade.trigger_post))
|
||||
.where(BotTrade.wallet_address == wallet)
|
||||
.where(BotTrade.closed_at.is_not(None))
|
||||
.order_by(BotTrade.opened_at.desc())
|
||||
.order_by(BotTrade.closed_at.desc()) # closed_at = settlement time; opened_at would bury recently-closed old trades
|
||||
.offset(offset)
|
||||
.limit(limit)
|
||||
)
|
||||
|
||||
+19
-11
@@ -16,7 +16,8 @@ from app.schemas import (
|
||||
)
|
||||
from app.services.crypto import encrypt_api_key
|
||||
from app.services.hyperliquid import HyperliquidTrader
|
||||
from app.services.signed_request import verify_signed_request
|
||||
from app.services.signed_request import (
|
||||
SignedReadCreds, signed_read_creds, verify_signed_request)
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -42,17 +43,18 @@ async def verify_hl_api_key_can_trade(api_key: str, account_address: str) -> Non
|
||||
|
||||
|
||||
def _trade_to_schema(trade: BotTrade) -> BotTradeSchema:
|
||||
# Preserve None — do NOT coerce to 0 / "". See trades.py comment.
|
||||
return BotTradeSchema(
|
||||
id=trade.id,
|
||||
asset=trade.asset,
|
||||
side=trade.side,
|
||||
entry_price=trade.entry_price,
|
||||
exit_price=trade.exit_price or 0.0,
|
||||
pnl_usd=trade.pnl_usd or 0.0,
|
||||
hold_seconds=trade.hold_seconds or 0,
|
||||
trigger_post_id=trade.trigger_post_id or 0,
|
||||
exit_price=trade.exit_price,
|
||||
pnl_usd=trade.pnl_usd,
|
||||
hold_seconds=trade.hold_seconds,
|
||||
trigger_post_id=trade.trigger_post_id,
|
||||
opened_at=iso_utc(trade.opened_at) or "",
|
||||
closed_at=iso_utc(trade.closed_at) or "",
|
||||
closed_at=iso_utc(trade.closed_at),
|
||||
)
|
||||
|
||||
|
||||
@@ -107,7 +109,7 @@ async def get_user_public(wallet: str, db: AsyncSession = Depends(get_db)):
|
||||
"wallet_address": wallet, "active": False, "hl_api_key_set": False,
|
||||
"paper_mode": False, "manual_window_until": None,
|
||||
"circuit_breaker_tripped_at": None, "circuit_breaker_reason": None,
|
||||
"auto_trade": False,
|
||||
"auto_trade": False, "trump_enabled": False, "macro_enabled": False,
|
||||
}
|
||||
return {
|
||||
"wallet_address": wallet,
|
||||
@@ -119,14 +121,19 @@ async def get_user_public(wallet: str, db: AsyncSession = Depends(get_db)):
|
||||
"circuit_breaker_tripped_at": iso_utc(sub.circuit_breaker_tripped_at),
|
||||
"circuit_breaker_reason": sub.circuit_breaker_reason,
|
||||
"auto_trade": bool(sub.auto_trade),
|
||||
# Per-system enable flags. bot_engine gates System-1 on trump_enabled
|
||||
# and System-2 on macro_enabled (see _execute_for_subscriber). Without
|
||||
# these, the UI claims "next Trump signal auto-opens" even when
|
||||
# trump_enabled=0, where the backend actually skips the trade.
|
||||
"trump_enabled": bool(sub.trump_enabled),
|
||||
"macro_enabled": bool(sub.macro_enabled),
|
||||
}
|
||||
|
||||
|
||||
@router.get("/user/{wallet}", response_model=UserResponse)
|
||||
async def get_user(
|
||||
wallet: str,
|
||||
ts: int = Query(..., description="Signed timestamp (ms)"),
|
||||
sig: str = Query(..., description="EIP-191 signature"),
|
||||
creds: SignedReadCreds = Depends(signed_read_creds),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
wallet = wallet.lower().strip()
|
||||
@@ -136,8 +143,8 @@ async def get_user(
|
||||
verify_signed_request(
|
||||
action=ACTION_VIEW_USER,
|
||||
wallet=wallet,
|
||||
timestamp_ms=ts,
|
||||
signature=sig,
|
||||
timestamp_ms=creds.ts,
|
||||
signature=creds.sig,
|
||||
body=None,
|
||||
allow_replay=True, # idempotent GET — allow sessionStorage caching for UX
|
||||
)
|
||||
@@ -179,6 +186,7 @@ async def get_user(
|
||||
hl_api_key_set=hl_api_key_set,
|
||||
hl_api_key_masked=masked,
|
||||
paper_mode=bool(sub.paper_mode),
|
||||
auto_trade=bool(sub.auto_trade),
|
||||
trades=[_trade_to_schema(t) for t in trades],
|
||||
settings=UserSettings(
|
||||
leverage=sub.leverage,
|
||||
|
||||
@@ -64,6 +64,41 @@ class Settings(BaseSettings):
|
||||
# (no trading details, no /adopt CTA). Leave empty to disable.
|
||||
telegram_public_channel_id: str = ""
|
||||
|
||||
# ── X (Twitter) auto-poster ──────────────────────────────────────────────
|
||||
# OAuth 1.0a User Context — for posting from a single fixed bot account
|
||||
# (@TrumpAlpha_signals). Get all four from developer.x.com → your App →
|
||||
# "Keys and tokens". App permissions MUST be "Read and Write", and the
|
||||
# Access Token/Secret must be regenerated AFTER setting write permission.
|
||||
# Bearer Token / Client ID / Client Secret are NOT used (those are read /
|
||||
# OAuth2 web-flow credentials). Empty = X posting disabled (no-op).
|
||||
x_api_key: str = "" # = Consumer Key
|
||||
x_api_secret: str = "" # = Consumer Secret
|
||||
x_access_token: str = ""
|
||||
x_access_secret: str = ""
|
||||
# Master switch. Even with keys set, posting stays off unless this is true.
|
||||
# Lets you load creds without going live (dry-run = logs the tweet text).
|
||||
x_enabled: bool = False
|
||||
# Only post when ai_confidence >= this (0–100). Keeps low-quality noise off
|
||||
# the public timeline. Scanner signals carry their own score.
|
||||
x_min_confidence: int = 60
|
||||
# Hard daily cap on tweets (initial + follow-ups counted together) to stay
|
||||
# under Free-tier 1,500/mo and avoid spam flags. In-memory, resets at UTC
|
||||
# midnight. Single-process by design so a plain counter is safe.
|
||||
x_daily_cap: int = 40
|
||||
# Minutes after the initial prediction tweet to post the result follow-up.
|
||||
x_followup_minutes: int = 15
|
||||
# Public site URL appended to the follow-up tweet. Falls back to frontend_url.
|
||||
x_link_url: str = ""
|
||||
|
||||
# ── X (Twitter) READ — KOL tweet ingestion (separate from the poster above) ──
|
||||
# twitterapi.io managed-scraper key, sent as the `x-api-key` header. Powers
|
||||
# kol_x.py: polls each tracked KOL's recent tweets → x_analysis.analyze_x_post
|
||||
# → KolPost(source="twitter") → kol_divergence. This is READ-ONLY third-party
|
||||
# data and is UNRELATED to the OAuth 1.0a poster creds above (those write our
|
||||
# own tweets; this reads other people's). Empty = X ingestion disabled (no-op).
|
||||
# Get a pay-as-you-go key at twitterapi.io (~$0.15 / 1k tweets).
|
||||
twitterapi_io_key: str = ""
|
||||
|
||||
model_config = {"env_file": ".env", "env_file_encoding": "utf-8", "extra": "ignore"}
|
||||
|
||||
|
||||
|
||||
+22
-11
@@ -139,17 +139,12 @@ async def lifespan(app: FastAPI):
|
||||
|
||||
# 3. Start Truth Social poller via APScheduler
|
||||
_scheduler = AsyncIOScheduler()
|
||||
# Signal monitor — polls every 5 minutes
|
||||
from app.services.funding_signal import poll_funding_signal
|
||||
_scheduler.add_job(
|
||||
poll_funding_signal,
|
||||
"interval",
|
||||
minutes=5,
|
||||
id="funding_signal_poll",
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
logger.info("Breakout signal monitor scheduled every 5 minutes.")
|
||||
# Breakout signal monitor (poll_funding_signal, ETH/LINK 5m) UNSCHEDULED
|
||||
# 2026-06-12: the feature has been disabled for months (_enabled=False,
|
||||
# operator-only toggle) and the frontend panel that displayed it was
|
||||
# removed — the 5-min Binance kline poll was pure waste. The /signal/*
|
||||
# API routes still work; to revive, re-add the add_job() here AND remount
|
||||
# SignalMonitor in the frontend's MacroVibesPageClient.
|
||||
|
||||
_scheduler.add_job(
|
||||
poll_truth_social,
|
||||
@@ -226,6 +221,20 @@ async def lifespan(app: FastAPI):
|
||||
)
|
||||
logger.info("KOL Substack poller scheduled daily at 01:15 UTC.")
|
||||
|
||||
# ── KOL X (Twitter) tweet ingestion (daily) ───────────────────────────
|
||||
# Polls X-native KOLs (andrewkang/@Rewkang, murad — no Substack feed) plus
|
||||
# Hayes's real-time position statements. Writes KolPost(source="twitter")
|
||||
# that the divergence scan (02:15) joins against on-chain data — this is the
|
||||
# post-side feed that lights up the andrewkang/murad wallets. No-op if
|
||||
# twitterapi_io_key is unset. Runs 01:30 — between substack (01:15) and
|
||||
# on-chain (02:00) so fresh posts are present for the 02:15 divergence scan.
|
||||
from app.services.kol_x import run_x_poll
|
||||
_scheduler.add_job(
|
||||
run_x_poll, "cron", hour=1, minute=30,
|
||||
id="kol_x_poll", max_instances=1, coalesce=True,
|
||||
)
|
||||
logger.info("KOL X (Twitter) poller scheduled daily at 01:30 UTC.")
|
||||
|
||||
# ── KOL A-tier: on-chain holdings snapshot (daily) ────────────────────
|
||||
# Polls HL public API (free) for perp positions; Arkham (key optional)
|
||||
# for full portfolio. Diffs against yesterday's snapshot → writes
|
||||
@@ -333,6 +342,8 @@ async def lifespan(app: FastAPI):
|
||||
await _telegram_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
from app.services.http_client import aclose as http_client_aclose
|
||||
await http_client_aclose()
|
||||
await engine.dispose()
|
||||
logger.info("Shutdown complete.")
|
||||
|
||||
|
||||
@@ -282,6 +282,12 @@ class KolPost(Base):
|
||||
analysis_model: Mapped[Optional[str]] = mapped_column(String(64), nullable=True)
|
||||
analysis_version: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||
|
||||
# x_analysis extended output (migration 027)
|
||||
tier: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||
post_type: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||
talks_vs_trades_flag: Mapped[Optional[bool]] = mapped_column(Boolean, nullable=True, default=False)
|
||||
sentiment: Mapped[Optional[str]] = mapped_column(String(16), nullable=True)
|
||||
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, nullable=False, default=utcnow)
|
||||
|
||||
|
||||
|
||||
+42
-5
@@ -42,6 +42,36 @@ class TrumpPost(BaseModel):
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class PostFilterCounts(BaseModel):
|
||||
# Count of posts matching the current non-signal filters. When the caller
|
||||
# sets ai_scored_only=true this already excludes off-topic/noise rows.
|
||||
all: int
|
||||
actionable: int
|
||||
buy: int
|
||||
short: int
|
||||
# Off-topic rows hidden by the "Signals only" toggle. Computed from the
|
||||
# same source/sentiment scope but ignoring ai_scored_only so the toggle
|
||||
# can stay visible while active.
|
||||
off_topic: int
|
||||
|
||||
|
||||
class SourceCount(BaseModel):
|
||||
source: str
|
||||
count: int
|
||||
latest: Optional[str] = None
|
||||
|
||||
|
||||
class PostListResponse(BaseModel):
|
||||
items: list[TrumpPost]
|
||||
total: int
|
||||
page: int
|
||||
limit: int
|
||||
counts: PostFilterCounts
|
||||
source_counts: list[SourceCount] = []
|
||||
|
||||
|
||||
|
||||
|
||||
class Candle(BaseModel):
|
||||
time: int
|
||||
open: float
|
||||
@@ -56,12 +86,18 @@ class BotTrade(BaseModel):
|
||||
asset: str
|
||||
side: str
|
||||
entry_price: float
|
||||
exit_price: float
|
||||
pnl_usd: float
|
||||
hold_seconds: int
|
||||
trigger_post_id: int
|
||||
# Nullable fields: None = "not yet known / externally closed / not applicable".
|
||||
# Do NOT coerce to 0 — the frontend uses null to distinguish genuine zeros
|
||||
# (break-even trade) from "we don't have this data yet".
|
||||
exit_price: Optional[float] = None # None while still open / extern-closed
|
||||
pnl_usd: Optional[float] = None # None = unsettled
|
||||
hold_seconds: Optional[int] = None # None = not yet computed
|
||||
trigger_post_id: Optional[int] = None # None = adopted/manual (no trigger post)
|
||||
opened_at: str
|
||||
closed_at: str
|
||||
closed_at: Optional[str] = None # None for still-open positions returned from /user
|
||||
# Optional short trigger snippet for table/list UIs. Comes from the joined
|
||||
# trigger Post row when present; omitted for adopted / deleted-post trades.
|
||||
trigger_post_text: Optional[str] = None
|
||||
# Source tag of the originating signal (e.g. 'truth', 'breakout', 'my_strategy').
|
||||
# Joined from posts.source on read; not stored on BotTrade itself.
|
||||
# 'unknown' when the trigger post has been deleted or trigger_post_id is null.
|
||||
@@ -152,6 +188,7 @@ class UserResponse(BaseModel):
|
||||
hl_api_key_set: bool
|
||||
hl_api_key_masked: Optional[str] = None
|
||||
paper_mode: bool = False
|
||||
auto_trade: bool = False # B50: was silently dropped, causing Settings to show ON as OFF
|
||||
trades: list[BotTrade]
|
||||
settings: UserSettings
|
||||
# Convex-strategy: ISO-UTC timestamp until which the bot is manually armed.
|
||||
|
||||
+41
-29
@@ -24,10 +24,12 @@ from datetime import datetime, timezone
|
||||
from email.utils import parsedate_to_datetime
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
|
||||
from app.scrapers.truth_social import _process_entry, _post_to_ws_payload
|
||||
from app.ws.manager import manager
|
||||
from app.scrapers.truth_social import (
|
||||
NOT_MODIFIED,
|
||||
_known_external_ids,
|
||||
_process_entry,
|
||||
dispatch_post,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -41,16 +43,31 @@ NS = {
|
||||
last_successful_poll_at: Optional[datetime] = None
|
||||
last_poll_error: Optional[str] = None
|
||||
|
||||
# Conditional-GET validators (same scheme as truth_social.py — most polls
|
||||
# come back 304 and skip the download + XML parse entirely).
|
||||
_etag: Optional[str] = None
|
||||
_last_modified: Optional[str] = None
|
||||
|
||||
async def _fetch_feed() -> Optional[str]:
|
||||
|
||||
async def _fetch_feed():
|
||||
"""Fetch the RSS body. Returns str, NOT_MODIFIED (304), or None on error."""
|
||||
global _etag, _last_modified
|
||||
headers = {
|
||||
"User-Agent": "Mozilla/5.0 (compatible; TrumpSignal/1.0)",
|
||||
"Accept": "application/rss+xml, application/xml",
|
||||
}
|
||||
if _etag:
|
||||
headers["If-None-Match"] = _etag
|
||||
if _last_modified:
|
||||
headers["If-Modified-Since"] = _last_modified
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=20, follow_redirects=True) as client:
|
||||
resp = await client.get(FEED_URL, headers=headers)
|
||||
from app.services.http_client import get_client
|
||||
resp = await get_client().get(FEED_URL, headers=headers, timeout=20)
|
||||
if resp.status_code == 304:
|
||||
return NOT_MODIFIED
|
||||
resp.raise_for_status()
|
||||
_etag = resp.headers.get("etag")
|
||||
_last_modified = resp.headers.get("last-modified")
|
||||
return resp.text
|
||||
except Exception as exc:
|
||||
# Include type name — httpx often raises bare ConnectError/RemoteProtocolError
|
||||
@@ -105,6 +122,11 @@ async def poll_trumpstruth(db_session_factory) -> None:
|
||||
global last_successful_poll_at, last_poll_error
|
||||
|
||||
raw = await _fetch_feed()
|
||||
if raw is NOT_MODIFIED:
|
||||
# Feed unchanged — successful cycle, nothing to do.
|
||||
last_successful_poll_at = datetime.now(timezone.utc)
|
||||
last_poll_error = None
|
||||
return
|
||||
if raw is None:
|
||||
last_poll_error = "fetch_feed returned None"
|
||||
return
|
||||
@@ -128,36 +150,26 @@ async def poll_trumpstruth(db_session_factory) -> None:
|
||||
|
||||
async with db_session_factory() as db:
|
||||
try:
|
||||
new_posts = []
|
||||
known_ids = await _known_external_ids(entries, db)
|
||||
# Newest-first: commit + dispatch each new post immediately so an
|
||||
# actionable post never queues behind older entries' AI analysis.
|
||||
# dispatch_post is the shared fan-out (WS + Telegram + X + trade)
|
||||
# from truth_social.py — delivery must not depend on which poller
|
||||
# wins the race.
|
||||
for entry in entries:
|
||||
try:
|
||||
post = await _process_entry(entry, db)
|
||||
post = await _process_entry(entry, db, known_ids)
|
||||
if post:
|
||||
new_posts.append(post)
|
||||
await db.commit()
|
||||
logger.info("[trumpstruth] beat CNN — new post id=%d",
|
||||
post.id)
|
||||
await dispatch_post(post, db)
|
||||
except Exception as exc:
|
||||
logger.error("trumpstruth: error on entry %s: %s",
|
||||
entry.get("id"), exc)
|
||||
|
||||
if new_posts:
|
||||
# Capture any remaining writes (entry-filter stub rows).
|
||||
await db.commit()
|
||||
for post in new_posts:
|
||||
await manager.broadcast(_post_to_ws_payload(post))
|
||||
logger.info("[trumpstruth] beat CNN — new post id=%d: %s",
|
||||
post.id, post.text[:60])
|
||||
# Telegram fan-out — matches truth_social.py. Without this,
|
||||
# whichever poller wins the race determines whether users
|
||||
# get pushed — flaky 50% delivery.
|
||||
try:
|
||||
from app.services.telegram import notify_signal
|
||||
notify_signal(post)
|
||||
except Exception as exc:
|
||||
logger.warning("Telegram notify failed for post %d: %s", post.id, exc)
|
||||
try:
|
||||
from app.services.bot_engine import process_post
|
||||
await process_post(post, db)
|
||||
except Exception as exc:
|
||||
logger.error("process_post failed for post %d: %s",
|
||||
post.id, exc)
|
||||
last_successful_poll_at = datetime.now(timezone.utc)
|
||||
last_poll_error = None
|
||||
except Exception as exc:
|
||||
|
||||
@@ -11,7 +11,6 @@ import re
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
@@ -29,6 +28,16 @@ ARCHIVE_URL = "https://ix.cnn.io/data/truth-social/truth_archive.json"
|
||||
last_successful_poll_at: Optional[datetime] = None
|
||||
last_poll_error: Optional[str] = None
|
||||
|
||||
# Conditional-GET validators from the last 200 response. The archive is 30k+
|
||||
# posts (~MBs of JSON); CNN only regenerates it every ~5 min, so most polls
|
||||
# can be answered with a 304 and skip download + parse entirely.
|
||||
_etag: Optional[str] = None
|
||||
_last_modified: Optional[str] = None
|
||||
|
||||
# Sentinel returned by _fetch_archive on HTTP 304 — distinct from None
|
||||
# (fetch failure) so the poller can count it as a successful, no-work cycle.
|
||||
NOT_MODIFIED = object()
|
||||
|
||||
|
||||
def _strip_html(text: str) -> str:
|
||||
text = re.sub(r"<[^>]+>", " ", text)
|
||||
@@ -52,15 +61,28 @@ def _parse_dt(iso: str) -> datetime:
|
||||
return datetime.now(timezone.utc).replace(tzinfo=None)
|
||||
|
||||
|
||||
async def _fetch_archive() -> Optional[list]:
|
||||
async def _fetch_archive(conditional: bool = True):
|
||||
"""Fetch the archive JSON. Returns a list, NOT_MODIFIED (304), or None
|
||||
on error. `conditional=False` (backfill) always downloads the full body
|
||||
so a poller-set ETag can't starve the startup backfill."""
|
||||
global _etag, _last_modified
|
||||
headers = {
|
||||
"User-Agent": "Mozilla/5.0 (compatible; TrumpSignal/1.0)",
|
||||
"Accept": "application/json",
|
||||
}
|
||||
if conditional:
|
||||
if _etag:
|
||||
headers["If-None-Match"] = _etag
|
||||
if _last_modified:
|
||||
headers["If-Modified-Since"] = _last_modified
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
|
||||
resp = await client.get(ARCHIVE_URL, headers=headers)
|
||||
from app.services.http_client import get_client
|
||||
resp = await get_client().get(ARCHIVE_URL, headers=headers, timeout=30)
|
||||
if conditional and resp.status_code == 304:
|
||||
return NOT_MODIFIED
|
||||
resp.raise_for_status()
|
||||
_etag = resp.headers.get("etag")
|
||||
_last_modified = resp.headers.get("last-modified")
|
||||
return resp.json()
|
||||
except Exception as exc:
|
||||
# Include type name — httpx often raises bare ConnectError/TimeoutException
|
||||
@@ -71,9 +93,27 @@ async def _fetch_archive() -> Optional[list]:
|
||||
return None
|
||||
|
||||
|
||||
async def _process_entry(entry: dict, db: AsyncSession) -> Optional[Post]:
|
||||
async def _known_external_ids(entries: list, db: AsyncSession) -> set:
|
||||
"""One batch query for the dedup pass. On a typical poll all ~50 entries
|
||||
already exist, so this replaces 50 per-entry SELECTs with 1."""
|
||||
ids = [hashlib.md5(str(e["id"]).encode()).hexdigest() for e in entries]
|
||||
if not ids:
|
||||
return set()
|
||||
rows = await db.execute(
|
||||
select(Post.external_id).where(Post.external_id.in_(ids)))
|
||||
return {r[0] for r in rows}
|
||||
|
||||
|
||||
async def _process_entry(entry: dict, db: AsyncSession,
|
||||
known_ids: Optional[set] = None) -> Optional[Post]:
|
||||
external_id = hashlib.md5(str(entry["id"]).encode()).hexdigest()
|
||||
|
||||
# Fast path: pre-fetched batch dedup set. New (unseen) ids still get the
|
||||
# confirming SELECT below — protects against a concurrent insert by the
|
||||
# other poller between the batch query and this entry.
|
||||
if known_ids is not None and external_id in known_ids:
|
||||
return None
|
||||
|
||||
result = await db.execute(select(Post).where(Post.external_id == external_id))
|
||||
if result.scalar_one_or_none():
|
||||
return None
|
||||
@@ -198,10 +238,40 @@ def _post_to_ws_payload(post: Post) -> dict:
|
||||
}
|
||||
|
||||
|
||||
async def dispatch_post(post: Post, db: AsyncSession) -> None:
|
||||
"""Broadcast + fan-out + trade for one freshly committed post. Shared by
|
||||
both pollers so delivery doesn't depend on which source wins the race."""
|
||||
await manager.broadcast(_post_to_ws_payload(post))
|
||||
logger.info("Saved new post id=%d: %s", post.id, post.text[:60])
|
||||
# Telegram fan-out (fire-and-forget). _dispatch filters internally:
|
||||
# buy/short → per-subscriber + public channel; relevant-but-hold →
|
||||
# public channel only; noise → dropped.
|
||||
try:
|
||||
from app.services.telegram import notify_signal
|
||||
notify_signal(post)
|
||||
except Exception as exc:
|
||||
logger.warning("Telegram notify failed for post %d: %s", post.id, exc)
|
||||
try:
|
||||
from app.services.x_poster import notify_x_signal
|
||||
notify_x_signal(post)
|
||||
except Exception as exc:
|
||||
logger.warning("X notify failed for post %d: %s", post.id, exc)
|
||||
try:
|
||||
from app.services.bot_engine import process_post
|
||||
await process_post(post, db)
|
||||
except Exception as exc:
|
||||
logger.error("process_post failed for post %d: %s", post.id, exc)
|
||||
|
||||
|
||||
async def poll_truth_social(db_session_factory) -> None:
|
||||
global last_successful_poll_at, last_poll_error
|
||||
logger.info("Polling CNN Truth Social archive...")
|
||||
entries = await _fetch_archive()
|
||||
if entries is NOT_MODIFIED:
|
||||
# Archive unchanged since last poll — successful cycle, nothing to do.
|
||||
last_successful_poll_at = datetime.now(timezone.utc)
|
||||
last_poll_error = None
|
||||
return
|
||||
if not entries:
|
||||
last_poll_error = "fetch_archive returned empty"
|
||||
return
|
||||
@@ -212,34 +282,24 @@ async def poll_truth_social(db_session_factory) -> None:
|
||||
|
||||
async with db_session_factory() as db:
|
||||
try:
|
||||
new_posts = []
|
||||
known_ids = await _known_external_ids(recent, db)
|
||||
found_new = False
|
||||
# Entries are newest-first. Commit + dispatch each new post
|
||||
# IMMEDIATELY rather than after the whole batch — an actionable
|
||||
# post must not wait behind the AI analysis of older entries.
|
||||
for entry in recent:
|
||||
try:
|
||||
post = await _process_entry(entry, db)
|
||||
post = await _process_entry(entry, db, known_ids)
|
||||
if post:
|
||||
new_posts.append(post)
|
||||
found_new = True
|
||||
await db.commit()
|
||||
await dispatch_post(post, db)
|
||||
except Exception as exc:
|
||||
logger.error("Error processing entry %s: %s", entry.get("id"), exc)
|
||||
|
||||
if new_posts:
|
||||
# Capture any remaining writes (entry-filter stub rows).
|
||||
await db.commit()
|
||||
for post in new_posts:
|
||||
await manager.broadcast(_post_to_ws_payload(post))
|
||||
logger.info("Saved new post id=%d: %s", post.id, post.text[:60])
|
||||
# Telegram fan-out (fire-and-forget). _dispatch filters
|
||||
# internally: buy/short → per-subscriber + public channel;
|
||||
# relevant-but-hold → public channel only; noise → dropped.
|
||||
try:
|
||||
from app.services.telegram import notify_signal
|
||||
notify_signal(post)
|
||||
except Exception as exc:
|
||||
logger.warning("Telegram notify failed for post %d: %s", post.id, exc)
|
||||
try:
|
||||
from app.services.bot_engine import process_post
|
||||
await process_post(post, db)
|
||||
except Exception as exc:
|
||||
logger.error("process_post failed for post %d: %s", post.id, exc)
|
||||
else:
|
||||
if not found_new:
|
||||
logger.info("No new posts found.")
|
||||
# Mark a successful poll cycle (separate from "found new posts").
|
||||
last_successful_poll_at = datetime.now(timezone.utc)
|
||||
@@ -253,7 +313,7 @@ async def poll_truth_social(db_session_factory) -> None:
|
||||
async def backfill_history(db_session_factory, limit: int = 500) -> None:
|
||||
"""One-time backfill of historical posts (no Claude analysis, no price impact)."""
|
||||
logger.info("Starting historical backfill (limit=%d)...", limit)
|
||||
entries = await _fetch_archive()
|
||||
entries = await _fetch_archive(conditional=False)
|
||||
if not entries:
|
||||
logger.error("Backfill failed: could not fetch archive")
|
||||
return
|
||||
@@ -263,10 +323,10 @@ async def backfill_history(db_session_factory, limit: int = 500) -> None:
|
||||
|
||||
async with db_session_factory() as db:
|
||||
try:
|
||||
known_ids = await _known_external_ids(to_process, db)
|
||||
for entry in to_process:
|
||||
external_id = hashlib.md5(str(entry["id"]).encode()).hexdigest()
|
||||
result = await db.execute(select(Post).where(Post.external_id == external_id))
|
||||
if result.scalar_one_or_none():
|
||||
if external_id in known_ids:
|
||||
continue
|
||||
|
||||
text = _strip_html(entry.get("content") or "").strip()
|
||||
|
||||
@@ -48,7 +48,7 @@ from app.services.signal_categories import (
|
||||
get_exit_profile, get_stop_ladder,
|
||||
sys2_normalize_mode, sys2_protective_stop_pct,
|
||||
sys2_derisk_ladder, sys2_addon_ladder, sys2_peak_trail,
|
||||
SYS2_MAX_CONCURRENT, SYS2_MODES,
|
||||
SYS2_MAX_CONCURRENT, SYS2_MODES, SYS2_MAX_LEVERAGE,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -244,6 +244,16 @@ async def _adopt_locked(wallet_l: str, asset_u: str, mode_n: str) -> AdoptionRes
|
||||
"no Hyperliquid position to manage. Turn off paper mode in "
|
||||
"Settings to use /adopt.")
|
||||
|
||||
# Macro Vibes (System-2) must be ENABLED to hand a position to the bot.
|
||||
# /adopt is the entry point to sys2 management, so honouring the
|
||||
# macro_enabled switch here is what makes that toggle real — otherwise
|
||||
# a user who turned Macro Vibes OFF could still /adopt and the bot would
|
||||
# manage it, contradicting their setting.
|
||||
if not getattr(sub, "macro_enabled", False):
|
||||
raise AdoptionError("macro_disabled",
|
||||
"Macro Vibes is turned off for this wallet. Enable Macro Vibes "
|
||||
"in Settings before using /adopt.")
|
||||
|
||||
# System-2 circuit breaker. Same gate the auto-open path used to run:
|
||||
# if recent losses tripped the sys2 breaker, block new adoptions for
|
||||
# the lockout window. Otherwise the breaker would be useless under
|
||||
|
||||
+28
-3
@@ -4,7 +4,6 @@ import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
import websockets
|
||||
|
||||
from app.config import settings
|
||||
@@ -31,6 +30,7 @@ last_tick_at: Optional[datetime] = None
|
||||
# on Binance; those require a separate HL price feed (see BUG-08 note in CLAUDE.md).
|
||||
# Until that feed is added, HYPE trades fall back to max-hold only.
|
||||
ASSET_MAP: dict[str, str] = {
|
||||
# Original 8
|
||||
"btcusdt": "BTC",
|
||||
"ethusdt": "ETH",
|
||||
"solusdt": "SOL",
|
||||
@@ -39,6 +39,31 @@ ASSET_MAP: dict[str, str] = {
|
||||
"dogeusdt": "DOGE",
|
||||
"linkusdt": "LINK",
|
||||
"aaveusdt": "AAVE",
|
||||
# Extended: all mainstream HL_PERPS that have Binance spot/perp pairs.
|
||||
# Without these, any bot trade on these assets loses TP/SL/trailing
|
||||
# protection — only max_hold remains as an exit.
|
||||
"avaxusdt": "AVAX",
|
||||
"arbusdt": "ARB",
|
||||
"opusdt": "OP",
|
||||
"suiusdt": "SUI",
|
||||
"aptusdt": "APT",
|
||||
"injusdt": "INJ",
|
||||
"atomusdt": "ATOM",
|
||||
"xrpusdt": "XRP",
|
||||
"ltcusdt": "LTC",
|
||||
"adausdt": "ADA",
|
||||
"maticusdt": "MATIC",
|
||||
"shibusdt": "SHIB",
|
||||
"pepeusdt": "PEPE",
|
||||
"wifusdt": "WIF",
|
||||
"bonkusdt": "BONK",
|
||||
"taousdt": "TAO",
|
||||
"jupusdt": "JUP",
|
||||
"renderusdt": "RENDER",
|
||||
"fetusdt": "FET",
|
||||
"tiausdt": "TIA",
|
||||
"seiusdt": "SEI",
|
||||
"pendleusdt": "PENDLE",
|
||||
}
|
||||
|
||||
# Build the combined-stream WS URL from ASSET_MAP so the two are always in sync.
|
||||
@@ -101,8 +126,8 @@ async def fetch_historical(asset: str, symbol: str, interval: str = "1m", limit:
|
||||
"""Fetch historical klines from Binance REST API to pre-fill price_store."""
|
||||
url = f"{settings.binance_rest_url}/api/v3/klines?symbol={symbol.upper()}&interval={interval}&limit={limit}"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=15) as client:
|
||||
resp = await client.get(url)
|
||||
from app.services.http_client import get_client
|
||||
resp = await get_client().get(url, timeout=15)
|
||||
resp.raise_for_status()
|
||||
for row in resp.json():
|
||||
candle = {
|
||||
|
||||
+154
-5
@@ -21,6 +21,16 @@ from app.services.price_store import price_store # noqa: F401 (used elsewhere)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def _broadcast_trade_alert(wallet: str, event: str, **kwargs) -> None:
|
||||
"""Broadcast a trade lifecycle event over WebSocket. Fire-and-forget — never raises."""
|
||||
try:
|
||||
from app.ws.manager import manager
|
||||
await manager.broadcast({"type": "trade_alert", "wallet": wallet, "event": event, **kwargs})
|
||||
except Exception as exc:
|
||||
logger.warning("trade_alert broadcast failed: %s", exc)
|
||||
|
||||
|
||||
# Platform-wide thresholds (per-user values in Subscription override where applicable)
|
||||
# No global confidence floor — users pick their own 0–100 threshold via settings.
|
||||
# Per-user max hold lives on Subscription.max_hold_hours (default 168h = 7 days
|
||||
@@ -230,6 +240,13 @@ async def process_post(post: Post, db: AsyncSession) -> None:
|
||||
sys2_leverage=s.sys2_leverage,
|
||||
sys2_mode=s.sys2_mode,
|
||||
auto_trade=bool(s.auto_trade),
|
||||
# B38: module toggles — must be in snapshot so the execution gate
|
||||
# can read them. trump_enabled gates System-1 (Trump); macro_enabled
|
||||
# gates System-2 (Macro Vibes / bottom reversal). Both default to
|
||||
# False (new subscribers start with bot idle) so a NULL in old rows
|
||||
# continues the old conservative behaviour.
|
||||
trump_enabled=bool(s.trump_enabled),
|
||||
macro_enabled=bool(s.macro_enabled),
|
||||
)
|
||||
for s in subscribers
|
||||
]
|
||||
@@ -298,6 +315,63 @@ async def process_post(post: Post, db: AsyncSession) -> None:
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
|
||||
def _is_in_active_window(sub: dict) -> bool:
|
||||
"""Return True if the current UTC time falls inside the subscriber's
|
||||
active window (schedule or manual override).
|
||||
|
||||
Priority (highest wins):
|
||||
1. manual_window_until — operator-armed override; if set and in the
|
||||
future, the bot is ALWAYS armed regardless of the schedule.
|
||||
2. active_from / active_until — user-configured recurring schedule.
|
||||
Both must be set; if only one is set we treat the schedule as
|
||||
unconfigured and return True (don't gate).
|
||||
3. If neither is set, always active.
|
||||
|
||||
B30: was defined but never called — entire feature was dead.
|
||||
"""
|
||||
now = datetime.now(timezone.utc).replace(tzinfo=None)
|
||||
|
||||
# 1. Manual override window (e.g. "arm for the next 4 hours")
|
||||
mwu = sub.get("manual_window_until")
|
||||
if mwu is not None:
|
||||
mwu_dt = mwu if isinstance(mwu, datetime) else None
|
||||
if mwu_dt is None:
|
||||
try:
|
||||
from datetime import datetime as _dt
|
||||
mwu_dt = _dt.fromisoformat(str(mwu).replace("Z", ""))
|
||||
except Exception:
|
||||
mwu_dt = None
|
||||
if mwu_dt is not None and mwu_dt > now:
|
||||
return True # manual override wins
|
||||
|
||||
# 2. Recurring schedule
|
||||
af = sub.get("active_from")
|
||||
au = sub.get("active_until")
|
||||
if af is None or au is None:
|
||||
return True # no schedule configured → always active
|
||||
|
||||
# Convert to time-of-day for comparison (schedule repeats daily)
|
||||
def _to_time(val):
|
||||
if isinstance(val, datetime):
|
||||
return val.time()
|
||||
try:
|
||||
from datetime import datetime as _dt
|
||||
return _dt.fromisoformat(str(val).replace("Z", "")).time()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
af_t = _to_time(af)
|
||||
au_t = _to_time(au)
|
||||
if af_t is None or au_t is None:
|
||||
return True # can't parse → don't block
|
||||
|
||||
now_t = now.time()
|
||||
if af_t <= au_t:
|
||||
return af_t <= now_t <= au_t # same-day window
|
||||
else:
|
||||
return now_t >= af_t or now_t <= au_t # crosses midnight
|
||||
|
||||
|
||||
async def _execute_for_subscriber(
|
||||
sub: dict,
|
||||
post_id: int,
|
||||
@@ -306,9 +380,27 @@ async def _execute_for_subscriber(
|
||||
side: str,
|
||||
) -> None:
|
||||
wallet = sub["wallet"]
|
||||
if not sub["hl_api_key"]:
|
||||
|
||||
# B38: module on/off switches gate BEFORE any expensive work.
|
||||
# trump_enabled → System-1 (source="truth"); macro_enabled → System-2.
|
||||
# Defaults: False. A NULL column (pre-migration row) is treated as False,
|
||||
# preserving the existing conservative behaviour.
|
||||
is_sys2 = bool(sub.get("_is_system_2"))
|
||||
if is_sys2:
|
||||
if not sub.get("macro_enabled"):
|
||||
logger.info("Sub %s: macro_enabled OFF — post %d not traded", wallet, post_id)
|
||||
return
|
||||
else:
|
||||
if not sub.get("trump_enabled"):
|
||||
logger.info("Sub %s: trump_enabled OFF — post %d not traded", wallet, post_id)
|
||||
return
|
||||
|
||||
# B29: paper users have no HL API key by design — skip the key check for
|
||||
# them. The paper-mode branch later uses price_store instead of HL.
|
||||
if not sub["hl_api_key"] and not sub.get("paper_mode"):
|
||||
logger.warning("Subscriber %s has no HL API key, skipping", wallet)
|
||||
return
|
||||
|
||||
# Required-setup guard. System 1 (Trump) needs take-profit and stop-loss.
|
||||
# System 2 (reversal) supplies its own stop + trailing from the category
|
||||
# profile, so only the two Trump exit fields are user-required.
|
||||
@@ -339,6 +431,15 @@ async def _execute_for_subscriber(
|
||||
wallet, post_id)
|
||||
return
|
||||
|
||||
# ── Schedule / manual-window gate (B30) ───────────────────────────────
|
||||
# active_from/active_until define a recurring daily window (e.g. 09:00-17:00 UTC).
|
||||
# manual_window_until is an operator override that arms the bot for a
|
||||
# fixed duration regardless of the schedule (e.g. "trade for the next 4h").
|
||||
# System-2 (manage-only / adopt model) is exempt — it never auto-opens.
|
||||
if _should_apply_schedule(sub) and not _is_in_active_window(sub):
|
||||
logger.info("Sub %s: outside active window — post %d not traded", wallet, post_id)
|
||||
return
|
||||
|
||||
# ── Circuit breaker gate (P1.1) ────────────────────────────────────────
|
||||
# Checked BEFORE key decryption (cheap fast-fail). If tripped, the wallet
|
||||
# has lost too much today or hit a losing streak — block until manual reset.
|
||||
@@ -410,8 +511,18 @@ async def _execute_for_subscriber(
|
||||
)
|
||||
)
|
||||
# Only count spend from THIS system against THIS system's slice.
|
||||
# M1 fix: adopted trades have trigger_post_id=NULL so the
|
||||
# outerjoin yields src=NULL → (src or "").lower()="" → not in
|
||||
# SYSTEM_2_SOURCES → t_is_s2=False. That miscounts every
|
||||
# adopted sys2 position against the sys1 budget, inflating
|
||||
# "spent" and prematurely blocking new Trump scalp opens.
|
||||
# Adopted trades are always sys2 — their hl_order_id starts
|
||||
# with "adopted:" by construction (see adoption.py).
|
||||
spent = 0.0
|
||||
for t, src in spent_result.all():
|
||||
if t.hl_order_id and str(t.hl_order_id).startswith("adopted:"):
|
||||
t_is_s2 = True
|
||||
else:
|
||||
t_is_s2 = (src or "").lower() in SYSTEM_2_SOURCES
|
||||
if t_is_s2 != is_s2:
|
||||
continue
|
||||
@@ -420,6 +531,11 @@ async def _execute_for_subscriber(
|
||||
logger.info("Sub %s [%s] daily budget reached: spent=%.2f + new=%.2f > cap=%.2f (%.0f%% of %.2f)",
|
||||
wallet, _system, spent, sized_position_usd, daily_cap,
|
||||
(sys2_pct if is_s2 else 100.0), total_cap)
|
||||
asyncio.create_task(_broadcast_trade_alert(
|
||||
wallet, "budget_reached",
|
||||
asset=asset, size_usd=sized_position_usd,
|
||||
spent_usd=round(spent, 2), cap_usd=round(daily_cap, 2),
|
||||
))
|
||||
return
|
||||
|
||||
# ── System-2 correlation / concentration cap ───────────────────────
|
||||
@@ -505,6 +621,24 @@ async def _execute_for_subscriber(
|
||||
logger.info("Subscriber %s already has open %s position, skipping", wallet, asset)
|
||||
return
|
||||
|
||||
balance = await trader.get_balance()
|
||||
# HL isolated-margin requires notional / leverage as collateral,
|
||||
# not the full notional value. Add a 10% buffer for fees + slippage.
|
||||
leverage = sub.get("leverage") or 1
|
||||
required_margin = round((sized_position_usd / max(leverage, 1)) * 1.1, 2)
|
||||
if balance < required_margin:
|
||||
logger.warning(
|
||||
"Sub %s: insufficient balance %.2f < required margin %.2f "
|
||||
"(notional=%.2f lev=%dx) for %s — skipping",
|
||||
wallet, balance, required_margin, sized_position_usd, leverage, asset,
|
||||
)
|
||||
asyncio.create_task(_broadcast_trade_alert(
|
||||
wallet, "insufficient_balance",
|
||||
asset=asset, balance_usd=round(balance, 2),
|
||||
required_usd=required_margin,
|
||||
))
|
||||
return
|
||||
|
||||
result = await trader.open_position(asset, side, sized_position_usd)
|
||||
entry_price = result.get('fill_price', 0.0)
|
||||
|
||||
@@ -529,7 +663,7 @@ async def _execute_for_subscriber(
|
||||
# for its stop math, but we still record the true value so
|
||||
# BotTrade.leverage reflects what HL actually applied.
|
||||
sub["leverage"] = effective_leverage
|
||||
if sys2:
|
||||
if sub.get("_is_system_2"):
|
||||
from app.services.signal_categories import (
|
||||
sys2_protective_stop_pct as _sps,
|
||||
)
|
||||
@@ -562,7 +696,7 @@ async def _execute_for_subscriber(
|
||||
# constructor before the later assignment used to throw
|
||||
# UnboundLocalError on every sys2 fire, leaving the HL
|
||||
# position open with NO DB record and NO watchdog. Critical.
|
||||
_sys2_mode = sub.get("_sys2_mode", "standard") if sys2 else None
|
||||
_sys2_mode = sub.get("_sys2_mode", "standard") if sub.get("_is_system_2") else None
|
||||
|
||||
trade = BotTrade(
|
||||
asset=asset,
|
||||
@@ -601,7 +735,8 @@ async def _execute_for_subscriber(
|
||||
_derisk = None
|
||||
_addon = None
|
||||
_peak_trail = None
|
||||
if sys2:
|
||||
_is_sys2 = bool(sub.get("_is_system_2"))
|
||||
if _is_sys2:
|
||||
_mode_for_ladders = _sys2_mode or "standard"
|
||||
from app.services.signal_categories import (
|
||||
sys2_derisk_ladder, sys2_addon_ladder, sys2_peak_trail,
|
||||
@@ -624,7 +759,7 @@ async def _execute_for_subscriber(
|
||||
invalidation=eff["invalidation"],
|
||||
invalidation_price=eff.get("invalidation_price"),
|
||||
min_hold_until_ts=eff["min_hold_until_ts"],
|
||||
stop_ladder=get_stop_ladder(post.category) if sys2 else None,
|
||||
stop_ladder=get_stop_ladder(sub.get("_category", "")) if _is_sys2 else None,
|
||||
derisk_ladder=_derisk,
|
||||
derisk_done=0,
|
||||
addon_ladder=_addon,
|
||||
@@ -656,6 +791,10 @@ async def _execute_for_subscriber(
|
||||
|
||||
except Exception as e:
|
||||
logger.error("Trade execution failed for %s: %s", wallet, e)
|
||||
asyncio.create_task(_broadcast_trade_alert(
|
||||
wallet, "execution_failed",
|
||||
asset=asset, reason=str(e)[:200],
|
||||
))
|
||||
|
||||
|
||||
async def partial_derisk(
|
||||
@@ -885,6 +1024,16 @@ async def pyramid_add(
|
||||
fill = r.get("fill_price")
|
||||
filled_coins = float(r.get("size_coins") or 0.0)
|
||||
if not fill or filled_coins <= 0:
|
||||
# Revert the pre-claim — HL didn't fill so the rung must
|
||||
# remain retryable. Without this, addon_steps_done stays
|
||||
# incremented and the rung is permanently burned (B48).
|
||||
await db.execute(
|
||||
update(BotTrade)
|
||||
.where(BotTrade.id == trade_id)
|
||||
.where(BotTrade.addon_steps_done == step_idx + 1)
|
||||
.values(addon_steps_done=step_idx)
|
||||
)
|
||||
await db.commit()
|
||||
return (False, None, None)
|
||||
# Use the ACTUAL filled notional, not the intended amount —
|
||||
# an IOC add can under-fill on thin/volatile books; assuming
|
||||
|
||||
@@ -129,7 +129,7 @@ async def check_and_trip(
|
||||
|
||||
today_trades = await _trades_for_system(db, wallet, start_of_day, system)
|
||||
today_pnl = sum(t.pnl_usd or 0 for t in today_trades)
|
||||
dd_limit = -CB_DAILY_DD_USD_PER_BASE * sub.position_size_usd * 10
|
||||
dd_limit = -CB_DAILY_DD_USD_PER_BASE * sub.position_size_usd
|
||||
if today_pnl < dd_limit:
|
||||
reason = "daily_dd"
|
||||
logger.warning("CB[%s] trip [daily_dd] %s: pnl=%.2f < %.2f",
|
||||
|
||||
+100
-21
@@ -1,54 +1,132 @@
|
||||
"""
|
||||
Envelope encryption for HL API private keys.
|
||||
|
||||
Plaintext keys never touch disk: stored values are Fernet-encrypted with a KEK
|
||||
loaded from env (ENCRYPTION_KEY). Rotate KEK → re-encrypt all keys offline.
|
||||
Plaintext keys never touch disk: stored values are Fernet-encrypted with a key
|
||||
derived from the env KEK (ENCRYPTION_KEY).
|
||||
|
||||
Blob formats:
|
||||
enc:v2:<salt_b64url>:<fernet_token> — current. Per-blob random 16-byte
|
||||
salt; Fernet key = PBKDF2-HMAC-SHA256(KEK, salt, 600k iters). Derived
|
||||
keys are cached per salt so steady-state decryption pays the KDF once.
|
||||
enc:v1:<fernet_token> — legacy. Fernet key = single unsalted
|
||||
SHA-256(KEK). Read-only: decrypt still works, encrypt always writes v2.
|
||||
Upgrade rows with scripts/reencrypt_keys.py (H4 fix).
|
||||
<plaintext> — pre-encryption rows. Refused in prod.
|
||||
|
||||
Rotate KEK → re-encrypt all keys offline (scripts/reencrypt_keys.py).
|
||||
"""
|
||||
import base64
|
||||
import hashlib
|
||||
import logging
|
||||
from typing import Optional
|
||||
import os
|
||||
from typing import Dict, Optional
|
||||
|
||||
from cryptography.fernet import Fernet, InvalidToken
|
||||
from cryptography.hazmat.primitives import hashes
|
||||
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
|
||||
|
||||
from app.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# OWASP-recommended floor for PBKDF2-HMAC-SHA256. The KEK is already a
|
||||
# high-entropy secret, so the KDF mainly buys defence-in-depth against a
|
||||
# weak / partially-leaked / reused KEK. ~0.2-0.4s, paid once per unique salt.
|
||||
_PBKDF2_ITERATIONS = 600_000
|
||||
_SALT_BYTES = 16
|
||||
|
||||
def _derive_fernet_key(raw: str) -> bytes:
|
||||
"""Accept any reasonably-long secret and derive a valid 32-byte Fernet key."""
|
||||
ENC_PREFIX_V1 = "enc:v1:"
|
||||
ENC_PREFIX_V2 = "enc:v2:"
|
||||
# Kept for older imports / scripts that reference the original name.
|
||||
ENC_PREFIX = ENC_PREFIX_V1
|
||||
|
||||
|
||||
def _check_kek(raw: str) -> str:
|
||||
if not raw or len(raw) < 32:
|
||||
raise RuntimeError(
|
||||
"ENCRYPTION_KEY must be set to at least 32 random chars (e.g. `openssl rand -hex 32`)"
|
||||
)
|
||||
digest = hashlib.sha256(raw.encode("utf-8")).digest()
|
||||
return raw
|
||||
|
||||
|
||||
def _derive_v1_key(raw: str) -> bytes:
|
||||
"""Legacy: single unsalted SHA-256 of the KEK."""
|
||||
digest = hashlib.sha256(_check_kek(raw).encode("utf-8")).digest()
|
||||
return base64.urlsafe_b64encode(digest)
|
||||
|
||||
|
||||
_fernet: Optional[Fernet] = None
|
||||
def _derive_v2_key(raw: str, salt: bytes) -> bytes:
|
||||
kdf = PBKDF2HMAC(
|
||||
algorithm=hashes.SHA256(),
|
||||
length=32,
|
||||
salt=salt,
|
||||
iterations=_PBKDF2_ITERATIONS,
|
||||
)
|
||||
return base64.urlsafe_b64encode(kdf.derive(_check_kek(raw).encode("utf-8")))
|
||||
|
||||
|
||||
def _cipher() -> Fernet:
|
||||
global _fernet
|
||||
if _fernet is None:
|
||||
_fernet = Fernet(_derive_fernet_key(settings.encryption_key))
|
||||
return _fernet
|
||||
_fernet_v1: Optional[Fernet] = None
|
||||
# salt → Fernet. One entry per unique salt actually seen: encryption reuses a
|
||||
# single process-lifetime salt, decryption adds one per distinct stored blob.
|
||||
_fernet_v2_cache: Dict[bytes, Fernet] = {}
|
||||
_V2_CACHE_MAX = 4096
|
||||
|
||||
# Salt for NEW encryptions in this process — generated once so the encrypt
|
||||
# path pays the 600k-iteration KDF a single time per process, while blobs
|
||||
# written by other processes/runs still carry their own salt.
|
||||
_encrypt_salt: Optional[bytes] = None
|
||||
|
||||
|
||||
# Prefix lets us distinguish encrypted blobs from any legacy plaintext rows during migration
|
||||
ENC_PREFIX = "enc:v1:"
|
||||
def _cipher_v1() -> Fernet:
|
||||
global _fernet_v1
|
||||
if _fernet_v1 is None:
|
||||
_fernet_v1 = Fernet(_derive_v1_key(settings.encryption_key))
|
||||
return _fernet_v1
|
||||
|
||||
|
||||
def _cipher_v2(salt: bytes) -> Fernet:
|
||||
cached = _fernet_v2_cache.get(salt)
|
||||
if cached is not None:
|
||||
return cached
|
||||
f = Fernet(_derive_v2_key(settings.encryption_key, salt))
|
||||
if len(_fernet_v2_cache) >= _V2_CACHE_MAX:
|
||||
_fernet_v2_cache.clear()
|
||||
_fernet_v2_cache[salt] = f
|
||||
return f
|
||||
|
||||
|
||||
def encrypt_api_key(plaintext: str) -> str:
|
||||
token = _cipher().encrypt(plaintext.encode("utf-8")).decode("utf-8")
|
||||
return ENC_PREFIX + token
|
||||
global _encrypt_salt
|
||||
if _encrypt_salt is None:
|
||||
_encrypt_salt = os.urandom(_SALT_BYTES)
|
||||
salt_b64 = base64.urlsafe_b64encode(_encrypt_salt).decode("ascii")
|
||||
token = _cipher_v2(_encrypt_salt).encrypt(plaintext.encode("utf-8")).decode("utf-8")
|
||||
return f"{ENC_PREFIX_V2}{salt_b64}:{token}"
|
||||
|
||||
|
||||
def decrypt_api_key(stored: str) -> str:
|
||||
if not stored:
|
||||
raise ValueError("Empty api key")
|
||||
if not stored.startswith(ENC_PREFIX):
|
||||
|
||||
if stored.startswith(ENC_PREFIX_V2):
|
||||
rest = stored[len(ENC_PREFIX_V2):]
|
||||
try:
|
||||
salt_b64, token = rest.split(":", 1)
|
||||
salt = base64.urlsafe_b64decode(salt_b64.encode("ascii"))
|
||||
except Exception as exc:
|
||||
raise RuntimeError("Malformed enc:v2 blob") from exc
|
||||
try:
|
||||
return _cipher_v2(salt).decrypt(token.encode("utf-8")).decode("utf-8")
|
||||
except InvalidToken as exc:
|
||||
raise RuntimeError("HL key decryption failed — wrong ENCRYPTION_KEY?") from exc
|
||||
|
||||
if stored.startswith(ENC_PREFIX_V1):
|
||||
try:
|
||||
return _cipher_v1().decrypt(
|
||||
stored[len(ENC_PREFIX_V1):].encode("utf-8")).decode("utf-8")
|
||||
except InvalidToken as exc:
|
||||
raise RuntimeError("HL key decryption failed — wrong ENCRYPTION_KEY?") from exc
|
||||
|
||||
# Legacy plaintext row (from before encryption was added). Refuse to use in prod.
|
||||
if settings.environment == "production":
|
||||
raise RuntimeError(
|
||||
@@ -56,7 +134,8 @@ def decrypt_api_key(stored: str) -> str:
|
||||
)
|
||||
logger.warning("Reading LEGACY plaintext HL key — migrate ASAP")
|
||||
return stored
|
||||
try:
|
||||
return _cipher().decrypt(stored[len(ENC_PREFIX):].encode("utf-8")).decode("utf-8")
|
||||
except InvalidToken as exc:
|
||||
raise RuntimeError("HL key decryption failed — wrong ENCRYPTION_KEY?") from exc
|
||||
|
||||
|
||||
def is_current_format(stored: Optional[str]) -> bool:
|
||||
"""True if the blob is already enc:v2 (used by scripts/reencrypt_keys.py)."""
|
||||
return bool(stored) and stored.startswith(ENC_PREFIX_V2)
|
||||
|
||||
@@ -11,7 +11,9 @@ Broadcasts alert via WebSocket. Gated by user on/off toggle.
|
||||
"""
|
||||
|
||||
import collections
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from typing import Deque, Optional
|
||||
|
||||
@@ -40,12 +42,49 @@ _enabled: bool = False
|
||||
_recent_signals: Deque[dict] = collections.deque(maxlen=50)
|
||||
_last_fired: dict[str, Optional[datetime]] = {s: None for s in WATCH_SYMBOLS}
|
||||
|
||||
# B52: persist _enabled across process restarts without a DB migration.
|
||||
# The file survives `systemctl restart` but is cleared by OS reboots (which is
|
||||
# acceptable — an operator who reboots the server is expected to re-arm the
|
||||
# monitor). Path is configurable via env so staging vs prod can differ.
|
||||
_STATE_FILE = os.environ.get(
|
||||
"BREAKOUT_MONITOR_STATE_FILE",
|
||||
"/tmp/trumpsignal-breakout-state.json",
|
||||
)
|
||||
|
||||
|
||||
def _load_persisted_state() -> None:
|
||||
"""Read _enabled from disk on startup. Called once at module import time."""
|
||||
global _enabled
|
||||
try:
|
||||
with open(_STATE_FILE) as f:
|
||||
data = json.load(f)
|
||||
_enabled = bool(data.get("enabled", False))
|
||||
logger.info("Breakout monitor: loaded persisted state enabled=%s", _enabled)
|
||||
except FileNotFoundError:
|
||||
pass # first run — start disabled
|
||||
except Exception as exc:
|
||||
logger.warning("Breakout monitor: failed to load state file: %s", exc)
|
||||
|
||||
|
||||
def _persist_state() -> None:
|
||||
"""Write _enabled to disk so it survives process restarts."""
|
||||
try:
|
||||
with open(_STATE_FILE, "w") as f:
|
||||
json.dump({"enabled": _enabled, "updated_at": datetime.now(timezone.utc).isoformat()}, f)
|
||||
except Exception as exc:
|
||||
logger.warning("Breakout monitor: failed to persist state: %s", exc)
|
||||
|
||||
|
||||
# Load persisted state at import time (called when main.py registers the scheduler).
|
||||
_load_persisted_state()
|
||||
|
||||
|
||||
# ── Public API ────────────────────────────────────────────────────────────────
|
||||
|
||||
def set_enabled(value: bool) -> None:
|
||||
global _enabled
|
||||
_enabled = value
|
||||
_persist_state() # B52: survive restarts
|
||||
logger.info("Funding signal monitor: %s", "ENABLED" if value else "DISABLED")
|
||||
|
||||
|
||||
|
||||
@@ -31,7 +31,6 @@ import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
|
||||
from app.services.price_store import price_store
|
||||
from app.ws.manager import manager
|
||||
@@ -96,8 +95,8 @@ async def _tick() -> None:
|
||||
"""Single price fetch + dispatch cycle for all HL_PRICE_ASSETS."""
|
||||
now_ms = int(time.time() * 1000)
|
||||
|
||||
async with httpx.AsyncClient(timeout=4.0) as c:
|
||||
r = await c.post(HL_API_URL, json={"type": "allMids"})
|
||||
from app.services.http_client import get_client
|
||||
r = await get_client().post(HL_API_URL, json={"type": "allMids"}, timeout=4.0)
|
||||
r.raise_for_status()
|
||||
mids: dict = r.json() # {"BTC": "74541.0", "HYPE": "13.5", …}
|
||||
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
"""Shared pooled httpx.AsyncClient.
|
||||
|
||||
Hot paths (scrapers, Telegram send/poll, price feeds, X poster) used to build
|
||||
a fresh AsyncClient per request, paying a TCP+TLS handshake every time. This
|
||||
module owns one process-wide client with keep-alive pooling; callers override
|
||||
the timeout per request (`client.get(url, timeout=10)`).
|
||||
|
||||
Lifecycle: lazily created on first use; main.py's lifespan closes it on
|
||||
shutdown. Low-frequency daily jobs (KOL/macro fetchers) may keep their own
|
||||
ad-hoc clients — pooling only matters on the per-second paths.
|
||||
"""
|
||||
import logging
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_client: Optional[httpx.AsyncClient] = None
|
||||
|
||||
|
||||
def get_client() -> httpx.AsyncClient:
|
||||
global _client
|
||||
if _client is None or _client.is_closed:
|
||||
_client = httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(20.0),
|
||||
follow_redirects=True,
|
||||
limits=httpx.Limits(max_connections=50, max_keepalive_connections=20),
|
||||
)
|
||||
return _client
|
||||
|
||||
|
||||
async def aclose() -> None:
|
||||
global _client
|
||||
if _client is not None and not _client.is_closed:
|
||||
await _client.aclose()
|
||||
_client = None
|
||||
@@ -185,7 +185,12 @@ class HyperliquidTrader:
|
||||
)
|
||||
logger.info("Set leverage %dx for %s (isolated)", effective, coin)
|
||||
except Exception as exc:
|
||||
logger.warning("set_leverage error (non-fatal): %s", exc)
|
||||
# DO NOT swallow — if leverage isn't set, the position would open
|
||||
# at whatever HL had previously. Downstream stop math
|
||||
# (sys2_protective_stop_pct) uses the return value; a wrong
|
||||
# value puts the stop outside the real liquidation line.
|
||||
logger.error("set_leverage FAILED for %s %dx: %s — aborting open", coin, effective, exc)
|
||||
raise RuntimeError(f"set_leverage failed for {coin} at {effective}×: {exc}") from exc
|
||||
return effective
|
||||
|
||||
async def open_position(
|
||||
|
||||
+132
-25
@@ -35,7 +35,7 @@ from app.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ANALYSIS_VERSION = "kol-v1"
|
||||
ANALYSIS_VERSION = "kol-v2"
|
||||
ANTHROPIC_MODEL = "claude-haiku-4-5-20251001"
|
||||
|
||||
_anthropic_client = None
|
||||
@@ -71,7 +71,8 @@ The author is a known crypto KOL. Your job: distill what they said and which tok
|
||||
Output **strict JSON only**, no markdown, no preface. Schema:
|
||||
|
||||
{
|
||||
"summary": "<one sentence in ENGLISH, ≤80 chars. State the author's current market thesis if they have one, or describe the post topic if no clear signal. Always English regardless of the post's original language.>",
|
||||
"summary": "<2-3 sentences in ENGLISH, ≤200 chars total. State the author's current market thesis if they have one, or describe the post topic. Capture the key directional call if any. Always English regardless of the post's original language.>",
|
||||
"post_type": "trade_update" | "macro_thesis" | "research" | "news_recap" | "opinion" | "other",
|
||||
"tickers": [
|
||||
{
|
||||
"ticker": "<UPPERCASE symbol, e.g. BTC, ETH, HYPE, SOL>",
|
||||
@@ -82,29 +83,80 @@ Output **strict JSON only**, no markdown, no preface. Schema:
|
||||
"quote": "<shortest verbatim sentence from the post supporting this call, ≤200 chars. Keep original language — do not translate.>"
|
||||
}
|
||||
],
|
||||
"talks_vs_trades_flag": <true if you detect a mismatch between the KOL's stated bullish/bearish narrative and signals of the opposite actual position (e.g. claiming bullish while describing reducing size, or bearish while mentioning recent buys) | false>
|
||||
"talks_vs_trades_score": <float 0.0-1.0. 0 = no divergence detected. 0.5 = moderate mismatch signals. 1.0 = clear contradiction between stated narrative and implied position action. See rules below.>
|
||||
}
|
||||
|
||||
Rules:
|
||||
- Always return summary in ENGLISH regardless of the post language.
|
||||
- If the post is macro commentary, news recap, or sponsored content with no specific token call, return tickers=[] and summary describing the topic.
|
||||
- IGNORE historical price references ("BTC bottomed at $60k earlier this year") — these are context, not current calls.
|
||||
- IGNORE advertising/sponsor sections — look for cues: "sponsor", "partner", "use code", "promo code", "this episode brought to you by", "ad", "广告", "赞助". Skip any ticker only mentioned inside such a section.
|
||||
|
||||
POST TYPE:
|
||||
- "trade_update" → author explicitly describes entering, exiting, or adjusting a position
|
||||
- "macro_thesis" → broad market view, cycle analysis, regime commentary without specific trade action
|
||||
- "research" → data-driven, analytical, with sources/charts (typical of Delphi, Glassnode, Blockworks)
|
||||
- "news_recap" → summarising recent events without strong personal view
|
||||
- "opinion" → personal take without data backing or explicit position
|
||||
- "other" → doesn't fit the above
|
||||
|
||||
SUMMARY:
|
||||
- Always English regardless of post language.
|
||||
- 2-3 sentences, ≤200 chars. Lead with the directional call if there is one.
|
||||
- If no signal: describe what the post covers in plain terms.
|
||||
|
||||
TICKERS:
|
||||
- IGNORE historical price references ("BTC bottomed at $60k earlier this year") — context, not current calls.
|
||||
- IGNORE advertising/sponsor sections — cues: "sponsor", "partner", "use code", "promo code", "this episode brought to you by", "ad", "广告", "赞助", "合作方". Skip any ticker only mentioned inside such a section.
|
||||
- action values:
|
||||
"buy"/"sell" → author explicitly states a position action ("I bought", "we are long", "我们减仓了", "added to my bag", "taking profits", "已建仓")
|
||||
"reduce" → author is partially exiting or taking profits on a long-held position (distinct from "sell" which implies closing)
|
||||
"bullish"/"bearish" → directional view without explicit position statement
|
||||
"mention" → ticker appears but no clear stance
|
||||
"buy"/"sell" → author explicitly states a position action ("I bought", "we are long", "我们减仓了", "added to my bag", "taking profits", "已建仓", "我加仓了", "建了仓")
|
||||
"reduce" → partially exiting or taking profits on an existing long ("trimming", "taking some off", "减了一部分", "止盈了一些")
|
||||
"bullish" → ANY expression of upside view, including SOFT/INDIRECT/CASUAL language. Err on the side of bullish over mention.
|
||||
English soft bullish: "X looks interesting here", "I like X at these levels", "X has room to run",
|
||||
"wouldn't be short X", "X is setting up well", "compelling setup in X", "X is undervalued",
|
||||
"I'd be a buyer of X", "X deserves attention here", "keeping an eye on X"
|
||||
Chinese soft bullish: "感觉还有空间", "这里可以关注", "值得关注", "看好", "还有机会",
|
||||
"这个位置不错", "可以考虑", "有上涨空间", "这波不错", "可以关注一下", "挺有意思的"
|
||||
"bearish" → ANY expression of downside view or caution, including soft signals.
|
||||
English soft bearish: "X looks extended", "I'd be careful here", "not touching X",
|
||||
"X has further to fall", "selling into strength on X", "X is a trap"
|
||||
Chinese soft bearish: "感觉要跌", "要小心", "不敢碰", "还会跌", "高位了", "风险很大"
|
||||
"mention" → ticker appears but author has ZERO directional lean — pure neutral reference,
|
||||
historical context only, or just used as an example. If there is ANY lean at all,
|
||||
use bullish or bearish instead. Reserve mention for truly neutral uses like:
|
||||
"BTC started in 2009" or "unlike ETH, which uses proof-of-stake"
|
||||
- Dedupe per ticker — at most one entry per symbol; pick the strongest action.
|
||||
- Do NOT invent tickers. If you see "$XYZ" but unsure it's a real token, skip it.
|
||||
- conviction: 0.8+ requires explicit + repeated + sized/timed view; 0.5-0.7 for clear directional view without commitment; <0.5 for passing references.
|
||||
- timeframe: "immediate" = author is acting now or within 24h; "days" = 1-7 days; "weeks" = 1-4 weeks; "months" = 1+ months; "unspecified" = no timeframe given.
|
||||
- talks_vs_trades_flag: set true when narrative reads one way but position signals read the other. Examples:
|
||||
- Author writes a bullish thesis but mentions "reducing", "taking profits", "trimming", "risk management"
|
||||
- Author is bearish on macro but "accumulating at these levels"
|
||||
- High-conviction public call ("this is THE entry") but with very low disclosed position size
|
||||
- Previously loudly bullish post, but this post avoids reaffirming the position
|
||||
- Do not include fiat (USD/CNY/JPY) or stablecoins (USDT/USDC/DAI/FRAX) unless the post's main thesis is about them.
|
||||
- Do NOT invent tickers. Skip "$XYZ" if unsure it is a real crypto token.
|
||||
- For CATEGORY-LEVEL calls (no specific ticker named), use these synthetic tickers:
|
||||
"MEME" → author is bullish/bearish on memecoins as a category ("memes look good", "memecoin season", "梗币最近不错")
|
||||
"ALTCOIN" → author is bullish/bearish on altcoins broadly ("alts are waking up", "山寨季", "altseason")
|
||||
"DEFI" → author is bullish/bearish on DeFi protocols broadly
|
||||
"AI" → author is bullish/bearish on AI-related tokens broadly
|
||||
Only use synthetic tickers when the view is genuinely about the category, not just a passing mention.
|
||||
- conviction: 0.8+ = explicit + repeated + sized or timed; 0.5–0.7 = clear view, no commitment; <0.5 = passing reference.
|
||||
- timeframe: "immediate" = acting now or within 24h; "days" = 1–7d; "weeks" = 1–4w; "months" = 1+ months; "unspecified" = not stated.
|
||||
- Do not include fiat (USD/CNY/JPY) or stablecoins (USDT/USDC/DAI/FRAX/USDE) unless the post's main thesis is about them.
|
||||
|
||||
TALKS-VS-TRADES SCORE (talks_vs_trades_score):
|
||||
This is the platform's most important signal. Score 0.0–1.0. Raise the score when you detect narrative-position mismatches:
|
||||
|
||||
Score 0.7–1.0 (strong divergence):
|
||||
- Author writes a bullish thesis but describes reducing, trimming, or taking profits on the same asset
|
||||
- Author is publicly bearish but mentions "accumulating", "adding at these levels", "买了一些"
|
||||
- Post is notably silent on an asset the author has been loudly bullish on recently (avoidance signal)
|
||||
- Stated high conviction ("this is THE entry", "strongest conviction of my career") but disclosed position size is very small
|
||||
- Author hedges every bullish statement with "but I could be wrong", "just my opinion", "not financial advice" applied unevenly (selective hedging)
|
||||
|
||||
Score 0.4–0.69 (moderate divergence):
|
||||
- Author mentions "risk management" or "waiting for confirmation" alongside a bullish thesis
|
||||
- Mixed signals: bullish long-term but explicitly neutral or cautious short-term on the same asset
|
||||
- Language shift: previous posts were assertive, this one uses softer language without explanation
|
||||
|
||||
Score 0.1–0.39 (weak divergence):
|
||||
- Minor hedges in an otherwise consistent narrative
|
||||
- Vague "taking profits" without clear contradiction of the main thesis
|
||||
|
||||
Score 0.0:
|
||||
- Narrative and any position signals are fully consistent
|
||||
- Post has no position signals at all (pure macro commentary)
|
||||
|
||||
Chinese-language divergence cues to detect: "减仓了" (reduced position), "止盈" (taking profit), "降低了仓位" (lowered position), "观望" (watching/waiting), "不着急" (in no hurry), "先不动" (staying put), "谨慎" (cautious), "控制仓位" (managing position size), "风控" (risk management).
|
||||
"""
|
||||
|
||||
|
||||
@@ -177,7 +229,7 @@ async def extract_kol_signal(
|
||||
if use_anth:
|
||||
msg = await _anth().messages.create(
|
||||
model=model,
|
||||
max_tokens=1500,
|
||||
max_tokens=2000, # raised: complex essays with many tickers were truncating at 1500
|
||||
temperature=0.1,
|
||||
system=SYSTEM_PROMPT,
|
||||
messages=[{"role": "user", "content": user}],
|
||||
@@ -190,7 +242,7 @@ async def extract_kol_signal(
|
||||
kwargs = {"model": model, "messages": [
|
||||
{"role": "system", "content": SYSTEM_PROMPT},
|
||||
{"role": "user", "content": user},
|
||||
], "max_tokens": 4000 if is_reasoning else 1500}
|
||||
], "max_tokens": 4000 if is_reasoning else 2000}
|
||||
if not is_reasoning:
|
||||
kwargs["temperature"] = 0.1
|
||||
# JSON mode — DeepSeek + OpenAI both support response_format.
|
||||
@@ -211,11 +263,20 @@ async def extract_kol_signal(
|
||||
cleaned = []
|
||||
valid_actions = {"buy", "sell", "reduce", "bullish", "bearish", "mention"}
|
||||
valid_timeframes = {"immediate", "days", "weeks", "months", "unspecified"}
|
||||
valid_post_types = {"trade_update", "macro_thesis", "research", "news_recap", "opinion", "other"}
|
||||
# Synthetic category tickers — allowed even though they're not real on-chain tokens.
|
||||
# Used when the KOL expresses a view on a category without naming specific coins.
|
||||
SYNTHETIC_TICKERS = {"MEME", "ALTCOIN", "DEFI", "AI"}
|
||||
|
||||
for t in tickers:
|
||||
if not isinstance(t, dict):
|
||||
continue
|
||||
sym = (t.get("ticker") or "").strip().upper()
|
||||
if not sym or len(sym) > 12:
|
||||
if not sym:
|
||||
continue
|
||||
# Allow synthetic category tickers; reject anything else > 12 chars
|
||||
# (12 chars covers most real tickers; synthetic ones are all ≤ 7 chars)
|
||||
if sym not in SYNTHETIC_TICKERS and len(sym) > 12:
|
||||
continue
|
||||
action = (t.get("action") or "mention").lower()
|
||||
if action not in valid_actions:
|
||||
@@ -238,12 +299,58 @@ async def extract_kol_signal(
|
||||
"quote": (t.get("quote") or "")[:200],
|
||||
})
|
||||
|
||||
talks_vs_trades = bool(data.get("talks_vs_trades_flag", False))
|
||||
# talks_vs_trades_score: new float field (v2). Backward-compat: if the
|
||||
# model returns the old boolean talks_vs_trades_flag, convert it so callers
|
||||
# that stored the old field still work. Clamp to [0, 1].
|
||||
raw_score = data.get("talks_vs_trades_score")
|
||||
if raw_score is None:
|
||||
# Fallback: old boolean field from v1 responses
|
||||
raw_score = 1.0 if bool(data.get("talks_vs_trades_flag", False)) else 0.0
|
||||
try:
|
||||
tvt_score = round(max(0.0, min(1.0, float(raw_score))), 2)
|
||||
except (TypeError, ValueError):
|
||||
tvt_score = 0.0
|
||||
|
||||
post_type = (data.get("post_type") or "other").lower()
|
||||
if post_type not in valid_post_types:
|
||||
post_type = "other"
|
||||
|
||||
tier = _derive_tier(cleaned, tvt_score)
|
||||
|
||||
return {
|
||||
"summary": (data.get("summary") or "").strip() or None,
|
||||
"post_type": post_type,
|
||||
"tickers": cleaned,
|
||||
"talks_vs_trades_flag": talks_vs_trades,
|
||||
"talks_vs_trades_score": tvt_score,
|
||||
# Keep old boolean for any callers that still check it
|
||||
"talks_vs_trades_flag": tvt_score >= 0.5,
|
||||
# tier mirrors x_analysis' vocabulary (trade_signal / directional /
|
||||
# noise) so blog/substack/podcast posts get the same SIGNAL/VIEW UI
|
||||
# badges + the "Signals only" filter that Twitter posts already have.
|
||||
"tier": tier,
|
||||
"model": model,
|
||||
"version": ANALYSIS_VERSION,
|
||||
}
|
||||
|
||||
|
||||
def _derive_tier(tickers: list[dict], tvt_score: float) -> str:
|
||||
"""Map kol_analysis output → the trade_signal/directional/noise tiers that
|
||||
x_analysis emits directly. Non-Twitter analyzers don't ask the model for a
|
||||
tier, so we derive one from the per-ticker conviction + the talks-vs-trades
|
||||
(divergence) score. Without this, blog/substack/podcast rows have tier=NULL
|
||||
and the "Signals only" filter + SIGNAL/VIEW badges never apply to them.
|
||||
|
||||
* directional ticker = an explicit non-"mention" action (buy/sell/
|
||||
reduce/bullish/bearish).
|
||||
* trade_signal = high conviction (>= 0.6) on a directional ticker, or a
|
||||
strong talks-vs-trades divergence (>= 0.6) — the platform's top signal.
|
||||
* directional = a directional view exists but below the trade_signal bar.
|
||||
* noise = no directional ticker and no notable divergence.
|
||||
"""
|
||||
directional = [t for t in tickers if (t.get("action") or "mention") != "mention"]
|
||||
max_conv = max((float(t.get("conviction") or 0) for t in directional), default=0.0)
|
||||
if max_conv >= 0.6 or tvt_score >= 0.6:
|
||||
return "trade_signal"
|
||||
if directional or tvt_score >= 0.5:
|
||||
return "directional"
|
||||
return "noise"
|
||||
|
||||
@@ -15,8 +15,11 @@ Two outcomes:
|
||||
Logic:
|
||||
|
||||
post side → action ∈ {buy, bullish} = LONG intent
|
||||
{sell, bearish} = SHORT intent
|
||||
{sell, bearish, reduce} = SHORT intent
|
||||
{mention} = skip (no clear view)
|
||||
(reduce = partial profit-take / trimming a long → a
|
||||
risk-reducing, short-leaning stance. Counting it lets us
|
||||
catch "publicly trimming BTC but wallet is adding" divergence.)
|
||||
|
||||
chain side → change_type ∈ {new_position, increased} = LONG action
|
||||
{decreased, closed} = SHORT action
|
||||
@@ -54,8 +57,11 @@ WINDOW_DAYS = 7
|
||||
MIN_USD_CHANGE = 10_000
|
||||
|
||||
# Post actions that map to a directional view (skip 'mention')
|
||||
# 'reduce' (kol-v2: partial profit-take / trimming a long) is a short-leaning
|
||||
# stance — include it so a "trimming publicly while accumulating on-chain"
|
||||
# mismatch is caught instead of silently dropped.
|
||||
_POST_LONG = {"buy", "bullish"}
|
||||
_POST_SHORT = {"sell", "bearish"}
|
||||
_POST_SHORT = {"sell", "bearish", "reduce"}
|
||||
|
||||
# On-chain actions that map to a direction
|
||||
_CHAIN_LONG = {"new_position", "increased"}
|
||||
|
||||
@@ -43,6 +43,16 @@ logger = logging.getLogger(__name__)
|
||||
# of body per post or it just hallucinates a topic line. (Headlines-only
|
||||
# feeds like Vitalik's blog need a follow-up HTML fetch, deferred.)
|
||||
# 3. Add with a sensible handle + display_name.
|
||||
# 4. ⚠️ The KOL feed COUNT is hardcoded in the FRONTEND for SEO/marketing
|
||||
# (it can't read this list cross-repo). If len(KOL_FEEDS) changes, update
|
||||
# every "N KOL feeds" mention in the frontend repo:
|
||||
# app/layout.tsx (JSON-LD ×2), app/page.tsx (×3 incl. metric),
|
||||
# app/[locale]/kol/page.tsx (×4), app/[locale]/kol/KolPageClient.tsx
|
||||
# (subtitle "and N more" = count-3), app/[locale]/glossary/page.tsx,
|
||||
# public/llms.txt + llms-full.txt, app/opengraph-image.tsx.
|
||||
# Currently len(KOL_FEEDS) == 25. (X-only KOLs live in kol_x.X_KOLS.)
|
||||
# (2026-06-09: dropped from 29 → 25; removed placeholder, dragonfly,
|
||||
# niccarter, eugene as dead feeds. Frontend count mentions need updating.)
|
||||
KOL_FEEDS: list[dict] = [
|
||||
# ── Substack essayists (long-form thesis pieces) ─────────────────────
|
||||
{
|
||||
@@ -50,38 +60,40 @@ KOL_FEEDS: list[dict] = [
|
||||
"display_name": "Arthur Hayes",
|
||||
"feed_url": "https://cryptohayes.substack.com/feed",
|
||||
},
|
||||
# Placeholder VC (Joel Monegro / Chris Burniske). Token-focused VC, posts
|
||||
# long-form thesis pieces every 1-3 months that map directly to their
|
||||
# portfolio bets (Solana staking, L1 monetary premium, etc.).
|
||||
# Raoul Pal — Real Vision / Global Macro Investor founder. "Short Excerpts
|
||||
# Raoul Pal — his public Substack (raoulpal.substack.com/feed) went STALE
|
||||
# (last post 2024-05). Replaced 2026-06-09 with the Real Vision official
|
||||
# podcast feed (feeds.megaphone.fm/realvision) — daily, free, 2000+ eps,
|
||||
# full episode descriptions with macro/crypto thesis. It's the whole Real
|
||||
# Vision channel (not Raoul-only), but high signal + fresh. Same canonical
|
||||
# handle as his X (@RaoulGMI in kol_x) so long-form + real-time aggregate.
|
||||
{
|
||||
"handle": "placeholder",
|
||||
"display_name": "Placeholder VC",
|
||||
"feed_url": "https://www.placeholder.vc/blog?format=rss",
|
||||
"handle": "raoulpal",
|
||||
"display_name": "Raoul Pal (Real Vision)",
|
||||
"feed_url": "https://feeds.megaphone.fm/realvision",
|
||||
"source": "podcast",
|
||||
},
|
||||
# Dragonfly Capital research blog on Medium — free, active (10+ posts).
|
||||
# dragonfly.xyz/blog/rss.xml returns 0 (paywall). medium.com/dragonfly-research
|
||||
# is the team's public research arm: airdrops, DeFi, protocol deep-dives.
|
||||
{
|
||||
"handle": "dragonfly",
|
||||
"display_name": "Dragonfly Capital",
|
||||
"feed_url": "https://medium.com/feed/dragonfly-research",
|
||||
},
|
||||
# Andy Constan's Substack is paywalled (RSS returns 0). Keeping for any
|
||||
# occasional public teaser. Forward Guidance podcast (Blockworks) features
|
||||
# him weekly but is macro/equities-focused — not crypto-coin-specific enough
|
||||
# to extract ticker signals from episode descriptions.
|
||||
# REMOVED 2026-06-09 — dead feeds, no active replacement exists:
|
||||
# • placeholder (Placeholder VC) — last post 2025-09, blog is their only
|
||||
# public source, no newsletter. ~266d stale.
|
||||
# • dragonfly (Dragonfly Research, Medium) — last post 2025-03 (~454d).
|
||||
# dragonfly.xyz has no working RSS (JS-rendered, all endpoints empty);
|
||||
# Haseeb's medium/@hosseeb is even older (2024).
|
||||
# If either resumes regular publishing with a real RSS, re-add here.
|
||||
# Andy Constan — his old Substack (dampedspring.substack.com) returned 0
|
||||
# entries (paywalled). Replaced 2026-06-09 with his FREE long-form Substack
|
||||
# "Damped Spring 101" (dampedspring101.substack.com), which he's publicly
|
||||
# committed to keeping free — active, real macro essays (liquidity, rates,
|
||||
# positioning). His deepest paid research stays gated, but this carries
|
||||
# genuine thesis content suitable for ticker/direction extraction.
|
||||
{
|
||||
"handle": "dampedspring",
|
||||
"display_name": "Damped Spring / Andy Constan",
|
||||
"feed_url": "https://dampedspring.substack.com/feed",
|
||||
},
|
||||
# Nic Carter's Substack is paywalled (RSS returns 0). His Medium feed is
|
||||
# FREE and active — different URL, same author, real content.
|
||||
{
|
||||
"handle": "niccarter",
|
||||
"display_name": "Nic Carter (Castle Island)",
|
||||
"feed_url": "https://medium.com/feed/@nic__carter",
|
||||
"feed_url": "https://dampedspring101.substack.com/feed",
|
||||
},
|
||||
# REMOVED 2026-06-09 — niccarter (Nic Carter / Castle Island). Substack
|
||||
# paywalled (0 entries); Medium feed went stale (last 2025-07, ~316d).
|
||||
# No active free replacement. Re-add if he resumes a public RSS.
|
||||
# Delphi Digital podcast (Buzzsprout) — 478 episodes, active May 2025.
|
||||
# Public, free. Episode descriptions name specific protocols / tokens with
|
||||
# thesis framing — good extraction signal. delphidigital.io/feed returns 0.
|
||||
@@ -104,12 +116,8 @@ KOL_FEEDS: list[dict] = [
|
||||
"display_name": "The DeFi Edge",
|
||||
"feed_url": "https://thedefiedge.com/feed/",
|
||||
},
|
||||
# Eugene Ng Ah Sio — trader/analyst, sporadic but specific.
|
||||
{
|
||||
"handle": "eugene",
|
||||
"display_name": "Eugene Ng Ah Sio",
|
||||
"feed_url": "https://eugene.substack.com/feed",
|
||||
},
|
||||
# REMOVED 2026-06-09 — eugene (Eugene Ng Ah Sio). Substack effectively
|
||||
# dormant: only 5 entries, last 2025-05 (~399d). No alternative source.
|
||||
# ── DeFi journalism (Substack-style RSS) ─────────────────────────────
|
||||
# The Defiant — Camila Russo's team. DeFi-focused news with frequent
|
||||
# protocol + token mentions. Free RSS, ~100 entries.
|
||||
@@ -149,12 +157,15 @@ KOL_FEEDS: list[dict] = [
|
||||
"feed_url": "https://feeds.megaphone.fm/lightspeed",
|
||||
"source": "podcast",
|
||||
},
|
||||
# Unchained — Laura Shin. Long interview format with founders and
|
||||
# traders. Show notes are 6K+ chars (near-transcript).
|
||||
# Unchained — Laura Shin. Long interview format with founders and traders.
|
||||
# The website RSS (unchainedcrypto.com/feed) is Cloudflare-blocked (403,
|
||||
# even HTTP/2 + browser UA). Switched 2026-06-09 to the canonical Megaphone
|
||||
# podcast feed (resolved via iTunes lookup id=1123922160) — 1100+ episodes,
|
||||
# active daily, 1.5-2K char show notes naming protocols/tokens.
|
||||
{
|
||||
"handle": "unchained",
|
||||
"display_name": "Unchained (Laura Shin)",
|
||||
"feed_url": "https://www.unchainedcrypto.com/feed/",
|
||||
"feed_url": "https://feeds.megaphone.fm/LSHML4761942757",
|
||||
"source": "podcast",
|
||||
},
|
||||
# Bankless podcast — Ryan Sean Adams + David Hoffman. ETH-focused but
|
||||
@@ -240,8 +251,12 @@ KOL_FEEDS: list[dict] = [
|
||||
# but extremely high quality. Mostly BTC-only signals.
|
||||
{
|
||||
"handle": "lynalden",
|
||||
# lynalden.com/feed is Cloudflare-blocked (202, no body). Switched
|
||||
# 2026-06-09 to her FeedBurner mirror, which serves fine. Note: bodies
|
||||
# are article EXCERPTS (~400 chars) not full text, but enough for the
|
||||
# AI to extract the BTC/macro thesis + direction.
|
||||
"display_name": "Lyn Alden",
|
||||
"feed_url": "https://www.lynalden.com/feed/",
|
||||
"feed_url": "https://feeds.feedburner.com/lynalden",
|
||||
"source": "blog",
|
||||
},
|
||||
# Bitcoin Magazine — high-frequency news + institutional adoption analysis.
|
||||
@@ -336,8 +351,19 @@ async def _fetch_feed(feed_url: str) -> list:
|
||||
"""feedparser is sync; do the HTTP fetch through httpx for timeout
|
||||
control + uniformity with the rest of the codebase, then hand bytes
|
||||
to feedparser."""
|
||||
async with httpx.AsyncClient(timeout=20.0, follow_redirects=True) as client:
|
||||
r = await client.get(feed_url, headers={"User-Agent": "TrumpSignal/1.0 KOL-tracker"})
|
||||
# http2=True matters: some CDN-fronted feeds (e.g. Glassnode) return 403
|
||||
# to HTTP/1.1 requests but serve fine over HTTP/2 (curl defaults to h2,
|
||||
# which is why they worked manually but not here). Requires the `h2` pkg.
|
||||
async with httpx.AsyncClient(timeout=20.0, follow_redirects=True, http2=True) as client:
|
||||
# Use a real browser UA. Several feeds (Glassnode, others behind a CDN)
|
||||
# return 403/202 to non-browser agents. A plain bot UA was silently
|
||||
# losing those feeds. This recovers them without per-feed special-casing.
|
||||
r = await client.get(feed_url, headers={
|
||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/124.0 Safari/537.36",
|
||||
"Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*",
|
||||
})
|
||||
r.raise_for_status()
|
||||
parsed = feedparser.parse(r.content)
|
||||
return list(parsed.entries or [])
|
||||
@@ -439,6 +465,12 @@ async def _ingest_kol(
|
||||
row.analyzed_at = utcnow()
|
||||
row.analysis_model = result.get("model")
|
||||
row.analysis_version = result.get("version")
|
||||
# Extended analysis fields (migration 027)
|
||||
row.post_type = result.get("post_type")
|
||||
row.talks_vs_trades_flag = bool(result.get("talks_vs_trades_flag", False))
|
||||
# tier (trade_signal/directional/noise) so the "Signals only"
|
||||
# filter + SIGNAL/VIEW badges work for non-Twitter KOLs too.
|
||||
row.tier = result.get("tier")
|
||||
stats["analyzed"] += 1
|
||||
except Exception as e:
|
||||
logger.warning("[kol_substack] analysis failed for %s post %s: %s",
|
||||
@@ -452,10 +484,14 @@ async def _ingest_kol(
|
||||
async def run_substack_poll(*, analyze: bool = True) -> list[dict]:
|
||||
"""Poll every configured KOL feed once. Despite the legacy name this now
|
||||
covers Substack essays, Medium blogs, and major crypto podcasts via RSS.
|
||||
Returns per-KOL stats."""
|
||||
Returns per-KOL stats.
|
||||
|
||||
Each KOL gets its own session so a commit failure for one does not leave
|
||||
a dirty session that breaks subsequent KOLs in the same run.
|
||||
"""
|
||||
results = []
|
||||
async with AsyncSessionLocal() as session:
|
||||
for kol in KOL_FEEDS:
|
||||
async with AsyncSessionLocal() as session:
|
||||
stats = await _ingest_kol(session, kol, analyze=analyze)
|
||||
results.append(stats)
|
||||
logger.info("[kol_substack] poll done: %s", results)
|
||||
|
||||
@@ -0,0 +1,284 @@
|
||||
"""KOL X (Twitter) ingester.
|
||||
|
||||
Polls each tracked KOL's recent tweets via twitterapi.io, dedupes by tweet id,
|
||||
runs `x_analysis.analyze_x_post`, and writes a `KolPost(source="twitter")`.
|
||||
|
||||
The `tickers_json` it stores uses the exact `{ticker, action, conviction}` shape
|
||||
that `kol_divergence` already consumes (it reads `t["ticker"]` / `t["action"]` /
|
||||
`t["conviction"]`) — so no mapping layer is needed. x_analysis was designed
|
||||
against that contract; this module just connects the pipe.
|
||||
|
||||
WHY THIS EXISTS
|
||||
`andrewkang` (@Rewkang) and `murad` publish ONLY on X — no Substack feed.
|
||||
Their on-chain wallets are already seeded (seed_kol_wallets.py), but with no
|
||||
post-side data the divergence scanner detects nothing for them — the wallets
|
||||
sit dark. This ingester is the missing post-side feed that lights them up.
|
||||
|
||||
DESIGN (mirrors kol_substack.py)
|
||||
- Disabled (full no-op) when `settings.twitterapi_io_key` is empty.
|
||||
- `KolPost.kol_handle` is the CANONICAL handle (e.g. "cryptohayes"), NOT the
|
||||
X screen name — it must match `KolWallet.handle` so divergence can join
|
||||
post-side ↔ on-chain for the same person.
|
||||
- Dedup by (source="twitter", external_id=<tweet id>).
|
||||
- Bare retweets ("RT @…") are skipped before the AI call to save spend.
|
||||
- Only the first page (~20 newest tweets) is fetched per run. Daily volume
|
||||
for these KOLs is < 20/day, and dedup makes re-runs cheap. Bump
|
||||
`max_pages` if you add a very high-frequency account.
|
||||
|
||||
COST
|
||||
~$0.15 / 1k tweets (twitterapi.io). 4 KOLs × ~20 tweets daily ≈ $0.015/mo.
|
||||
|
||||
CADENCE
|
||||
Daily 01:30 UTC — after substack (01:15), before on-chain (02:00) and
|
||||
divergence (02:15), so fresh X posts are present when divergence runs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from email.utils import parsedate_to_datetime
|
||||
from typing import AsyncGenerator, Optional
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.config import settings
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.models import KolPost, utcnow
|
||||
from app.services import x_analysis
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
X_API = "https://api.twitterapi.io/twitter/user/last_tweets"
|
||||
|
||||
# Tracked X-native KOLs. `handle` MUST match the canonical handle used in
|
||||
# KolWallet / KOL_FEEDS so divergence can join against on-chain data.
|
||||
# `x_username` is the actual X screen name (no @).
|
||||
#
|
||||
# cryptohayes is also on Substack (long-form) — X adds his real-time position
|
||||
# statements ("just dumped my $HYPE"), which Substack essays don't carry.
|
||||
# andrewkang + murad are X-ONLY and have seeded wallets waiting for this feed.
|
||||
X_KOLS: list[dict] = [
|
||||
{"handle": "cryptohayes", "x_username": "CryptoHayes", "display_name": "Arthur Hayes"},
|
||||
{"handle": "andrewkang", "x_username": "Rewkang", "display_name": "Andrew Kang"},
|
||||
{"handle": "murad", "x_username": "MustStopMurad", "display_name": "Murad Mahmudov"},
|
||||
# Raoul Pal — Real Vision / GMI founder. Macro/liquidity/cycle thinker
|
||||
# (mostly DIRECTIONAL, rarely position TRADE_SIGNAL). Same canonical handle
|
||||
# as his Substack feed below so his X takes + long-form theses aggregate.
|
||||
{"handle": "raoulpal", "x_username": "RaoulGMI", "display_name": "Raoul Pal"},
|
||||
]
|
||||
|
||||
|
||||
def _parse_created_at(raw: Optional[str]) -> datetime:
|
||||
"""X `createdAt` looks like 'Thu Jun 04 12:29:13 +0000 2026'. Normalise to
|
||||
naive UTC (same convention as kol_substack._parse_pub — divergence window
|
||||
matching and digest 'since' filters all assume naive UTC)."""
|
||||
if not raw:
|
||||
return utcnow()
|
||||
try:
|
||||
dt = parsedate_to_datetime(raw)
|
||||
if dt.tzinfo:
|
||||
dt = dt.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return dt
|
||||
except Exception:
|
||||
return utcnow()
|
||||
|
||||
|
||||
async def _fetch_tweet_page(
|
||||
client: httpx.AsyncClient,
|
||||
x_username: str,
|
||||
cursor: Optional[str],
|
||||
key: str,
|
||||
) -> tuple[list[dict], Optional[str]]:
|
||||
"""Fetch one page of tweets. Returns (tweets, next_cursor).
|
||||
next_cursor is None when there are no more pages or on any error."""
|
||||
params: dict = {"userName": x_username, "includeReplies": "false"}
|
||||
if cursor:
|
||||
params["cursor"] = cursor
|
||||
try:
|
||||
r = await client.get(X_API, params=params, headers={"x-api-key": key})
|
||||
if r.status_code != 200:
|
||||
logger.warning("[kol_x] fetch %s HTTP %d: %s",
|
||||
x_username, r.status_code, r.text[:200])
|
||||
return [], None
|
||||
data = r.json()
|
||||
# Shape: {status, data: {tweets: [...], next_cursor?: str}, ...}
|
||||
if data.get("status") != "success":
|
||||
logger.warning("[kol_x] fetch %s non-success: %s",
|
||||
x_username, str(data.get("msg") or data)[:200])
|
||||
return [], None
|
||||
inner = data.get("data") or {}
|
||||
tweets = inner.get("tweets") or []
|
||||
next_cursor = inner.get("next_cursor") or None
|
||||
return tweets, next_cursor
|
||||
except Exception as e:
|
||||
logger.warning("[kol_x] fetch %s exception: %s", x_username, e)
|
||||
return [], None
|
||||
|
||||
|
||||
async def _iter_tweet_pages(
|
||||
x_username: str,
|
||||
max_pages: int = 3,
|
||||
) -> AsyncGenerator[list[dict], None]:
|
||||
"""Async generator: yields one page of tweets at a time.
|
||||
|
||||
Callers can break out of the loop early (page-level early-stop) without
|
||||
fetching unnecessary pages. max_pages=3 is the gap-fill ceiling — normal
|
||||
daily runs stop after page 1 because _ingest_kol_x breaks on all-dedup.
|
||||
|
||||
Never raises — a network error yields nothing and ends the iteration.
|
||||
"""
|
||||
key = settings.twitterapi_io_key
|
||||
if not key:
|
||||
return
|
||||
cursor: Optional[str] = None
|
||||
async with httpx.AsyncClient(timeout=20.0) as client:
|
||||
for _ in range(max_pages):
|
||||
tweets, cursor = await _fetch_tweet_page(client, x_username, cursor, key)
|
||||
if tweets:
|
||||
yield tweets
|
||||
if not cursor or not tweets:
|
||||
break
|
||||
|
||||
|
||||
async def _ingest_kol_x(
|
||||
session: AsyncSession,
|
||||
kol: dict,
|
||||
*,
|
||||
analyze: bool = True,
|
||||
max_new: int = 20,
|
||||
max_pages: int = 3,
|
||||
) -> dict:
|
||||
"""Ingest one KOL's recent tweets. Mirrors kol_substack._ingest_kol.
|
||||
|
||||
Pages are fetched one at a time. If an entire page consists only of
|
||||
already-stored tweets (dedup hits), fetching stops — subsequent pages
|
||||
would be even older and equally known, so the API call is skipped.
|
||||
This keeps the normal daily run to a single page fetch.
|
||||
"""
|
||||
handle = kol["handle"]
|
||||
x_username = kol["x_username"]
|
||||
stats = {"handle": handle, "x_username": x_username,
|
||||
"new": 0, "skipped": 0, "analyzed": 0, "errors": 0}
|
||||
|
||||
async for page_tweets in _iter_tweet_pages(x_username, max_pages=max_pages):
|
||||
if stats["new"] >= max_new:
|
||||
break
|
||||
|
||||
page_new = 0 # new tweets stored on this page
|
||||
page_deduped = 0 # already-stored tweets hit on this page (dedup)
|
||||
|
||||
for tw in page_tweets:
|
||||
if stats["new"] >= max_new:
|
||||
break
|
||||
|
||||
tweet_id = str(tw.get("id") or "").strip()
|
||||
text = (tw.get("text") or "").strip()
|
||||
if not tweet_id or not text:
|
||||
continue
|
||||
|
||||
# Bare retweet → noise. Skip before the AI call to save spend.
|
||||
if text.startswith("RT @"):
|
||||
stats["skipped"] += 1
|
||||
continue
|
||||
|
||||
# Dedup by (source, external_id).
|
||||
existing = await session.execute(
|
||||
select(KolPost).where(
|
||||
KolPost.source == "twitter",
|
||||
KolPost.external_id == tweet_id,
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none() is not None:
|
||||
stats["skipped"] += 1
|
||||
page_deduped += 1
|
||||
continue
|
||||
|
||||
author = tw.get("author") or {}
|
||||
follower_count = author.get("followers")
|
||||
url = tw.get("url") or f"https://x.com/{x_username}/status/{tweet_id}"
|
||||
pub = _parse_created_at(tw.get("createdAt"))
|
||||
body_hash = hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
row = KolPost(
|
||||
kol_handle=handle,
|
||||
source="twitter",
|
||||
external_id=tweet_id,
|
||||
url=url,
|
||||
title=None,
|
||||
published_at=pub,
|
||||
raw_text=text,
|
||||
content_hash=body_hash,
|
||||
)
|
||||
session.add(row)
|
||||
await session.flush()
|
||||
stats["new"] += 1
|
||||
page_new += 1
|
||||
logger.info("[kol_x] new tweet %s id=%s tweet_id=%s", handle, row.id, tweet_id)
|
||||
|
||||
if analyze:
|
||||
try:
|
||||
result = await x_analysis.analyze_x_post(
|
||||
handle=handle,
|
||||
text=text,
|
||||
posted_at=pub.isoformat(),
|
||||
follower_count=follower_count,
|
||||
)
|
||||
if result.get("error"):
|
||||
stats["errors"] += 1
|
||||
else:
|
||||
row.summary = result.get("summary")
|
||||
row.tickers_json = json.dumps(result.get("tickers") or [],
|
||||
ensure_ascii=False)
|
||||
row.analyzed_at = utcnow()
|
||||
row.analysis_model = result.get("model")
|
||||
row.analysis_version = result.get("version")
|
||||
# Extended x_analysis fields (migration 027)
|
||||
row.tier = result.get("tier")
|
||||
row.post_type = result.get("post_type")
|
||||
row.talks_vs_trades_flag = bool(result.get("talks_vs_trades_flag", False))
|
||||
row.sentiment = result.get("sentiment")
|
||||
stats["analyzed"] += 1
|
||||
except Exception as e:
|
||||
logger.warning("[kol_x] analysis failed %s tweet %s: %s",
|
||||
handle, row.id, e)
|
||||
stats["errors"] += 1
|
||||
|
||||
# Page-level early stop: this page yielded NO new tweets AND hit at
|
||||
# least one dedup → we've caught up to already-stored data, so older
|
||||
# pages are known too. A page that is ALL bare-RTs (page_deduped == 0)
|
||||
# must NOT early-stop — the next page may still hold original posts.
|
||||
if page_new == 0 and page_deduped > 0:
|
||||
logger.debug("[kol_x] %s page fully deduped — stopping early", handle)
|
||||
break
|
||||
|
||||
await session.commit()
|
||||
return stats
|
||||
|
||||
|
||||
async def run_x_poll(*, analyze: bool = True) -> list[dict]:
|
||||
"""Poll every tracked X KOL once. Full no-op (returns []) when the
|
||||
twitterapi.io key is unset. Returns per-KOL stats.
|
||||
|
||||
Each KOL gets its own session so a commit failure for one does not leave
|
||||
a dirty session that breaks subsequent KOLs in the same run.
|
||||
"""
|
||||
if not settings.twitterapi_io_key:
|
||||
logger.info("[kol_x] twitterapi_io_key not set — X ingestion disabled.")
|
||||
return []
|
||||
results = []
|
||||
for kol in X_KOLS:
|
||||
try:
|
||||
async with AsyncSessionLocal() as session:
|
||||
stats = await _ingest_kol_x(session, kol, analyze=analyze)
|
||||
results.append(stats)
|
||||
except Exception as e:
|
||||
# One KOL's DB/commit failure must not abort the rest of the run.
|
||||
logger.error("[kol_x] ingest failed for %s: %s", kol.get("handle"), e)
|
||||
results.append({"handle": kol.get("handle"), "error": str(e)})
|
||||
logger.info("[kol_x] poll done: %s", results)
|
||||
return results
|
||||
@@ -161,63 +161,54 @@ async def fetch_ahr999() -> dict:
|
||||
}
|
||||
|
||||
|
||||
# ── 2. Altcoin Season Index (blockchaincenter.net formula) ───────────────────
|
||||
# Take top-50 coins by market cap (excluding stablecoins + wrapped). Count how
|
||||
# many beat BTC's 90-day return. Result is the count, projected to 0-100.
|
||||
# 75+ = altseason, <25 = bitcoin season, middle = neutral.
|
||||
# ── 2. Altcoin Season Index (blockchaincenter.net — official source) ─────────
|
||||
# Scrape the value directly from blockchaincenter.net, which is the canonical
|
||||
# publisher of this index (90-day window: how many of the top 50 alts beat BTC
|
||||
# over 90 days). 75+ = altseason, <25 = bitcoin season.
|
||||
#
|
||||
# Previous implementation computed the index from CoinGecko /coins/markets
|
||||
# using a 30d window (CoinGecko doesn't return 90d per-coin data on that
|
||||
# endpoint). The 30d vs 90d discrepancy caused readings up to ~30 points
|
||||
# higher than the official index during BTC-dominated markets. Scraping the
|
||||
# actual page is more reliable than re-implementing the formula.
|
||||
#
|
||||
# Fallback: if the page scrape fails, return None (the @_none_on_fail
|
||||
# decorator handles that gracefully).
|
||||
|
||||
_STABLE_OR_WRAPPED = {
|
||||
"USDT", "USDC", "DAI", "BUSD", "TUSD", "USDD", "FDUSD", "PYUSD", "USDE",
|
||||
"WBTC", "WETH", "STETH", "WSTETH", "WEETH", "RETH",
|
||||
}
|
||||
_BCC_URL = "https://www.blockchaincenter.net/altcoin-season-index/"
|
||||
# Regex for the server-rendered value in the Next.js HTML:
|
||||
# "Season<!-- -->41" or "Season (<!-- -->41" inside the page markup.
|
||||
_BCC_RE = re.compile(r"Season[^<]{0,20}<!--\s*-->\s*(\d{1,3})")
|
||||
|
||||
|
||||
@_none_on_fail("altcoin_season_index")
|
||||
async def fetch_altcoin_season_index() -> dict:
|
||||
"""Compute the Altcoin Season Index from CoinGecko /coins/markets.
|
||||
"""Fetch the Altcoin Season Index from blockchaincenter.net.
|
||||
|
||||
Original blockchaincenter.net formula uses a 90-day window, but
|
||||
CoinGecko's /coins/markets `price_change_percentage` parameter only
|
||||
accepts 1h/24h/7d/14d/30d/200d/1y — 90d returns HTTP 400. We use 30d
|
||||
as the closest practical proxy. Long-horizon altseason (which 90d
|
||||
captures better) would need per-coin /market_chart calls — 50× the
|
||||
API budget for a marginal definition improvement.
|
||||
The site is Next.js SSR — the value is embedded in the initial HTML as a
|
||||
server-rendered text node. We parse it with a tight regex and fall back to
|
||||
None on any parse failure so the rest of the snapshot is unaffected.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=DEFAULT_TIMEOUT, headers=UA) as c:
|
||||
r = await c.get(
|
||||
"https://api.coingecko.com/api/v3/coins/markets",
|
||||
params={"vs_currency": "usd", "order": "market_cap_desc",
|
||||
"per_page": 60, "page": 1,
|
||||
"price_change_percentage": "30d"},
|
||||
)
|
||||
async with httpx.AsyncClient(
|
||||
timeout=DEFAULT_TIMEOUT,
|
||||
headers={**UA, "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"},
|
||||
follow_redirects=True,
|
||||
) as c:
|
||||
r = await c.get(_BCC_URL)
|
||||
r.raise_for_status()
|
||||
rows = r.json()
|
||||
html = r.text
|
||||
|
||||
# Drop stablecoins + wrapped, keep top 50 of the remainder.
|
||||
eligible = [
|
||||
row for row in rows
|
||||
if (row.get("symbol") or "").upper() not in _STABLE_OR_WRAPPED
|
||||
and row.get("price_change_percentage_30d_in_currency") is not None
|
||||
][:50]
|
||||
if len(eligible) < 30:
|
||||
return {"value": None, "raw": {"error": "insufficient eligible coins",
|
||||
"have": len(eligible)}}
|
||||
m = _BCC_RE.search(html)
|
||||
if not m:
|
||||
return {"value": None, "raw": {"error": "regex did not match", "url": _BCC_URL}}
|
||||
|
||||
btc_row = next((x for x in rows if x.get("symbol", "").upper() == "BTC"), None)
|
||||
btc_30d = btc_row.get("price_change_percentage_30d_in_currency") if btc_row else None
|
||||
if btc_30d is None:
|
||||
return {"value": None, "raw": {"error": "BTC 30d return missing"}}
|
||||
value = int(m.group(1))
|
||||
if not 0 <= value <= 100:
|
||||
return {"value": None, "raw": {"error": f"parsed value out of range: {value}"}}
|
||||
|
||||
n_outperform = sum(
|
||||
1 for row in eligible
|
||||
if (row["price_change_percentage_30d_in_currency"] or -999) > btc_30d
|
||||
)
|
||||
# Project the count over `len(eligible)` to a 0–100 scale.
|
||||
index = (n_outperform / len(eligible)) * 100
|
||||
return {
|
||||
"value": round(index, 1),
|
||||
"raw": {"n_outperform": n_outperform, "of": len(eligible),
|
||||
"btc_30d_pct": round(btc_30d, 2), "window": "30d"},
|
||||
"value": float(value),
|
||||
"raw": {"source": "blockchaincenter.net", "window": "90d", "parsed": value},
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -74,11 +74,16 @@ def _eth_btc_signal(v: Optional[float]) -> Optional[float]:
|
||||
|
||||
|
||||
def _stablecoin_supply_signal(v: Optional[float]) -> Optional[float]:
|
||||
"""Absolute supply tells us little day-over-day; we need the delta. Since
|
||||
this scorer sees only the snapshot, we treat presence as 0 and let the
|
||||
visual chart show the trend. Returns 0 if we have any value at all."""
|
||||
if v is None: return None
|
||||
return 0.0 # contribution = 0 until we wire in a trend lookup
|
||||
"""Absolute supply tells us little day-over-day; we need the delta, which
|
||||
this snapshot-only scorer doesn't have yet.
|
||||
|
||||
Return None (NOT 0.0) so this indicator is EXCLUDED from the weighted sum
|
||||
and its weight is renormalised away — exactly the "a dead indicator must
|
||||
not drag the score toward zero" rule stated in the module docstring.
|
||||
Returning 0.0 would keep its 0.05 weight in the denominator and silently
|
||||
compress every other indicator's contribution. Wire in a trend lookup to
|
||||
re-activate it."""
|
||||
return None
|
||||
|
||||
|
||||
def _etf_flow_signal(v: Optional[float]) -> Optional[float]:
|
||||
@@ -95,9 +100,10 @@ def _etf_flow_signal(v: Optional[float]) -> Optional[float]:
|
||||
|
||||
def _open_interest_signal(v: Optional[float]) -> Optional[float]:
|
||||
"""OI in isolation doesn't tell us direction — we'd need OI vs price
|
||||
correlation. Until we have a trend window, contribute 0."""
|
||||
if v is None: return None
|
||||
return 0.0
|
||||
correlation. Return None (NOT 0.0) until we have a trend window, so this
|
||||
indicator is excluded + its weight renormalised away rather than diluting
|
||||
every other indicator. (Same reasoning as _stablecoin_supply_signal.)"""
|
||||
return None
|
||||
|
||||
|
||||
# Weights (sum to 1.0 across all). When an indicator is missing, we drop its
|
||||
|
||||
@@ -94,7 +94,7 @@ def verify_signed_request(
|
||||
|
||||
# 1. Freshness
|
||||
now_ms = int(time.time() * 1000)
|
||||
if abs(now_ms - timestamp_ms) > MAX_SKEW_SECONDS * 1000:
|
||||
if timestamp_ms > now_ms + 30_000 or now_ms - timestamp_ms > MAX_SKEW_SECONDS * 1000: # allow 30s future drift for clock skew
|
||||
raise HTTPException(401, "Signed request expired or clock skew too large")
|
||||
|
||||
# 2. Recover signer
|
||||
@@ -144,3 +144,58 @@ def verify_signed_request_any(
|
||||
last_exc = exc
|
||||
if last_exc is not None:
|
||||
raise last_exc
|
||||
|
||||
|
||||
# ── Signed-read credential extraction (C3) ────────────────────────────────
|
||||
# Read endpoints historically took the signature as `?ts=&sig=` query params,
|
||||
# which leaks signatures into access logs / proxies / browser history. The
|
||||
# canonical transport is now the X-Sig-Ts / X-Sig-Sig HEADERS; the query
|
||||
# params are kept as a deprecated fallback so older clients keep working.
|
||||
from dataclasses import dataclass
|
||||
|
||||
from fastapi import Header, Query
|
||||
|
||||
|
||||
@dataclass
|
||||
class SignedReadCreds:
|
||||
ts: int
|
||||
sig: str
|
||||
|
||||
|
||||
def signed_read_creds(
|
||||
ts: Optional[int] = Query(default=None, deprecated=True,
|
||||
description="DEPRECATED — use X-Sig-Ts header"),
|
||||
sig: Optional[str] = Query(default=None, deprecated=True,
|
||||
description="DEPRECATED — use X-Sig-Sig header"),
|
||||
x_sig_ts: Optional[int] = Header(default=None, alias="X-Sig-Ts",
|
||||
description="Signed timestamp (ms)"),
|
||||
x_sig_sig: Optional[str] = Header(default=None, alias="X-Sig-Sig",
|
||||
description="EIP-191 signature"),
|
||||
) -> SignedReadCreds:
|
||||
"""FastAPI dependency: required signed-read credentials.
|
||||
|
||||
Headers win over query params when both are present."""
|
||||
t = x_sig_ts if x_sig_ts is not None else ts
|
||||
s = x_sig_sig if x_sig_sig else sig
|
||||
if t is None or not s:
|
||||
raise HTTPException(
|
||||
422, "Missing signed-read credentials (X-Sig-Ts / X-Sig-Sig headers)")
|
||||
return SignedReadCreds(ts=t, sig=s)
|
||||
|
||||
|
||||
def optional_signed_read_creds(
|
||||
# Legacy query names used by /telegram/{wallet}/status.
|
||||
timestamp: Optional[int] = Query(default=None, deprecated=True,
|
||||
description="DEPRECATED — use X-Sig-Ts header"),
|
||||
signature: Optional[str] = Query(default=None, deprecated=True,
|
||||
description="DEPRECATED — use X-Sig-Sig header"),
|
||||
x_sig_ts: Optional[int] = Header(default=None, alias="X-Sig-Ts"),
|
||||
x_sig_sig: Optional[str] = Header(default=None, alias="X-Sig-Sig"),
|
||||
) -> Optional[SignedReadCreds]:
|
||||
"""Like signed_read_creds but returns None when absent — for endpoints
|
||||
that serve a redacted response to unauthenticated callers."""
|
||||
t = x_sig_ts if x_sig_ts is not None else timestamp
|
||||
s = x_sig_sig if x_sig_sig else signature
|
||||
if t is None or not s:
|
||||
return None
|
||||
return SignedReadCreds(ts=t, sig=s)
|
||||
|
||||
+33
-11
@@ -22,11 +22,11 @@ Source → user-toggle mapping:
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import html
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import select, update
|
||||
|
||||
from app.config import settings
|
||||
@@ -100,13 +100,21 @@ def format_post(post: Post) -> str:
|
||||
asset = post.target_asset or "?"
|
||||
conf = post.ai_confidence or 0
|
||||
|
||||
# Heading: emoji + asset + direction + confidence
|
||||
head = f"{emoji} <b>{asset} · {sig}</b> · conf <b>{conf}</b>"
|
||||
# Heading: emoji + asset + direction + confidence.
|
||||
# asset comes from post.target_asset (scanner-written) — escape defensively;
|
||||
# sig/conf/src are controlled enums/ints/labels.
|
||||
head = f"{emoji} <b>{html.escape(asset)} · {sig}</b> · conf <b>{conf}</b>"
|
||||
sub = f"<i>{src}</i>"
|
||||
|
||||
# Truncate the RAW text first, THEN escape — escaping first could let a
|
||||
# 5-char entity (&) get sliced mid-sequence by the length cap.
|
||||
body = (post.text or "").strip()
|
||||
if len(body) > 600:
|
||||
body = body[:600].rstrip() + "…"
|
||||
# parse_mode=HTML: Trump posts routinely contain & < > ("Law & Order").
|
||||
# Without escaping, Telegram rejects the whole message (400 can't parse
|
||||
# entities) and every subscriber silently misses the alert.
|
||||
body = html.escape(body)
|
||||
|
||||
# Move size hint if present (BTC bottom & funding emit expected_move_pct)
|
||||
extra = ""
|
||||
@@ -157,6 +165,7 @@ def format_trump_mention(post: Post) -> str:
|
||||
body = (post.text or "").strip()
|
||||
if len(body) > 300:
|
||||
body = body[:300].rstrip() + "…"
|
||||
body = html.escape(body) # see format_post — escape user text for HTML mode
|
||||
|
||||
fe = (settings.frontend_url or "").rstrip("/")
|
||||
link = f'\n\n<a href="{fe}/en/trump">→ view on TrumpAlpha</a>' if fe else ""
|
||||
@@ -197,7 +206,7 @@ def format_public_post(post: Post) -> str:
|
||||
else:
|
||||
conf_label = "—"
|
||||
|
||||
head = f"{emoji} <b>{asset} · {sig}</b> · conf <b>{conf_label}</b>"
|
||||
head = f"{emoji} <b>{html.escape(asset)} · {sig}</b> · conf <b>{conf_label}</b>"
|
||||
sub = f"<i>{src}</i>"
|
||||
|
||||
body = (post.text or "").strip()
|
||||
@@ -212,6 +221,10 @@ def format_public_post(post: Post) -> str:
|
||||
reason = reason[:200].rstrip() + "…"
|
||||
body = reason
|
||||
|
||||
# Escape AFTER choosing text-vs-reason — both are user/AI content going
|
||||
# into an HTML-parse-mode message. (See format_post.)
|
||||
body = html.escape(body)
|
||||
|
||||
fe = (settings.frontend_url or "").rstrip("/")
|
||||
link = ""
|
||||
if fe:
|
||||
@@ -260,8 +273,8 @@ async def send_message(chat_id: int | str, text: str, *,
|
||||
if reply_markup is not None:
|
||||
payload["reply_markup"] = reply_markup
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10) as client:
|
||||
r = await client.post(url, json=payload)
|
||||
from app.services.http_client import get_client
|
||||
r = await get_client().post(url, json=payload, timeout=10)
|
||||
if r.status_code != 200:
|
||||
logger.warning("Telegram sendMessage failed chat=%s status=%d body=%s",
|
||||
chat_id, r.status_code, r.text[:200])
|
||||
@@ -288,8 +301,8 @@ async def edit_message(chat_id: int, message_id: int, text: str, *,
|
||||
if reply_markup is not None:
|
||||
payload["reply_markup"] = reply_markup
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10) as client:
|
||||
r = await client.post(url, json=payload)
|
||||
from app.services.http_client import get_client
|
||||
r = await get_client().post(url, json=payload, timeout=10)
|
||||
if r.status_code != 200:
|
||||
# Telegram returns 400 on "message is not modified" — harmless.
|
||||
if "message is not modified" not in r.text:
|
||||
@@ -317,8 +330,8 @@ async def answer_callback(callback_query_id: str, text: str = "",
|
||||
payload["text"] = text
|
||||
payload["show_alert"] = show_alert
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=10) as client:
|
||||
await client.post(url, json=payload)
|
||||
from app.services.http_client import get_client
|
||||
await get_client().post(url, json=payload, timeout=10)
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.debug("Telegram answerCallback exception: %s", exc)
|
||||
@@ -447,6 +460,13 @@ async def _dispatch(post_id: int) -> None:
|
||||
post_id, channel_id)
|
||||
|
||||
|
||||
# Strong references to in-flight dispatch tasks. asyncio.create_task() only
|
||||
# keeps a WEAK reference, so a fire-and-forget task can be garbage-collected
|
||||
# mid-await — "Task was destroyed but it is pending!" — silently dropping a
|
||||
# subscriber fan-out. We hold the task until it completes, then auto-discard.
|
||||
_dispatch_tasks: set[asyncio.Task] = set()
|
||||
|
||||
|
||||
def notify_signal(post: Post) -> None:
|
||||
"""Fire-and-forget. Schedules `_dispatch(post.id)` on the running loop
|
||||
and returns immediately. Safe to call from any async context — falls
|
||||
@@ -459,7 +479,9 @@ def notify_signal(post: Post) -> None:
|
||||
if not post or not post.id:
|
||||
return
|
||||
try:
|
||||
asyncio.create_task(_dispatch(post.id))
|
||||
task = asyncio.create_task(_dispatch(post.id))
|
||||
_dispatch_tasks.add(task)
|
||||
task.add_done_callback(_dispatch_tasks.discard)
|
||||
except RuntimeError:
|
||||
# No running loop — extremely unusual in our FastAPI context.
|
||||
logger.warning("notify_signal: no running event loop, skipping post=%s", post.id)
|
||||
|
||||
@@ -393,6 +393,15 @@ async def _cmd_status(chat_id: int) -> None:
|
||||
b = (await db.execute(
|
||||
select(TelegramBinding).where(TelegramBinding.chat_id == chat_id)
|
||||
)).scalar_one_or_none()
|
||||
# Also fetch the subscription if the binding is wallet-linked, so we
|
||||
# can show auto_trade + paper_mode — the two most important trading
|
||||
# states that were previously invisible in /status.
|
||||
sub = None
|
||||
if b and b.wallet_address:
|
||||
sub = (await db.execute(
|
||||
select(Subscription).where(Subscription.wallet_address == b.wallet_address)
|
||||
)).scalar_one_or_none()
|
||||
|
||||
if not b:
|
||||
await send_message(chat_id,
|
||||
"Not subscribed yet. Send /start to begin (no wallet required).")
|
||||
@@ -418,6 +427,23 @@ async def _cmd_status(chat_id: int) -> None:
|
||||
f"🟢 ON @ {b.digest_hour_utc:02d}:00 UTC"
|
||||
if b.digest_enabled else "🔴 OFF"
|
||||
)
|
||||
|
||||
# Pro-only trading state block — only shown when wallet-linked.
|
||||
trading_block = ""
|
||||
if sub:
|
||||
auto = "🟢 ON" if sub.auto_trade else "🔴 OFF"
|
||||
mode = "📝 Paper" if sub.paper_mode else "💰 Live"
|
||||
cb_line = ""
|
||||
if sub.circuit_breaker_tripped_at:
|
||||
cb_line = f"\n🚨 Circuit breaker: tripped ({sub.circuit_breaker_reason or 'risk limit'})"
|
||||
else:
|
||||
cb_line = "\n✓ Circuit breaker: clear"
|
||||
trading_block = (
|
||||
f"\n\n<b>— Auto-Trader —</b>\n"
|
||||
f"Auto-Trade: {auto}\n"
|
||||
f"Mode: {mode}{cb_line}"
|
||||
)
|
||||
|
||||
await send_message(
|
||||
chat_id,
|
||||
f"📡 <b>Status</b>\n\n"
|
||||
@@ -425,7 +451,8 @@ async def _cmd_status(chat_id: int) -> None:
|
||||
f"Alerts: {on}\n"
|
||||
f"Sources: {src_line}\n"
|
||||
f"Min confidence: {b.min_confidence}{mute}\n"
|
||||
f"Daily brief: {digest_state}\n\n"
|
||||
f"Daily brief: {digest_state}"
|
||||
f"{trading_block}\n\n"
|
||||
f"Sent: {b.total_alerts_sent} · Failed: {b.total_alerts_failed}\n\n"
|
||||
f"Toggle anything with /trump /btc /funding /kol /conf /quiet "
|
||||
f"/digest /digest_time — send /help for the full list.",
|
||||
@@ -787,9 +814,17 @@ async def _handle_callback(cb: dict) -> None:
|
||||
await answer_callback(cb_id)
|
||||
return
|
||||
if sub == "dup":
|
||||
await answer_callback(cb_id,
|
||||
"This position is already managed.",
|
||||
show_alert=True)
|
||||
# Asset is already adopted — tell the user how to release,
|
||||
# and also edit the picker message so the UI doesn't stay stuck.
|
||||
asset_dup = parts[2] if len(parts) > 2 else "this position"
|
||||
await edit_message(
|
||||
chat_id, msg_id,
|
||||
f"✅ <b>{asset_dup}</b> is already under bot management.\n\n"
|
||||
f"To stop managing it and take back manual control:\n"
|
||||
f"<code>/release</code> — pick from list\n"
|
||||
f"or send <code>/release <trade_id></code> directly."
|
||||
)
|
||||
await answer_callback(cb_id, "Already managed — see message.", show_alert=False)
|
||||
return
|
||||
if sub == "pick" and len(parts) == 3:
|
||||
asset = parts[2]
|
||||
@@ -963,6 +998,30 @@ async def run_bot_loop() -> None:
|
||||
return
|
||||
|
||||
logger.info("Telegram bot loop starting (long-poll mode).")
|
||||
|
||||
# ── Startup drain ────────────────────────────────────────────────────
|
||||
# On restart, Telegram may replay up to 24h of unacknowledged updates.
|
||||
# The most dangerous replays are stale /adopt callback_query items that
|
||||
# would re-fire an adoption the user already dealt with. Fix: pull all
|
||||
# pending updates with timeout=0 (non-blocking), advance offset past
|
||||
# them without processing, then start the real loop fresh.
|
||||
try:
|
||||
drain_url = TG_API.format(token=token, method="getUpdates")
|
||||
from app.services.http_client import get_client
|
||||
r = await get_client().get(drain_url, params={"timeout": 0, "limit": 100}, timeout=10)
|
||||
if r.status_code == 200:
|
||||
pending = r.json().get("result", [])
|
||||
if pending:
|
||||
drain_offset = pending[-1]["update_id"] + 1
|
||||
# ACK by sending offset back — Telegram won't re-deliver these.
|
||||
await get_client().get(drain_url, params={"timeout": 0, "offset": drain_offset}, timeout=10)
|
||||
logger.info(
|
||||
"Startup drain: skipped %d stale update(s), offset now %d",
|
||||
len(pending), drain_offset,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("Startup drain failed (non-fatal): %s", exc)
|
||||
|
||||
offset: Optional[int] = None
|
||||
backoff = 1.0
|
||||
|
||||
@@ -972,8 +1031,8 @@ async def run_bot_loop() -> None:
|
||||
params: dict = {"timeout": 25}
|
||||
if offset is not None:
|
||||
params["offset"] = offset
|
||||
async with httpx.AsyncClient(timeout=35) as client:
|
||||
r = await client.get(url, params=params)
|
||||
from app.services.http_client import get_client
|
||||
r = await get_client().get(url, params=params, timeout=35)
|
||||
if r.status_code != 200:
|
||||
logger.warning("Telegram getUpdates HTTP %d: %s", r.status_code, r.text[:200])
|
||||
await asyncio.sleep(backoff)
|
||||
|
||||
@@ -29,6 +29,7 @@ so a coalesced cron or worker restart can't double-send within a day.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import html
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timedelta, timezone
|
||||
@@ -138,26 +139,43 @@ async def build_global_digest(now: Optional[datetime] = None) -> GlobalDigest:
|
||||
)
|
||||
|
||||
# ── KOL: last 24h posts + divergences ────────────────────────────
|
||||
# Exclude twitter noise (gm / RT / jokes) from the count. Substack
|
||||
# posts have tier=NULL (kept); twitter signal posts have tier!='noise'
|
||||
# (kept); twitter noise is dropped. Without this filter a KOL's gm/RT
|
||||
# spam inflates the daily "N KOL posts" stat. Mirrors the /kol/posts
|
||||
# signals_only filter and the frontend.
|
||||
kol_posts = (await db.execute(
|
||||
select(KolPost).where(KolPost.published_at >= cutoff_24h)
|
||||
select(KolPost).where(
|
||||
KolPost.published_at >= cutoff_24h,
|
||||
(KolPost.tier.is_(None)) | (KolPost.tier != "noise"),
|
||||
)
|
||||
)).scalars().all()
|
||||
# action lives inside tickers_json; easier: classify on summary text.
|
||||
# The structured action is on KolDivergence rows; for the bare count
|
||||
# we just split posts by whether divergence rows reference them.
|
||||
# Count bullish/bearish/neutral at POST level, not divergence-pair level.
|
||||
# A single post can match multiple on-chain events (one per ticker/wallet),
|
||||
# producing several KolDivergence rows. Counting rows would inflate the
|
||||
# numbers — a post mentioning 3 tickers was previously counted 3× (bug).
|
||||
bullish = bearish = neutral = 0
|
||||
divergence_rows = (await db.execute(
|
||||
select(KolDivergence)
|
||||
.where(KolDivergence.post_at >= cutoff_24h)
|
||||
.order_by(KolDivergence.created_at.desc())
|
||||
)).scalars().all()
|
||||
# Deduplicate by post_id; for posts with mixed directions take "long"
|
||||
# first (a bullish call that also hedges short is net bullish).
|
||||
post_direction: dict[int, str] = {}
|
||||
for d in divergence_rows:
|
||||
if d.direction == "long":
|
||||
if d.post_id not in post_direction:
|
||||
post_direction[d.post_id] = d.direction
|
||||
elif d.direction == "long":
|
||||
post_direction[d.post_id] = "long" # long overrides neutral
|
||||
for direction in post_direction.values():
|
||||
if direction == "long":
|
||||
bullish += 1
|
||||
elif d.direction == "short":
|
||||
elif direction == "short":
|
||||
bearish += 1
|
||||
else:
|
||||
neutral += 1
|
||||
# Posts not yet classified into divergences just count as neutral.
|
||||
# Posts not yet cross-referenced with on-chain data count as neutral.
|
||||
neutral += max(0, len(kol_posts) - (bullish + bearish + neutral))
|
||||
|
||||
divergences_24h = sum(
|
||||
@@ -168,7 +186,9 @@ async def build_global_digest(now: Optional[datetime] = None) -> GlobalDigest:
|
||||
if d.signal_type != "divergence":
|
||||
continue
|
||||
# "Hayes publicly bullish BTC, on-chain selling"
|
||||
sample_divergence = (
|
||||
# Escape — ticker is AI-extracted (kol_analysis) so not guaranteed
|
||||
# HTML-safe; the digest is sent with parse_mode=HTML.
|
||||
sample_divergence = html.escape(
|
||||
f"{d.handle} publicly {d.post_action} {d.ticker}, "
|
||||
f"on-chain {d.onchain_action}"
|
||||
)
|
||||
|
||||
@@ -158,6 +158,19 @@ def register_trade(
|
||||
and not derisk_ladder):
|
||||
return
|
||||
|
||||
# Warn when the price feed doesn't cover this asset — TP/SL/trailing
|
||||
# stops will NEVER fire; only max_hold will exit the position.
|
||||
from app.services.binance import ASSET_MAP
|
||||
from app.services.hl_price_feed import HL_PRICE_ASSETS
|
||||
covered = set(ASSET_MAP.values()) | HL_PRICE_ASSETS
|
||||
if asset not in covered:
|
||||
logger.error(
|
||||
"PRICE FEED GAP: trade %d asset=%s has no price coverage. "
|
||||
"TP/SL/trailing-stop will NOT trigger — only max_hold protects "
|
||||
"this position. Add %susdt to ASSET_MAP in binance.py.",
|
||||
trade_id, asset, asset.lower(),
|
||||
)
|
||||
|
||||
_watched[trade_id] = WatchedTrade(
|
||||
trade_id=trade_id, wallet=wallet, api_key=api_key, leverage=leverage,
|
||||
asset=asset, side=side, entry_price=entry_price,
|
||||
|
||||
@@ -35,7 +35,7 @@ from app.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ANALYSIS_VERSION = "x-v1"
|
||||
ANALYSIS_VERSION = "x-v2" # v2: tone-is-not-content rule + meme calibration example
|
||||
ANTHROPIC_MODEL = "claude-haiku-4-5-20251001"
|
||||
|
||||
_anthropic_client = None
|
||||
@@ -89,6 +89,13 @@ ALL of the above → tier: "noise", tickers: []
|
||||
Only extract signals when the post contains CLEAR, EXPLICIT information
|
||||
about what the KOL is doing or believes RIGHT NOW. Ambiguity → noise.
|
||||
|
||||
TONE IS NOT CONTENT. A joke, meme, celebratory, or emoji-spam tone
|
||||
("Arise Chikun!", "Yachtzee", "Meow", "😘😘😘", "WAGMI lfg") does NOT make a
|
||||
post noise when it still carries an EXPLICIT ticker + direction. Score the
|
||||
claim, ignore the wrapper. "$WLD initiate bull market 😘😘😘" is a directional
|
||||
bullish call on WLD — NOT noise. Only drop to noise when the ticker or the
|
||||
direction is genuinely absent/vague, never merely because the wording is silly.
|
||||
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
THREE SIGNAL TIERS
|
||||
═══════════════════════════════════════════════════════════════════════
|
||||
@@ -194,6 +201,11 @@ POST: "ETH dominance will crush alts this cycle. The flippening is real."
|
||||
→ tier: "directional", action: "bullish", asset: ETH, conviction: 0.52,
|
||||
timeframe: "months"
|
||||
|
||||
POST: "Arise Chikun! $WLD initiate bull market. Yachtzee 😘😘😘😘"
|
||||
→ tier: "directional", action: "bullish", asset: WLD, conviction: 0.7,
|
||||
timeframe: "immediate" (meme/celebration tone does NOT override the
|
||||
explicit "$WLD initiate bull market" call — score the claim, not the vibe)
|
||||
|
||||
[NOISE — output noise, empty tickers]
|
||||
|
||||
POST: "gm everyone 🌅"
|
||||
|
||||
@@ -0,0 +1,306 @@
|
||||
"""
|
||||
X (Twitter) auto-poster — viral "live prediction" tweets for actionable signals.
|
||||
|
||||
Mirrors the design of telegram.py: fire-and-forget, fully degradable. If X
|
||||
creds are missing or `x_enabled` is False, every entry point becomes a no-op
|
||||
(dry-run logs the tweet text but sends nothing). A signal MUST never be blocked
|
||||
by an X API failure.
|
||||
|
||||
notify_x_signal(post) ← call right after notify_signal(post) in the ingest path
|
||||
|
||||
Flow per actionable signal:
|
||||
1. Immediately post the PREDICTION tweet (entry / TP / SL, hard time window).
|
||||
2. Schedule a FOLLOW-UP tweet `x_followup_minutes` later that reports the
|
||||
actual move (price_impact_m15) — the "told you" brag that drives the link.
|
||||
|
||||
Auth: OAuth 1.0a User Context, signed by hand with stdlib hmac/hashlib (no extra
|
||||
dependency — httpx is already vendored). Posting uses POST /2/tweets.
|
||||
|
||||
Tone: deliberately provocative / "黑红" — invites ratios, taunts CT, no soft CTA.
|
||||
All templates are hard-capped at 280 chars (no X Premium = 280 limit).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import secrets
|
||||
import time
|
||||
import urllib.parse
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.config import settings
|
||||
from app.database import AsyncSessionLocal as async_session
|
||||
from app.models import Post
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TWEET_URL = "https://api.twitter.com/2/tweets"
|
||||
MAX_TWEET = 280
|
||||
|
||||
# In-memory daily cap counter. Single-process by design (see main.py leader
|
||||
# guard), so a plain dict is safe. Resets when the UTC date rolls over.
|
||||
_sent_today = 0
|
||||
_sent_date: Optional[str] = None
|
||||
|
||||
|
||||
# ── credential / enablement gate ──────────────────────────────────────────
|
||||
|
||||
|
||||
def _creds_ok() -> bool:
|
||||
return bool(
|
||||
settings.x_api_key
|
||||
and settings.x_api_secret
|
||||
and settings.x_access_token
|
||||
and settings.x_access_secret
|
||||
)
|
||||
|
||||
|
||||
def _under_daily_cap() -> bool:
|
||||
"""True if we still have budget today. Increments are done by _record_sent."""
|
||||
global _sent_today, _sent_date
|
||||
today = datetime.now(timezone.utc).strftime("%Y-%m-%d")
|
||||
if today != _sent_date:
|
||||
_sent_date = today
|
||||
_sent_today = 0
|
||||
return _sent_today < settings.x_daily_cap
|
||||
|
||||
|
||||
def _record_sent() -> None:
|
||||
global _sent_today
|
||||
_sent_today += 1
|
||||
|
||||
|
||||
# ── OAuth 1.0a signing ─────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _percent(s: str) -> str:
|
||||
# RFC 3986 — OAuth requires ~ unescaped and everything else strict.
|
||||
return urllib.parse.quote(str(s), safe="~")
|
||||
|
||||
|
||||
def _oauth_header(method: str, url: str) -> str:
|
||||
"""Build the OAuth 1.0a Authorization header for a JSON-body request.
|
||||
|
||||
For POST /2/tweets the body is JSON (not form-encoded), so per the OAuth
|
||||
spec the body params are NOT part of the signature base string — only the
|
||||
oauth_* params are. This is the documented X v2 behaviour.
|
||||
"""
|
||||
oauth = {
|
||||
"oauth_consumer_key": settings.x_api_key,
|
||||
"oauth_nonce": secrets.token_hex(16),
|
||||
"oauth_signature_method": "HMAC-SHA1",
|
||||
"oauth_timestamp": str(int(time.time())),
|
||||
"oauth_token": settings.x_access_token,
|
||||
"oauth_version": "1.0",
|
||||
}
|
||||
# Signature base string
|
||||
param_str = "&".join(
|
||||
f"{_percent(k)}={_percent(v)}" for k, v in sorted(oauth.items())
|
||||
)
|
||||
base = "&".join([method.upper(), _percent(url), _percent(param_str)])
|
||||
signing_key = f"{_percent(settings.x_api_secret)}&{_percent(settings.x_access_secret)}"
|
||||
digest = hmac.new(
|
||||
signing_key.encode(), base.encode(), hashlib.sha1
|
||||
).digest()
|
||||
oauth["oauth_signature"] = base64.b64encode(digest).decode()
|
||||
|
||||
header = "OAuth " + ", ".join(
|
||||
f'{_percent(k)}="{_percent(v)}"' for k, v in sorted(oauth.items())
|
||||
)
|
||||
return header
|
||||
|
||||
|
||||
async def _post_tweet(text: str, reply_to: Optional[str] = None) -> Optional[str]:
|
||||
"""POST a single tweet. Returns the new tweet id, or None on failure / dry-run.
|
||||
|
||||
`reply_to` chains the follow-up under the prediction tweet (a thread), which
|
||||
reads better and keeps the brag attached to the original call.
|
||||
"""
|
||||
text = text[:MAX_TWEET]
|
||||
|
||||
if not settings.x_enabled:
|
||||
logger.info("[x_poster] DRY-RUN (x_enabled=False), would tweet:\n%s", text)
|
||||
return None
|
||||
if not _creds_ok():
|
||||
logger.warning("[x_poster] missing X creds — skipping tweet")
|
||||
return None
|
||||
if not _under_daily_cap():
|
||||
logger.warning("[x_poster] daily cap (%d) reached — skipping tweet",
|
||||
settings.x_daily_cap)
|
||||
return None
|
||||
|
||||
payload: dict = {"text": text}
|
||||
if reply_to:
|
||||
payload["reply"] = {"in_reply_to_tweet_id": reply_to}
|
||||
|
||||
headers = {
|
||||
"Authorization": _oauth_header("POST", TWEET_URL),
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
try:
|
||||
from app.services.http_client import get_client
|
||||
resp = await get_client().post(TWEET_URL, json=payload, headers=headers, timeout=10)
|
||||
if resp.status_code in (200, 201):
|
||||
_record_sent()
|
||||
tid = resp.json().get("data", {}).get("id")
|
||||
logger.info("[x_poster] tweeted id=%s (%d/%d today)",
|
||||
tid, _sent_today, settings.x_daily_cap)
|
||||
return tid
|
||||
logger.warning("[x_poster] tweet failed %s: %s",
|
||||
resp.status_code, resp.text[:300])
|
||||
return None
|
||||
except Exception as exc: # noqa: BLE001 — never let X break ingestion
|
||||
logger.warning("[x_poster] tweet exception: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
# ── win-rate helper (for the brag in the follow-up) ────────────────────────
|
||||
|
||||
|
||||
async def _recent_hit_rate(db) -> tuple[int, Optional[float]]:
|
||||
"""(count, hit_rate_pct) over recent scored directional posts.
|
||||
|
||||
A 'hit' = the realised 15-min move agreed with the call direction
|
||||
(buy → up, short → down). Cheap, audit-friendly, and honest: it reads the
|
||||
same price_impact_m15 we already record. Returns (n, None) if too few.
|
||||
"""
|
||||
rows = (
|
||||
await db.execute(
|
||||
select(Post.signal, Post.price_impact_m15)
|
||||
.where(
|
||||
Post.signal.in_(("buy", "short")),
|
||||
Post.price_impact_m15.isnot(None),
|
||||
)
|
||||
.order_by(Post.id.desc())
|
||||
.limit(200)
|
||||
)
|
||||
).all()
|
||||
n = len(rows)
|
||||
if n < 5:
|
||||
return n, None
|
||||
hits = 0
|
||||
for sig, move in rows:
|
||||
if move is None:
|
||||
continue
|
||||
if (sig == "buy" and move > 0) or (sig == "short" and move < 0):
|
||||
hits += 1
|
||||
return n, round(100.0 * hits / n, 1)
|
||||
|
||||
|
||||
# ── tweet templates (≤280, 黑红 tone) ───────────────────────────────────────
|
||||
|
||||
|
||||
def _dir_word(signal: Optional[str]) -> str:
|
||||
return "LONG" if signal == "buy" else "SHORT" if signal == "short" else "WATCH"
|
||||
|
||||
|
||||
def format_prediction(post: Post) -> str:
|
||||
"""The instant 'Trump just posted, here's my call' tweet."""
|
||||
asset = (post.target_asset or "BTC").upper()
|
||||
d = _dir_word(post.signal)
|
||||
entry = post.price_at_post
|
||||
mins = settings.x_followup_minutes
|
||||
|
||||
entry_line = f"Entry ~${entry:,.0f}\n" if entry else ""
|
||||
# Hard, falsifiable, time-boxed → invites screenshots and ratios.
|
||||
text = (
|
||||
f"Trump just posted. 🟠\n\n"
|
||||
f"{asset}: {d} — moves within {mins} min.\n"
|
||||
f"{entry_line}"
|
||||
f"\nScreenshot this. Wrong? Ratio me.\n"
|
||||
f"Right? You already know. ⏰"
|
||||
)
|
||||
return text[:MAX_TWEET]
|
||||
|
||||
|
||||
def format_followup(post: Post, move_pct: Optional[float],
|
||||
n: int, hit_rate: Optional[float]) -> str:
|
||||
"""The '15 min later, told you' brag. Drives the only link we ever post."""
|
||||
asset = (post.target_asset or "BTC").upper()
|
||||
mins = settings.x_followup_minutes
|
||||
link = (settings.x_link_url or settings.frontend_url or "").rstrip("/")
|
||||
|
||||
if move_pct is None:
|
||||
result = f"{asset}: still cooking. Check the board 👇"
|
||||
else:
|
||||
arrow = "✅" if (
|
||||
(post.signal == "buy" and move_pct > 0)
|
||||
or (post.signal == "short" and move_pct < 0)
|
||||
) else "🤡"
|
||||
result = f"{asset}: {move_pct:+.1f}% {arrow}"
|
||||
|
||||
rate_line = ""
|
||||
if hit_rate is not None:
|
||||
rate_line = f"\n{hit_rate:.0f}% hit rate over {n} calls. No CT badge needed."
|
||||
|
||||
tail = f"\n{link}" if link else ""
|
||||
text = f"{mins} min later.\n\n{result}{rate_line}\n\nBookmarked yet?{tail}"
|
||||
return text[:MAX_TWEET]
|
||||
|
||||
|
||||
# ── dispatch ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def _x_dispatch(post_id: int) -> None:
|
||||
"""Post the prediction now, then schedule the follow-up brag."""
|
||||
async with async_session() as db:
|
||||
post = await db.get(Post, post_id)
|
||||
if not post:
|
||||
return
|
||||
# Only post Trump-sourced actionable calls — that's the viral hook.
|
||||
# (BTC/funding scanner signals are slow-burn; they don't fit the
|
||||
# "Trump just posted, watch this" format.)
|
||||
if post.source != "truth" or post.signal not in ("buy", "short"):
|
||||
return
|
||||
if post.ai_confidence and post.ai_confidence < settings.x_min_confidence:
|
||||
logger.info("[x_poster] post=%d conf %s < %d — skipping",
|
||||
post_id, post.ai_confidence, settings.x_min_confidence)
|
||||
return
|
||||
|
||||
pred_text = format_prediction(post)
|
||||
|
||||
tweet_id = await _post_tweet(pred_text)
|
||||
|
||||
# Schedule the follow-up. Fire-and-forget asyncio task (single process by
|
||||
# design). A restart within the window loses the pending follow-up — that's
|
||||
# acceptable for a marketing post; the signal itself is already safe.
|
||||
delay = max(0, settings.x_followup_minutes) * 60
|
||||
asyncio.create_task(_followup_later(post_id, tweet_id, delay))
|
||||
|
||||
|
||||
async def _followup_later(post_id: int, reply_to: Optional[str], delay: int) -> None:
|
||||
try:
|
||||
await asyncio.sleep(delay)
|
||||
async with async_session() as db:
|
||||
post = await db.get(Post, post_id)
|
||||
if not post:
|
||||
return
|
||||
move = post.price_impact_m15
|
||||
n, rate = await _recent_hit_rate(db)
|
||||
text = format_followup(post, move, n, rate)
|
||||
await _post_tweet(text, reply_to=reply_to)
|
||||
except asyncio.CancelledError: # shutdown — drop quietly
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("[x_poster] follow-up failed post=%d: %s", post_id, exc)
|
||||
|
||||
|
||||
def notify_x_signal(post: Post) -> None:
|
||||
"""Fire-and-forget entry point. Mirrors telegram.notify_signal. Safe to call
|
||||
unconditionally — degrades to a no-op when X is disabled/unconfigured."""
|
||||
if not _creds_ok() and not settings.x_enabled:
|
||||
# Neither configured nor in dry-run intent → silent skip.
|
||||
return
|
||||
if not post or not post.id:
|
||||
return
|
||||
try:
|
||||
asyncio.create_task(_x_dispatch(post.id))
|
||||
except RuntimeError:
|
||||
logger.warning("[x_poster] no running event loop, skipping post=%s", post.id)
|
||||
@@ -8,6 +8,7 @@ alembic==1.16.5
|
||||
pydantic==2.13.2
|
||||
pydantic-settings==2.11.0
|
||||
httpx==0.28.1
|
||||
h2==4.3.0 # enables httpx http2=True in kol_substack (Glassnode etc. 403 on HTTP/1.1)
|
||||
feedparser==6.0.12
|
||||
websockets==15.0.1
|
||||
websocket-client==1.9.0
|
||||
@@ -23,3 +24,4 @@ apscheduler==3.11.2
|
||||
python-dotenv==1.2.1
|
||||
cryptography==46.0.7
|
||||
slowapi==0.1.9
|
||||
packaging
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"""
|
||||
Re-encrypt all stored HL API keys to the current enc:v2 format (H4 fix).
|
||||
|
||||
Upgrades, in place:
|
||||
* enc:v1 blobs (legacy unsalted-SHA256 KEK derivation)
|
||||
* plaintext rows (pre-encryption era)
|
||||
to enc:v2 (PBKDF2-HMAC-SHA256, per-blob salt). Rows already in enc:v2 are
|
||||
skipped — the script is idempotent and safe to re-run.
|
||||
|
||||
KEK rotation: set OLD_ENCRYPTION_KEY in the environment to decrypt with the
|
||||
old KEK while encrypting with the current ENCRYPTION_KEY.
|
||||
|
||||
Usage (BACK UP THE DB FIRST):
|
||||
DATABASE_URL=<prod-url> python scripts/reencrypt_keys.py [--dry-run]
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.models import Subscription
|
||||
from app.services import crypto
|
||||
|
||||
|
||||
async def main(dry_run: bool) -> None:
|
||||
old_kek = os.environ.get("OLD_ENCRYPTION_KEY")
|
||||
if old_kek:
|
||||
# Decrypt with the old KEK, encrypt with the new one. Swapping the
|
||||
# setting around each call keeps crypto.py's caching simple.
|
||||
print("KEK rotation mode: decrypting with OLD_ENCRYPTION_KEY")
|
||||
|
||||
upgraded = skipped = failed = 0
|
||||
async with AsyncSessionLocal() as db:
|
||||
rows = (await db.execute(
|
||||
select(Subscription).where(Subscription.hl_api_key.is_not(None))
|
||||
)).scalars().all()
|
||||
print(f"{len(rows)} subscription(s) with a stored HL key")
|
||||
|
||||
for sub in rows:
|
||||
stored = sub.hl_api_key
|
||||
if crypto.is_current_format(stored) and not old_kek:
|
||||
skipped += 1
|
||||
continue
|
||||
try:
|
||||
if old_kek:
|
||||
current = crypto.settings.encryption_key
|
||||
crypto.settings.encryption_key = old_kek
|
||||
crypto._fernet_v1 = None
|
||||
crypto._fernet_v2_cache.clear()
|
||||
try:
|
||||
plaintext = crypto.decrypt_api_key(stored)
|
||||
finally:
|
||||
crypto.settings.encryption_key = current
|
||||
crypto._fernet_v1 = None
|
||||
crypto._fernet_v2_cache.clear()
|
||||
crypto._encrypt_salt = None
|
||||
else:
|
||||
plaintext = crypto.decrypt_api_key(stored)
|
||||
new_blob = crypto.encrypt_api_key(plaintext)
|
||||
# Round-trip check before touching the row.
|
||||
assert crypto.decrypt_api_key(new_blob) == plaintext
|
||||
if not dry_run:
|
||||
sub.hl_api_key = new_blob
|
||||
upgraded += 1
|
||||
print(f" {sub.wallet_address}: upgraded"
|
||||
f"{' (dry-run)' if dry_run else ''}")
|
||||
except Exception as exc:
|
||||
failed += 1
|
||||
print(f" {sub.wallet_address}: FAILED — {type(exc).__name__}: {exc}")
|
||||
|
||||
if not dry_run and upgraded:
|
||||
await db.commit()
|
||||
print("Committed.")
|
||||
|
||||
print(f"Done: upgraded={upgraded} skipped(already v2)={skipped} failed={failed}")
|
||||
if failed:
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("--dry-run", action="store_true",
|
||||
help="report what would change without writing")
|
||||
args = ap.parse_args()
|
||||
asyncio.run(main(args.dry_run))
|
||||
@@ -155,7 +155,7 @@ def test_adoption_error_codes_used_by_callers_are_all_defined():
|
||||
# These are the codes referenced from app/api/positions.py +
|
||||
# app/services/telegram_bot.py. Update both sites if you add one.
|
||||
promised_codes = {
|
||||
"no_subscription", "no_hl_key", "paper_mode",
|
||||
"no_subscription", "no_hl_key", "paper_mode", "macro_disabled",
|
||||
"key_decrypt_failed", "hl_read_failed",
|
||||
"bad_mode", "circuit_breaker",
|
||||
"already_adopted", "concurrency_cap",
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
"""enc:v2 envelope encryption (H4): roundtrip, legacy v1 compat, format checks."""
|
||||
import base64
|
||||
import hashlib
|
||||
|
||||
import pytest
|
||||
from cryptography.fernet import Fernet
|
||||
|
||||
from app.services import crypto
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _kek(monkeypatch):
|
||||
monkeypatch.setattr(crypto.settings, "encryption_key", "k" * 64)
|
||||
# Reset module caches so each test derives from the patched KEK.
|
||||
crypto._fernet_v1 = None
|
||||
crypto._fernet_v2_cache.clear()
|
||||
crypto._encrypt_salt = None
|
||||
yield
|
||||
crypto._fernet_v1 = None
|
||||
crypto._fernet_v2_cache.clear()
|
||||
crypto._encrypt_salt = None
|
||||
|
||||
|
||||
def test_v2_roundtrip():
|
||||
blob = crypto.encrypt_api_key("0x" + "ab" * 32)
|
||||
assert blob.startswith(crypto.ENC_PREFIX_V2)
|
||||
assert crypto.decrypt_api_key(blob) == "0x" + "ab" * 32
|
||||
assert crypto.is_current_format(blob)
|
||||
# Fits the hl_api_key String(256) column.
|
||||
assert len(blob) <= 256
|
||||
|
||||
|
||||
def test_v1_blob_still_decrypts():
|
||||
# Build a v1 blob exactly the way the legacy code did.
|
||||
digest = hashlib.sha256(("k" * 64).encode()).digest()
|
||||
f = Fernet(base64.urlsafe_b64encode(digest))
|
||||
blob = crypto.ENC_PREFIX_V1 + f.encrypt(b"secret-key").decode()
|
||||
assert not crypto.is_current_format(blob)
|
||||
assert crypto.decrypt_api_key(blob) == "secret-key"
|
||||
|
||||
|
||||
def test_wrong_kek_raises(monkeypatch):
|
||||
blob = crypto.encrypt_api_key("topsecret")
|
||||
monkeypatch.setattr(crypto.settings, "encryption_key", "x" * 64)
|
||||
crypto._fernet_v2_cache.clear()
|
||||
with pytest.raises(RuntimeError):
|
||||
crypto.decrypt_api_key(blob)
|
||||
|
||||
|
||||
def test_malformed_v2_raises():
|
||||
with pytest.raises(RuntimeError):
|
||||
crypto.decrypt_api_key(crypto.ENC_PREFIX_V2 + "no-salt-separator")
|
||||
|
||||
|
||||
def test_plaintext_refused_in_production(monkeypatch):
|
||||
monkeypatch.setattr(crypto.settings, "environment", "production")
|
||||
with pytest.raises(RuntimeError):
|
||||
crypto.decrypt_api_key("raw-plaintext-key")
|
||||
@@ -0,0 +1,38 @@
|
||||
"""Unit tests for kol_analysis._derive_tier — the non-Twitter tier derivation
|
||||
that lets blog/substack/podcast posts get the same trade_signal/directional/
|
||||
noise tiers (and SIGNAL/VIEW badges + "Signals only" filter) as Twitter posts.
|
||||
"""
|
||||
|
||||
from app.services.kol_analysis import _derive_tier
|
||||
|
||||
|
||||
def _tk(action="mention", conviction=0.0):
|
||||
return {"action": action, "conviction": conviction}
|
||||
|
||||
|
||||
def test_no_tickers_is_noise():
|
||||
assert _derive_tier([], 0.0) == "noise"
|
||||
|
||||
|
||||
def test_only_mentions_is_noise():
|
||||
assert _derive_tier([_tk("mention", 0.9)], 0.1) == "noise"
|
||||
|
||||
|
||||
def test_high_conviction_directional_is_trade_signal():
|
||||
assert _derive_tier([_tk("buy", 0.7)], 0.0) == "trade_signal"
|
||||
|
||||
|
||||
def test_strong_divergence_is_trade_signal_even_without_ticker():
|
||||
assert _derive_tier([], 0.8) == "trade_signal"
|
||||
|
||||
|
||||
def test_low_conviction_directional_is_directional():
|
||||
assert _derive_tier([_tk("bullish", 0.4)], 0.0) == "directional"
|
||||
|
||||
|
||||
def test_moderate_divergence_is_directional():
|
||||
assert _derive_tier([_tk("mention", 0.0)], 0.55) == "directional"
|
||||
|
||||
|
||||
def test_boundary_conviction_0_6_is_trade_signal():
|
||||
assert _derive_tier([_tk("sell", 0.6)], 0.0) == "trade_signal"
|
||||
@@ -0,0 +1,272 @@
|
||||
"""Tests for the X (Twitter) KOL ingester (kol_x).
|
||||
|
||||
The twitterapi.io fetch and the AI scorer are mocked so we exercise the
|
||||
storage / dedup / mapping logic deterministically — no network, no AI spend.
|
||||
|
||||
The contract these lock down:
|
||||
- tweets land as KolPost(source="twitter")
|
||||
- kol_handle is the CANONICAL handle (joins to KolWallet), not the X username
|
||||
- bare retweets are skipped before scoring
|
||||
- tickers_json keeps the {ticker, action, conviction} shape kol_divergence reads
|
||||
- tier / post_type / talks_vs_trades_flag / sentiment are persisted (migration 027)
|
||||
- re-running is idempotent (dedup by tweet id)
|
||||
- page-level early-stop fires when a full page is all-dedup
|
||||
- no twitterapi key → full no-op
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from typing import AsyncGenerator
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import (
|
||||
AsyncSession, async_sessionmaker, create_async_engine,
|
||||
)
|
||||
|
||||
from app.config import settings
|
||||
from app.models import Base, KolPost
|
||||
from app.services import kol_x
|
||||
|
||||
|
||||
_FAKE_TWEETS = [
|
||||
{ # actionable position statement → should be stored + scored
|
||||
"id": "111",
|
||||
"text": "I just dumped my entire $HYPE position",
|
||||
"url": "https://x.com/CryptoHayes/status/111",
|
||||
"createdAt": "Thu Jun 04 05:49:13 +0000 2026",
|
||||
"author": {"followers": 797330},
|
||||
},
|
||||
{ # bare retweet → skipped before the AI call
|
||||
"id": "222",
|
||||
"text": "RT @someone: not my words",
|
||||
"createdAt": "Thu Jun 04 06:00:00 +0000 2026",
|
||||
"author": {"followers": 797330},
|
||||
},
|
||||
{ # low-signal but original → stored (AI decides noise/not)
|
||||
"id": "333",
|
||||
"text": "gm crypto fam",
|
||||
"createdAt": "Thu Jun 04 07:00:00 +0000 2026",
|
||||
"author": {"followers": 797330},
|
||||
},
|
||||
]
|
||||
|
||||
_FAKE_SCORE = {
|
||||
"post_type": "original",
|
||||
"tier": "trade_signal",
|
||||
"summary": "Dumped HYPE",
|
||||
"tickers": [{"ticker": "HYPE", "action": "sell", "conviction": 0.95}],
|
||||
"talks_vs_trades_flag": True,
|
||||
"sentiment": "bearish",
|
||||
"model": "test-model",
|
||||
"version": "x-test",
|
||||
"error": None,
|
||||
}
|
||||
|
||||
_KOL = {"handle": "cryptohayes", "x_username": "CryptoHayes", "display_name": "Hayes"}
|
||||
|
||||
|
||||
async def _fresh_session_factory():
|
||||
"""Each test gets its own isolated in-memory DB."""
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:", future=True)
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
return async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
|
||||
def _make_page_iter(pages: list[list[dict]]):
|
||||
"""Return an async generator that yields one page at a time from `pages`."""
|
||||
async def fake_iter(_username, max_pages=3) -> AsyncGenerator[list[dict], None]:
|
||||
for page in pages:
|
||||
yield page
|
||||
return fake_iter
|
||||
|
||||
|
||||
def _patch_io(monkeypatch, pages: list[list[dict]] | None = None):
|
||||
if pages is None:
|
||||
pages = [list(_FAKE_TWEETS)] # one page containing all fake tweets
|
||||
|
||||
async def fake_score(**_kw):
|
||||
return dict(_FAKE_SCORE)
|
||||
|
||||
monkeypatch.setattr(settings, "twitterapi_io_key", "test-key")
|
||||
monkeypatch.setattr(kol_x, "_iter_tweet_pages", _make_page_iter(pages))
|
||||
monkeypatch.setattr(kol_x.x_analysis, "analyze_x_post", fake_score)
|
||||
|
||||
|
||||
# ── Core contract ─────────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_run_x_poll_noop_without_key(monkeypatch):
|
||||
monkeypatch.setattr(settings, "twitterapi_io_key", "")
|
||||
assert await kol_x.run_x_poll() == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ingest_writes_divergence_ready_kolposts(monkeypatch):
|
||||
_patch_io(monkeypatch)
|
||||
sf = await _fresh_session_factory()
|
||||
|
||||
async with sf() as s:
|
||||
stats = await kol_x._ingest_kol_x(s, _KOL)
|
||||
|
||||
# 111 + 333 stored; 222 (bare RT) skipped before scoring
|
||||
assert stats["new"] == 2
|
||||
assert stats["skipped"] == 1
|
||||
assert stats["analyzed"] == 2
|
||||
assert stats["errors"] == 0
|
||||
|
||||
rows = (await s.execute(
|
||||
select(KolPost).where(KolPost.source == "twitter")
|
||||
)).scalars().all()
|
||||
assert {r.external_id for r in rows} == {"111", "333"}
|
||||
# canonical handle (joins to KolWallet), NOT the X screen name
|
||||
assert all(r.kol_handle == "cryptohayes" for r in rows)
|
||||
|
||||
signal = next(r for r in rows if r.external_id == "111")
|
||||
tk = json.loads(signal.tickers_json)
|
||||
assert tk[0]["ticker"] == "HYPE"
|
||||
assert tk[0]["action"] == "sell" # in kol_divergence._POST_SHORT
|
||||
assert tk[0]["conviction"] == 0.95
|
||||
assert signal.analysis_model == "test-model"
|
||||
assert signal.analysis_version == "x-test"
|
||||
# published_at parsed from the X date string into a naive datetime
|
||||
assert signal.published_at.year == 2026 and signal.published_at.month == 6
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ingest_stores_extended_analysis_fields(monkeypatch):
|
||||
"""migration 027 fields — tier / post_type / talks_vs_trades_flag / sentiment."""
|
||||
_patch_io(monkeypatch)
|
||||
sf = await _fresh_session_factory()
|
||||
|
||||
async with sf() as s:
|
||||
await kol_x._ingest_kol_x(s, _KOL)
|
||||
signal = (await s.execute(
|
||||
select(KolPost).where(KolPost.external_id == "111")
|
||||
)).scalar_one()
|
||||
|
||||
assert signal.tier == "trade_signal"
|
||||
assert signal.post_type == "original"
|
||||
assert signal.talks_vs_trades_flag is True
|
||||
assert signal.sentiment == "bearish"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ingest_is_idempotent_on_rerun(monkeypatch):
|
||||
_patch_io(monkeypatch)
|
||||
sf = await _fresh_session_factory()
|
||||
|
||||
async with sf() as s:
|
||||
first = await kol_x._ingest_kol_x(s, _KOL)
|
||||
second = await kol_x._ingest_kol_x(s, _KOL)
|
||||
|
||||
assert first["new"] == 2
|
||||
assert second["new"] == 0 # everything already stored
|
||||
assert second["skipped"] == 3 # 2 existing + 1 bare RT
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_failure_does_not_raise(monkeypatch):
|
||||
"""A dead/blocked KOL must not abort the run — empty pages → empty stats."""
|
||||
monkeypatch.setattr(settings, "twitterapi_io_key", "test-key")
|
||||
_patch_io(monkeypatch, pages=[]) # generator yields nothing
|
||||
sf = await _fresh_session_factory()
|
||||
async with sf() as s:
|
||||
stats = await kol_x._ingest_kol_x(s, _KOL)
|
||||
assert stats["new"] == 0 and stats["errors"] == 0
|
||||
|
||||
|
||||
# ── Multi-page & early-stop ───────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multipage_fetches_all_new_tweets(monkeypatch):
|
||||
"""Two pages of distinct tweets → both pages are stored."""
|
||||
page1 = [{"id": "p1_1", "text": "SOL is the play", "createdAt": "Thu Jun 04 05:00:00 +0000 2026", "author": {"followers": 100000}}]
|
||||
page2 = [{"id": "p2_1", "text": "ETH too slow ngmi", "createdAt": "Wed Jun 03 10:00:00 +0000 2026", "author": {"followers": 100000}}]
|
||||
_patch_io(monkeypatch, pages=[page1, page2])
|
||||
sf = await _fresh_session_factory()
|
||||
|
||||
async with sf() as s:
|
||||
stats = await kol_x._ingest_kol_x(s, _KOL)
|
||||
|
||||
assert stats["new"] == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_early_stop_when_page_fully_deduped(monkeypatch):
|
||||
"""After page1 is stored, a second run should detect all-dedup on page1
|
||||
and break before page2 is consumed — verified by keeping a 'new' tweet
|
||||
on page2 that must NOT appear in the DB if early-stop fired correctly."""
|
||||
page1 = [{"id": "old1", "text": "BTC to the moon",
|
||||
"createdAt": "Thu Jun 04 05:00:00 +0000 2026", "author": {}}]
|
||||
page2 = [{"id": "brand_new", "text": "Sold everything",
|
||||
"createdAt": "Wed Jun 03 09:00:00 +0000 2026", "author": {}}]
|
||||
|
||||
run = 0
|
||||
pages_yielded: list[list] = []
|
||||
|
||||
async def counting_iter(_username, max_pages=3):
|
||||
nonlocal run
|
||||
run += 1
|
||||
if run == 1:
|
||||
# First run: only page1 (simulates single-page normal daily run)
|
||||
pages_yielded.append(page1)
|
||||
yield page1
|
||||
else:
|
||||
# Second run: both pages available, but early-stop should fire at page1
|
||||
for page in [page1, page2]:
|
||||
pages_yielded.append(page)
|
||||
yield page
|
||||
|
||||
async def fake_score(**_kw):
|
||||
return dict(_FAKE_SCORE)
|
||||
|
||||
monkeypatch.setattr(settings, "twitterapi_io_key", "test-key")
|
||||
monkeypatch.setattr(kol_x, "_iter_tweet_pages", counting_iter)
|
||||
monkeypatch.setattr(kol_x.x_analysis, "analyze_x_post", fake_score)
|
||||
|
||||
sf = await _fresh_session_factory()
|
||||
async with sf() as s:
|
||||
first = await kol_x._ingest_kol_x(s, _KOL)
|
||||
assert first["new"] == 1 # page1 stored (1 tweet)
|
||||
|
||||
pages_yielded.clear()
|
||||
second = await kol_x._ingest_kol_x(s, _KOL)
|
||||
assert second["new"] == 0 # page1 all deduped → early stop
|
||||
# Only page1 should have been yielded — early-stop broke before page2
|
||||
assert pages_yielded == [page1]
|
||||
|
||||
# page2's tweet must NOT be in the DB (generator closed before it was yielded)
|
||||
rows = (await s.execute(
|
||||
select(KolPost).where(KolPost.source == "twitter")
|
||||
)).scalars().all()
|
||||
assert {r.external_id for r in rows} == {"old1"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_all_retweet_page_does_not_early_stop(monkeypatch):
|
||||
"""A page that is ALL bare retweets (page_new=0, page_deduped=0) must NOT
|
||||
trigger early-stop — the next page can still hold original posts. Regression
|
||||
guard: previously 'page_new == 0' alone stopped paging on RT-heavy pages,
|
||||
silently dropping originals on older pages."""
|
||||
page1 = [
|
||||
{"id": "rt1", "text": "RT @a: gm", "createdAt": "Thu Jun 04 05:00:00 +0000 2026", "author": {}},
|
||||
{"id": "rt2", "text": "RT @b: wagmi", "createdAt": "Thu Jun 04 04:00:00 +0000 2026", "author": {}},
|
||||
]
|
||||
page2 = [
|
||||
{"id": "orig1", "text": "Just sold all my $SOL", "createdAt": "Wed Jun 03 09:00:00 +0000 2026", "author": {}},
|
||||
]
|
||||
_patch_io(monkeypatch, pages=[page1, page2])
|
||||
sf = await _fresh_session_factory()
|
||||
|
||||
async with sf() as s:
|
||||
stats = await kol_x._ingest_kol_x(s, _KOL)
|
||||
# page1 all-RT → skipped 2, NO early-stop → page2 consumed → orig1 stored
|
||||
assert stats["new"] == 1
|
||||
assert stats["skipped"] == 2
|
||||
rows = (await s.execute(
|
||||
select(KolPost).where(KolPost.source == "twitter")
|
||||
)).scalars().all()
|
||||
assert {r.external_id for r in rows} == {"orig1"}
|
||||
@@ -0,0 +1,232 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
|
||||
from app.api.posts import router as posts_router
|
||||
from app.database import get_db
|
||||
from app.models import Base, Post
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_posts_paged_filters_and_counts():
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:", future=True)
|
||||
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
now = datetime(2026, 5, 30, 12, 0, 0)
|
||||
async with session_factory() as session:
|
||||
session.add_all([
|
||||
Post(
|
||||
external_id="truth-buy",
|
||||
text="Bullish signal",
|
||||
source="truth",
|
||||
published_at=now,
|
||||
sentiment="bullish",
|
||||
signal="buy",
|
||||
ai_confidence=91,
|
||||
ai_reasoning="AI saw upside",
|
||||
relevant=True,
|
||||
),
|
||||
Post(
|
||||
external_id="truth-short",
|
||||
text="Bearish signal",
|
||||
source="truth",
|
||||
published_at=now - timedelta(minutes=1),
|
||||
sentiment="bearish",
|
||||
signal="short",
|
||||
ai_confidence=88,
|
||||
ai_reasoning="AI saw downside",
|
||||
relevant=True,
|
||||
),
|
||||
Post(
|
||||
external_id="truth-hold",
|
||||
text="Neutral but scored",
|
||||
source="truth",
|
||||
published_at=now - timedelta(minutes=2),
|
||||
sentiment="neutral",
|
||||
signal="hold",
|
||||
ai_confidence=45,
|
||||
ai_reasoning="No trade edge",
|
||||
relevant=True,
|
||||
),
|
||||
Post(
|
||||
external_id="truth-noise",
|
||||
text="Off-topic golf post",
|
||||
source="truth",
|
||||
published_at=now - timedelta(minutes=3),
|
||||
sentiment="neutral",
|
||||
signal=None,
|
||||
ai_confidence=0,
|
||||
ai_reasoning=None,
|
||||
relevant=False,
|
||||
),
|
||||
Post(
|
||||
external_id="macro-buy",
|
||||
text="Macro buy",
|
||||
source="btc_bottom_reversal",
|
||||
published_at=now - timedelta(minutes=4),
|
||||
sentiment="bullish",
|
||||
signal="buy",
|
||||
ai_confidence=97,
|
||||
ai_reasoning="Macro setup",
|
||||
relevant=True,
|
||||
),
|
||||
Post(
|
||||
external_id="archive-breakout",
|
||||
text="Legacy breakout signal",
|
||||
source="breakout",
|
||||
published_at=now - timedelta(minutes=5),
|
||||
sentiment="bullish",
|
||||
signal="buy",
|
||||
ai_confidence=73,
|
||||
ai_reasoning="Old scanner",
|
||||
relevant=True,
|
||||
),
|
||||
Post(
|
||||
external_id="archive-sma",
|
||||
text="Legacy sma reclaim signal",
|
||||
source="sma_reclaim",
|
||||
published_at=now - timedelta(minutes=6),
|
||||
sentiment="bullish",
|
||||
signal="short",
|
||||
ai_confidence=64,
|
||||
ai_reasoning="Old scanner 2",
|
||||
relevant=True,
|
||||
),
|
||||
])
|
||||
await session.commit()
|
||||
|
||||
app = FastAPI()
|
||||
app.include_router(posts_router, prefix="/api")
|
||||
|
||||
async def override_get_db():
|
||||
async with session_factory() as session:
|
||||
yield session
|
||||
|
||||
app.dependency_overrides[get_db] = override_get_db
|
||||
|
||||
transport = ASGITransport(app=app)
|
||||
async with AsyncClient(transport=transport, base_url="http://testserver") as client:
|
||||
res = await client.get("/api/posts-paged", params={"source": "truth", "limit": 10, "page": 1})
|
||||
assert res.status_code == 200
|
||||
data = res.json()
|
||||
assert data["total"] == 4
|
||||
assert len(data["items"]) == 4
|
||||
assert data["counts"] == {
|
||||
"all": 4,
|
||||
"actionable": 2,
|
||||
"buy": 1,
|
||||
"short": 1,
|
||||
"off_topic": 1,
|
||||
}
|
||||
assert data["source_counts"] == [{"source": "truth", "count": 4, "latest": "2026-05-30T12:00:00.000Z"}]
|
||||
|
||||
actionable = await client.get(
|
||||
"/api/posts-paged",
|
||||
params={"source": "truth", "signal": "actionable", "limit": 10, "page": 1},
|
||||
)
|
||||
assert actionable.status_code == 200
|
||||
actionable_data = actionable.json()
|
||||
assert actionable_data["total"] == 2
|
||||
assert {item["signal"] for item in actionable_data["items"]} == {"buy", "short"}
|
||||
assert actionable_data["counts"]["all"] == 4
|
||||
|
||||
scored_only = await client.get(
|
||||
"/api/posts-paged",
|
||||
params={"source": "truth", "ai_scored_only": "true", "limit": 10, "page": 1},
|
||||
)
|
||||
assert scored_only.status_code == 200
|
||||
scored_data = scored_only.json()
|
||||
assert scored_data["total"] == 3
|
||||
assert all(item["ai_confidence"] > 0 or item["ai_reasoning"] for item in scored_data["items"])
|
||||
assert scored_data["counts"] == {
|
||||
"all": 3,
|
||||
"actionable": 2,
|
||||
"buy": 1,
|
||||
"short": 1,
|
||||
"off_topic": 1,
|
||||
}
|
||||
|
||||
bearish_buy = await client.get(
|
||||
"/api/posts-paged",
|
||||
params={
|
||||
"source": "truth",
|
||||
"sentiment": "bearish",
|
||||
"signal": "buy",
|
||||
"limit": 10,
|
||||
"page": 1,
|
||||
},
|
||||
)
|
||||
assert bearish_buy.status_code == 200
|
||||
bearish_buy_data = bearish_buy.json()
|
||||
assert bearish_buy_data["total"] == 0
|
||||
assert bearish_buy_data["counts"] == {
|
||||
"all": 1,
|
||||
"actionable": 1,
|
||||
"buy": 0,
|
||||
"short": 1,
|
||||
"off_topic": 0,
|
||||
}
|
||||
|
||||
archive = await client.get(
|
||||
"/api/posts-paged",
|
||||
params={
|
||||
"archive_only": "true",
|
||||
"limit": 10,
|
||||
"page": 1,
|
||||
},
|
||||
)
|
||||
assert archive.status_code == 200
|
||||
archive_data = archive.json()
|
||||
assert archive_data["total"] == 2
|
||||
assert len(archive_data["items"]) == 2
|
||||
assert {item["source"] for item in archive_data["items"]} == {"breakout", "sma_reclaim"}
|
||||
assert archive_data["source_counts"] == [
|
||||
{"source": "breakout", "count": 1, "latest": "2026-05-30T11:55:00.000Z"},
|
||||
{"source": "sma_reclaim", "count": 1, "latest": "2026-05-30T11:54:00.000Z"},
|
||||
]
|
||||
|
||||
# Regression: selecting one archive source via source_in must narrow the
|
||||
# paged items/total, but source_counts (the chip bar) must STILL list
|
||||
# every archived source so the UI can offer a way back to "all".
|
||||
archive_one = await client.get(
|
||||
"/api/posts-paged",
|
||||
params={
|
||||
"archive_only": "true",
|
||||
"source_in": "breakout",
|
||||
"limit": 10,
|
||||
"page": 1,
|
||||
},
|
||||
)
|
||||
assert archive_one.status_code == 200
|
||||
archive_one_data = archive_one.json()
|
||||
assert archive_one_data["total"] == 1
|
||||
assert {item["source"] for item in archive_one_data["items"]} == {"breakout"}
|
||||
# Chip bar is NOT collapsed to the selected source.
|
||||
assert archive_one_data["source_counts"] == [
|
||||
{"source": "breakout", "count": 1, "latest": "2026-05-30T11:55:00.000Z"},
|
||||
{"source": "sma_reclaim", "count": 1, "latest": "2026-05-30T11:54:00.000Z"},
|
||||
]
|
||||
|
||||
archive_compat = await client.get(
|
||||
"/api/posts-paged",
|
||||
params={
|
||||
"archive_only": "true",
|
||||
"source_not_in": "truth",
|
||||
"limit": 10,
|
||||
"page": 1,
|
||||
},
|
||||
)
|
||||
assert archive_compat.status_code == 200
|
||||
archive_compat_data = archive_compat.json()
|
||||
assert archive_compat_data["total"] == 2
|
||||
assert {item["source"] for item in archive_compat_data["items"]} == {"breakout", "sma_reclaim"}
|
||||
|
||||
await engine.dispose()
|
||||
@@ -41,6 +41,12 @@ class _Trade:
|
||||
pnl_usd = 2.25
|
||||
|
||||
|
||||
class _ClosedTrade(_Trade):
|
||||
"""Same as _Trade but with closed_at set, simulating a committed close."""
|
||||
from datetime import datetime, timezone
|
||||
closed_at = datetime(2026, 1, 1, 0, 0, 0)
|
||||
|
||||
|
||||
class _Sub:
|
||||
leverage = 3
|
||||
hl_api_key = None
|
||||
@@ -65,7 +71,10 @@ async def test_manual_close_returns_close_result(monkeypatch):
|
||||
|
||||
monkeypatch.setattr(bot_engine, "close_and_finalize", fake_close_and_finalize)
|
||||
|
||||
db = _Db([_Trade(), _Sub(), _Trade()])
|
||||
# Responses in order: load trade → load sub → populate_existing re-read
|
||||
# (B45 fix: one query with populate_existing=True instead of old stale cache).
|
||||
# _ClosedTrade has closed_at set so the B46 success guard passes.
|
||||
db = _Db([_Trade(), _Sub(), _ClosedTrade()])
|
||||
|
||||
result = await positions.manual_close(7, _Request(), db)
|
||||
|
||||
@@ -88,10 +97,10 @@ async def test_open_positions_requires_signed_wallet_read(monkeypatch):
|
||||
monkeypatch.setattr(positions, "verify_signed_request_any", fake_verify)
|
||||
db = _Db([[]])
|
||||
|
||||
from app.services.signed_request import SignedReadCreds
|
||||
result = await positions.get_open_positions(
|
||||
wallet="0xABC",
|
||||
ts=123,
|
||||
sig="0xsig",
|
||||
creds=SignedReadCreds(ts=123, sig="0xsig"),
|
||||
db=db,
|
||||
)
|
||||
|
||||
|
||||
@@ -10,6 +10,8 @@ change accidentally turns "BTC bottom triggers: 3/3 firing" into a less
|
||||
clear phrasing, CI catches it.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from app.services.telegram_digest import (
|
||||
GlobalDigest, MacroBlock, KolBlock, TrumpBlock, format_digest,
|
||||
)
|
||||
@@ -174,3 +176,48 @@ def test_total_length_well_under_telegram_cap():
|
||||
"open_pnl_summary": "3 open (+125.3 USD)",
|
||||
})
|
||||
assert len(out) < 4096
|
||||
|
||||
|
||||
# ── build_global_digest DB aggregation (twitter-noise exclusion) ───────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_build_global_digest_excludes_twitter_noise(monkeypatch):
|
||||
"""KOL posts_24h must NOT count twitter noise (gm / RT / jokes). Substack
|
||||
essays (tier=NULL) and twitter signal posts (tier!='noise') count; twitter
|
||||
noise (tier='noise') is excluded. Regression guard for the digest count
|
||||
being inflated by a KOL's gm/RT spam after X ingestion landed."""
|
||||
from datetime import datetime, timedelta
|
||||
from sqlalchemy.ext.asyncio import (
|
||||
AsyncSession, async_sessionmaker, create_async_engine,
|
||||
)
|
||||
from app.models import Base, KolPost
|
||||
from app.services import telegram_digest
|
||||
|
||||
engine = create_async_engine("sqlite+aiosqlite:///:memory:", future=True)
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
sf = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
now = datetime(2026, 6, 4, 12, 0, 0)
|
||||
recent = now - timedelta(hours=2)
|
||||
|
||||
def _post(ext, source, tier=None, handle="cryptohayes"):
|
||||
return KolPost(
|
||||
kol_handle=handle, source=source, external_id=ext,
|
||||
url=f"https://x.com/x/status/{ext}", published_at=recent,
|
||||
raw_text="body", content_hash=ext, tier=tier,
|
||||
)
|
||||
|
||||
async with sf() as s:
|
||||
s.add_all([
|
||||
_post("sub1", "substack", tier=None, handle="raoulpal"), # essay → count
|
||||
_post("tw1", "twitter", tier="directional"), # signal → count
|
||||
_post("tw2", "twitter", tier="noise"), # gm noise → exclude
|
||||
_post("tw3", "twitter", tier="noise"), # RT noise → exclude
|
||||
])
|
||||
await s.commit()
|
||||
|
||||
monkeypatch.setattr(telegram_digest, "async_session", sf)
|
||||
g = await telegram_digest.build_global_digest(now=now)
|
||||
# 2 real posts (substack essay + twitter signal); 2 noise excluded
|
||||
assert g.kol.posts_24h == 2
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
"""Tests for trade-alert broadcasts and balance pre-check logic added 2026-06-01.
|
||||
|
||||
Coverage targets:
|
||||
1. _broadcast_trade_alert — fire-and-forget, must not raise
|
||||
2. Balance pre-check maths — required_margin = (notional / leverage) * 1.1
|
||||
3. Startup drain in the Telegram bot loop — offset advances past pending updates
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import pytest
|
||||
|
||||
# ── 1. _broadcast_trade_alert ─────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_broadcast_trade_alert_no_connections(monkeypatch):
|
||||
"""broadcast_trade_alert must silently succeed when no WS clients are connected."""
|
||||
from app.services.bot_engine import _broadcast_trade_alert
|
||||
from app.ws import manager as mgr_mod
|
||||
|
||||
# Patch manager.broadcast to verify it's called with correct payload
|
||||
calls: list[dict] = []
|
||||
async def fake_broadcast(msg: dict):
|
||||
calls.append(msg)
|
||||
|
||||
monkeypatch.setattr(mgr_mod.manager, "broadcast", fake_broadcast)
|
||||
|
||||
await _broadcast_trade_alert("0xabc", "execution_failed", asset="BTC", reason="test error")
|
||||
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["type"] == "trade_alert"
|
||||
assert calls[0]["wallet"] == "0xabc"
|
||||
assert calls[0]["event"] == "execution_failed"
|
||||
assert calls[0]["asset"] == "BTC"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_broadcast_trade_alert_swallows_exceptions(monkeypatch):
|
||||
"""broadcast_trade_alert must not propagate exceptions — trade flow must continue."""
|
||||
from app.services.bot_engine import _broadcast_trade_alert
|
||||
from app.ws import manager as mgr_mod
|
||||
|
||||
async def exploding_broadcast(msg: dict):
|
||||
raise RuntimeError("WS layer crashed")
|
||||
|
||||
monkeypatch.setattr(mgr_mod.manager, "broadcast", exploding_broadcast)
|
||||
|
||||
# Must not raise
|
||||
await _broadcast_trade_alert("0xabc", "budget_reached", asset="BTC")
|
||||
|
||||
|
||||
# ── 2. Balance pre-check maths ────────────────────────────────────────────────
|
||||
|
||||
def test_required_margin_formula():
|
||||
"""required_margin = (notional / leverage) * 1.1 — verify key scenarios."""
|
||||
def required_margin(notional: float, leverage: int) -> float:
|
||||
return round((notional / max(leverage, 1)) * 1.1, 2)
|
||||
|
||||
# 2× leverage, $100 position → $50 margin + 10% buffer = $55
|
||||
assert required_margin(100, 2) == 55.0
|
||||
|
||||
# 5× leverage, $500 position → $100 margin + 10% = $110
|
||||
assert required_margin(500, 5) == 110.0
|
||||
|
||||
# 1× leverage, $20 position → $20 + 10% = $22
|
||||
assert required_margin(20, 1) == 22.0
|
||||
|
||||
# edge: leverage=0 treated as 1 (no divide-by-zero)
|
||||
assert required_margin(100, 0) == 110.0
|
||||
|
||||
|
||||
def test_balance_check_does_not_block_low_leverage():
|
||||
"""With $100 balance and 10× leverage on a $200 notional, margin = $22 — should PASS."""
|
||||
balance = 100.0
|
||||
notional = 200.0
|
||||
leverage = 10
|
||||
required = (notional / max(leverage, 1)) * 1.1
|
||||
assert balance >= required, (
|
||||
f"Balance ${balance} should cover ${required:.2f} margin "
|
||||
f"(${notional} notional at {leverage}×)"
|
||||
)
|
||||
|
||||
|
||||
def test_balance_check_fires_at_truly_insufficient_balance():
|
||||
"""With $5 balance and 2× on $20 notional, margin = $11 — should BLOCK."""
|
||||
balance = 5.0
|
||||
notional = 20.0
|
||||
leverage = 2
|
||||
required = (notional / max(leverage, 1)) * 1.1
|
||||
assert balance < required, (
|
||||
f"Balance ${balance} should NOT cover ${required:.2f} margin"
|
||||
)
|
||||
|
||||
|
||||
# ── 3. Telegram startup drain ────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_telegram_startup_drain_advances_offset(monkeypatch):
|
||||
"""Verify the drain calls getUpdates with timeout=0 and ACKs the last update_id."""
|
||||
import httpx
|
||||
|
||||
drain_calls: list[dict] = []
|
||||
|
||||
class FakeResponse:
|
||||
status_code = 200
|
||||
def json(self):
|
||||
if len(drain_calls) == 1: # first call: return pending updates
|
||||
return {"result": [{"update_id": 100}, {"update_id": 101}]}
|
||||
return {"result": []} # second call (ACK): no pending
|
||||
|
||||
class FakeClient:
|
||||
async def __aenter__(self): return self
|
||||
async def __aexit__(self, *a): pass
|
||||
async def get(self, url, params=None):
|
||||
drain_calls.append(params or {})
|
||||
return FakeResponse()
|
||||
|
||||
monkeypatch.setattr(httpx, "AsyncClient", lambda **kw: FakeClient())
|
||||
|
||||
# Simulate only the drain portion (extract the logic)
|
||||
from app.config import settings
|
||||
monkeypatch.setattr(settings, "telegram_bot_token", "test-token")
|
||||
|
||||
import httpx as _httpx
|
||||
|
||||
# Re-run the drain logic inline (mirrors telegram_bot.py startup drain)
|
||||
token = "test-token"
|
||||
TG_API = "https://api.telegram.org/bot{token}/{method}"
|
||||
|
||||
async with _httpx.AsyncClient(timeout=10) as client:
|
||||
r = await client.get(
|
||||
TG_API.format(token=token, method="getUpdates"),
|
||||
params={"timeout": 0, "limit": 100},
|
||||
)
|
||||
if r.status_code == 200:
|
||||
pending = r.json().get("result", [])
|
||||
if pending:
|
||||
drain_offset = pending[-1]["update_id"] + 1
|
||||
async with _httpx.AsyncClient(timeout=10) as client:
|
||||
await client.get(
|
||||
TG_API.format(token=token, method="getUpdates"),
|
||||
params={"timeout": 0, "offset": drain_offset},
|
||||
)
|
||||
|
||||
# First call: drain request (timeout=0, limit=100)
|
||||
assert drain_calls[0].get("timeout") == 0
|
||||
assert drain_calls[0].get("limit") == 100
|
||||
|
||||
# Second call: ACK with offset = last_update_id + 1
|
||||
assert drain_calls[1].get("offset") == 102 # 101 + 1
|
||||
@@ -0,0 +1,217 @@
|
||||
"""Tests for x_analysis — the DETERMINISTIC normalization layer.
|
||||
|
||||
The LLM call is mocked so each test feeds a controlled raw model response and
|
||||
asserts how analyze_x_post normalizes it. This locks down the enforcement rules
|
||||
that protect downstream consumers (kol_x → kol_divergence), independent of
|
||||
whatever the model actually returns:
|
||||
|
||||
- retweet post_type → forced noise
|
||||
- trade_signal requires a buy/sell/reduce ticker with conviction ≥ 0.7,
|
||||
else it downgrades to directional (never silently dropped)
|
||||
- noise → tickers cleared + talks_vs_trades_flag forced false
|
||||
- ticker hygiene: conviction clamped 0..1, bad action → mention,
|
||||
overlong symbol dropped
|
||||
- invalid tier/post_type → safe defaults
|
||||
- bad JSON / empty text → graceful fallback, never raises
|
||||
|
||||
These are pure logic (no network, no AI spend) so they're fast and stable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from app.config import settings
|
||||
from app.services import x_analysis
|
||||
|
||||
|
||||
# ── helpers ────────────────────────────────────────────────────────────────
|
||||
|
||||
def _patch_llm(monkeypatch, raw: dict):
|
||||
"""Force analyze_x_post's LLM call to return `raw` (serialized). We patch
|
||||
the OpenAI path (anthropic key blanked) since that's the default in CI."""
|
||||
payload = json.dumps(raw)
|
||||
|
||||
class _Msg:
|
||||
content = payload
|
||||
|
||||
class _Choice:
|
||||
message = _Msg()
|
||||
|
||||
class _Resp:
|
||||
choices = [_Choice()]
|
||||
|
||||
class _Completions:
|
||||
async def create(self, **_kw):
|
||||
return _Resp()
|
||||
|
||||
class _Chat:
|
||||
completions = _Completions()
|
||||
|
||||
class _Client:
|
||||
chat = _Chat()
|
||||
|
||||
monkeypatch.setattr(settings, "anthropic_api_key", "") # → OpenAI path
|
||||
monkeypatch.setattr(x_analysis, "_oai", lambda: _Client())
|
||||
|
||||
|
||||
def _raw(**over):
|
||||
"""Minimal well-formed raw model response; override per test."""
|
||||
base = {
|
||||
"post_type": "original",
|
||||
"tier": "directional",
|
||||
"summary": "x",
|
||||
"tickers": [],
|
||||
"talks_vs_trades_flag": False,
|
||||
"has_price_target": False,
|
||||
"price_targets": [],
|
||||
"sentiment": "neutral",
|
||||
"reasoning": "y",
|
||||
}
|
||||
base.update(over)
|
||||
return base
|
||||
|
||||
|
||||
# ── tier enforcement ───────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retweet_post_type_forced_to_noise(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
post_type="retweet", tier="trade_signal",
|
||||
tickers=[{"ticker": "BTC", "action": "buy", "conviction": 0.9}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="some original-looking text body")
|
||||
assert r["tier"] == "noise"
|
||||
assert r["tickers"] == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trade_signal_downgrades_without_strong_action(monkeypatch):
|
||||
# tier=trade_signal but only a 'bullish' ticker (not buy/sell/reduce)
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="trade_signal",
|
||||
tickers=[{"ticker": "SOL", "action": "bullish", "conviction": 0.9}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="SOL looking strong")
|
||||
assert r["tier"] == "directional"
|
||||
assert r["tickers"][0]["ticker"] == "SOL"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trade_signal_downgrades_on_low_conviction(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="trade_signal",
|
||||
tickers=[{"ticker": "SOL", "action": "buy", "conviction": 0.5}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="bought a little SOL maybe")
|
||||
assert r["tier"] == "directional" # 0.5 < 0.7 floor
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_trade_signal_kept_when_strong(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="trade_signal",
|
||||
tickers=[{"ticker": "SOL", "action": "buy", "conviction": 0.9}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="aped SOL full size lfg")
|
||||
assert r["tier"] == "trade_signal"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_invalid_tier_falls_back_to_noise(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(tier="超级买入"))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="ambiguous content here")
|
||||
assert r["tier"] == "noise"
|
||||
|
||||
|
||||
# ── noise enforcement ──────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_noise_clears_tickers_and_flag(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="noise", talks_vs_trades_flag=True,
|
||||
tickers=[{"ticker": "BTC", "action": "buy", "conviction": 0.9}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="gm frens beautiful day")
|
||||
assert r["tickers"] == []
|
||||
assert r["talks_vs_trades_flag"] is False
|
||||
|
||||
|
||||
# ── ticker hygiene ─────────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_conviction_clamped_to_unit_interval(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="directional",
|
||||
tickers=[{"ticker": "BTC", "action": "bullish", "conviction": 1.8}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="BTC going parabolic")
|
||||
assert r["tickers"][0]["conviction"] == 1.0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_invalid_action_becomes_mention(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="directional",
|
||||
tickers=[{"ticker": "BTC", "action": "yolo", "conviction": 0.5}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="BTC yolo time")
|
||||
assert r["tickers"][0]["action"] == "mention"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_overlong_ticker_dropped(monkeypatch):
|
||||
_patch_llm(monkeypatch, _raw(
|
||||
tier="directional",
|
||||
tickers=[{"ticker": "THISISWAYTOOLONG", "action": "bullish", "conviction": 0.5}],
|
||||
))
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="some long token mention")
|
||||
assert r["tickers"] == []
|
||||
|
||||
|
||||
# ── graceful failure ───────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_empty_text_short_circuits_without_llm(monkeypatch):
|
||||
# no _patch_llm → if it called the LLM it would error; it must not.
|
||||
r = await x_analysis.analyze_x_post(handle="k", text=" ")
|
||||
assert r["tier"] == "noise"
|
||||
assert r["error"] == "empty post"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bare_retweet_prefiltered_without_llm(monkeypatch):
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="RT @someone: gm")
|
||||
assert r["tier"] == "noise"
|
||||
assert r["post_type"] == "retweet"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bad_json_returns_fallback(monkeypatch):
|
||||
class _Msg:
|
||||
content = "not json at all {{{ "
|
||||
|
||||
class _Choice:
|
||||
message = _Msg()
|
||||
|
||||
class _Resp:
|
||||
choices = [_Choice()]
|
||||
|
||||
class _Completions:
|
||||
async def create(self, **_kw):
|
||||
return _Resp()
|
||||
|
||||
class _Chat:
|
||||
completions = _Completions()
|
||||
|
||||
class _Client:
|
||||
chat = _Chat()
|
||||
|
||||
monkeypatch.setattr(settings, "anthropic_api_key", "")
|
||||
monkeypatch.setattr(x_analysis, "_oai", lambda: _Client())
|
||||
|
||||
r = await x_analysis.analyze_x_post(handle="k", text="real content that triggers llm")
|
||||
assert r["tier"] == "noise"
|
||||
assert r["error"] and "parse_error" in r["error"]
|
||||
Reference in New Issue
Block a user