k
54884f3e24
Feed-health pass over KOL_FEEDS: - raoulpal: stale Substack (last 2024-05) → Real Vision podcast feed - dampedspring: paywalled (0 entries) → free "Damped Spring 101" Substack - unchained: Cloudflare 403 → canonical Megaphone podcast feed - lynalden: Cloudflare 202 → FeedBurner mirror - glassnode: recovered via httpx http2=True (was 403 on HTTP/1.1) - browser User-Agent + Accept headers on feed fetch - removed dead feeds with no active replacement: placeholder, dragonfly, niccarter, eugene - pin h2==4.3.0 (required by http2=True) All 25 remaining feeds verified fetching real body content; newest post per feed ≤88d. Bundles in-flight KOL-module work already in the working tree (kol_x ingest, migration 027, tests). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
546 lines
28 KiB
Markdown
546 lines
28 KiB
Markdown
# Trump Alpha — Backend
|
||
|
||
> AI-powered crypto signal aggregator. Surfaces four uncorrelated signal
|
||
> streams (Trump Truth Social, Macro Vibes, KOL talks-vs-trades, funding
|
||
> reversal) and runs an optional execution layer on Hyperliquid perps.
|
||
> Real money — handle every change to the trading layer like surgery.
|
||
|
||
This file is the **first thing an AI agent should read** when entering this
|
||
repo. It encodes the invariants that aren't visible from any single file.
|
||
|
||
---
|
||
|
||
## 🛑 Read this BEFORE touching anything trading-related
|
||
|
||
This backend manages real Hyperliquid leveraged positions for real users.
|
||
Bugs cost users money. **Five non-negotiable rules:**
|
||
|
||
1. **Two systems, one execution layer.** System 1 (Trump scalp) auto-opens.
|
||
System 2 (Macro Vibes / reversal) is **MANAGE-ONLY since v2.0** — the
|
||
user opens manually on Hyperliquid, then `/adopt` hands it to the bot.
|
||
`process_post()` early-returns for sys2 sources. Re-enabling sys2
|
||
auto-open would silently reintroduce all the leverage/budget/concurrency
|
||
race conditions we excised. Don't do it without an ADR.
|
||
|
||
2. **`released_at` is the "user took back control" marker.** A trade with
|
||
`released_at IS NOT NULL` is **OUT OF BOUNDS for the bot**:
|
||
- `recovery.rehydrate_open_trades` skips it
|
||
- `reconciler` skips it
|
||
- `close_and_finalize`'s atomic claim requires `released_at IS NULL`
|
||
unless `force=True` (only manual_close passes that)
|
||
- `partial_derisk` and `pyramid_add` early-return idempotent-success
|
||
- `/positions/open`, `/positions/today`, telegram_digest all filter it
|
||
If you add ANY new code path that touches BotTrade rows, ask yourself
|
||
"does this respect released_at?" Almost always yes.
|
||
|
||
3. **Effective exit params are FROZEN on the BotTrade row at open time.**
|
||
See the `eff_*` columns on `BotTrade`. Recovery rebuilds the watchdog
|
||
from these, NOT from the live Subscription. Without this, restarting
|
||
the backend silently rewrites every open System-2 reversal's stop loss
|
||
to the user's Trump scalp setting (1.5%). NEVER read live
|
||
`Subscription.stop_loss_pct` etc. in the close path.
|
||
|
||
4. **HL leverage is what HL says, not what the user requested.** Hyperliquid
|
||
silently clips the requested leverage to the asset's max (memes capped
|
||
at 3×). `hyperliquid.open_position()` returns `effective_leverage` —
|
||
bot_engine and adoption must use THAT value to compute
|
||
`sys2_protective_stop_pct(lev)` and the derisk ladder. Using the
|
||
requested value puts the stop OUTSIDE the real liquidation line.
|
||
|
||
5. **Per-wallet asyncio locks wrap "check + open" critical sections.**
|
||
See `_wallet_lock` in `bot_engine` and `_wallet_adopt_lock` in
|
||
`adoption`. Without them, two concurrent signals can both pass the
|
||
daily-budget / concurrency / already-open check before either commits.
|
||
|
||
---
|
||
|
||
## What this product does (90 seconds)
|
||
|
||
```
|
||
Four signal sources → one bot → optional Hyperliquid execution
|
||
|
||
┌──────────────────────────┐
|
||
│ 1. Trump Truth Social │── auto-classify (DeepSeek) → "buy"/"short"/"noise"
|
||
│ (every post, <3s) │ if actionable: Trump scalp auto-open (System 1)
|
||
└──────────────────────────┘ Tight 1.5% SL, 12h cooldown, ≥30min min-hold
|
||
Optional: post prediction tweet via x_poster.py
|
||
|
||
┌──────────────────────────┐
|
||
│ 2. Macro Vibes │── 8 daily macro indicators (AHR999, F&G, etc.)
|
||
│ (BTC bottom + funding)│ + 2-of-3 bottom-reversal trigger
|
||
└──────────────────────────┘ Telegram alert with /adopt CTA — NO auto-open.
|
||
User opens on HL → /adopt → bot manages with
|
||
5-rung stop ladder, de-risk, pyramid, peak-trail.
|
||
|
||
┌──────────────────────────┐
|
||
│ 3. KOL talks-vs-trades │── Substack/podcast/X ingest + ETH on-chain diff
|
||
│ (29 feeds, daily) │ Divergence (publicly bullish, secretly selling)
|
||
└──────────────────────────┘ is the platform's highest-conviction signal.
|
||
Telegram alert only — never auto-trades.
|
||
x_analysis.py adds real-time X post scoring.
|
||
|
||
┌──────────────────────────┐
|
||
│ 4. Funding extreme │── Hourly BTC perp funding scan
|
||
└──────────────────────────┘ Alert only (manage-only via /adopt like Macro)
|
||
|
||
┌──────────────────────────┐
|
||
│ Telegram daily digest │── Once-a-day 3-section brief (Macro/KOL/Trump)
|
||
│ (per-user hour, opt-out)│ to every subscriber. Cron at minute=0 each hour.
|
||
└──────────────────────────┘
|
||
```
|
||
|
||
Free tier = read everything + Telegram alerts. Pro tier (Hyperliquid wallet
|
||
linked) = Trump auto-trade + /adopt manage-only flow for sys2.
|
||
|
||
---
|
||
|
||
## Stack
|
||
|
||
- **Python 3.9+** / FastAPI / async SQLAlchemy 2.x / APScheduler
|
||
- **DB**: SQLite dev, **Postgres prod**. All schema lives in
|
||
`alembic/versions/NNN_*.py`, ordered. Currently at head **026**.
|
||
- **AI**: DeepSeek via OpenAI-compatible API (`AI_BASE_URL`, `AI_MODEL`).
|
||
- Live scoring uses `AI_LIVE_MODEL` (~2s, latency-critical)
|
||
- Batch / reanalysis uses `AI_MODEL` (quality, ~10s)
|
||
- **Trading**: Hyperliquid SDK; API-wallet keys are envelope-encrypted with
|
||
`ENCRYPTION_KEY` (KEK), per-user DEK derivation via `crypto.py`.
|
||
- **Prices**: Two feeds:
|
||
- `binance.py` WebSocket → BTC, ETH, SOL, TRUMP, BNB, DOGE, LINK, AAVE
|
||
- `hl_price_feed.py` polls HL `allMids` every 2s → HYPE, PURR (HL-native assets not on Binance)
|
||
Both pump `price_store` + `tp_sl_monitor` on every tick.
|
||
- **Telegram**: long-poll mode (single instance), HTML messages, inline
|
||
keyboards. `telegram.py` send/edit/answer + `telegram_bot.py` commands.
|
||
Public channel broadcast (`TELEGRAM_PUBLIC_CHANNEL_ID` env) sends a
|
||
sanitised `format_public_post` version (no execution details, tier label
|
||
instead of raw confidence) after every per-user fan-out in `_dispatch`.
|
||
- **X (Twitter)**: `x_poster.py` — optional viral prediction tweets after each
|
||
actionable Trump signal. Gated by `x_enabled=False` (off by default). Full
|
||
no-op if creds missing. OAuth 1.0a hand-signed with stdlib hmac — no extra deps.
|
||
|
||
---
|
||
|
||
## Module map (where things live)
|
||
|
||
```
|
||
app/
|
||
├── api/ HTTP routes
|
||
│ ├── signals.py POST /api/signals/ingest ← scanners write here
|
||
│ ├── positions.py /positions/open|today|close|grow|adopt|release|hl
|
||
│ ├── user.py /subscribe|settings|manual-window|auto-trade
|
||
│ ├── telegram.py /telegram/{preferences,bind,unbind,test}
|
||
│ ├── macro.py /macro/{snapshot,history}
|
||
│ ├── kol.py /kol/{posts,digest,wallets,divergence}
|
||
│ ├── performance.py /performance ← wallet-scoped real-money stats (30d)
|
||
│ ├── funding_reversal.py /funding/snapshot ← live funding state + 7d history
|
||
│ ├── funding_signal.py /signal/{status,toggle,history} ← breakout monitor
|
||
│ └── dev.py Dev-only routes (only mounted in development env)
|
||
├── services/
|
||
│ ├── bot_engine.py ★ TRADING CORE — process_post, _execute_for_subscriber,
|
||
│ │ _broadcast_trade_alert (WS failure notifications),
|
||
│ │ close_and_finalize, partial_derisk, pyramid_add
|
||
│ ├── adoption.py ★ /adopt + /release flow (sys2 manage-only)
|
||
│ ├── tp_sl_monitor.py Per-price-tick close evaluator. on_price_tick is
|
||
│ │ called from binance.py + hl_price_feed.py once/sec
|
||
│ ├── hyperliquid.py HL trader (open/close/reduce/leverage)
|
||
│ ├── recovery.py Startup rehydration of open BotTrades into watchdog
|
||
│ ├── reconciler.py Every 60s: compare DB ↔ HL state, mark drift
|
||
│ ├── circuit_breaker.py Per-system (sys1/sys2) CB, daily DD + N-loss streak
|
||
│ ├── signal_categories.py CRITICAL CONFIG — sys1/sys2 sources, ladders,
|
||
│ │ leverage clamping, protective stop formulas
|
||
│ ├── regime_filter.py Sys1 only — recent-move / vol-contraction gates
|
||
│ ├── analysis.py AI signal scoring (DeepSeek) for Trump posts
|
||
│ ├── x_analysis.py AI scoring for X (Twitter) KOL posts. Three tiers:
|
||
│ │ TRADE_SIGNAL / DIRECTIONAL / NOISE. Strict NOISE
|
||
│ │ default — most X posts should be filtered out.
|
||
│ │ Consumed by kol_x.py. tickers come out in the
|
||
│ │ {ticker,action,conviction} shape kol_divergence reads.
|
||
│ ├── x_poster.py X (Twitter) auto-poster for Trump signals.
|
||
│ │ Fires a prediction tweet then a follow-up at
|
||
│ │ +x_followup_minutes with the actual move. Gated by
|
||
│ │ x_enabled env var (False by default). Full no-op
|
||
│ │ if creds missing — never blocks signal flow.
|
||
│ ├── entry_filter.py Cheap text-based pre-filter (skip RT/URL-only)
|
||
│ ├── telegram.py send_message / edit_message / answer_callback
|
||
│ ├── telegram_bot.py Long-poll loop + /start /digest /adopt /release ...
|
||
│ ├── telegram_digest.py Daily 3-section brief (rule-based; no LLM)
|
||
│ ├── price_store.py In-memory latest price per asset
|
||
│ ├── price_backfill.py Backfill historical 5min bars from Binance
|
||
│ ├── hl_price_feed.py Supplemental HL price feed for HL-native assets
|
||
│ │ (HYPE, PURR). Polls allMids every 2s. Runs
|
||
│ │ alongside binance.py. Without this, TP/SL
|
||
│ │ silently stops protecting HL-native trades.
|
||
│ ├── backtest.py Single-post backtest harness. Fetches 1m Binance
|
||
│ │ candles for [published_at, +max_hold_h] and
|
||
│ │ replays current exit rules. Conservative (uses
|
||
│ │ HIGH/LOW within bar). No fees. Batch runner on top.
|
||
│ ├── crypto.py HL API-key envelope encryption
|
||
│ ├── scanner_state.py In-memory toggle + observability for scanners
|
||
│ ├── macro/
|
||
│ │ ├── fetchers.py 8 macro indicator HTTP fetchers (each @_none_on_fail)
|
||
│ │ ├── scoring.py Weighted composite -100..+100
|
||
│ │ └── poll.py Daily UPSERT into macro_snapshots
|
||
│ ├── scanners/
|
||
│ │ ├── btc_bottom_reversal.py 2-of-3 AHR999 + 200WMA + Pi Bottom
|
||
│ │ ├── funding_reversal.py Hourly funding extreme
|
||
│ │ └── sma_reclaim.py (archive — not scheduled)
|
||
│ ├── kol_substack.py RSS ingest for 29 KOL feeds (substack/blog/podcast)
|
||
│ ├── kol_x.py X (Twitter) ingest via twitterapi.io → x_analysis →
|
||
│ │ KolPost(source="twitter"). Daily 01:30 UTC. No-op
|
||
│ │ if twitterapi_io_key unset. Provides the post-side
|
||
│ │ feed for X-only KOLs (andrewkang, murad).
|
||
│ ├── kol_onchain.py HL public API + Etherscan diff
|
||
│ ├── kol_divergence.py Cross-ref talks vs trades within ±7d
|
||
│ ├── kol_analysis.py AI ticker/direction/conviction extract (Substack)
|
||
│ ├── bottom_indicators.py AHR999 / Pi Cycle / 200WMA math
|
||
│ ├── funding_signal.py Real-time funding extreme detector
|
||
│ └── signed_request.py EIP-191 signature verification (+ replay cache)
|
||
├── scrapers/
|
||
│ ├── truth_social.py CNN archive poller (5s interval) — primary
|
||
│ └── trumpstruth.py trumpstruth.org RSS fallback poller. Same post id
|
||
│ hash → automatic dedup. Whoever sees first wins.
|
||
│ Offset by half the interval so the two pollers
|
||
│ don't hit upstream simultaneously.
|
||
├── ws/
|
||
│ └── manager.py WebSocket fan-out for live UI updates.
|
||
│ Broadcasts trade_alert events (execution_failed /
|
||
│ insufficient_balance / budget_reached) via
|
||
│ _broadcast_trade_alert() in bot_engine.py.
|
||
├── models.py ★ All SQLAlchemy models in one file
|
||
├── database.py Async engine + session factory
|
||
├── config.py Pydantic Settings — reads .env
|
||
└── main.py FastAPI lifespan, scheduler setup, route mount
|
||
Includes: singleton lock guard (one-leader, multi-
|
||
worker safe), deep health check /api/health/deep,
|
||
boot-grace window for price feeds.
|
||
|
||
alembic/versions/ Migrations (numbered NNN). Latest = 026
|
||
026 = composite index (wallet_address, closed_at) on bot_trades
|
||
scripts/ One-shot ops
|
||
├── preflight.py Pre-launch readiness gate (env / DB / TG / AI)
|
||
├── launch_smoke.py End-to-end smoke (14 checks against running API)
|
||
├── launch_seed.py Pre-launch data prep: drops test sources, trims KOL
|
||
│ window to last 30d, refetches all upstream sources.
|
||
│ Run ONCE before flipping traffic to a fresh DB.
|
||
├── seed_kol_wallets.py Seeds the KOL wallet table with known addresses.
|
||
│ Idempotent (INSERT OR IGNORE). Run once at first deploy.
|
||
├── rescore_v5.py Re-score every Post with current AI prompt
|
||
├── backfill_signals.py Fill in signal for posts missing it
|
||
└── verify_sys2_lifecycle.py Manual System-2 lifecycle walk-through
|
||
|
||
tests/ pytest, 83 tests, fast (<3s total)
|
||
├── test_adoption.py Adoption + release flow (snapshot-style, no real HL/AI)
|
||
├── test_telegram_digest.py Daily digest formatting
|
||
├── test_kol_x.py X ingest: dedup / mapping / no-op (mocked fetch+AI)
|
||
├── test_ratelimit.py Rate limit coverage (BUG-02 fix)
|
||
├── test_bottom_reversal_strategy.py btc_bottom_reversal 2-of-3 logic
|
||
├── test_macro_fetchers_timing.py Macro fetcher timeout + error handling
|
||
└── test_production_readiness.py Environment / config sanity checks
|
||
```
|
||
|
||
---
|
||
|
||
## The two trading systems (memorise this)
|
||
|
||
```
|
||
System 1 System 2
|
||
──────── ────────
|
||
Source "truth" "btc_bottom_reversal"
|
||
(+"funding_reversal" alert-only)
|
||
Trigger Trump posts a thing Daily scanner: 2-of-3 confluence
|
||
Latency need Seconds (price moves fast) Days/weeks (signal lives a long time)
|
||
Open path Auto (bot_engine.process_post fires _execute_for_subscriber)
|
||
MANUAL: user opens on HL UI, then
|
||
/adopt hands it to the bot
|
||
Stop loss User-configured + tight sys2_protective_stop_pct(actual_lev)
|
||
1.5% floor (TRUMP_*) = 85% × (100/lev), capped at 35%
|
||
Exit model TP / trailing / SL 5-rung stop ladder + downside de-risk
|
||
+ pyramid + peak-trail. NO TP.
|
||
Min hold 30 min (suppresses TP) n/a
|
||
Max hold 6h 18 months (ladder is the real exit)
|
||
Sizing base × regime multiplier Whatever user opened on HL
|
||
Concurrency cap n/a 3 positions / wallet (correlated beta)
|
||
Confidence min 88 (platform) / user 85 (platform)
|
||
Circuit breaker sys1_* sys2_* (independent)
|
||
Daily budget Full daily_budget_usd n/a — user controls notional on HL
|
||
Telegram alert Trump alert format Macro/funding alert + /adopt CTA
|
||
X tweet Optional prediction tweet n/a
|
||
```
|
||
|
||
**If you're tempted to put sys2 logic in `_execute_for_subscriber`**: stop.
|
||
`process_post()` early-returns for sys2. The function only runs for sys1 now.
|
||
The dead sys2 branches inside `_execute_for_subscriber` are kept for diff
|
||
minimalism — don't extend them.
|
||
|
||
---
|
||
|
||
## The /adopt flow (System-2 lifecycle in detail)
|
||
|
||
```
|
||
1. Scanner fires
|
||
└─ POST /api/signals/ingest (source=btc_bottom_reversal, signal=buy)
|
||
└─ Post row created
|
||
└─ process_post() early-returns for sys2 (no auto-open)
|
||
└─ notify_signal() → Telegram fan-out with /adopt CTA appended
|
||
|
||
2. User opens BTC long on Hyperliquid manually
|
||
└─ size / leverage of their choice
|
||
|
||
3. User in bot: /adopt
|
||
└─ adoption.list_hl_positions(wallet) reads HL state
|
||
└─ Telegram inline keyboard: tap [🟢 BTC long $1500 @72k · 2x]
|
||
└─ Mode picker: [📈 Standard] or [🚀 Aggressive]
|
||
└─ adoption.adopt_position(wallet, asset, mode):
|
||
a. Per-wallet asyncio lock acquired
|
||
b. Pre-flight: no_subscription / no_hl_key / paper_mode /
|
||
circuit_breaker (sys2 CB still gates adopt!) /
|
||
already_adopted / concurrency_cap (3)
|
||
c. Re-read HL state inside lock (fresh entry/size/lev)
|
||
d. Reject if leverage > SYS2_MAX_LEVERAGE (BUG-09 fix)
|
||
e. Resolve sys2_protective_stop_pct(HL_actual_leverage)
|
||
f. INSERT BotTrade with eff_* frozen + sys2_mode + hl_order_id="adopted:<ts>"
|
||
+ trigger_post_id=NULL
|
||
g. register_trade() with full ladder/de-risk/addon/peak_trail
|
||
|
||
4. tp_sl_monitor drives the position
|
||
└─ Stop ratchet, downside de-risk partial reduces, pyramid add-ons,
|
||
peak-trail close, max_hold backstop. All staged through the
|
||
lock-protected partial_derisk / pyramid_add / close_and_finalize.
|
||
|
||
5a. User wants out: /release
|
||
└─ Sets BotTrade.released_at = now; unregister(trade_id) from watchdog
|
||
└─ HL position UNTOUCHED — bot stops driving, user has manual control
|
||
|
||
5b. Bot drives the close (ladder / max-hold)
|
||
└─ close_and_finalize() atomic claim sets closed_at, computes pnl
|
||
|
||
5c. User force-closes via UI: POST /api/positions/{id}/close
|
||
└─ manual_close calls close_and_finalize(force=True) — bypasses
|
||
released_at guard. Works on adopted-and-released trades too.
|
||
|
||
6. Recovery on restart:
|
||
└─ recovery.rehydrate_open_trades reads BotTrade WHERE closed_at IS NULL
|
||
AND released_at IS NULL
|
||
└─ For each sys2 trade: rebuild ladder from sys2_mode + adopted fallback.
|
||
Also reschedules _time_stop_check with remaining seconds (elapsed windows
|
||
fire immediately with delay=0). This is the critical fix — without it
|
||
adopted trades lose their entire sys2 ladder on restart.
|
||
```
|
||
|
||
---
|
||
|
||
## Singleton lock guard (critical infrastructure detail)
|
||
|
||
`main.py:lifespan` calls `_acquire_singleton_lock()` at boot using an
|
||
advisory file lock (`/tmp/trumpsignal-backend.lock`, configurable via
|
||
`SINGLETON_LOCK_PATH`). Only ONE process — "the leader" — starts the
|
||
scheduler, scrapers, price feeds, and Telegram poller. Any additional
|
||
worker (e.g. accidental `--workers 2`) serves HTTP reads only and logs
|
||
`SINGLETON LOCK NOT ACQUIRED`. The OS releases the lock automatically
|
||
when the leader exits, so a crashed leader unblocks the next start.
|
||
|
||
The `/api/health/deep` endpoint surfaces `"is_leader": false` and adds
|
||
it to `problems[]` so uptime monitors catch mis-configured multi-worker
|
||
deployments.
|
||
|
||
---
|
||
|
||
## Critical invariants checklist (when reviewing any trading change)
|
||
|
||
- [ ] Does this code path respect `released_at IS NULL`?
|
||
- [ ] Does it use `eff_*` (frozen) not live `Subscription.*` for exit math?
|
||
- [ ] If it opens a new position, does it use HL's actual leverage (not requested)?
|
||
- [ ] If it touches an open position concurrently, is it wrapped in the
|
||
per-trade `_lock_for(trade_id)` lock?
|
||
- [ ] If it opens, is it inside `_wallet_lock(wallet)` so the budget /
|
||
concurrency check is atomic with the write?
|
||
- [ ] If it closes, does it use the conditional `UPDATE ... WHERE closed_at
|
||
IS NULL` atomic claim?
|
||
- [ ] Does it handle the `already_closed` path from HL gracefully (preserve
|
||
banked partial PnL)?
|
||
- [ ] Does it correctly check sys1 vs sys2 CB independently?
|
||
|
||
---
|
||
|
||
## Common workflows
|
||
|
||
### Add a new signal source
|
||
|
||
1. Decide: System 1 (auto-trade) or System 2 (alert + /adopt) or alert-only?
|
||
2. Write a scanner under `app/services/scanners/NEW.py` that posts to
|
||
`POST /api/signals/ingest` with `{source: "your_new_source", ...}`.
|
||
3. Schedule it in `app/main.py` (`_scheduler.add_job`).
|
||
4. Add the source to `signal_categories.SYSTEM_1_SOURCES` or
|
||
`SYSTEM_2_SOURCES` if it should trade. Leave it out if alert-only.
|
||
5. Add a Telegram preference column to `TelegramBinding` (migration)
|
||
+ a mapping entry in `telegram._pref_column_for_source`.
|
||
6. Add a label to `telegram._source_label` and `_signal_emoji`.
|
||
7. Add a `/yoursource on|off` command in `telegram_bot.py`.
|
||
8. If sys2: extend `signal_categories._CATEGORY_EXITS` if it needs a custom
|
||
exit profile (otherwise default works).
|
||
9. Add the deep-link path in `telegram.format_post` AND `telegram.format_public_post`.
|
||
|
||
### Add a new bot command
|
||
|
||
1. `_cmd_x` async function in `telegram_bot.py`.
|
||
2. Route it in `_handle_message`.
|
||
3. If it needs inline buttons: build `reply_markup` payload, handle
|
||
callbacks in `_handle_callback` (route by `callback_data` prefix).
|
||
4. Update `HELP_TEXT` and remind the user to add it to BotFather
|
||
`/setcommands` after deploy.
|
||
|
||
### Add a column to an existing table
|
||
|
||
1. New migration `alembic/versions/NNN_description.py`.
|
||
- Use `op.batch_alter_table` (sqlite-compatible).
|
||
- Default values via `server_default=` so backfill is implicit.
|
||
2. Mirror the field on the SQLAlchemy model in `app/models.py`.
|
||
3. Apply locally: `DATABASE_URL=<sqlite> alembic upgrade head`.
|
||
|
||
### Enable X (Twitter) posting
|
||
|
||
1. Create a Twitter developer app with OAuth 1.0a user context permissions.
|
||
2. Set in `.env`: `X_API_KEY`, `X_API_SECRET`, `X_ACCESS_TOKEN`,
|
||
`X_ACCESS_SECRET`, `X_ENABLED=true`.
|
||
3. Optionally tune: `X_DAILY_CAP` (default 40), `X_FOLLOWUP_MINUTES` (default 15).
|
||
4. Verify with a dry-run: set `X_ENABLED=false` and check logs — the poster
|
||
logs what it WOULD tweet without sending.
|
||
|
||
### Deploy
|
||
|
||
```bash
|
||
# On the server:
|
||
DATABASE_URL=$PROD_URL alembic upgrade head
|
||
systemctl restart trumpalpha-backend # or whatever the unit is
|
||
python scripts/preflight.py # MUST pass before flipping traffic
|
||
python scripts/launch_smoke.py --base https://api.trumpalpha.io
|
||
```
|
||
|
||
---
|
||
|
||
## Running it
|
||
|
||
```bash
|
||
cd backend && python -m venv venv && source venv/bin/activate
|
||
pip install -r requirements.txt
|
||
cp .env.example .env # fill in: ENCRYPTION_KEY, AI_API_KEY, INGEST_API_KEY
|
||
uvicorn app.main:app --reload # dev only; prod uses --workers 1, no --reload
|
||
```
|
||
|
||
---
|
||
|
||
## Testing
|
||
|
||
```bash
|
||
source venv/bin/activate
|
||
python -m pytest tests/ -q # 79 tests, ~2s
|
||
python scripts/preflight.py # env + DB + TG + AI auth checks
|
||
python scripts/launch_smoke.py # 14 end-to-end checks vs running API
|
||
```
|
||
|
||
Adoption + telegram_digest are snapshot-style (no real HL/AI).
|
||
End-to-end trading is verified manually via the bot.
|
||
|
||
---
|
||
|
||
## Telegram bot mechanics (since it's a custom integration)
|
||
|
||
- **Long-poll mode** via `getUpdates`. Only ONE process can long-poll a
|
||
given bot token at a time — if you horizontally scale, switch to
|
||
webhook (not done yet).
|
||
- Bot must be re-bound via `@BotFather` `/setcommands` whenever new
|
||
commands are added (the slash-menu users see is separate from what
|
||
the bot internally handles).
|
||
- `send_message` returns `False` on failure; per-user binding rows track
|
||
`total_alerts_sent` / `total_alerts_failed` counters.
|
||
- **Inline keyboards** = the `reply_markup` payload to `sendMessage`.
|
||
Callback data is capped at 64 bytes; keep it short (`adopt:mode:BTC:standard`).
|
||
`_handle_callback` MUST end with `answer_callback` or the button spins
|
||
forever on the user's client.
|
||
- Free tier = walletless `/start` (chat_id only). Pro tier = wallet bound
|
||
via `/start CODE` where CODE comes from Settings UI.
|
||
|
||
---
|
||
|
||
## Why "Macro Vibes" became manage-only (the ADR)
|
||
|
||
V1.0: System 2 auto-opened sys2 trades on user wallets. Carried real
|
||
execution surface: leverage clipping, daily budget split, concurrency caps,
|
||
sys2 paper branches, key handling per user. Audit surfaced ~6 bugs.
|
||
|
||
V2.0 (current): sys2 manage-only. The strategy is day-K — entry delay of
|
||
24h doesn't matter. The valuable part is multi-month exit management
|
||
(5-rung ladder, de-risk, pyramid, peak-trail), which still runs against
|
||
positions the user adopts.
|
||
|
||
**Net effect**: massive reduction in execution risk surface; same alpha
|
||
(strategy logic unchanged); legal/responsibility shifts from "bot opened
|
||
this for you" to "you opened it, bot manages your discipline".
|
||
|
||
---
|
||
|
||
## Things that LOOK like bugs but aren't
|
||
|
||
- **`_execute_for_subscriber` has lots of `if sub["_is_system_2"]` branches.**
|
||
Dead code under v2.0 (process_post early-returns for sys2). Kept for diff
|
||
minimalism — don't extend or re-enable.
|
||
- **`kol_x.X_KOLS` handle ≠ X username.** `handle` is the CANONICAL key
|
||
(e.g. "cryptohayes") that MUST match `KolWallet.handle` so divergence can
|
||
join post-side ↔ on-chain; `x_username` is the screen name fetched from X.
|
||
andrewkang/murad wallets stay dark (zero divergence detections) until X
|
||
ingestion supplies their post side — that's the whole reason kol_x exists.
|
||
Empty `twitterapi_io_key` → kol_x is a full no-op (no error).
|
||
- **`funding_reversal` source is in `SYSTEM_2_SOURCES`? No.** It's
|
||
intentionally NOT in either supported set — it ingests as a Post for
|
||
audit + sends Telegram alert via the CTA path, but doesn't trigger any
|
||
auto-trading. Adopt still works (it's asset-based, not signal-based).
|
||
- **`Subscription.sys2_budget_pct` defaults to 0.7.** Legacy field from the
|
||
auto-open era. With v2.0 manage-only, it's effectively unused — sys1
|
||
(Trump) reads full `daily_budget_usd`. Don't read it for new code.
|
||
- **Adopted trades have `hl_order_id` starting with `"adopted:"`.** Distinct
|
||
from auto-opened (HL order id integer) and paper (`"paper"` literal).
|
||
Useful for telemetry filtering.
|
||
- **`telegram.send_message` accepts `int | str` for `chat_id`.** Intentional.
|
||
Integer = private chat, string = public channel username (e.g. `"@trumpalpha"`).
|
||
- **`format_public_post` deliberately omits `expected_move_pct`,
|
||
`invalidation_price`, and `/adopt` CTA.** Execution-sensitive data stays
|
||
private. The public version shows confidence tier (HIGH/MED/LOW) instead
|
||
of the raw score.
|
||
- **`_adopt_locks` in adoption.py** is an `OrderedDict` capped at 512 with
|
||
LRU eviction — matches the `_WALLET_LOCK_MAX` pattern in `bot_engine`.
|
||
- **`trumpstruth.py` runs at half-interval offset** — the two CNN + trumpstruth
|
||
pollers are deliberately staggered so they don't hammer upstream simultaneously.
|
||
The offset is `truth_social_poll_seconds // 2`, set via APScheduler
|
||
`next_run_time` at boot.
|
||
|
||
---
|
||
|
||
## Open known issues (not blocking launch but worth fixing later)
|
||
|
||
- **`adopt:choose:BTC` callback may show stale prices** if user takes >60s
|
||
to tap (HL fees, partial fills can change entry/size). adopt_position
|
||
re-reads HL at mode-tap time so the FROZEN BotTrade is always fresh,
|
||
but the picker label could be outdated.
|
||
|
||
- ~~**Telegram bot offset on restart**~~ **FIXED 2026-06-01**: startup drain
|
||
added. Stale `/adopt` replays suppressed.
|
||
|
||
- **`/adopt` picker label** can show stale price if user waits >60s to tap
|
||
(frozen BotTrade is always fresh, but the Telegram picker label may be outdated).
|
||
|
||
**Deferred security items (need interface change or DB migration):**
|
||
- **C3**: Read endpoints pass `ts`/`sig` as URL query params → access log exposure.
|
||
- **H4**: HL API key KEK uses single unsalted SHA-256. Needs re-encryption migration.
|
||
- **M1**: Adopted sys2 positions miscounted against sys1 daily budget.
|
||
- **M5**: `GET /telegram/{wallet}/status` unauthenticated — exposes chat_id/tg_username.
|
||
|
||
---
|
||
|
||
## Repos in this project
|
||
|
||
- **This repo** (`/Users/k/Public/trumpsignal/backend`) — Python/FastAPI backend
|
||
- **Sibling frontend** (`/Users/k/Public/trumpsignal/frontend`) — Next.js 16
|
||
dashboard at trumpsignal.com. See its own CLAUDE.md.
|
||
|
||
Both deployed independently. Backend serves the JSON API + Telegram bot.
|
||
Frontend is a thin SPA over the API + WebSocket.
|