Pre-launch hardening: KOL module, Telegram, scanners, WS resilience

Big-picture changes since b941223:

KOL pipeline (new) — Substack/podcast/blog RSS → AI ticker extraction →
on-chain wallet diff → talks-vs-trades divergence detection. Daily polls,
19 feeds, divergence emits Post + Telegram fan-out.

Telegram push (new) — walletless free tier + wallet-linked Pro upgrade,
in-bot preference commands (/trump /btc /funding /kol /conf /quiet),
signed-envelope API for dashboard. Disconnect-wallet keeps free
subscription.

BTC funding-rate reversal scanner (new) — hourly cron, 30d cumulative
funding threshold + mean-revert + 7d price confirm, emits via
/api/signals/ingest. BTC bottom-reversal scanner promoted to System 2.

WS broadcast rewrite — per-client send timeout + parallel fan-out
(asyncio.gather). Fixes "Binance WS no close frame" reconnect storms +
APScheduler 11-min job misses, both caused by one slow client stalling
the kline loop.

Error visibility — three silent-error sites (trumpstruth/truth_social
fetchers, funding_reversal scanner) now include exception type name so
httpx ConnectError-style empty-message errors stop logging blank lines.
Telegram bot loop now classifies ReadTimeout vs network vs unknown +
logger.exception for the unknown bucket.

Security hygiene — trumpsignal.db untracked from git (held subscriber
wallets + encrypted HL keys + 22 bot trades); .gitignore now blocks
*.db/.next/backups. CORS only allows FRONTEND_URL in production.

New ops scripts —
  - scripts/preflight.py: env/DB/Telegram/AI auth verification gate
  - scripts/backup_db.sh: cron-friendly daily DB backup (SQLite + Postgres)
  - scripts/seed_kol_wallets.py: idempotent KOL on-chain wallet seeder

15 new Alembic migrations (007-021) covering convex strategy fields,
phase-1 safety, two-system frozen exits, invalidation prices, dynamic
SYS2 leverage, staged de-risk + pyramiding, peak gain tracking, risk
mode, auto-trade + grow flags, KOL module, KOL on-chain, KOL divergence,
Telegram bindings + walletless.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
k
2026-05-25 00:52:56 +08:00
parent b941223c88
commit 5fb1d52026
81 changed files with 13251 additions and 158 deletions
+397 -23
View File
@@ -1,21 +1,39 @@
"""
Take-profit / stop-loss monitor.
Take-profit / stop-loss / trailing-stop monitor.
Subscribes to the Binance price stream; for every open trade that has TP/SL set,
closes the HL position as soon as the live mark price crosses the threshold.
Subscribes to the Binance price stream; for every open trade, closes the HL
position when the live mark price crosses one of:
1. Fixed stop-loss (always active).
2. Fixed take-profit (optional — set to None for trend capture).
3. Trailing stop (optional — once peak profit ≥ trailing_activate_at_pct,
close if price drops trailing_stop_pct from the peak).
The trailing branch is the centrepiece of the convex-payoff redesign:
fixed TP at +2% kills every potential runner before it starts. Instead we
let unrealised PnL grow, watching the peak, and only exit on a real
retracement.
Lifecycle:
- bot_engine.open_position calls register_trade(...) after each new trade
- price callback in binance.py calls on_price_tick() once per second
- when TP or SL hits, we enqueue close_and_finalize(...) and de-register
- binance.py's price callback calls on_price_tick() once per second
- when a rule fires, _fire_close(...) calls bot_engine.close_and_finalize
"""
import asyncio
import logging
from dataclasses import dataclass
import time
from dataclasses import dataclass, field
from typing import Dict, Optional
logger = logging.getLogger(__name__)
# Throttled peak persistence: write peak_gain_pct to the DB at most once per
# PEAK_FLUSH_SECS per trade, and only after it advances ≥ PEAK_FLUSH_DELTA pp.
# Peak is monotonic so writes are rare after the initial run-up; this is
# enough to keep the post-restart regime correct without per-tick I/O.
PEAK_FLUSH_DELTA = 1.0
PEAK_FLUSH_SECS = 30.0
@dataclass
class WatchedTrade:
@@ -26,14 +44,83 @@ class WatchedTrade:
asset: str
side: str # "long" | "short"
entry_price: float
take_profit_pct: Optional[float]
stop_loss_pct: Optional[float]
take_profit_pct: Optional[float] # legacy fixed TP (may be None)
stop_loss_pct: Optional[float] # always required in practice
trailing_stop_pct: Optional[float] # trail distance, e.g. 2.5
trailing_activate_at_pct: Optional[float] # activation threshold, e.g. 5.0
# System-2 only. "below_entry": exit immediately if price crosses back
# through entry BEFORE trailing arms — the reversal thesis (e.g. SMA
# reclaim) is dead, no reason to wait for the wide hard stop. None for
# System-1 / unset.
invalidation: Optional[str] = None
invalidation_price: Optional[float] = None
# Staged stop-loss ladder (System-2, optional). List of
# (peak_gain_trigger_pct, stop_floor_pct) sorted ascending. When set,
# this trade has NO take-profit and NO from-peak trailing: the only
# market exit is "price fell back to the highest staged floor that the
# peak gain has unlocked". See signal_categories.get_stop_ladder.
stop_ladder: Optional[list] = None
# Staged DOWNSIDE de-risk ladder (System-2, optional). List of
# (threshold_signed_pct_negative, frac_of_original, is_final) ordered by
# increasing adversity. While the trade is underwater (peak below the
# first stop_ladder trigger) it is scaled OUT in partial reduces at each
# rung; the final rung is a full close at the protective level (inside
# liquidation). See signal_categories.sys2_derisk_ladder.
derisk_ladder: Optional[list] = None
derisk_done: int = 0
derisk_in_flight: bool = field(default=False)
# Pyramiding (做对了往上加仓): add to a confirmed winner. List of
# (peak_gain_trigger_pct, frac_of_base, is_last). Only active in the
# profit regime AND while derisk_done==0. Each add blends entry up; the
# monitor then floors the stop at breakeven (eff_stop ≥ 0) so a pyramided
# winner can never become a loser.
addon_ladder: Optional[list] = None
addon_done: int = 0
addon_in_flight: bool = field(default=False)
# Per-trade "Grow" switch. Pyramiding only runs when True. De-risk +
# ratchet stop are unaffected (safety floor is always on). Toggled live
# by the user via POST /positions/{id}/grow.
grow_mode: bool = field(default=False)
# Parabolic-top trailing: (activate_peak_gain_pct, price_drawdown_frac).
# In the profit regime, once peak ≥ activate, the floor also trails the
# peak PRICE by at most drawdown_frac (scale-invariant). None = use only
# the fixed stop_ladder rungs. See signal_categories.sys2_peak_trail.
peak_trail: Optional[tuple] = None
# Throttled persistence of peak_gain_pct (monotonic). peak_persisted is
# the last value written to the DB; peak_persist_ts the last write time.
peak_persisted: float = field(default=0.0)
peak_persist_ts: float = field(default=0.0)
# System-1 (Trump) min-hold floor. Epoch seconds; before this time,
# take-profit and trailing exits are SUPPRESSED (don't scalp out of a
# developing move). Hard stop-loss + invalidation always fire — capital
# protection isn't deferrable. None = no floor (System 2 / legacy).
min_hold_until_ts: Optional[float] = None
# Runtime state — peak unrealised gain seen so far, in %. Used by the
# trailing branch. Long: peak = max(over time) of pct_gain. Short: same,
# but pct_gain is already signed correctly by the caller.
peak_gain_pct: float = field(default=0.0)
trailing_armed: bool = field(default=False)
# Buffer below entry before "below_entry" invalidation fires. Entry price is
# only a PROXY for the true thesis-invalidation level (e.g. the 200d SMA for
# an sma_reclaim). 0.75% is roughly one normal noise band on a major coin —
# big enough to survive the fill-tick wobble, small enough to still cut a
# failed reclaim well before the wide hard stop.
INVALIDATION_BUFFER_PCT = 0.75
# trade_id -> WatchedTrade
_watched: Dict[int, WatchedTrade] = {}
# Strong references to fire-close tasks to prevent GC before completion
# Strong refs to fire-close tasks (prevent GC before completion)
_background_tasks: set = set()
@@ -45,18 +132,58 @@ def register_trade(
asset: str,
side: str,
entry_price: float,
take_profit_pct: Optional[float],
stop_loss_pct: Optional[float],
take_profit_pct: Optional[float],
stop_loss_pct: Optional[float],
trailing_stop_pct: Optional[float] = None,
trailing_activate_at_pct: Optional[float] = None,
invalidation: Optional[str] = None,
invalidation_price: Optional[float] = None,
min_hold_until_ts: Optional[float] = None,
stop_ladder: Optional[list] = None,
derisk_ladder: Optional[list] = None,
derisk_done: int = 0,
addon_ladder: Optional[list] = None,
addon_done: int = 0,
initial_peak: float = 0.0,
peak_trail: Optional[tuple] = None,
grow_mode: bool = False,
) -> None:
if take_profit_pct is None and stop_loss_pct is None:
return # nothing to watch
# Nothing to monitor → skip. We require AT LEAST a stop-loss for any
# registered trade; protocol still works without TP if trailing or a
# staged-stop / de-risk ladder is set.
if (stop_loss_pct is None
and take_profit_pct is None
and trailing_stop_pct is None
and not stop_ladder
and not derisk_ladder):
return
_watched[trade_id] = WatchedTrade(
trade_id=trade_id, wallet=wallet, api_key=api_key, leverage=leverage,
asset=asset, side=side, entry_price=entry_price,
take_profit_pct=take_profit_pct, stop_loss_pct=stop_loss_pct,
take_profit_pct=take_profit_pct,
stop_loss_pct=stop_loss_pct,
trailing_stop_pct=trailing_stop_pct,
trailing_activate_at_pct=trailing_activate_at_pct,
invalidation=invalidation,
invalidation_price=invalidation_price,
min_hold_until_ts=min_hold_until_ts,
stop_ladder=stop_ladder,
derisk_ladder=derisk_ladder,
derisk_done=derisk_done or 0,
addon_ladder=addon_ladder,
addon_done=addon_done or 0,
peak_gain_pct=max(0.0, float(initial_peak or 0.0)),
peak_persisted=max(0.0, float(initial_peak or 0.0)),
peak_trail=peak_trail,
grow_mode=bool(grow_mode),
)
logger.info(
"Watching trade %d (%s %s @ %.4f, sl=%s, tp=%s, trail=%s/@%s)",
trade_id, side, asset, entry_price,
stop_loss_pct, take_profit_pct,
trailing_stop_pct, trailing_activate_at_pct,
)
logger.info("TP/SL watching trade %d (%s %s @ %.2f, tp=%s, sl=%s)",
trade_id, side, asset, entry_price, take_profit_pct, stop_loss_pct)
def unregister(trade_id: int) -> None:
@@ -64,21 +191,163 @@ def unregister(trade_id: int) -> None:
def on_price_tick(asset: str, price: float) -> None:
"""Called from binance.py on every price update. Fires close for any trade that hit TP/SL."""
"""Called from binance.py on every price update.
Iterates watched trades on this asset, evaluates each rule, and triggers
a close on the first match. Rule evaluation order: stop-loss → trailing
→ fixed TP. (Stop-loss first so a sudden gap kills the position before
we celebrate hitting trailing.)
"""
if not _watched:
return
triggered = []
derisk_actions = [] # (wt, step_idx, frac_of_original)
addon_actions = [] # (wt, step_idx, frac_of_base)
for tid, wt in list(_watched.items()):
if wt.asset != asset:
continue
pct = (price - wt.entry_price) / wt.entry_price
signed_pct = pct if wt.side == 'long' else -pct # gain in position's direction
pct_x100 = signed_pct * 100
# Convert raw price → signed gain in position's direction, in %.
raw_pct = (price - wt.entry_price) / wt.entry_price
signed_pct = (raw_pct if wt.side == "long" else -raw_pct) * 100
if wt.take_profit_pct is not None and pct_x100 >= wt.take_profit_pct:
triggered.append((wt, "take_profit"))
elif wt.stop_loss_pct is not None and pct_x100 <= -wt.stop_loss_pct:
# Update peak BEFORE evaluating trailing — a tick that pushes the peak
# AND retraces in the same step should still arm trailing at the new peak.
if signed_pct > wt.peak_gain_pct:
wt.peak_gain_pct = signed_pct
# Throttled monotonic persistence so a restart keeps the regime.
now_s = time.time()
if (wt.peak_gain_pct - wt.peak_persisted >= PEAK_FLUSH_DELTA
and now_s - wt.peak_persist_ts >= PEAK_FLUSH_SECS):
wt.peak_persisted = wt.peak_gain_pct
wt.peak_persist_ts = now_s
_pt = asyncio.create_task(
_persist_peak(wt.trade_id, wt.peak_gain_pct))
_background_tasks.add(_pt)
_pt.add_done_callback(_background_tasks.discard)
# 0. System-2 self-contained exit model: NO take-profit, NO from-peak
# trailing. Two regimes, fully replacing branches 1/1b/2/3:
# • Underwater → STAGED DE-RISK ("分段式减仓"): partial reduces
# at each downside rung, full close at the final (protective,
# inside-liquidation) rung.
# • In profit → STAGED STOP: floor ratchets up as peak gain
# crosses each upside rung; full close when price falls back.
if wt.stop_ladder or wt.derisk_ladder:
first_up = (min(t for t, _ in wt.stop_ladder)
if wt.stop_ladder else None)
in_profit_regime = (first_up is not None
and wt.peak_gain_pct >= first_up)
if wt.derisk_ladder and not in_profit_regime:
ladder = wt.derisk_ladder
# Gap protection: blown straight through to the final
# protective level → full close NOW regardless of step
# bookkeeping (still inside the exchange liquidation line).
if signed_pct <= ladder[-1][0]:
triggered.append((wt, "staged_stop"))
continue
if (not wt.derisk_in_flight
and 0 <= wt.derisk_done < len(ladder)):
thr, frac, is_final = ladder[wt.derisk_done]
if signed_pct <= thr:
if is_final:
triggered.append((wt, "staged_stop"))
continue
# Partial reduce — spawn async, guard re-entry until
# it completes (done-callback clears the flag).
wt.derisk_in_flight = True
derisk_actions.append((wt, wt.derisk_done, frac))
continue # underwater: no profit-ratchet / TP / trailing
# Profit regime — ratchet floor (base + unlocked upside rungs).
eff_stop = -wt.stop_loss_pct if wt.stop_loss_pct is not None else float("-inf")
if wt.stop_ladder:
for trig, floor in wt.stop_ladder: # sorted ascending
if wt.peak_gain_pct >= trig and floor > eff_stop:
eff_stop = floor
# Parabolic-top trailing: once deep in profit, also trail the PEAK
# PRICE by ≤ drawdown_frac (scale-invariant — self-adjusts to a
# +120% or +900% move so the fixed top rung doesn't cap it, while
# a normal ~2530% bull pullback still doesn't trip it).
if wt.peak_trail is not None:
start_pp, dd = wt.peak_trail
if wt.peak_gain_pct >= start_pp:
trail_floor = ((1.0 + wt.peak_gain_pct / 100.0)
* (1.0 - dd) - 1.0) * 100.0
if trail_floor > eff_stop:
eff_stop = trail_floor
# Pyramided → never let a winner become a loser: blended position
# is floored at breakeven once any add-on has filled.
if wt.addon_done > 0 and eff_stop < 0.0:
eff_stop = 0.0
if signed_pct <= eff_stop:
triggered.append((wt, "staged_stop"))
continue
# Pyramiding — add to a confirmed winner. Gated by the per-trade
# Grow switch (off by default — user opts a position in). Evaluated
# ONLY after the stop check above passed (never add on a tick that's
# also exiting). Peak-gain triggered; only while no de-risk has
# fired. Structural confirmation is re-checked in the executor.
if (wt.grow_mode and wt.addon_ladder and wt.derisk_done == 0
and not wt.addon_in_flight
and 0 <= wt.addon_done < len(wt.addon_ladder)):
a_trig, a_frac, _a_last = wt.addon_ladder[wt.addon_done]
if wt.peak_gain_pct >= a_trig:
wt.addon_in_flight = True
addon_actions.append((wt, wt.addon_done, a_frac))
continue
# 1. Stop loss — first priority.
if wt.stop_loss_pct is not None and signed_pct <= -wt.stop_loss_pct:
triggered.append((wt, "stop_loss"))
continue
# 1b. Signal invalidation (System-2 only). Prefer the real thesis
# level when available (e.g. 200d SMA reclaim price). Fall back
# to entry-buffer proxy for legacy rows.
if wt.invalidation == "below_entry" and not wt.trailing_armed:
if wt.invalidation_price is not None:
invalidation_hit = (
(wt.side == "long" and price <= wt.invalidation_price) or
(wt.side == "short" and price >= wt.invalidation_price)
)
if invalidation_hit:
triggered.append((wt, "invalidation"))
continue
elif signed_pct <= -INVALIDATION_BUFFER_PCT:
triggered.append((wt, "invalidation"))
continue
# Min-hold gate (System-1 Trump). Before the floor elapses we let a
# winner develop — suppress TP + trailing. SL / invalidation above
# already ran, so downside is still protected; we only defer the
# PROFIT-side exits here.
if wt.min_hold_until_ts is not None:
import time as _t
if _t.time() < wt.min_hold_until_ts:
continue # still in min-hold; skip TP/trailing this tick
# 2. Trailing stop — only once armed.
if (wt.trailing_stop_pct is not None
and wt.trailing_activate_at_pct is not None):
if not wt.trailing_armed and wt.peak_gain_pct >= wt.trailing_activate_at_pct:
wt.trailing_armed = True
logger.info(
"Trade %d: trailing armed at peak %.2f%% (asset=%s)",
wt.trade_id, wt.peak_gain_pct, asset,
)
if wt.trailing_armed:
drawdown_from_peak = wt.peak_gain_pct - signed_pct
if drawdown_from_peak >= wt.trailing_stop_pct:
triggered.append((wt, "trailing_stop"))
continue
# 3. Fixed TP — legacy / fallback when no trailing config.
if wt.take_profit_pct is not None and signed_pct >= wt.take_profit_pct:
triggered.append((wt, "take_profit"))
continue
for wt, reason in triggered:
unregister(wt.trade_id)
@@ -86,10 +355,115 @@ def on_price_tick(asset: str, price: float) -> None:
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
# Partial de-risk steps — trade stays registered (it's not closed).
for wt, step_idx, frac in derisk_actions:
task = asyncio.create_task(_fire_partial(wt, step_idx, frac))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
# Pyramid add-ons — trade stays registered (size grows, entry blends).
for wt, step_idx, frac in addon_actions:
task = asyncio.create_task(_fire_pyramid(wt, step_idx, frac))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
async def _fire_partial(wt: WatchedTrade, step_idx: int, frac: float) -> None:
"""Execute one staged de-risk partial reduce. The trade stays open and
registered; on success advance derisk_done, always clear the in-flight
guard so the next rung can fire on a later tick."""
from app.services.bot_engine import partial_derisk
ok = False
try:
ok = await partial_derisk(
trade_id=wt.trade_id,
api_key=wt.api_key,
asset=wt.asset,
wallet=wt.wallet,
step_idx=step_idx,
frac_of_original=frac,
reason="derisk",
)
except Exception as exc:
logger.error("De-risk partial failed for trade %d step %d: %s",
wt.trade_id, step_idx, exc)
finally:
# Re-fetch: the trade may have been closed/unregistered meanwhile.
live = _watched.get(wt.trade_id)
if live is not None:
if ok and live.derisk_done == step_idx:
live.derisk_done = step_idx + 1
live.derisk_in_flight = False
async def _fire_pyramid(wt: WatchedTrade, step_idx: int, frac: float) -> None:
"""Execute one pyramid add-on. On success re-base the in-memory entry to
the blended average and reset peak to the post-add gain so the ratchet /
regime use the NEW entry (and we stay in the profit regime). Always clear
the in-flight guard."""
from app.services.bot_engine import pyramid_add
ok, new_entry, ref_price = False, None, None
try:
ok, new_entry, ref_price = await pyramid_add(
trade_id=wt.trade_id,
api_key=wt.api_key,
asset=wt.asset,
wallet=wt.wallet,
step_idx=step_idx,
frac_of_base=frac,
reason="pyramid",
)
except Exception as exc:
logger.error("Pyramid add failed for trade %d step %d: %s",
wt.trade_id, step_idx, exc)
finally:
live = _watched.get(wt.trade_id)
if live is not None:
if ok and live.addon_done == step_idx:
# Step is RESOLVED (added, notional-capped, or already-done).
# Advance regardless of whether an add actually filled, else
# the monitor re-schedules this step every tick forever.
if new_entry:
live.entry_price = new_entry
if ref_price:
raw = (ref_price - new_entry) / new_entry
g = (raw if live.side == "long" else -raw) * 100.0
else:
g = 0.0
# Stay in the profit regime; ratchet re-accumulates here.
first_up = (min(t for t, _ in live.stop_ladder)
if live.stop_ladder else 0.0)
live.peak_gain_pct = max(g, first_up)
live.addon_done = step_idx + 1
live.addon_in_flight = False
async def _persist_peak(trade_id: int, peak: float) -> None:
"""Monotonic best-effort write of peak_gain_pct. Race-safe: only raises
the stored value (WHERE peak_gain_pct < :peak), never lowers it."""
try:
from sqlalchemy import update
from app.database import AsyncSessionLocal
from app.models import BotTrade
async with AsyncSessionLocal() as db:
await db.execute(
update(BotTrade)
.where(BotTrade.id == trade_id)
.where(BotTrade.peak_gain_pct < peak)
.values(peak_gain_pct=peak)
)
await db.commit()
except Exception as exc: # never let persistence break the monitor
logger.debug("peak persist failed trade %d: %s", trade_id, exc)
async def _fire_close(wt: WatchedTrade, reason: str) -> None:
from app.services.bot_engine import close_and_finalize
try:
logger.info(
"Closing trade %d on %s (peak=%.2f%%, reason=%s)",
wt.trade_id, wt.asset, wt.peak_gain_pct, reason,
)
await close_and_finalize(
trade_id=wt.trade_id,
api_key=wt.api_key,