Files
k 54884f3e24 KOL feeds: fix dead/blocked sources, drop stale feeds (29→25)
Feed-health pass over KOL_FEEDS:
- raoulpal: stale Substack (last 2024-05) → Real Vision podcast feed
- dampedspring: paywalled (0 entries) → free "Damped Spring 101" Substack
- unchained: Cloudflare 403 → canonical Megaphone podcast feed
- lynalden: Cloudflare 202 → FeedBurner mirror
- glassnode: recovered via httpx http2=True (was 403 on HTTP/1.1)
- browser User-Agent + Accept headers on feed fetch
- removed dead feeds with no active replacement: placeholder,
  dragonfly, niccarter, eugene
- pin h2==4.3.0 (required by http2=True)

All 25 remaining feeds verified fetching real body content; newest
post per feed ≤88d. Bundles in-flight KOL-module work already in the
working tree (kol_x ingest, migration 027, tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 22:55:16 +08:00

501 lines
22 KiB
Python
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""
Take-profit / stop-loss / trailing-stop monitor.
Subscribes to the Binance price stream; for every open trade, closes the HL
position when the live mark price crosses one of:
1. Fixed stop-loss (always active).
2. Fixed take-profit (optional — set to None for trend capture).
3. Trailing stop (optional — once peak profit ≥ trailing_activate_at_pct,
close if price drops trailing_stop_pct from the peak).
The trailing branch is the centrepiece of the convex-payoff redesign:
fixed TP at +2% kills every potential runner before it starts. Instead we
let unrealised PnL grow, watching the peak, and only exit on a real
retracement.
Lifecycle:
- bot_engine.open_position calls register_trade(...) after each new trade
- binance.py's price callback calls on_price_tick() once per second
- when a rule fires, _fire_close(...) calls bot_engine.close_and_finalize
"""
import asyncio
import logging
import time
from dataclasses import dataclass, field
from typing import Dict, Optional
logger = logging.getLogger(__name__)
# Throttled peak persistence: write peak_gain_pct to the DB at most once per
# PEAK_FLUSH_SECS per trade, and only after it advances ≥ PEAK_FLUSH_DELTA pp.
# Peak is monotonic so writes are rare after the initial run-up; this is
# enough to keep the post-restart regime correct without per-tick I/O.
PEAK_FLUSH_DELTA = 1.0
PEAK_FLUSH_SECS = 30.0
@dataclass
class WatchedTrade:
trade_id: int
wallet: str
api_key: str
leverage: int
asset: str
side: str # "long" | "short"
entry_price: float
take_profit_pct: Optional[float] # legacy fixed TP (may be None)
stop_loss_pct: Optional[float] # always required in practice
trailing_stop_pct: Optional[float] # trail distance, e.g. 2.5
trailing_activate_at_pct: Optional[float] # activation threshold, e.g. 5.0
# System-2 only. "below_entry": exit immediately if price crosses back
# through entry BEFORE trailing arms — the reversal thesis (e.g. SMA
# reclaim) is dead, no reason to wait for the wide hard stop. None for
# System-1 / unset.
invalidation: Optional[str] = None
invalidation_price: Optional[float] = None
# Staged stop-loss ladder (System-2, optional). List of
# (peak_gain_trigger_pct, stop_floor_pct) sorted ascending. When set,
# this trade has NO take-profit and NO from-peak trailing: the only
# market exit is "price fell back to the highest staged floor that the
# peak gain has unlocked". See signal_categories.get_stop_ladder.
stop_ladder: Optional[list] = None
# Staged DOWNSIDE de-risk ladder (System-2, optional). List of
# (threshold_signed_pct_negative, frac_of_original, is_final) ordered by
# increasing adversity. While the trade is underwater (peak below the
# first stop_ladder trigger) it is scaled OUT in partial reduces at each
# rung; the final rung is a full close at the protective level (inside
# liquidation). See signal_categories.sys2_derisk_ladder.
derisk_ladder: Optional[list] = None
derisk_done: int = 0
derisk_in_flight: bool = field(default=False)
# Pyramiding (做对了往上加仓): add to a confirmed winner. List of
# (peak_gain_trigger_pct, frac_of_base, is_last). Only active in the
# profit regime AND while derisk_done==0. Each add blends entry up; the
# monitor then floors the stop at breakeven (eff_stop ≥ 0) so a pyramided
# winner can never become a loser.
addon_ladder: Optional[list] = None
addon_done: int = 0
addon_in_flight: bool = field(default=False)
# Per-trade "Grow" switch. Pyramiding only runs when True. De-risk +
# ratchet stop are unaffected (safety floor is always on). Toggled live
# by the user via POST /positions/{id}/grow.
grow_mode: bool = field(default=False)
# Parabolic-top trailing: (activate_peak_gain_pct, price_drawdown_frac).
# In the profit regime, once peak ≥ activate, the floor also trails the
# peak PRICE by at most drawdown_frac (scale-invariant). None = use only
# the fixed stop_ladder rungs. See signal_categories.sys2_peak_trail.
peak_trail: Optional[tuple] = None
# Throttled persistence of peak_gain_pct (monotonic). peak_persisted is
# the last value written to the DB; peak_persist_ts the last write time.
peak_persisted: float = field(default=0.0)
peak_persist_ts: float = field(default=0.0)
# System-1 (Trump) min-hold floor. Epoch seconds; before this time,
# take-profit and trailing exits are SUPPRESSED (don't scalp out of a
# developing move). Hard stop-loss + invalidation always fire — capital
# protection isn't deferrable. None = no floor (System 2 / legacy).
min_hold_until_ts: Optional[float] = None
# Runtime state — peak unrealised gain seen so far, in %. Used by the
# trailing branch. Long: peak = max(over time) of pct_gain. Short: same,
# but pct_gain is already signed correctly by the caller.
peak_gain_pct: float = field(default=0.0)
trailing_armed: bool = field(default=False)
# Buffer below entry before "below_entry" invalidation fires. Entry price is
# only a PROXY for the true thesis-invalidation level (e.g. the 200d SMA for
# an sma_reclaim). 0.75% is roughly one normal noise band on a major coin —
# big enough to survive the fill-tick wobble, small enough to still cut a
# failed reclaim well before the wide hard stop.
INVALIDATION_BUFFER_PCT = 0.75
# trade_id -> WatchedTrade
_watched: Dict[int, WatchedTrade] = {}
# Strong refs to fire-close tasks (prevent GC before completion)
_background_tasks: set = set()
def register_trade(
trade_id: int,
wallet: str,
api_key: str,
leverage: int,
asset: str,
side: str,
entry_price: float,
take_profit_pct: Optional[float],
stop_loss_pct: Optional[float],
trailing_stop_pct: Optional[float] = None,
trailing_activate_at_pct: Optional[float] = None,
invalidation: Optional[str] = None,
invalidation_price: Optional[float] = None,
min_hold_until_ts: Optional[float] = None,
stop_ladder: Optional[list] = None,
derisk_ladder: Optional[list] = None,
derisk_done: int = 0,
addon_ladder: Optional[list] = None,
addon_done: int = 0,
initial_peak: float = 0.0,
peak_trail: Optional[tuple] = None,
grow_mode: bool = False,
) -> None:
# Nothing to monitor → skip. We require AT LEAST a stop-loss for any
# registered trade; protocol still works without TP if trailing or a
# staged-stop / de-risk ladder is set.
if (stop_loss_pct is None
and take_profit_pct is None
and trailing_stop_pct is None
and not stop_ladder
and not derisk_ladder):
return
# Warn when the price feed doesn't cover this asset — TP/SL/trailing
# stops will NEVER fire; only max_hold will exit the position.
from app.services.binance import ASSET_MAP
from app.services.hl_price_feed import HL_PRICE_ASSETS
covered = set(ASSET_MAP.values()) | HL_PRICE_ASSETS
if asset not in covered:
logger.error(
"PRICE FEED GAP: trade %d asset=%s has no price coverage. "
"TP/SL/trailing-stop will NOT trigger — only max_hold protects "
"this position. Add %susdt to ASSET_MAP in binance.py.",
trade_id, asset, asset.lower(),
)
_watched[trade_id] = WatchedTrade(
trade_id=trade_id, wallet=wallet, api_key=api_key, leverage=leverage,
asset=asset, side=side, entry_price=entry_price,
take_profit_pct=take_profit_pct,
stop_loss_pct=stop_loss_pct,
trailing_stop_pct=trailing_stop_pct,
trailing_activate_at_pct=trailing_activate_at_pct,
invalidation=invalidation,
invalidation_price=invalidation_price,
min_hold_until_ts=min_hold_until_ts,
stop_ladder=stop_ladder,
derisk_ladder=derisk_ladder,
derisk_done=derisk_done or 0,
addon_ladder=addon_ladder,
addon_done=addon_done or 0,
peak_gain_pct=max(0.0, float(initial_peak or 0.0)),
peak_persisted=max(0.0, float(initial_peak or 0.0)),
peak_trail=peak_trail,
grow_mode=bool(grow_mode),
)
logger.info(
"Watching trade %d (%s %s @ %.4f, sl=%s, tp=%s, trail=%s/@%s)",
trade_id, side, asset, entry_price,
stop_loss_pct, take_profit_pct,
trailing_stop_pct, trailing_activate_at_pct,
)
def unregister(trade_id: int) -> None:
wt = _watched.pop(trade_id, None)
# Flush the latest peak to DB on unregister so a restart between the last
# throttled write and this point doesn't lose progress.
if wt is not None and wt.peak_gain_pct > wt.peak_persisted:
task = asyncio.create_task(_persist_peak(trade_id, wt.peak_gain_pct))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
# Clean up the per-trade close lock from bot_engine to prevent memory leak.
# Trades closed via reconciler bypass close_and_finalize, so the lock would
# otherwise remain in the dict forever.
from app.services.bot_engine import _close_locks
_close_locks.pop(trade_id, None)
def on_price_tick(asset: str, price: float) -> None:
"""Called from binance.py on every price update.
Iterates watched trades on this asset, evaluates each rule, and triggers
a close on the first match. Rule evaluation order: stop-loss → trailing
→ fixed TP. (Stop-loss first so a sudden gap kills the position before
we celebrate hitting trailing.)
"""
if not _watched:
return
triggered = []
derisk_actions = [] # (wt, step_idx, frac_of_original)
addon_actions = [] # (wt, step_idx, frac_of_base)
for tid, wt in list(_watched.items()):
if wt.asset != asset:
continue
# Convert raw price → signed gain in position's direction, in %.
raw_pct = (price - wt.entry_price) / wt.entry_price
signed_pct = (raw_pct if wt.side == "long" else -raw_pct) * 100
# Update peak BEFORE evaluating trailing — a tick that pushes the peak
# AND retraces in the same step should still arm trailing at the new peak.
if signed_pct > wt.peak_gain_pct:
wt.peak_gain_pct = signed_pct
# Throttled monotonic persistence so a restart keeps the regime.
now_s = time.time()
if (wt.peak_gain_pct - wt.peak_persisted >= PEAK_FLUSH_DELTA
and now_s - wt.peak_persist_ts >= PEAK_FLUSH_SECS):
wt.peak_persisted = wt.peak_gain_pct
wt.peak_persist_ts = now_s
_pt = asyncio.create_task(
_persist_peak(wt.trade_id, wt.peak_gain_pct))
_background_tasks.add(_pt)
_pt.add_done_callback(_background_tasks.discard)
# 0. System-2 self-contained exit model: NO take-profit, NO from-peak
# trailing. Two regimes, fully replacing branches 1/1b/2/3:
# • Underwater → STAGED DE-RISK ("分段式减仓"): partial reduces
# at each downside rung, full close at the final (protective,
# inside-liquidation) rung.
# • In profit → STAGED STOP: floor ratchets up as peak gain
# crosses each upside rung; full close when price falls back.
if wt.stop_ladder or wt.derisk_ladder:
first_up = (min(t for t, _ in wt.stop_ladder)
if wt.stop_ladder else None)
in_profit_regime = (first_up is not None
and wt.peak_gain_pct >= first_up)
if wt.derisk_ladder and not in_profit_regime:
ladder = wt.derisk_ladder
# Gap protection: blown straight through to the final
# protective level → full close NOW regardless of step
# bookkeeping (still inside the exchange liquidation line).
if signed_pct <= ladder[-1][0]:
triggered.append((wt, "staged_stop"))
continue
if (not wt.derisk_in_flight
and 0 <= wt.derisk_done < len(ladder)):
thr, frac, is_final = ladder[wt.derisk_done]
if signed_pct <= thr:
if is_final:
triggered.append((wt, "staged_stop"))
continue
# Partial reduce — spawn async, guard re-entry until
# it completes (done-callback clears the flag).
wt.derisk_in_flight = True
derisk_actions.append((wt, wt.derisk_done, frac))
continue # underwater: no profit-ratchet / TP / trailing
# Profit regime — ratchet floor (base + unlocked upside rungs).
eff_stop = -wt.stop_loss_pct if wt.stop_loss_pct is not None else float("-inf")
if wt.stop_ladder:
for trig, floor in wt.stop_ladder: # sorted ascending
if wt.peak_gain_pct >= trig and floor > eff_stop:
eff_stop = floor
# Parabolic-top trailing: once deep in profit, also trail the PEAK
# PRICE by ≤ drawdown_frac (scale-invariant — self-adjusts to a
# +120% or +900% move so the fixed top rung doesn't cap it, while
# a normal ~2530% bull pullback still doesn't trip it).
if wt.peak_trail is not None:
start_pp, dd = wt.peak_trail
if wt.peak_gain_pct >= start_pp:
trail_floor = ((1.0 + wt.peak_gain_pct / 100.0)
* (1.0 - dd) - 1.0) * 100.0
if trail_floor > eff_stop:
eff_stop = trail_floor
# Pyramided → never let a winner become a loser: blended position
# is floored at breakeven once any add-on has filled.
if wt.addon_done > 0 and eff_stop < 0.0:
eff_stop = 0.0
if signed_pct <= eff_stop:
triggered.append((wt, "staged_stop"))
continue
# Pyramiding — add to a confirmed winner. Gated by the per-trade
# Grow switch (off by default — user opts a position in). Evaluated
# ONLY after the stop check above passed (never add on a tick that's
# also exiting). Peak-gain triggered; only while no de-risk has
# fired. Structural confirmation is re-checked in the executor.
if (wt.grow_mode and wt.addon_ladder and wt.derisk_done == 0
and not wt.addon_in_flight
and 0 <= wt.addon_done < len(wt.addon_ladder)):
a_trig, a_frac, _a_last = wt.addon_ladder[wt.addon_done]
if wt.peak_gain_pct >= a_trig:
wt.addon_in_flight = True
addon_actions.append((wt, wt.addon_done, a_frac))
continue
# 1. Stop loss — first priority.
if wt.stop_loss_pct is not None and signed_pct <= -wt.stop_loss_pct:
triggered.append((wt, "stop_loss"))
continue
# 1b. Signal invalidation (System-2 only). Prefer the real thesis
# level when available (e.g. 200d SMA reclaim price). Fall back
# to entry-buffer proxy for legacy rows.
if wt.invalidation == "below_entry" and not wt.trailing_armed:
if wt.invalidation_price is not None:
invalidation_hit = (
(wt.side == "long" and price <= wt.invalidation_price) or
(wt.side == "short" and price >= wt.invalidation_price)
)
if invalidation_hit:
triggered.append((wt, "invalidation"))
continue
elif signed_pct <= -INVALIDATION_BUFFER_PCT:
triggered.append((wt, "invalidation"))
continue
# Min-hold gate (System-1 Trump). Before the floor elapses we let a
# winner develop — suppress TP + trailing. SL / invalidation above
# already ran, so downside is still protected; we only defer the
# PROFIT-side exits here.
if wt.min_hold_until_ts is not None:
import time as _t
if _t.time() < wt.min_hold_until_ts:
continue # still in min-hold; skip TP/trailing this tick
# 2. Trailing stop — only once armed.
if (wt.trailing_stop_pct is not None
and wt.trailing_activate_at_pct is not None):
if not wt.trailing_armed and wt.peak_gain_pct >= wt.trailing_activate_at_pct:
wt.trailing_armed = True
logger.info(
"Trade %d: trailing armed at peak %.2f%% (asset=%s)",
wt.trade_id, wt.peak_gain_pct, asset,
)
if wt.trailing_armed:
drawdown_from_peak = wt.peak_gain_pct - signed_pct
if drawdown_from_peak >= wt.trailing_stop_pct:
triggered.append((wt, "trailing_stop"))
continue
# 3. Fixed TP — legacy / fallback when no trailing config.
if wt.take_profit_pct is not None and signed_pct >= wt.take_profit_pct:
triggered.append((wt, "take_profit"))
continue
for wt, reason in triggered:
unregister(wt.trade_id)
task = asyncio.create_task(_fire_close(wt, reason))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
# Partial de-risk steps — trade stays registered (it's not closed).
for wt, step_idx, frac in derisk_actions:
task = asyncio.create_task(_fire_partial(wt, step_idx, frac))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
# Pyramid add-ons — trade stays registered (size grows, entry blends).
for wt, step_idx, frac in addon_actions:
task = asyncio.create_task(_fire_pyramid(wt, step_idx, frac))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
async def _fire_partial(wt: WatchedTrade, step_idx: int, frac: float) -> None:
"""Execute one staged de-risk partial reduce. The trade stays open and
registered; on success advance derisk_done, always clear the in-flight
guard so the next rung can fire on a later tick."""
from app.services.bot_engine import partial_derisk
ok = False
try:
ok = await partial_derisk(
trade_id=wt.trade_id,
api_key=wt.api_key,
asset=wt.asset,
wallet=wt.wallet,
step_idx=step_idx,
frac_of_original=frac,
reason="derisk",
)
except Exception as exc:
logger.error("De-risk partial failed for trade %d step %d: %s",
wt.trade_id, step_idx, exc)
finally:
# Re-fetch: the trade may have been closed/unregistered meanwhile.
live = _watched.get(wt.trade_id)
if live is not None:
if ok and live.derisk_done == step_idx:
live.derisk_done = step_idx + 1
live.derisk_in_flight = False
async def _fire_pyramid(wt: WatchedTrade, step_idx: int, frac: float) -> None:
"""Execute one pyramid add-on. On success re-base the in-memory entry to
the blended average and reset peak to the post-add gain so the ratchet /
regime use the NEW entry (and we stay in the profit regime). Always clear
the in-flight guard."""
from app.services.bot_engine import pyramid_add
ok, new_entry, ref_price = False, None, None
try:
ok, new_entry, ref_price = await pyramid_add(
trade_id=wt.trade_id,
api_key=wt.api_key,
asset=wt.asset,
wallet=wt.wallet,
step_idx=step_idx,
frac_of_base=frac,
reason="pyramid",
)
except Exception as exc:
logger.error("Pyramid add failed for trade %d step %d: %s",
wt.trade_id, step_idx, exc)
finally:
live = _watched.get(wt.trade_id)
if live is not None:
if ok and live.addon_done == step_idx:
# Step is RESOLVED (added, notional-capped, or already-done).
# Advance regardless of whether an add actually filled, else
# the monitor re-schedules this step every tick forever.
if new_entry:
live.entry_price = new_entry
if ref_price:
raw = (ref_price - new_entry) / new_entry
g = (raw if live.side == "long" else -raw) * 100.0
else:
g = 0.0
# Stay in the profit regime; ratchet re-accumulates here.
first_up = (min(t for t, _ in live.stop_ladder)
if live.stop_ladder else 0.0)
live.peak_gain_pct = max(g, first_up)
live.addon_done = step_idx + 1
live.addon_in_flight = False
async def _persist_peak(trade_id: int, peak: float) -> None:
"""Monotonic best-effort write of peak_gain_pct. Race-safe: only raises
the stored value (WHERE peak_gain_pct < :peak), never lowers it."""
try:
from sqlalchemy import update
from app.database import AsyncSessionLocal
from app.models import BotTrade
async with AsyncSessionLocal() as db:
await db.execute(
update(BotTrade)
.where(BotTrade.id == trade_id)
.where(BotTrade.peak_gain_pct < peak)
.values(peak_gain_pct=peak)
)
await db.commit()
except Exception as exc: # never let persistence break the monitor
logger.debug("peak persist failed trade %d: %s", trade_id, exc)
async def _fire_close(wt: WatchedTrade, reason: str) -> None:
from app.services.bot_engine import close_and_finalize
try:
logger.info(
"Closing trade %d on %s (peak=%.2f%%, reason=%s)",
wt.trade_id, wt.asset, wt.peak_gain_pct, reason,
)
await close_and_finalize(
trade_id=wt.trade_id,
api_key=wt.api_key,
leverage=wt.leverage,
asset=wt.asset,
wallet=wt.wallet,
reason=reason,
)
except Exception as exc:
logger.error("TP/SL close failed for trade %d: %s", wt.trade_id, exc)