Files
trumpsignal-backend/app/services/recovery.py
T
k 3754d2caf8 feat: Telegram daily digest + System-2 manage-only refactor
Two changes ship together — both reshape the Telegram-bot surface.

──── 1. Telegram daily digest (3-section brief)
Once-a-day push covering Macro Vibes / KOL talks-vs-trades / Trump 24h.
Body is rule-based templating (no LLM); each section reads structured DB
fields and picks a phrasing. Per-user opt-out + per-user UTC hour.

  - 023 migration: TelegramBinding gains digest_enabled, digest_hour_utc,
    last_digest_sent_at (idempotent against coalesced cron / restarts).
  - New services/telegram_digest.py: build_global_digest +
    format_digest + send_daily_digest + send_preview_for.
  - Hourly cron at :00 fans out to bindings whose digest_hour_utc matches.
  - New bot commands: /digest (preview), /digest on|off, /digest_time HH.
  - HELP_TEXT + /status updated to surface the new prefs.

──── 2. System-2 manage-only refactor
The Macro Vibes (BTC bottom + funding) signal no longer auto-opens
positions. Strategy is day-K — a 24h entry delay is irrelevant — but the
auto-open path carried real execution surface (leverage clipping, daily
budget split, concurrency caps, paper branches, key handling) for ~zero
alpha. The valuable part — multi-month exit management (5-rung stop
ladder + de-risk + pyramid + peak-trail) — is preserved and runs
against positions the user adopts.

Flow: scanner fires → Telegram alert with "/adopt" CTA → user opens
manually on Hyperliquid → /adopt picks the position via inline keyboard
→ picks Standard or Aggressive mode → bot creates BotTrade + registers
watchdog. Escape hatch: /release marks released_at, unregisters
watchdog, leaves the HL position open under user control.

  - 024 migration: BotTrade.released_at — "user took back control" marker.
  - bot_engine.process_post early-returns for sys2 (no auto-open path).
  - New services/adoption.py: list_hl_positions + adopt_position +
    release_management + AdoptionError. Per-wallet asyncio lock prevents
    dual-adopt race. Pre-flight checks: no_subscription / no_hl_key /
    paper_mode / circuit_breaker / already_adopted / concurrency_cap.
    Protective stop + de-risk + addon + peak-trail ladders all built
    against the ACTUAL HL leverage so the "inside liquidation" guarantee
    holds.
  - New API: GET /positions/hl/{wallet}, POST /positions/adopt,
    POST /positions/{id}/release (all signed).
  - telegram.py: send_message supports reply_markup; new edit_message +
    answer_callback for the inline-keyboard pickers; sys2 alert format
    now ends with the "/adopt" CTA.
  - telegram_bot.py: /adopt + /release commands with picker → confirm
    → execute flow via inline keyboards. New _handle_callback dispatches
    on "adopt:*" / "release:*" callback_data; run_bot_loop now consumes
    callback_query updates alongside messages.
  - recovery.py + reconciler.py skip released_at IS NOT NULL rows so a
    restart doesn't silently re-attach the watchdog to a released trade.
  - /positions/open, /positions/today, and telegram_digest's user-state
    line all filter released_at so they don't lie about what the bot is
    actually managing.

Tests: 26 new (8 digest snapshot + 14 adoption + 4 absorbed via existing
suites). All 64 pass.

Deploy: alembic upgrade head (runs 023 + 024) → restart backend. Existing
TelegramBindings get digest_enabled=true / digest_hour_utc=12 via server
defaults. In-flight auto-opened System-2 positions continue to be managed
(recovery rehydrates them) — no in-flight trade is abandoned.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 12:53:16 +08:00

177 lines
8.3 KiB
Python

"""
Startup rehydration: re-register TP/SL + max-hold for every open trade in DB.
Called once from lifespan() after tables are ensured. Without this, any deploy
or crash silently drops TP/SL protection on live positions.
"""
import asyncio
import logging
from datetime import datetime, timezone
from sqlalchemy import select
from app.database import AsyncSessionLocal
from app.models import BotTrade, Post, Subscription
logger = logging.getLogger(__name__)
async def rehydrate_open_trades() -> None:
# Imported locally to avoid circular imports at module load
from app.services.bot_engine import close_and_finalize
from app.services.crypto import decrypt_api_key
from app.services.signal_categories import (
get_stop_ladder as _get_stop_ladder,
sys2_derisk_ladder as _sys2_derisk_ladder,
sys2_addon_ladder as _sys2_addon_ladder,
sys2_peak_trail as _sys2_peak_trail,
)
from app.services.tp_sl_monitor import register_trade
async with AsyncSessionLocal() as db:
# Skip released trades: the user has taken back manual control of
# the HL position; re-registering would re-attach the watchdog and
# silently start managing again after a restart.
result = await db.execute(
select(BotTrade).where(
BotTrade.closed_at.is_(None),
BotTrade.released_at.is_(None),
)
)
open_trades = result.scalars().all()
if not open_trades:
logger.info("Rehydration: no open trades.")
return
now = datetime.now(timezone.utc)
for t in open_trades:
sub_res = await db.execute(
select(Subscription).where(Subscription.wallet_address == t.wallet_address)
)
sub = sub_res.scalar_one_or_none()
if sub is None or not sub.hl_api_key:
logger.warning(
"Open trade %d has no subscription/key — marking as closed(abandoned)", t.id
)
t.closed_at = now.replace(tzinfo=None)
continue
try:
api_key = decrypt_api_key(sub.hl_api_key)
except Exception as exc:
logger.error("Cannot decrypt key for trade %d: %s", t.id, exc)
continue
# Use the leverage snapshot from the trade row (stamped at open time).
# Fall back to current Subscription only for legacy rows (pre-migration 005).
trade_leverage = t.leverage if t.leverage is not None else sub.leverage
# Re-register from the trade's FROZEN exit profile (eff_* columns),
# NOT the live Subscription. The trade may be a 90-day System-2
# reversal; using sub.* would rehydrate it with the user's Trump
# stop (1.5%) and 7-day max-hold — silently rewriting its risk on
# every restart. eff_* is stamped at open and never changes.
#
# Legacy rows (opened before this migration) have NULL eff_* —
# fall back to the Subscription so they still get *some* watcher.
eff_sl = t.eff_stop_loss_pct if t.eff_stop_loss_pct is not None else sub.stop_loss_pct
eff_tp = t.eff_take_profit_pct # may legitimately be None (sys2 pure-trail)
eff_tr = t.eff_trailing_stop_pct if t.eff_trailing_stop_pct is not None else sub.trailing_stop_pct
eff_tra = t.eff_trailing_activate_pct if t.eff_trailing_activate_pct is not None else sub.trailing_activate_at_pct
eff_mh = t.eff_max_hold_hours if t.eff_max_hold_hours is not None else (sub.max_hold_hours or 168)
# NOTE: peak_gain_pct is now RESTORED from the row (throttled
# persistence) so a pyramided / in-profit System-2 trade keeps its
# regime across restarts. min_hold still resets on rehydrate, but
# it only matters in the first 30 min of a Trump trade; a restart
# inside that window is rare and the downside (TP can fire early)
# is bounded.
post_res = await db.execute(
select(Post).where(Post.id == t.trigger_post_id)
)
trigger_post = post_res.scalar_one_or_none()
register_trade(
trade_id=t.id,
wallet=t.wallet_address,
api_key=api_key,
leverage=trade_leverage,
asset=t.asset,
side=t.side,
entry_price=t.entry_price,
take_profit_pct=eff_tp,
stop_loss_pct=eff_sl,
trailing_stop_pct=eff_tr,
trailing_activate_at_pct=eff_tra,
invalidation=t.eff_invalidation,
invalidation_price=(
t.eff_invalidation_price
if t.eff_invalidation_price is not None
else (trigger_post.invalidation_price if trigger_post else None)
),
min_hold_until_ts=t.eff_min_hold_until_ts,
stop_ladder=_get_stop_ladder(
trigger_post.category if trigger_post else None
),
# Restore staged de-risk: rebuild the ladder from the trade's
# FROZEN leverage and skip steps already executed before the
# restart (derisk_steps_done is persisted on the row).
# Restore staged de-risk/pyramid/peak-trail with the trade's
# FROZEN risk mode + leverage, skipping steps already executed
# before the restart (persisted on the row).
derisk_ladder=(
_sys2_derisk_ladder(t.leverage or 1, t.sys2_mode)
if _get_stop_ladder(trigger_post.category if trigger_post else None)
else None
),
derisk_done=(t.derisk_steps_done or 0),
addon_ladder=(
_sys2_addon_ladder(t.sys2_mode)
if _get_stop_ladder(trigger_post.category if trigger_post else None)
else None
),
addon_done=(t.addon_steps_done or 0),
# Restore the monotonic peak so a pyramided / in-profit trade
# doesn't fall back to the underwater de-risk regime on restart.
initial_peak=(t.peak_gain_pct or 0.0),
peak_trail=(
_sys2_peak_trail(t.sys2_mode)
if _get_stop_ladder(trigger_post.category if trigger_post else None)
else None
),
grow_mode=bool(getattr(t, "grow_mode", False)),
)
# Remaining hold from the trade's FROZEN max_hold (e.g. 2160h for
# an sma_reclaim), not the user's Trump setting.
opened_aware = t.opened_at.replace(tzinfo=timezone.utc)
elapsed = (now - opened_aware).total_seconds()
max_hold_seconds = int(eff_mh) * 3600
remaining = max_hold_seconds - elapsed
from app.services.bot_engine import _background_tasks
if remaining <= 0:
logger.info("Trade %d past max-hold on startup — closing now", t.id)
task = asyncio.create_task(
close_and_finalize(
trade_id=t.id, api_key=api_key, leverage=trade_leverage,
asset=t.asset, wallet=t.wallet_address, reason="max_hold_recovery",
)
)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
else:
async def _delayed_close(trade_id=t.id, key=api_key, lev=trade_leverage,
asset=t.asset, wallet=t.wallet_address, delay=remaining):
await asyncio.sleep(delay)
await close_and_finalize(
trade_id=trade_id, api_key=key, leverage=lev,
asset=asset, wallet=wallet, reason="max_hold",
)
task = asyncio.create_task(_delayed_close())
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
await db.commit()
logger.info("Rehydrated %d open trades.", len(open_trades))