Files
trumpsignal-backend/app/services/adoption.py
T
k 3754d2caf8 feat: Telegram daily digest + System-2 manage-only refactor
Two changes ship together — both reshape the Telegram-bot surface.

──── 1. Telegram daily digest (3-section brief)
Once-a-day push covering Macro Vibes / KOL talks-vs-trades / Trump 24h.
Body is rule-based templating (no LLM); each section reads structured DB
fields and picks a phrasing. Per-user opt-out + per-user UTC hour.

  - 023 migration: TelegramBinding gains digest_enabled, digest_hour_utc,
    last_digest_sent_at (idempotent against coalesced cron / restarts).
  - New services/telegram_digest.py: build_global_digest +
    format_digest + send_daily_digest + send_preview_for.
  - Hourly cron at :00 fans out to bindings whose digest_hour_utc matches.
  - New bot commands: /digest (preview), /digest on|off, /digest_time HH.
  - HELP_TEXT + /status updated to surface the new prefs.

──── 2. System-2 manage-only refactor
The Macro Vibes (BTC bottom + funding) signal no longer auto-opens
positions. Strategy is day-K — a 24h entry delay is irrelevant — but the
auto-open path carried real execution surface (leverage clipping, daily
budget split, concurrency caps, paper branches, key handling) for ~zero
alpha. The valuable part — multi-month exit management (5-rung stop
ladder + de-risk + pyramid + peak-trail) — is preserved and runs
against positions the user adopts.

Flow: scanner fires → Telegram alert with "/adopt" CTA → user opens
manually on Hyperliquid → /adopt picks the position via inline keyboard
→ picks Standard or Aggressive mode → bot creates BotTrade + registers
watchdog. Escape hatch: /release marks released_at, unregisters
watchdog, leaves the HL position open under user control.

  - 024 migration: BotTrade.released_at — "user took back control" marker.
  - bot_engine.process_post early-returns for sys2 (no auto-open path).
  - New services/adoption.py: list_hl_positions + adopt_position +
    release_management + AdoptionError. Per-wallet asyncio lock prevents
    dual-adopt race. Pre-flight checks: no_subscription / no_hl_key /
    paper_mode / circuit_breaker / already_adopted / concurrency_cap.
    Protective stop + de-risk + addon + peak-trail ladders all built
    against the ACTUAL HL leverage so the "inside liquidation" guarantee
    holds.
  - New API: GET /positions/hl/{wallet}, POST /positions/adopt,
    POST /positions/{id}/release (all signed).
  - telegram.py: send_message supports reply_markup; new edit_message +
    answer_callback for the inline-keyboard pickers; sys2 alert format
    now ends with the "/adopt" CTA.
  - telegram_bot.py: /adopt + /release commands with picker → confirm
    → execute flow via inline keyboards. New _handle_callback dispatches
    on "adopt:*" / "release:*" callback_data; run_bot_loop now consumes
    callback_query updates alongside messages.
  - recovery.py + reconciler.py skip released_at IS NOT NULL rows so a
    restart doesn't silently re-attach the watchdog to a released trade.
  - /positions/open, /positions/today, and telegram_digest's user-state
    line all filter released_at so they don't lie about what the bot is
    actually managing.

Tests: 26 new (8 digest snapshot + 14 adoption + 4 absorbed via existing
suites). All 64 pass.

Deploy: alembic upgrade head (runs 023 + 024) → restart backend. Existing
TelegramBindings get digest_enabled=true / digest_hour_utc=12 via server
defaults. In-flight auto-opened System-2 positions continue to be managed
(recovery rehydrates them) — no in-flight trade is abandoned.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 12:53:16 +08:00

440 lines
18 KiB
Python

"""Adopt / release flow for System-2 manage-only positions.
The user opens a position MANUALLY on Hyperliquid (any size / leverage).
They then ask the bot to manage it — either via the dashboard adoption
button or the Telegram /adopt command. From that moment on, tp_sl_monitor
runs the full System-2 lifecycle against it: 5-rung stop ladder, downside
de-risk, pyramid, peak-trail, full close.
`release` is the escape hatch. It marks released_at on the BotTrade row,
unregisters the watchdog, and leaves the HL position untouched — the user
takes back manual control without closing.
The flow is intentionally LIGHT:
- We don't validate signal timing (any HL position can be adopted)
- We don't enforce direction (both long and short work; ladder is signed)
- We DO enforce the same concurrency cap as the old auto-open path
(SYS2_MAX_CONCURRENT) — adopting unlimited correlated crypto-beta is
the same disaster regardless of who pulled the trigger
- We DO block paper-mode subscribers (paper has no HL state to read)
Race notes:
- Adopt creates the BotTrade row, then register_trade. If a price tick
arrives between those two events, no harm — register_trade is
idempotent on trade_id.
- Two concurrent adopts on the same wallet+asset are guarded by the
"already adopted?" check inside the wallet-keyed asyncio lock reused
from bot_engine.
"""
from __future__ import annotations
import asyncio
import logging
from dataclasses import dataclass
from datetime import datetime, timezone
from typing import Dict, Optional
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import AsyncSessionLocal
from app.models import BotTrade, Subscription
from app.config import settings
from app.services.crypto import decrypt_api_key
from app.services.hyperliquid import HyperliquidTrader
from app.services.signal_categories import (
get_exit_profile, get_stop_ladder,
sys2_normalize_mode, sys2_protective_stop_pct,
sys2_derisk_ladder, sys2_addon_ladder, sys2_peak_trail,
SYS2_MAX_CONCURRENT, SYS2_MODES,
)
logger = logging.getLogger(__name__)
# Default category — the System-2 strategy is BTC bottom-reversal long.
# An adopted short or non-BTC asset still gets the same generous ladder
# (it's direction-agnostic), but we tag the category for telemetry.
ADOPTED_CATEGORY = "btc_bottom_reversal_long"
# Per-wallet asyncio lock to serialise adopt_position calls for the same
# wallet. Without this, two concurrent /adopt requests on the same wallet
# can both pass the duplicate-check and end up writing two BotTrade rows
# pointing at the same HL position — corrupting the ladder / de-risk math
# (each row would race to reduce the position). Process-local; with the
# atomic duplicate check inside the lock, multi-process deploys are still
# safe (losers would just see "already_adopted" on the second attempt).
_adopt_locks: Dict[str, asyncio.Lock] = {}
def _wallet_adopt_lock(wallet: str) -> asyncio.Lock:
lock = _adopt_locks.get(wallet)
if lock is None:
lock = asyncio.Lock()
_adopt_locks[wallet] = lock
return lock
# ── Errors ──────────────────────────────────────────────────────────────────
class AdoptionError(Exception):
"""User-facing adoption failure. .code is a stable short token suitable
for switching on in the API / Telegram layers; .message is the
human-readable text."""
def __init__(self, code: str, message: str):
super().__init__(message)
self.code = code
self.message = message
# ── Reads ───────────────────────────────────────────────────────────────────
@dataclass
class HLPositionView:
"""Lightweight projection of a Hyperliquid open position, for adoption
pickers. Kept thin so the UI / Telegram layers don't depend on the
HL SDK's dict shape."""
asset: str
side: str # "long" | "short"
size_coins: float # absolute (always positive)
entry_price: float
leverage: int
size_usd: float # size_coins * entry_price
already_adopted: bool # there's a BotTrade open + not released for it
async def list_hl_positions(wallet: str) -> list[HLPositionView]:
"""Read the wallet's current HL open positions, annotated with whether
each is already under bot management. Raises AdoptionError on missing
subscription / missing key / HL read failure."""
wallet_l = wallet.lower().strip()
async with AsyncSessionLocal() as db:
sub = (await db.execute(
select(Subscription).where(Subscription.wallet_address == wallet_l)
)).scalar_one_or_none()
if sub is None:
raise AdoptionError("no_subscription",
"Wallet not subscribed. Open Settings on the dashboard first.")
if not sub.hl_api_key:
raise AdoptionError("no_hl_key",
"This wallet has no Hyperliquid API key on file. "
"Add one in Settings before adopting positions.")
# Adopted positions only — closed/released rows don't block re-adopt.
existing = (await db.execute(
select(BotTrade.asset).where(
BotTrade.wallet_address == wallet_l,
BotTrade.closed_at.is_(None),
BotTrade.released_at.is_(None),
)
)).scalars().all()
adopted_assets = {a.upper() for a in existing}
try:
api_key = decrypt_api_key(sub.hl_api_key)
except Exception as exc:
raise AdoptionError("key_decrypt_failed",
f"Could not decrypt the HL API key: {exc}")
trader = HyperliquidTrader(
api_private_key=api_key,
account_address=wallet_l,
leverage=int(sub.leverage or 2),
mainnet=settings.hl_mainnet,
)
try:
raw = await trader.get_open_positions()
except Exception as exc:
raise AdoptionError("hl_read_failed",
f"Could not read HL positions: {exc}")
out: list[HLPositionView] = []
for p in raw:
coin = (p.get("coin") or "").upper()
szi = float(p.get("szi") or 0.0)
entry = float(p.get("entry_px") or 0.0)
lev_info = p.get("leverage") or {}
# HL position.leverage shape is {"type": "isolated"|"cross", "value": N}
lev = int(lev_info.get("value") or sub.leverage or 2)
if szi == 0 or entry <= 0:
continue
out.append(HLPositionView(
asset = coin,
side = "long" if szi > 0 else "short",
size_coins = abs(szi),
entry_price = entry,
leverage = lev,
size_usd = round(abs(szi) * entry, 2),
already_adopted = coin in adopted_assets,
))
return out
# ── Adopt ───────────────────────────────────────────────────────────────────
@dataclass
class AdoptionResult:
trade_id: int
asset: str
side: str
entry_price: float
size_usd: float
leverage: int
mode: str
protective_stop: float # positive % distance
derisk_ladder: list # the rungs we just registered
stop_ladder: list # the upside ratchet rungs
async def adopt_position(wallet: str, asset: str, mode: str) -> AdoptionResult:
"""Read the user's current HL position for `asset`, create a BotTrade
row with the System-2 ladder for `mode`, and register it with the
watchdog. Raises AdoptionError on any failure path.
Wallet-level locking serialises concurrent adopts for the same wallet
so the duplicate / concurrency-cap checks below are race-free.
"""
wallet_l = wallet.lower().strip()
asset_u = asset.upper().strip()
mode_n = sys2_normalize_mode(mode)
if mode_n not in SYS2_MODES:
raise AdoptionError("bad_mode",
f"mode must be one of {SYS2_MODES}; got {mode!r}")
async with _wallet_adopt_lock(wallet_l):
return await _adopt_locked(wallet_l, asset_u, mode_n)
async def _adopt_locked(wallet_l: str, asset_u: str, mode_n: str) -> AdoptionResult:
async with AsyncSessionLocal() as db:
sub = (await db.execute(
select(Subscription).where(Subscription.wallet_address == wallet_l)
)).scalar_one_or_none()
if sub is None:
raise AdoptionError("no_subscription",
"Wallet not subscribed. Open Settings on the dashboard first.")
if not sub.hl_api_key:
raise AdoptionError("no_hl_key",
"This wallet has no Hyperliquid API key on file.")
# Paper-mode wallets have no real HL position to manage. The HL read
# below would return empty and surface a confusing "not_on_hl" error
# — better to reject explicitly with a clear cause up front.
if getattr(sub, "paper_mode", False):
raise AdoptionError("paper_mode",
"Adopt isn't available in paper mode — paper trades have "
"no Hyperliquid position to manage. Turn off paper mode in "
"Settings to use /adopt.")
# System-2 circuit breaker. Same gate the auto-open path used to run:
# if recent losses tripped the sys2 breaker, block new adoptions for
# the lockout window. Otherwise the breaker would be useless under
# the manage-only model.
from app.services.circuit_breaker import is_tripped as _cb_tripped
sub_snapshot = {
"sys2_cb_tripped_at": sub.sys2_cb_tripped_at,
"sys2_cb_reason": sub.sys2_cb_reason,
}
tripped, cb_msg = _cb_tripped(sub_snapshot, "sys2")
if tripped:
raise AdoptionError("circuit_breaker",
f"Adopt blocked: {cb_msg}. Turning Auto-Trade ON or arming "
"Manual Window in Settings clears the breaker.")
# Already managing this wallet+asset?
dup = (await db.execute(
select(BotTrade).where(
BotTrade.wallet_address == wallet_l,
BotTrade.asset == asset_u,
BotTrade.closed_at.is_(None),
BotTrade.released_at.is_(None),
)
)).scalar_one_or_none()
if dup is not None:
raise AdoptionError("already_adopted",
f"{asset_u} is already managed by the bot (trade #{dup.id}). "
"Send /release first if you want to re-adopt.")
# Wallet-level concurrency cap. Same reasoning as the auto-open
# path: every System-2 asset is correlated crypto-beta and 3
# simultaneous positions ≈ 1 leveraged macro bet, not 3 bets.
open_count = (await db.execute(
select(BotTrade).where(
BotTrade.wallet_address == wallet_l,
BotTrade.closed_at.is_(None),
BotTrade.released_at.is_(None),
)
)).scalars().all()
if len(open_count) >= SYS2_MAX_CONCURRENT:
raise AdoptionError("concurrency_cap",
f"You already have {len(open_count)} managed positions "
f"(cap is {SYS2_MAX_CONCURRENT}). /release one first.")
try:
api_key = decrypt_api_key(sub.hl_api_key)
except Exception as exc:
raise AdoptionError("key_decrypt_failed",
f"Could not decrypt the HL API key: {exc}")
trader = HyperliquidTrader(
api_private_key=api_key,
account_address=wallet_l,
leverage=int(sub.leverage or 2),
mainnet=settings.hl_mainnet,
)
try:
raw = await trader.get_open_positions()
except Exception as exc:
raise AdoptionError("hl_read_failed",
f"Could not read HL positions: {exc}")
target = next((p for p in raw if (p.get("coin") or "").upper() == asset_u),
None)
if target is None:
raise AdoptionError("not_on_hl",
f"No open {asset_u} position found on Hyperliquid. "
"Make sure you've filled the order, then try again.")
szi = float(target.get("szi") or 0.0)
entry = float(target.get("entry_px") or 0.0)
if szi == 0 or entry <= 0:
raise AdoptionError("bad_hl_state",
f"HL returned a zero-size or zero-price {asset_u} position. "
"Refresh and try again in a moment.")
side = "long" if szi > 0 else "short"
size_coins = abs(szi)
size_usd = round(size_coins * entry, 2)
lev_info = target.get("leverage") or {}
leverage = int(lev_info.get("value") or sub.leverage or 2)
# Build the System-2 exit profile against the ACTUAL HL leverage so the
# protective stop is always inside the real liquidation line. Identical
# math to bot_engine's old sys2 open path; reused so behaviour stays
# consistent across the two entry points.
exit_profile = get_exit_profile(ADOPTED_CATEGORY)
prot_stop = sys2_protective_stop_pct(leverage)
derisk = sys2_derisk_ladder(leverage, mode_n)
addon = sys2_addon_ladder(mode_n)
peak_trail = sys2_peak_trail(mode_n)
stop_ladder = get_stop_ladder(ADOPTED_CATEGORY)
now_naive = datetime.now(timezone.utc).replace(tzinfo=None)
async with AsyncSessionLocal() as db:
trade = BotTrade(
asset=asset_u,
side=side,
entry_price=entry,
wallet_address=wallet_l,
trigger_post_id=None, # adopted, not signal-triggered
opened_at=now_naive,
hl_order_id=f"adopted:{int(now_naive.timestamp())}",
size_usd=size_usd,
base_size_usd=size_usd,
sys2_mode=mode_n,
leverage=leverage,
# Freeze the System-2 exit profile on the row. No TP / no
# trailing-from-peak — pure staged stop + de-risk + peak-trail.
eff_take_profit_pct = None,
eff_stop_loss_pct = prot_stop,
eff_trailing_stop_pct = None,
eff_trailing_activate_pct = None,
eff_max_hold_hours = exit_profile.max_hold_hours,
eff_invalidation = None,
eff_invalidation_price = None,
eff_min_hold_until_ts = None,
)
db.add(trade)
await db.commit()
await db.refresh(trade)
trade_id = trade.id
# Register with the watchdog AFTER the row is committed so a tick fired
# in the next millisecond can find the trade on disk if it needs to.
from app.services.tp_sl_monitor import register_trade
register_trade(
trade_id=trade_id,
wallet=wallet_l,
api_key=api_key,
leverage=leverage,
asset=asset_u,
side=side,
entry_price=entry,
take_profit_pct=None,
stop_loss_pct=prot_stop,
trailing_stop_pct=None,
trailing_activate_at_pct=None,
invalidation=None,
invalidation_price=None,
min_hold_until_ts=None,
stop_ladder=stop_ladder,
derisk_ladder=derisk,
derisk_done=0,
addon_ladder=addon,
addon_done=0,
peak_trail=peak_trail,
grow_mode=False, # user opts a position into Grow explicitly
)
logger.warning(
"ADOPTED trade %d: wallet=%s asset=%s side=%s entry=%.4f size=%.2f "
"lev=%dx mode=%s prot_stop=%.2f%% derisk=%s",
trade_id, wallet_l, asset_u, side, entry, size_usd,
leverage, mode_n, prot_stop, derisk,
)
return AdoptionResult(
trade_id=trade_id, asset=asset_u, side=side, entry_price=entry,
size_usd=size_usd, leverage=leverage, mode=mode_n,
protective_stop=prot_stop, derisk_ladder=derisk,
stop_ladder=stop_ladder or [],
)
# ── Release ─────────────────────────────────────────────────────────────────
async def release_management(wallet: str, trade_id: int) -> dict:
"""Stop managing trade_id without closing the HL position. Marks
released_at, unregisters the watchdog. Idempotent — re-releasing a
released trade is a no-op."""
wallet_l = wallet.lower().strip()
async with AsyncSessionLocal() as db:
trade = (await db.execute(
select(BotTrade).where(BotTrade.id == trade_id)
)).scalar_one_or_none()
if trade is None:
raise AdoptionError("not_found", f"trade {trade_id} not found")
if trade.wallet_address.lower() != wallet_l:
raise AdoptionError("not_owner",
f"trade {trade_id} belongs to a different wallet")
if trade.closed_at is not None:
raise AdoptionError("already_closed",
f"trade {trade_id} is already closed — nothing to release")
if trade.released_at is not None:
# Idempotent — return success.
return {"status": "ok", "trade_id": trade_id,
"asset": trade.asset, "already_released": True}
now_naive = datetime.now(timezone.utc).replace(tzinfo=None)
await db.execute(
update(BotTrade)
.where(BotTrade.id == trade_id)
.values(released_at=now_naive)
)
await db.commit()
from app.services.tp_sl_monitor import unregister
unregister(trade_id)
logger.warning(
"RELEASED trade %d: wallet=%s asset=%s — bot stops managing, "
"HL position untouched.",
trade_id, wallet_l, trade.asset,
)
return {"status": "ok", "trade_id": trade_id,
"asset": trade.asset, "already_released": False}