fix: de-risk launch_seed, monitor price feeds, enforce single-process

Three pre-launch audit findings:

launch_seed.py — destructive cleanup is now OPT-IN
  Previously `launch_seed.py --yes` truncated KOL history (divergence /
  holdings / posts older than 30d) by default — an accidental run erased
  unrecoverable data, and abort_if_live_user_state() only guards on
  subscriptions/bindings, not KOL history. Now nothing is deleted unless
  --wipe is passed; the safe path is --seed-only (pure fetch). Bare/--yes
  without --wipe refuses and prints guidance.

/api/health/deep — monitor the price feeds, not just scrapers
  The deep healthcheck only watched the (redundant) Trump scrapers + DB, so a
  dead Binance/HL price feed — which silently stops ALL tp_sl_monitor stop-loss
  / take-profit firing on live trades — left health green. Added per-feed
  liveness (binance.last_tick_at, hl_price_feed.last_tick_at) with a 180s boot
  grace so startup doesn't false-503. Body now includes price_feeds[].

Single-process enforcement (multi-worker safety)
  The backend is single-process by design (in-memory scheduler, replay cache,
  tp_sl table, price_store). systemd unit lacked the --workers 1 + rationale
  that supervisor.conf already had; added it. Added a runtime advisory file
  lock (app.main._acquire_singleton_lock): only the leader starts background
  tasks; extra workers serve HTTP reads only and log CRITICAL. health/deep now
  reports is_leader so the misconfig is visible to monitors.

72 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
k
2026-05-29 14:29:24 +08:00
parent 520bd7243d
commit 6876c0c280
5 changed files with 167 additions and 16 deletions
+10
View File
@@ -1,6 +1,8 @@
import asyncio
import json
import logging
from datetime import datetime, timezone
from typing import Optional
import httpx
import websockets
@@ -11,6 +13,11 @@ from app.ws.manager import manager
logger = logging.getLogger(__name__)
# Liveness signal for /api/health/deep. Updated on every processed kline tick.
# A stale value means the price feed is down → TP/SL/trailing stops are NOT
# firing for live trades, which is a real-money incident, not a soft warning.
last_tick_at: Optional[datetime] = None
# symbol (lowercase Binance ticker) → asset (HL perp name, uppercase).
#
# WHY THIS MATTERS: tp_sl_monitor.on_price_tick(asset, price) is the sole
@@ -68,6 +75,9 @@ async def _process_message(raw: str):
}
price_store.update(asset, candle)
global last_tick_at
last_tick_at = datetime.now(timezone.utc)
# TP/SL check on every tick (cheap no-op when no trades are being watched)
from app.services.tp_sl_monitor import on_price_tick
on_price_tick(asset, candle["close"])
+10
View File
@@ -28,6 +28,7 @@ from __future__ import annotations
import asyncio
import logging
import time
from datetime import datetime, timezone
from typing import Optional
import httpx
@@ -42,6 +43,10 @@ HL_API_URL = "https://api.hyperliquid.xyz/info"
# HL-native assets not covered by the Binance WS. Add new ones here as needed.
HL_PRICE_ASSETS: frozenset[str] = frozenset({"HYPE", "PURR"})
# Liveness signal for /api/health/deep. Set on every successful allMids poll.
# Stale → HYPE/PURR trades lose TP/SL just like a dead Binance feed would.
last_tick_at: Optional[datetime] = None
# Poll cadence — 2 s gives ~same freshness as a Binance 1-min kline close
# without hammering the free HL REST endpoint (30 req/min well within limit).
_POLL_INTERVAL = 2.0
@@ -96,6 +101,11 @@ async def _tick() -> None:
r.raise_for_status()
mids: dict = r.json() # {"BTC": "74541.0", "HYPE": "13.5", …}
# Feed is alive the moment we successfully fetch mids, even if a specific
# asset is momentarily absent from the response.
global last_tick_at
last_tick_at = datetime.now(timezone.utc)
for asset in HL_PRICE_ASSETS:
raw: Optional[str] = mids.get(asset)
if raw is None: