fix: de-risk launch_seed, monitor price feeds, enforce single-process
Three pre-launch audit findings: launch_seed.py — destructive cleanup is now OPT-IN Previously `launch_seed.py --yes` truncated KOL history (divergence / holdings / posts older than 30d) by default — an accidental run erased unrecoverable data, and abort_if_live_user_state() only guards on subscriptions/bindings, not KOL history. Now nothing is deleted unless --wipe is passed; the safe path is --seed-only (pure fetch). Bare/--yes without --wipe refuses and prints guidance. /api/health/deep — monitor the price feeds, not just scrapers The deep healthcheck only watched the (redundant) Trump scrapers + DB, so a dead Binance/HL price feed — which silently stops ALL tp_sl_monitor stop-loss / take-profit firing on live trades — left health green. Added per-feed liveness (binance.last_tick_at, hl_price_feed.last_tick_at) with a 180s boot grace so startup doesn't false-503. Body now includes price_feeds[]. Single-process enforcement (multi-worker safety) The backend is single-process by design (in-memory scheduler, replay cache, tp_sl table, price_store). systemd unit lacked the --workers 1 + rationale that supervisor.conf already had; added it. Added a runtime advisory file lock (app.main._acquire_singleton_lock): only the leader starts background tasks; extra workers serve HTTP reads only and log CRITICAL. health/deep now reports is_leader so the misconfig is visible to monitors. 72 tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,8 @@
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
import websockets
|
||||
@@ -11,6 +13,11 @@ from app.ws.manager import manager
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Liveness signal for /api/health/deep. Updated on every processed kline tick.
|
||||
# A stale value means the price feed is down → TP/SL/trailing stops are NOT
|
||||
# firing for live trades, which is a real-money incident, not a soft warning.
|
||||
last_tick_at: Optional[datetime] = None
|
||||
|
||||
# symbol (lowercase Binance ticker) → asset (HL perp name, uppercase).
|
||||
#
|
||||
# WHY THIS MATTERS: tp_sl_monitor.on_price_tick(asset, price) is the sole
|
||||
@@ -68,6 +75,9 @@ async def _process_message(raw: str):
|
||||
}
|
||||
price_store.update(asset, candle)
|
||||
|
||||
global last_tick_at
|
||||
last_tick_at = datetime.now(timezone.utc)
|
||||
|
||||
# TP/SL check on every tick (cheap no-op when no trades are being watched)
|
||||
from app.services.tp_sl_monitor import on_price_tick
|
||||
on_price_tick(asset, candle["close"])
|
||||
|
||||
@@ -28,6 +28,7 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
import httpx
|
||||
@@ -42,6 +43,10 @@ HL_API_URL = "https://api.hyperliquid.xyz/info"
|
||||
# HL-native assets not covered by the Binance WS. Add new ones here as needed.
|
||||
HL_PRICE_ASSETS: frozenset[str] = frozenset({"HYPE", "PURR"})
|
||||
|
||||
# Liveness signal for /api/health/deep. Set on every successful allMids poll.
|
||||
# Stale → HYPE/PURR trades lose TP/SL just like a dead Binance feed would.
|
||||
last_tick_at: Optional[datetime] = None
|
||||
|
||||
# Poll cadence — 2 s gives ~same freshness as a Binance 1-min kline close
|
||||
# without hammering the free HL REST endpoint (30 req/min well within limit).
|
||||
_POLL_INTERVAL = 2.0
|
||||
@@ -96,6 +101,11 @@ async def _tick() -> None:
|
||||
r.raise_for_status()
|
||||
mids: dict = r.json() # {"BTC": "74541.0", "HYPE": "13.5", …}
|
||||
|
||||
# Feed is alive the moment we successfully fetch mids, even if a specific
|
||||
# asset is momentarily absent from the response.
|
||||
global last_tick_at
|
||||
last_tick_at = datetime.now(timezone.utc)
|
||||
|
||||
for asset in HL_PRICE_ASSETS:
|
||||
raw: Optional[str] = mids.get(asset)
|
||||
if raw is None:
|
||||
|
||||
Reference in New Issue
Block a user