Pre-launch hardening: KOL module, Telegram, scanners, WS resilience

Big-picture changes since b941223: KOL pipeline (new) — Substack/podcast/blog RSS → AI ticker extraction → on-chain wallet diff → talks-vs-trades divergence detection. Daily polls, 19 feeds, divergence emits Post + Telegram fan-out. Telegram push (new) — walletless free tier + wallet-linked Pro upgrade, in-bot preference commands (/trump /btc /funding /kol /conf /quiet), signed-envelope API for dashboard. Disconnect-wallet keeps free subscription. BTC funding-rate reversal scanner (new) — hourly cron, 30d cumulative funding threshold + mean-revert + 7d price confirm, emits via /api/signals/ingest. BTC bottom-reversal scanner promoted to System 2. WS broadcast rewrite — per-client send timeout + parallel fan-out (asyncio.gather). Fixes "Binance WS no close frame" reconnect storms + APScheduler 11-min job misses, both caused by one slow client stalling the kline loop. Error visibility — three silent-error sites (trumpstruth/truth_social fetchers, funding_reversal scanner) now include exception type name so httpx ConnectError-style empty-message errors stop logging blank lines. Telegram bot loop now classifies ReadTimeout vs network vs unknown + logger.exception for the unknown bucket. Security hygiene — trumpsignal.db untracked from git (held subscriber wallets + encrypted HL keys + 22 bot trades); .gitignore now blocks *.db/.next/backups. CORS only allows FRONTEND_URL in production. New ops scripts — - scripts/preflight.py: env/DB/Telegram/AI auth verification gate - scripts/backup_db.sh: cron-friendly daily DB backup (SQLite + Postgres) - scripts/seed_kol_wallets.py: idempotent KOL on-chain wallet seeder 15 new Alembic migrations (007-021) covering convex strategy fields, phase-1 safety, two-system frozen exits, invalidation prices, dynamic SYS2 leverage, staged de-risk + pyramiding, peak gain tracking, risk mode, auto-trade + grow flags, KOL module, KOL on-chain, KOL divergence, Telegram bindings + walletless. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 00:52:56 +08:00
parent b941223c88
commit 5fb1d52026
81 changed files with 13251 additions and 158 deletions
@@ -0,0 +1,196 @@
+"""
+System-2 生命周期·小额真单端到端验证
+
+验证我们新写的三个动钱路径在真实 Hyperliquid 上的行为，并和账面记账对账：
+  开仓 → 加仓(pyramid) → 部分减仓(de-risk) → 全平
+每一步都把"预期"和"HL 实际"并排打印，并用和 bot_engine 完全一致的
+公式做 PnL 自洽校验。
+
+用法:
+  source venv/bin/activate
+  HL_API_PRIVATE_KEY="0x..." HL_ACCOUNT_ADDRESS="0x..." \
+  python scripts/verify_sys2_lifecycle.py            # 干跑(不下单, 只打印计划)
+  ... python scripts/verify_sys2_lifecycle.py --live  # 真下单(小额, 需确认)
+
+安全:
+  - 默认 DRY-RUN, 不下任何单
+  - --live 才真下单, 且会要求手动输入 YES 确认
+  - 名义金额默认 $20, 上限 $40 (超过需 --force), 杠杆默认 2x
+  - 任何异常 / 结束都会尝试把仓位平掉 (best-effort flatten)
+  - mainnet/testnet 跟随 settings.hl_mainnet
+"""
+
+import argparse
+import asyncio
+import os
+import sys
+
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from app.config import settings
+from app.services.hyperliquid import HyperliquidTrader
+from app.services.bot_engine import HL_TAKER_FEE_RATE
+
+ASSET = "BTC"
+SIDE = "long"
+HARD_CAP_USD = 40.0
+
+
+def _slice_pnl(notional: float, entry: float, exit_px: float, side: str) -> float:
+    """Identical to bot_engine: notional × signed move − round-trip taker."""
+    pct = (exit_px - entry) / entry if entry else 0.0
+    signed = pct if side == "long" else -pct
+    return notional * signed - notional * HL_TAKER_FEE_RATE * 2
+
+
+def _pos(positions: list):
+    return next((p for p in positions if p.get("coin") == ASSET), None)
+
+
+def _row(label, expected, actual):
+    print(f"  {label:<28} expected={expected!s:<22} actual={actual!s}")
+
+
+async def _flatten(trader, why: str):
+    try:
+        pos = _pos(await trader.get_open_positions())
+        if pos:
+            print(f"\n🧹 安全平仓 ({why}) — 当前 szi={pos['szi']}")
+            r = await trader.close_position(ASSET)
+            print(f"   平仓结果: {r}")
+        else:
+            print(f"\n🧹 无残留仓位 ({why})")
+    except Exception as exc:
+        print(f"\n⚠️  安全平仓失败 ({why}): {exc} — 请手动检查 HL!")
+
+
+async def main(live: bool, size_usd: float, leverage: int, force: bool):
+    api_key = os.getenv("HL_API_PRIVATE_KEY") or os.getenv("HL_API_KEY", "")
+    account = os.getenv("HL_ACCOUNT_ADDRESS", "")
+    if not api_key or not account:
+        print("❌ 需要 HL_API_PRIVATE_KEY 和 HL_ACCOUNT_ADDRESS 环境变量")
+        sys.exit(1)
+    if size_usd > HARD_CAP_USD and not force:
+        print(f"❌ size_usd ${size_usd} 超过安全上限 ${HARD_CAP_USD}（加 --force 才允许）")
+        sys.exit(1)
+
+    net = "MAINNET 真钱" if settings.hl_mainnet else "TESTNET"
+    add_usd = round(size_usd * 0.30, 2)          # 模拟 pyramid 第1档 (+30% base)
+    derisk_frac_of_cur = 1.0 / 3.0               # 模拟 de-risk 第1档 (减当前 1/3)
+
+    print("=" * 64)
+    print(f"System-2 生命周期验证 · {net} · {ASSET} {SIDE} {leverage}x")
+    print(f"计划: 开 ${size_usd}  → 加 ${add_usd}  → 减当前 1/3  → 全平")
+    print(f"模式: {'🔴 LIVE 真下单' if live else '🟢 DRY-RUN 仅打印'}")
+    print("=" * 64)
+
+    if not live:
+        print("\n干跑结束。确认计划无误后加 --live 真跑（小额）。")
+        return
+
+    confirm = input(f"\n⚠️  将在 {net} 下真单（约 ${size_usd}）。输入大写 YES 继续: ")
+    if confirm.strip() != "YES":
+        print("已取消。")
+        return
+
+    trader = HyperliquidTrader(
+        api_private_key=api_key, account_address=account,
+        leverage=leverage, mainnet=settings.hl_mainnet,
+    )
+
+    bal = await trader.get_balance()
+    print(f"\n账户可用 USDC: ${bal:.2f}")
+    if bal < size_usd:
+        print("❌ 余额不足，放弃。")
+        return
+    if _pos(await trader.get_open_positions()):
+        print(f"❌ 已存在 {ASSET} 持仓 — 为避免干扰，请先手动清空后再跑。")
+        return
+
+    try:
+        # ── 1. 开仓 ──────────────────────────────────────────────────────
+        print("\n[1] 开仓 open_position")
+        o = await trader.open_position(ASSET, SIDE, size_usd)
+        entry = float(o["fill_price"])
+        base_coins = float(o["size_coins"])
+        base_notional = base_coins * entry
+        pos = _pos(await trader.get_open_positions())
+        _row("entry fill", "~mkt", entry)
+        _row("size_coins", f"~{size_usd/entry:.6f}", base_coins)
+        _row("HL szi", f"~{base_coins:.6f}", pos and pos["szi"])
+        assert pos and abs(abs(pos["szi"]) - base_coins) / base_coins < 0.05, "开仓后 HL 仓位不符"
+
+        # ── 2. 加仓 (pyramid 第1档) ──────────────────────────────────────
+        print("\n[2] 加仓 open_position（模拟 pyramid +30%）")
+        a = await trader.open_position(ASSET, SIDE, add_usd)
+        add_fill = float(a["fill_price"])
+        add_coins = float(a["size_coins"])
+        actual_add_notional = add_coins * add_fill          # ← 和修过的 pyramid_add 一致
+        # 混合均价：按名义加权（与 bot_engine.pyramid_add 完全相同的公式）
+        old_notional = base_notional
+        new_notional = old_notional + actual_add_notional
+        blended = (old_notional * entry + actual_add_notional * add_fill) / new_notional
+        pos = _pos(await trader.get_open_positions())
+        exp_coins = base_coins + add_coins
+        _row("add fill", "~mkt", add_fill)
+        _row("add size_coins", f"~{add_usd/add_fill:.6f}", add_coins)
+        _row("blended entry", f"{blended:.2f}", "(账面)")
+        _row("HL szi", f"~{exp_coins:.6f}", pos and pos["szi"])
+        assert pos and abs(abs(pos["szi"]) - exp_coins) / exp_coins < 0.05, "加仓后 HL 仓位不符"
+
+        # ── 3. 部分减仓 (de-risk 第1档：减当前 1/3) ──────────────────────
+        print("\n[3] 部分减仓 reduce_position(1/3)")
+        pre_coins = abs(pos["szi"])
+        r = await trader.reduce_position(ASSET, derisk_frac_of_cur)
+        cut_fill = float(r["fill_price"])
+        closed_frac = float(r["closed_fraction"])
+        pos = _pos(await trader.get_open_positions())
+        post_coins = abs(pos["szi"]) if pos else 0.0
+        actually_cut = pre_coins - post_coins
+        slice_notional = actually_cut * cut_fill
+        slice_pnl = _slice_pnl(slice_notional, blended, cut_fill, SIDE)
+        _row("reduce fill", "~mkt", cut_fill)
+        _row("closed_fraction", f"~{derisk_frac_of_cur:.3f}", round(closed_frac, 4))
+        _row("coins cut", f"~{pre_coins/3:.6f}", round(actually_cut, 6))
+        _row("剩余 szi", f"~{pre_coins*2/3:.6f}", pos and pos["szi"])
+        _row("该片已实现PnL($)", "—", round(slice_pnl, 4))
+        assert 0.25 < closed_frac < 0.42, "减仓比例偏离 1/3 过大"
+
+        # ── 4. 全平 ──────────────────────────────────────────────────────
+        print("\n[4] 全平 close_position")
+        c = await trader.close_position(ASSET)
+        close_fill = float(c["fill_price"])
+        remaining_notional = post_coins * close_fill
+        remaining_pnl = _slice_pnl(remaining_notional, blended, close_fill, SIDE)
+        pos = _pos(await trader.get_open_positions())
+        _row("close fill", "~mkt", close_fill)
+        _row("收尾后持仓", "None", pos)
+        assert pos is None, "全平后仍有残留仓位！"
+
+        total_pnl = slice_pnl + remaining_pnl
+        print("\n" + "=" * 64)
+        print("对账小结（账面 vs 行为）")
+        print(f"  混合均价        : {blended:.2f}")
+        print(f"  分片PnL(减1/3)  : {slice_pnl:+.4f} USD")
+        print(f"  收尾PnL(剩2/3)  : {remaining_pnl:+.4f} USD")
+        print(f"  合计PnL         : {total_pnl:+.4f} USD（应≈HL账户实际变动，含滑点/费）")
+        print("  ✅ 开/加/减/平 四步与 HL 实际持仓全部一致")
+        print("=" * 64)
+        print("\n说明: 这里用市价瞬时往返，PnL≈ -往返taker费(~0.09%)±滑点，属正常。")
+
+    except AssertionError as e:
+        print(f"\n❌ 校验失败: {e}")
+    except Exception as e:
+        print(f"\n❌ 异常: {e}")
+    finally:
+        await _flatten(trader, "收尾")
+
+
+if __name__ == "__main__":
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--live", action="store_true", help="真下单（默认干跑）")
+    ap.add_argument("--size", type=float, default=20.0, help="开仓名义USD（默认20）")
+    ap.add_argument("--leverage", type=int, default=2, help="杠杆（默认2x）")
+    ap.add_argument("--force", action="store_true", help="允许 size 超过安全上限")
+    a = ap.parse_args()
+    asyncio.run(main(a.live, a.size, a.leverage, a.force))