Pre-launch hardening: KOL module, Telegram, scanners, WS resilience

Big-picture changes since b941223:

KOL pipeline (new) — Substack/podcast/blog RSS → AI ticker extraction →
on-chain wallet diff → talks-vs-trades divergence detection. Daily polls,
19 feeds, divergence emits Post + Telegram fan-out.

Telegram push (new) — walletless free tier + wallet-linked Pro upgrade,
in-bot preference commands (/trump /btc /funding /kol /conf /quiet),
signed-envelope API for dashboard. Disconnect-wallet keeps free
subscription.

BTC funding-rate reversal scanner (new) — hourly cron, 30d cumulative
funding threshold + mean-revert + 7d price confirm, emits via
/api/signals/ingest. BTC bottom-reversal scanner promoted to System 2.

WS broadcast rewrite — per-client send timeout + parallel fan-out
(asyncio.gather). Fixes "Binance WS no close frame" reconnect storms +
APScheduler 11-min job misses, both caused by one slow client stalling
the kline loop.

Error visibility — three silent-error sites (trumpstruth/truth_social
fetchers, funding_reversal scanner) now include exception type name so
httpx ConnectError-style empty-message errors stop logging blank lines.
Telegram bot loop now classifies ReadTimeout vs network vs unknown +
logger.exception for the unknown bucket.

Security hygiene — trumpsignal.db untracked from git (held subscriber
wallets + encrypted HL keys + 22 bot trades); .gitignore now blocks
*.db/.next/backups. CORS only allows FRONTEND_URL in production.

New ops scripts —
  - scripts/preflight.py: env/DB/Telegram/AI auth verification gate
  - scripts/backup_db.sh: cron-friendly daily DB backup (SQLite + Postgres)
  - scripts/seed_kol_wallets.py: idempotent KOL on-chain wallet seeder

15 new Alembic migrations (007-021) covering convex strategy fields,
phase-1 safety, two-system frozen exits, invalidation prices, dynamic
SYS2 leverage, staged de-risk + pyramiding, peak gain tracking, risk
mode, auto-trade + grow flags, KOL module, KOL on-chain, KOL divergence,
Telegram bindings + walletless.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
k
2026-05-25 00:52:56 +08:00
parent b941223c88
commit 5fb1d52026
81 changed files with 13251 additions and 158 deletions
+62
View File
@@ -0,0 +1,62 @@
#!/usr/bin/env bash
#
# Daily DB backup. Cron-friendly. Detects SQLite vs Postgres from DATABASE_URL.
#
# Example crontab:
# 15 3 * * * /path/to/scripts/backup_db.sh >> /var/log/trumpsignal-backup.log 2>&1
#
# Retention: keeps the last RETAIN_DAYS daily backups (default 14).
# Storage: writes to $BACKUP_DIR (default ./backups/).
#
# DATABASE_URL must be available in env. Loads .env if present.
set -euo pipefail
cd "$(dirname "$0")/.."
# Load .env if it exists (so cron jobs without inherited env still work)
if [[ -f .env ]]; then
set -a
# shellcheck disable=SC1091
source .env
set +a
fi
: "${DATABASE_URL:?DATABASE_URL not set — refusing to back up nothing}"
BACKUP_DIR="${BACKUP_DIR:-./backups}"
RETAIN_DAYS="${RETAIN_DAYS:-14}"
TS="$(date -u +%Y%m%d_%H%M%S)"
mkdir -p "$BACKUP_DIR"
if [[ "$DATABASE_URL" == sqlite* ]]; then
# Extract path from sqlite+aiosqlite:///./trumpsignal.db
DB_PATH="$(echo "$DATABASE_URL" | sed -E 's#^sqlite(\+[^:]+)?:///+##')"
if [[ ! -f "$DB_PATH" ]]; then
echo "[backup] SQLite file not found: $DB_PATH" >&2
exit 1
fi
OUT="$BACKUP_DIR/sqlite_${TS}.db"
# sqlite3 .backup is online-safe (uses backup API, not a raw copy).
# Fall back to cp if sqlite3 CLI isn't installed.
if command -v sqlite3 >/dev/null 2>&1; then
sqlite3 "$DB_PATH" ".backup '$OUT'"
else
cp "$DB_PATH" "$OUT"
fi
gzip -f "$OUT"
echo "[backup] wrote $OUT.gz ($(du -h "$OUT.gz" | cut -f1))"
elif [[ "$DATABASE_URL" == postgres* ]] || [[ "$DATABASE_URL" == postgresql* ]]; then
OUT="$BACKUP_DIR/pg_${TS}.sql.gz"
# pg_dump reads DATABASE_URL natively if we strip the driver prefix.
PG_URL="${DATABASE_URL/+asyncpg/}"
PG_URL="${PG_URL/+psycopg2/}"
pg_dump "$PG_URL" --no-owner --no-privileges | gzip > "$OUT"
echo "[backup] wrote $OUT ($(du -h "$OUT" | cut -f1))"
else
echo "[backup] unsupported DATABASE_URL scheme: $DATABASE_URL" >&2
exit 1
fi
# Prune backups older than RETAIN_DAYS days
find "$BACKUP_DIR" -name "*.gz" -mtime "+$RETAIN_DAYS" -print -delete