feat: add daily budget, active window, trade snapshots, and price impact monitor
- New migrations for daily_budget, active_window, and bottrade snapshot - Add trumpstruth scraper and price_impact_monitor service - Expand bot_engine, hyperliquid, recovery, and tp_sl_monitor logic - Update API/schemas/models for new features Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
k
@@ -0,0 +1,155 @@
|
||||
# Backend deployment + uptime monitoring
|
||||
|
||||
This folder collects the moving pieces that turn the backend from "runs on
|
||||
my laptop" into "runs 24/7 with auto-restart and external alerting".
|
||||
|
||||
## What the bot needs to be reliable
|
||||
|
||||
1. **Auto-restart on crash** — handled by systemd (`Restart=always`) or
|
||||
Docker Compose (`restart: unless-stopped`). Pick one.
|
||||
2. **Health endpoints** — exposed by `app/main.py`:
|
||||
- `GET /api/health` (shallow): 200 as long as the FastAPI process is alive.
|
||||
Used by Docker's internal `HEALTHCHECK`.
|
||||
- `GET /api/health/deep`: 200 only if DB ping works AND at least one
|
||||
Truth Social scraper has polled in the last 90s. Used by external
|
||||
uptime monitors.
|
||||
3. **Dual-source scraper** — `truth_social.py` (CNN archive) +
|
||||
`trumpstruth.py` (RSS). Both insert into `posts` with the same
|
||||
`external_id` hash, so duplicates are dropped. Whoever sees the post
|
||||
first wins.
|
||||
4. **External uptime monitor** — pings the deep health endpoint from
|
||||
outside the box. If we lose internet or the box is on fire, we get
|
||||
alerted. Recommended: UptimeRobot (free tier covers this).
|
||||
|
||||
---
|
||||
|
||||
## Option A — Docker Compose (recommended)
|
||||
|
||||
`docker-compose.yml` is already configured with:
|
||||
|
||||
```yaml
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8000/api/health"]
|
||||
interval: 15s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
```
|
||||
|
||||
Deploy steps on a fresh server:
|
||||
|
||||
```bash
|
||||
git clone <repo>
|
||||
cd backend
|
||||
cp .env.example .env
|
||||
# Fill in: SERVICE_USER_POSTGRES, SERVICE_PASSWORD_POSTGRES, FRONTEND_URL,
|
||||
# ENCRYPTION_KEY, AI_API_KEY, HL_MAINNET=true
|
||||
docker compose up -d
|
||||
docker compose logs -f api # tail logs
|
||||
```
|
||||
|
||||
To verify auto-restart works:
|
||||
|
||||
```bash
|
||||
docker compose kill api # simulate crash
|
||||
sleep 12 # wait for compose to relaunch
|
||||
curl http://localhost:8000/api/health # should return {"status":"ok"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Option B — systemd (bare-metal, no Docker)
|
||||
|
||||
Use `trumpsignal-api.service` from this folder.
|
||||
|
||||
```bash
|
||||
# 1. Code lives at /opt/trumpsignal/backend
|
||||
sudo useradd -m -d /opt/trumpsignal trumpsignal
|
||||
sudo -u trumpsignal git clone <repo> /opt/trumpsignal/backend
|
||||
cd /opt/trumpsignal/backend
|
||||
sudo -u trumpsignal python3.11 -m venv venv
|
||||
sudo -u trumpsignal venv/bin/pip install -r requirements.txt
|
||||
|
||||
# 2. Configure
|
||||
sudo -u trumpsignal cp .env.example .env
|
||||
sudo -u trumpsignal nano .env # fill in secrets
|
||||
|
||||
# 3. Install + start the service
|
||||
sudo cp deploy/trumpsignal-api.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now trumpsignal-api
|
||||
|
||||
# 4. Tail logs
|
||||
sudo journalctl -u trumpsignal-api -f
|
||||
```
|
||||
|
||||
Test auto-restart:
|
||||
|
||||
```bash
|
||||
sudo systemctl kill --signal=SIGKILL trumpsignal-api
|
||||
sleep 12
|
||||
curl http://localhost:8000/api/health # should return {"status":"ok"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## UptimeRobot setup
|
||||
|
||||
1. Sign up at https://uptimerobot.com (free tier = 50 monitors, 5-min interval).
|
||||
2. Add new monitor:
|
||||
- **Type:** HTTP(s)
|
||||
- **URL:** `https://api.trump-signal.bitnews.day/api/health/deep`
|
||||
(replace with your real domain)
|
||||
- **Monitoring interval:** 5 min
|
||||
- **Alert contacts:** add your email and/or Telegram
|
||||
3. UptimeRobot will alert you when the deep healthcheck returns 503 — i.e.
|
||||
when both scrapers have been silent for >90s, OR when the DB is
|
||||
unreachable.
|
||||
|
||||
### Why deep, not shallow
|
||||
|
||||
The shallow `/api/health` returns 200 even if the scraper has died but the
|
||||
FastAPI process is still serving HTTP. A monitor pointed at the shallow
|
||||
endpoint would happily report "all green" while the bot silently misses
|
||||
posts. Always point external monitors at `/api/health/deep`.
|
||||
|
||||
---
|
||||
|
||||
## Reading the deep healthcheck
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"now": "2026-04-25T07:35:01.123456+00:00",
|
||||
"db_ok": true,
|
||||
"db_error": null,
|
||||
"scrapers": [
|
||||
{ "name": "cnn", "last_poll": "2026-04-25T07:34:58Z", "age_sec": 3, "last_error": null },
|
||||
{ "name": "trumpstruth", "last_poll": "2026-04-25T07:34:52Z", "age_sec": 9, "last_error": null }
|
||||
],
|
||||
"freshest_age_sec": 3,
|
||||
"problems": []
|
||||
}
|
||||
```
|
||||
|
||||
`status: "degraded"` + HTTP 503 means at least one item in `problems` is
|
||||
non-empty. The most common failure modes:
|
||||
|
||||
- `db: <error>` — Postgres/SQLite can't be reached. Check `DATABASE_URL`.
|
||||
- `scrapers: all stale (freshest=Ns)` — both upstream feeds silent. Check
|
||||
network egress, then check if both `ix.cnn.io` and `trumpstruth.org` are
|
||||
responding from the box: `curl -sI https://ix.cnn.io/data/truth-social/truth_archive.json`
|
||||
|
||||
---
|
||||
|
||||
## Burst-protection note (systemd)
|
||||
|
||||
The unit file caps restarts at 5 within 60s, so a hard misconfiguration
|
||||
(missing env var, etc.) stops the boot loop instead of thrashing forever.
|
||||
After fixing the underlying issue:
|
||||
|
||||
```bash
|
||||
sudo systemctl reset-failed trumpsignal-api
|
||||
sudo systemctl start trumpsignal-api
|
||||
```
|
||||
Reference in New Issue
Block a user