KOL feeds: fix dead/blocked sources, drop stale feeds (29→25)

Feed-health pass over KOL_FEEDS:
- raoulpal: stale Substack (last 2024-05) → Real Vision podcast feed
- dampedspring: paywalled (0 entries) → free "Damped Spring 101" Substack
- unchained: Cloudflare 403 → canonical Megaphone podcast feed
- lynalden: Cloudflare 202 → FeedBurner mirror
- glassnode: recovered via httpx http2=True (was 403 on HTTP/1.1)
- browser User-Agent + Accept headers on feed fetch
- removed dead feeds with no active replacement: placeholder,
  dragonfly, niccarter, eugene
- pin h2==4.3.0 (required by http2=True)

All 25 remaining feeds verified fetching real body content; newest
post per feed ≤88d. Bundles in-flight KOL-module work already in the
working tree (kol_x ingest, migration 027, tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
k
2026-06-09 22:55:16 +08:00
parent 213bb911e3
commit 54884f3e24
38 changed files with 2340 additions and 322 deletions
+14
View File
@@ -226,6 +226,20 @@ async def lifespan(app: FastAPI):
)
logger.info("KOL Substack poller scheduled daily at 01:15 UTC.")
# ── KOL X (Twitter) tweet ingestion (daily) ───────────────────────────
# Polls X-native KOLs (andrewkang/@Rewkang, murad — no Substack feed) plus
# Hayes's real-time position statements. Writes KolPost(source="twitter")
# that the divergence scan (02:15) joins against on-chain data — this is the
# post-side feed that lights up the andrewkang/murad wallets. No-op if
# twitterapi_io_key is unset. Runs 01:30 — between substack (01:15) and
# on-chain (02:00) so fresh posts are present for the 02:15 divergence scan.
from app.services.kol_x import run_x_poll
_scheduler.add_job(
run_x_poll, "cron", hour=1, minute=30,
id="kol_x_poll", max_instances=1, coalesce=True,
)
logger.info("KOL X (Twitter) poller scheduled daily at 01:30 UTC.")
# ── KOL A-tier: on-chain holdings snapshot (daily) ────────────────────
# Polls HL public API (free) for perp positions; Arkham (key optional)
# for full portfolio. Diffs against yesterday's snapshot → writes