KOL feeds: fix dead/blocked sources, drop stale feeds (29→25)

Feed-health pass over KOL_FEEDS:
- raoulpal: stale Substack (last 2024-05) → Real Vision podcast feed
- dampedspring: paywalled (0 entries) → free "Damped Spring 101" Substack
- unchained: Cloudflare 403 → canonical Megaphone podcast feed
- lynalden: Cloudflare 202 → FeedBurner mirror
- glassnode: recovered via httpx http2=True (was 403 on HTTP/1.1)
- browser User-Agent + Accept headers on feed fetch
- removed dead feeds with no active replacement: placeholder,
  dragonfly, niccarter, eugene
- pin h2==4.3.0 (required by http2=True)

All 25 remaining feeds verified fetching real body content; newest
post per feed ≤88d. Bundles in-flight KOL-module work already in the
working tree (kol_x ingest, migration 027, tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
k
2026-06-09 22:55:16 +08:00
parent 213bb911e3
commit 54884f3e24
38 changed files with 2340 additions and 322 deletions
+47
View File
@@ -10,6 +10,8 @@ change accidentally turns "BTC bottom triggers: 3/3 firing" into a less
clear phrasing, CI catches it.
"""
import pytest
from app.services.telegram_digest import (
GlobalDigest, MacroBlock, KolBlock, TrumpBlock, format_digest,
)
@@ -174,3 +176,48 @@ def test_total_length_well_under_telegram_cap():
"open_pnl_summary": "3 open (+125.3 USD)",
})
assert len(out) < 4096
# ── build_global_digest DB aggregation (twitter-noise exclusion) ───────────────
@pytest.mark.asyncio
async def test_build_global_digest_excludes_twitter_noise(monkeypatch):
"""KOL posts_24h must NOT count twitter noise (gm / RT / jokes). Substack
essays (tier=NULL) and twitter signal posts (tier!='noise') count; twitter
noise (tier='noise') is excluded. Regression guard for the digest count
being inflated by a KOL's gm/RT spam after X ingestion landed."""
from datetime import datetime, timedelta
from sqlalchemy.ext.asyncio import (
AsyncSession, async_sessionmaker, create_async_engine,
)
from app.models import Base, KolPost
from app.services import telegram_digest
engine = create_async_engine("sqlite+aiosqlite:///:memory:", future=True)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
sf = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
now = datetime(2026, 6, 4, 12, 0, 0)
recent = now - timedelta(hours=2)
def _post(ext, source, tier=None, handle="cryptohayes"):
return KolPost(
kol_handle=handle, source=source, external_id=ext,
url=f"https://x.com/x/status/{ext}", published_at=recent,
raw_text="body", content_hash=ext, tier=tier,
)
async with sf() as s:
s.add_all([
_post("sub1", "substack", tier=None, handle="raoulpal"), # essay → count
_post("tw1", "twitter", tier="directional"), # signal → count
_post("tw2", "twitter", tier="noise"), # gm noise → exclude
_post("tw3", "twitter", tier="noise"), # RT noise → exclude
])
await s.commit()
monkeypatch.setattr(telegram_digest, "async_session", sf)
g = await telegram_digest.build_global_digest(now=now)
# 2 real posts (substack essay + twitter signal); 2 noise excluded
assert g.kol.posts_24h == 2