KOL feeds: fix dead/blocked sources, drop stale feeds (29→25)

Feed-health pass over KOL_FEEDS:
- raoulpal: stale Substack (last 2024-05) → Real Vision podcast feed
- dampedspring: paywalled (0 entries) → free "Damped Spring 101" Substack
- unchained: Cloudflare 403 → canonical Megaphone podcast feed
- lynalden: Cloudflare 202 → FeedBurner mirror
- glassnode: recovered via httpx http2=True (was 403 on HTTP/1.1)
- browser User-Agent + Accept headers on feed fetch
- removed dead feeds with no active replacement: placeholder,
  dragonfly, niccarter, eugene
- pin h2==4.3.0 (required by http2=True)

All 25 remaining feeds verified fetching real body content; newest
post per feed ≤88d. Bundles in-flight KOL-module work already in the
working tree (kol_x ingest, migration 027, tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
k
2026-06-09 22:55:16 +08:00
parent 213bb911e3
commit 54884f3e24
38 changed files with 2340 additions and 322 deletions
+17 -2
View File
@@ -31,6 +31,7 @@ from typing import Optional
from fastapi import APIRouter, Depends, Header, HTTPException
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
@@ -174,8 +175,22 @@ async def ingest_signal(
prefilter_reason="external_signal", # bypasses entry-filter audit
)
db.add(post)
await db.commit()
await db.refresh(post)
try:
await db.commit()
await db.refresh(post)
except IntegrityError:
# Concurrent request beat us to the INSERT — fetch the existing row.
await db.rollback()
existing2 = await db.execute(select(Post).where(Post.external_id == ext_id_hashed))
prior2 = existing2.scalar_one_or_none()
if prior2:
return SignalIngestResponse(
status="duplicate",
post_id=prior2.id,
dedup_against=prior2.id,
note=f"concurrent ingest: external_id {body.external_id!r} already exists",
)
raise # unexpected — re-raise if we still can't find the row
logger.info(
"Ingested signal: source=%s id=%s → post_id=%d, %s/%s conf=%d category=%s",