Recover omitted DBLP discovery families from pinned snapshots

Reviewed-on: #42

.gitea/workflows/check.yml

@@ -50,8 +50,10 @@ jobs:
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','cvf','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','www-archive','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--help'],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','dblp-family','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--year','2025','--snapshots',str(src/'tests/fixtures/aamas2025'),'--out',str(root/'installed-aamas2025')],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','iclr-archive','--help'],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','acl','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','postgres','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','edition-tracks','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','joint-editions','--help'],cwd=outside,check=True)

README.md

@@ -10,6 +10,8 @@ Paper Atlas 将 arXiv、AI 会议论文集及公开投稿决定整理到可追
 
 ## 项目结构
 
+ACL 1978 的年会与 TINLAP-2 来源关系按精确卷适配，保留原系列和不明逐篇角色；1965 的替代关系不转成全局别名。来源、排除条目和恢复边界见 [ACL 早期来源核查](docs/coverage/acl-early-sources-2026-11-09.md)。
+
 ```text
 paper_atlas/
   cli.py                 统一命令入口
@@ -68,6 +70,8 @@ EWSL、EPS、ECP、EWSP 的[具体前身卷与完整快照要求](docs/coverage/
 
 AAMAS 的[官方论文集采集器](docs/aamas-proceedings.md)支持已核验的 2012、2025 版式，可从缓存或已校验快照采集元数据。2025 年目录含 476 条非 keynote 出版物，保留完整作者和机构，HTML 未提供摘要时留空；发表形式与投稿 track 分开，不根据主题推定主会轨道或研究类型。
 
+旧 DBLP 候选文件可能漏掉新命名空间。`collect dblp-family` 可从已校验的固定压缩快照和完整目录中[恢复指定 family 的原始候选](docs/dblp-family-recovery.md)，例如 `ifaamas`；恢复文件保留卷、keynote 和 workshop 等原始条目，需另做来源范围与身份核验。
+
 `paper-atlas collect --help` 查看来源；`paper-atlas import --help` 查看导入方式。完整批次步骤见[使用指南](docs/usage.md)。
 
 共享实例名为 `foundry-postgres`，数据位于 `foundry_data` 库的 `research` schema；其他项目可按权限复用同一组表。库名通过 libpq service 配置、schema 通过 search_path 或 `--pg-schema` 指定，不写死在查询中。PostgreSQL 凭据放在服务器权限为 `0600` 的 libpq service 文件中；命令行只使用服务名称。`paper-atlas maintain postgres --help` 提供迁移入口：先完整核验 21 张业务表，再发布目标 schema。SQLite FTS5 内部表由 PostgreSQL GIN 索引替代。切换后 SQLite 为只读备份；`import records`、`import arxiv` 和 `maintain backfill` 使用 `--pg-service foundry-research` 写入主库，整个写入会话只允许一个协调写入者，会议文件失败时整体回滚并保留失败批次记录。

docs/coverage/acl-early-sources-2026-11-09.md

@@ -0,0 +1,52 @@
+# ACL 早期年份：会议存在、出版目录与逐篇资料分开核验
+
+固定普查中 ACL 1963–1978 仍有年份缺口。不能因为会议年表连续，就假设每年都有独立的正式论文集；也不能因为当前目录始于 1979，就断言此前没有摘要、通知或其他公开资料。本次不修改固定的 1,266 个目标年份基准。
+
+## 1978 年：限定 T78-1 的来源关系
+
+[同期年会通知](https://aclanthology.org/J77-4001.pdf)的 PDF 第 9–10 页（印刷页码 9–10）明确说明，1978 年 ACL 年会采用第二届 Theoretical Issues in Natural Language Processing 的形式，地点为 Urbana/Champaign，日期为 7 月 25–27 日。通知中的 ACM 联合赞助确认仍处于待确认状态，不能将该预告改写为已完成的行政决定。
+
+[1982 年 Donald E. Walker 的历史回顾](https://aclanthology.org/www.mt-archive.info/70/ACL-1982-Walker-1.pdf)第 3 页（印刷页码 91）的年会表及缩写说明，独立对应了 1978 年的时间、地点与 TNLP/TINLAP-2 关系。两份扫描件相关页已渲染并逐页目视核对，不仅依靠 OCR。
+
+[官方卷目录](https://aclanthology.org/volumes/T78-1/)与固定 XML `T78.xml` 保留原系列名 TINLAP、完整卷名、1978 年及编辑 David L. Waltz。适配仅接受精确的 collection `T78`、volume `1` 和上述字段组合；关键元数据变化会拒绝处理，等待重新核查。
+
+- 规范事件名使用 `ACL+TINLAP`，原系列名 `TINLAP`、原始 XML 的 `venue_codes: [tinlap]` 及证据均保留。
+- ACL/TINLAP 别名只挂在这一规范事件系列上，不把所有 TINLAP 年份都算成 ACL。T75、T87 及 T78 其他卷没有此例外。
+- 通知描述了邀请与提交论文两种参与方式，XML 没有逐篇角色。统一暂存为 `proceedings_unspecified`，不猜测每篇属于 invited、main 或 workshop。
+- XML 共 36 条条目，其中 T78-1036 是无作者的场次讨论问题列表。精确识别并在 manifest 的 `excluded_materials` 保留其标识、标题与原因；其余 35 条作为具名贡献整理。该排除不推广到所有无作者条目，已识别条目发生变化也须重新核查。
+- 全部 35 条 XML 均无摘要。出版目录中的 accepted 是来源观察；通知提及录用／不录用通知时间，不提供任何可识别的拒稿决定，不能制造 rejected 或 withdrawn。
+
+采集器仅生成元数据文件，不写业务数据库。真实来源逐条验收、PostgreSQL 身份预检、串行导入与独立验收仍是后续门槛。35 条来源不等于 35 个新增论文实体，更不等于完成全库去重。
+
+## 1965 年：替代关系不改写为别名
+
+Walker 回顾第 1 页（印刷页码 89）说明，1965 年 ICCL（后来称 COLING）替代了当年的年会。[官方 C65-1 目录](https://aclanthology.org/volumes/C65-1/)及 XML 仍以 COLING 标记该卷。这是限定年份的历史关系，不证明 COLING 与 ACL 是两个可全局互换的名字。
+
+本次保留 C65 原有 `COLING` 归属，不添加 ACL 全局别名、不重放已导入记录、不将某一年的日历关系推广到整个系列，也不据此悄悄减少固定普查缺口。其他 1963–1977 年资料仍须逐年核查；本次不能证明这些年份整体属于非归档例外。
+
+## 固定来源与恢复
+
+服务器证据目录：`/data/paper-index/collected/acl-early-source-evidence-v1`。五份官方响应均 HTTP 200，另有两份来自既有官方 XML checkout 的文件，已与 Git 对象逐字节核对。未下载论文全文集合；两份 PDF 是本次历史／行政取证资料，不进入论文全文库。
+
+| 文件 | SHA256 |
+|---|---|
+| `acl-venue.html` | `c60f64cb4d72e677aeb84a5844cde28a8d50579e1995f08e8c7801eed1e3fac5` |
+| `acl-history-1982.pdf` | `35c21bd4983975bcaccbb94bd8ab91c9019fc190feec1791255407cf18a7e49a` |
+| `acl-1978-notice.pdf` | `14b5deb471c55afb07d789529b95d0462a2a1a77df3dfba7fa1f5d3ab75eb411` |
+| `tinlap1978-volume.html` | `7e20d709f72f415819e4d17e3674b19dc7d871eabcf0cf8b4935f0392d775729` |
+| `coling1965-volume.html` | `0d2f2f49f2a311f9c0b80a8832f6476eb32f94f35abf36f6d2f2e10063ac880f` |
+| `T78.xml` | `f712f94cff0cba05f786b7ac31dad2ea9408fde2a17a04531130df623f57dd84` |
+| `C65.xml` | `572791610e20b7c6045d984d968e73a18a144bcb5c3da64112001c781826808e` |
+
+XML commit：`a7be623d01c48c61405dec31c871121cdbed5a01`。测试 fixture `tests/fixtures/acl-t78.xml` 是该原始文件，不是手工模拟论文；不包含 PDF。PDF 哈希建立完整性，不单独证明语义，语义依据是上述具体页面。
+
+在服务器使用既有命令，固定 checkout 并输出到新目录：
+
+```sh
+paper-atlas collect acl \
+  --checkout /data/paper-index/raw/metadata-checkouts/acl-anthology \
+  --collection T78 \
+  --out /data/paper-index/collected/acl-t78-source-v1
+```
+
+该命令会校验／更新官方 metadata checkout，实际 commit 必须以生成清单为准；严格重现本次取证时，使用已冻结的 XML 和上述 commit，不把更新后的内容冒充旧快照。不要将无 collection 限制的全部历史输出直接作为增量队列。采集完成不代表导入完成，恢复前先检查当前单元和报告，不因日志安静重启已执行的写入。

docs/dblp-family-recovery.md

@@ -0,0 +1,37 @@
+# DBLP family 候选恢复
+
+DBLP 的会议命名空间会变化。旧候选过滤器包含 `atal`、`aamas`，遗漏了 `ifaamas`；2026-11-01 固定快照的完整目录实际已有 `conf/ifaamas/2025` 和 `conf/ifaamas/2026`。新目录盘点已把 `ifaamas` 纳入广泛候选范围，已有快照也可按需恢复，无需重新下载或重建完整目录。
+
+## 运行
+
+在已有元数据的服务器运行：
+
+```sh
+paper-atlas collect dblp-family \
+  --archive /data/paper-index/raw/dblp-2026-11-01.xml.gz \
+  --dtd /data/paper-index/raw/dblp-2023-06-28.dtd \
+  --catalogue /data/paper-index/collected/dblp/dblp-catalogue.sqlite \
+  --family ifaamas \
+  --out /data/paper-index/collected/ifaamas-recovered
+```
+
+`--family` 可以重复，但仅接受项目明确列出的候选 family。`--out` 必须尚不存在。同名输出由文件锁防止并发恢复；任务运行中应观测原任务，不能因为日志间隔长就重复启动。完整扫描会定期打印进度。
+
+命令重新验证固定压缩包 SHA256、DTD MD5 与大小，并只读打开目录。目录旁的 `dblp-ai-candidates.manifest.json` 必须属于同一个快照和 DTD，包含完整记录类型计数。旧候选文件不作为完整性依据，也不会被改写。
+
+恢复扫描使用项目原有 XML 流式解析器，仅允许已校验的本地 DTD，不联网读取 XML 实体。最终会比对完整快照的记录总数、逐 family 数量和全部选中卷的原始元数据。全部一致后才原子发布输出目录；失败时不发布半成品，也不覆盖既有文件。排查明确终止的失败任务后，可用尚不存在的输出目录重新运行。
+
+## 输出与后续流程
+
+- `dblp-family-candidates.jsonl`：原始 `inproceedings` 和 `proceedings`，保留书目 key、原始作者、标题、年份、卷引用、出版链接等字段。
+- `recovery-manifest.json`：快照、DTD、目录凭据哈希，完整扫描数量、逐 family 数量、选中卷、旧候选文件的已知数量及输出哈希。状态为 `recovered_raw_candidates_pending_scope_review`。
+
+恢复结果没有新增规范化的 `venue`、`track` 或决定标签，也不是 `import records` 的输入队列。`ifaamas` 的整卷可能混有研究论文、扩展摘要、keynote、博士生、演示和 JAAMAS 展示。目录过滤修复不会把这些条目统一标成 main；规范化的范围映射仍需第一方逐栏目证据。
+
+后续先用官方目录核对标题、页码、作者及发表形式，再查数据库稳定标识和候选实体，保留每次投稿、决定和标签来源。只有审核后的新增分区才能原生导入并独立验收；已验收的来源记录不能重放。单个年份有记录不代表逐篇、track、摘要或公开拒稿覆盖完整。
+
+## 2026-11-09 核查实例
+
+对既有固定快照的独立原始元数据恢复发现 `ifaamas` 有 1,122 条出版物候选及 2 卷：2025 年 479 条、2026 年 643 条。2025 年的 479 条标题与起始页均对应官方目录，其中 3 条 keynote 应排除，余下 476 条出版物需逐项处理。该数字包含已经验收的 225 条来源，不是 1,122 篇可新增论文。
+
+此实例说明过滤遗漏的影响，不代表恢复命令已完成生产导入。2026 年仍需核验官方出版清单、栏目、身份及逐次决定；缺少录用记录不能推断为拒稿。最终全库去重、覆盖和查询审计继续单独进行。

paper_atlas/cli.py

@@ -27,6 +27,7 @@ COMMANDS = {
     ('collect','aamas-proceedings'): ('paper_atlas.collectors.aamas', {}, '按官方目录区分 AAMAS 发表形式和摘要'),
     ('collect','www-archive'): ('paper_atlas.collectors.www_archive', {}, '核验早期WWW官方档案并分开待核查条目'),
     ('collect','dblp'): ('paper_atlas.collectors.dblp_records', {}, '核对并整理 DBLP 历史发表记录'),
+    ('collect','dblp-family'): ('paper_atlas.collectors.dblp_recovery', {}, '从固定快照恢复指定 DBLP family 的原始候选'),
     ('catalogue','dblp'): ('paper_atlas.collectors.dblp', {}, '盘点 DBLP 历史会议目录'),
     ('import','arxiv'): ('paper_atlas.ingestion.arxiv', {}, '导入 arXiv 元数据 ZIP'),
     ('import','records'): ('paper_atlas.ingestion.records', {}, '导入核验过的会议文件'),

paper_atlas/collectors/acl.py

@@ -6,6 +6,7 @@ import xml.etree.ElementTree as ET
 from pathlib import Path
 
 from paper_atlas.io.git_metadata import checkout_metadata
+from paper_atlas.collectors.acl_historical import event_scope, material_reason
 
 VENUES={'acl':'ACL','emnlp':'EMNLP','naacl':'NAACL','eacl':'EACL','coling':'COLING','conll':'CoNLL'}
 
@@ -29,7 +30,10 @@ def paper_id(collection,volume,number):
     return f'{collection}-{int(volume):0{width}d}{int(number):0{digits}d}'
 
 
-def volume_scope(meta,volume):
+def volume_scope(meta,volume,collection=None):
+    historical=event_scope(meta,volume,collection)
+    if historical:
+        return historical['canonical_venue'],historical['venue_aliases'],historical['track']
     codes=[text(n).lower() for n in meta.findall('venue')]
     book=text(meta.find('booktitle'))
     lower=book.lower()
@@ -59,7 +63,7 @@ def collect_xml(path,out,commit):
         vid=volume.attrib['id']
         meta=volume.find('meta')
         if meta is None:raise ValueError('Volume metadata missing')
-        scope=volume_scope(meta,vid)
+        scope=volume_scope(meta,vid,collection)
         key=f'{collection}-{vid}'
         if scope is None:
             results.append({'volume':key,'status':'excluded_by_venue','venue_codes':[text(n) for n in meta.findall('venue')],'booktitle':text(meta.find('booktitle')),'year':text(meta.find('year'))})
@@ -70,12 +74,18 @@ def collect_xml(path,out,commit):
         year=int(year_text)
         records=[]
         frontmatter=0
+        excluded_materials=[]
+        historical=event_scope(meta,vid,collection)
         for paper in volume.findall('paper'):
             pid=paper.attrib['id']
             if int(pid)==0:
                 frontmatter+=1
                 continue
             identifier=paper_id(collection,vid,pid)
+            reason=material_reason(collection,vid,paper)
+            if reason:
+                excluded_materials.append({'external_id':identifier,'title':text(paper.find('title')),'reason':reason})
+                continue
             doi=text(paper.find('doi'))
             if doi.startswith('10.18653/v1/') and doi.split('10.18653/v1/',1)[1].casefold()!=identifier.casefold():
                 raise ValueError('Official DOI disagrees with constructed Anthology ID: '+identifier)
@@ -83,6 +93,8 @@ def collect_xml(path,out,commit):
             if not title:raise ValueError('Missing paper title: '+identifier)
             authors=[{'given':text(a.find('first')),'family':text(a.find('last'))} for a in paper.findall('author')]
             records.append({'source':'acl_anthology','external_id':identifier,'url':f'https://aclanthology.org/{identifier}/','title':title,'abstract':text(paper.find('abstract')) or None,'authors':authors,'identifiers':{'acl':identifier,'doi':doi},'venue':venue,'venue_aliases':aliases if len(aliases)>1 else [],'year':year,'track':track,'status':'accepted','raw_decision':'listed in authoritative ACL Anthology proceedings XML','metadata':{'collection':collection,'volume':vid,'booktitle':text(meta.find('booktitle')),'venue_codes':[text(n) for n in meta.findall('venue')],'pages':text(paper.find('pages')),'publisher':text(meta.find('publisher')),'metadata_commit':commit,'source_xml':f'https://github.com/acl-org/acl-anthology/blob/{commit}/data/xml/{collection}.xml'}})
+            if historical:
+                records[-1]['metadata']['historical_event_scope']=historical
         if not records:
             results.append({'volume':key,'status':'unavailable','reason':'no research paper elements after excluding front matter'})
             continue
@@ -93,7 +105,7 @@ def collect_xml(path,out,commit):
             for record in records:output.write(json.dumps(record,ensure_ascii=False)+'\n')
         part.replace(target)
         missing=sum(not row['abstract'] for row in records)
-        manifest={'source':'acl_anthology','venue':venue,'year':year,'track':track,'count':len(records),'xml_paper_elements':len(volume.findall('paper')),'excluded_frontmatter':frontmatter,'missing_abstract_count':missing,'list_url':f'https://aclanthology.org/{key}/','metadata_commit':commit,'file':str(target),'status':'partial' if missing else 'collected'}
+        manifest={'source':'acl_anthology','venue':venue,'year':year,'track':track,'count':len(records),'xml_paper_elements':len(volume.findall('paper')),'excluded_frontmatter':frontmatter,'excluded_materials':excluded_materials,'missing_abstract_count':missing,'list_url':f'https://aclanthology.org/{key}/','metadata_commit':commit,'file':str(target),'status':'partial' if missing else 'collected'}
         target.with_suffix('.manifest.json').write_text(json.dumps(manifest,indent=2))
         results.append(manifest)
         print(json.dumps(manifest),flush=True)

paper_atlas/collectors/acl_historical.py

@@ -0,0 +1,45 @@
+"""Edition-specific historical scope; never alias every TINLAP year to ACL."""
+
+HISTORY_URL = 'https://aclanthology.org/www.mt-archive.info/70/ACL-1982-Walker-1.pdf'
+NOTICE_URL = 'https://aclanthology.org/J77-4001.pdf'
+BOOK = 'Theoretical Issues in Natural Language Processing-2'
+MATERIAL_TITLE = 'List of questions suggested for consideration in each session'
+
+
+def text(node):
+    return ' '.join(''.join(node.itertext()).split()) if node is not None else ''
+
+
+def event_scope(meta, volume, collection):
+    if (collection, volume) != ('T78', '1'):
+        return None
+    if ([text(n) for n in meta.findall('venue')] != ['tinlap']
+            or [text(n) for n in meta.findall('year')] != ['1978']
+            or [text(n) for n in meta.findall('booktitle')] != [BOOK]
+            or [(text(n.find('first')), text(n.find('last'))) for n in meta.findall('editor')]
+               != [('David L.', 'Waltz')]):
+        raise ValueError('Reviewed T78-1 scope metadata changed; independent review required')
+    return {
+        'collection': 'T78', 'volume': '1', 'original_venue': 'TINLAP',
+        'event_year': 1978, 'event_dates': ['1978-07-25', '1978-07-27'],
+        'relationship': 'ACL annual meeting took the form of TINLAP-2',
+        'canonical_venue': 'ACL+TINLAP', 'venue_aliases': ['ACL', 'TINLAP'],
+        'track': 'proceedings_unspecified',
+        'track_scope_note': 'Invited and submitted contributions are described at volume level; individual roles are not identified in XML.',
+        'evidence': [
+            {'url': NOTICE_URL, 'sha256': '14b5deb471c55afb07d789529b95d0462a2a1a77df3dfba7fa1f5d3ab75eb411',
+             'pdf_pages': [9, 10], 'printed_pages': ['9', '10'],
+             'scope': 'Contemporary annual meeting notice, not individual decision notes; ACM confirmation was pending in this notice.'},
+            {'url': HISTORY_URL, 'sha256': '35c21bd4983975bcaccbb94bd8ab91c9019fc190feec1791255407cf18a7e49a',
+             'pdf_pages': [3], 'printed_pages': ['91'],
+             'scope': 'Retrospective annual meeting table and TNLP abbreviation confirm the 1978 event association.'},
+        ],
+    }
+
+
+def material_reason(collection, volume, paper):
+    if (collection, volume, paper.attrib.get('id')) != ('T78', '1', '36'):
+        return None
+    if text(paper.find('title')) != MATERIAL_TITLE or paper.findall('author'):
+        raise ValueError('Reviewed T78-1036 material changed; independent review required')
+    return 'Session question list, not an authored research contribution; no decision inferred.'

paper_atlas/collectors/dblp.py

@@ -20,7 +20,7 @@ DTD_MD5='96e4283e71886dec2173718957293648'
 DTD_NAMES={'dblp.dtd','dblp-2023-06-28.dtd'}
 RELEASE='https://doi.org/10.4230/dblp.xml.2026-11-01'
 # Broad discovery candidates, including historical names; not a venue mapping.
-FAMILIES=set('nips icml iclr colm colt aistats uai clear acl emnlp naacl eacl coling conll cvpr iccv eccv aaai ijcai ecai kdd sigir www wsdm recsys cikm icdm sdm ecml pkdd ecmlpkdd icra iros rss corl aips icaps ecp ewsp kr atal aamas icassp interspeech eurospeech icslp ismir miccai fat facct lrec mm mlsys'.split())
+FAMILIES=set('nips icml iclr colm colt aistats uai clear acl emnlp naacl eacl coling conll cvpr iccv eccv aaai ijcai ecai kdd sigir www wsdm recsys cikm icdm sdm ecml pkdd ecmlpkdd icra iros rss corl aips icaps ecp ewsp kr atal aamas ifaamas icassp interspeech eurospeech icslp ismir miccai fat facct lrec mm mlsys'.split())
 
 
 def records(archive,dtd_bytes):

paper_atlas/collectors/dblp_recovery.py

@@ -0,0 +1,156 @@
+"""Recover raw family candidates from a pinned snapshot and its existing catalogue.
+
+The old candidate file is not an exhaustive source: earlier discovery filters
+can omit an entire namespace. This command checks the complete snapshot and
+catalogue, without assigning venues, tracks, acceptance or paper identities.
+"""
+import argparse
+from collections import Counter
+from contextlib import closing
+import fcntl
+import json
+from pathlib import Path
+import re
+import sqlite3
+import tempfile
+
+from paper_atlas.collectors.dblp import DTD_MD5, FAMILIES, RELEASE, XML_SHA, family, records
+from paper_atlas.db.submissions import timestamp
+from paper_atlas.io.snapshots import digest_file
+
+
+def catalogue_evidence(catalogue, families, expected_sha, expected_dtd_md5):
+    manifest_path = catalogue.with_name('dblp-ai-candidates.manifest.json')
+    manifest = json.loads(manifest_path.read_text())
+    if (manifest.get('snapshot_sha256') != expected_sha
+            or manifest.get('dtd_md5') != expected_dtd_md5):
+        raise ValueError('Catalogue belongs to a different snapshot or DTD')
+    totals = manifest.get('record_types', {})
+    if (set(totals) != {'proceedings', 'inproceedings'}
+            or any(type(n) is not int or n < 0 for n in totals.values())
+            or type(manifest.get('records_scanned')) is not int
+            or sum(totals.values()) != manifest['records_scanned']):
+        raise ValueError('Incomplete catalogue record totals')
+    counts = Counter()
+    volumes = {}
+    # The catalogue is evidence, never a writable recovery destination.
+    with closing(sqlite3.connect(catalogue.resolve().as_uri() + '?mode=ro', uri=True)) as c:
+        for name in families:
+            found = c.execute('SELECT record_type,count FROM family_counts WHERE family=?',
+                              (name,)).fetchall()
+            if not found or not any(n for _, n in found):
+                raise ValueError('Family absent from complete catalogue: ' + name)
+            for kind, count in found:
+                if kind not in totals or type(count) is not int or count < 0:
+                    raise ValueError('Invalid catalogue family count')
+                counts[(name, kind)] = count
+            for key, raw in c.execute('SELECT key,metadata_json FROM proceedings WHERE family=?',
+                                      (name,)):
+                row = json.loads(raw)
+                if (family(key) != name or row.get('key') != key
+                        or row.get('record_type') != 'proceedings'):
+                    raise ValueError('Invalid catalogue proceedings evidence')
+                volumes[key] = row
+    if len(volumes) != sum(counts[(name, 'proceedings')] for name in families):
+        raise ValueError('Catalogue proceedings count differs from volume records')
+    return manifest, digest_file(manifest_path)['sha256'], counts, volumes
+
+
+def recover(archive, dtd, catalogue, out, families, *, expected_sha=XML_SHA,
+            expected_dtd_md5=DTD_MD5):
+    families = list(families)
+    if (not families or any(not isinstance(name, str)
+                           or not re.fullmatch(r'[a-z][a-z0-9-]*', name)
+                           or name not in FAMILIES for name in families)):
+        raise ValueError('Select an explicitly supported discovery family')
+    families = sorted(set(families))
+    archive, dtd, catalogue = Path(archive), Path(dtd), Path(catalogue)
+    output = Path(out).absolute()
+    if output.exists() or output.is_symlink():
+        raise ValueError('Recovery output must not already exist')
+    output.parent.mkdir(parents=True, exist_ok=True)
+    lock_path = output.parent / ('.' + output.name + '.recovery.lock')
+    with lock_path.open('a') as lock:
+        fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        if output.exists() or output.is_symlink():
+            raise ValueError('Recovery output appeared before scan')
+        actual = digest_file(archive)
+        if actual['sha256'] != expected_sha:
+            raise ValueError('DBLP snapshot checksum mismatch')
+        if dtd.stat().st_size > 128 * 1024 or digest_file(dtd)['md5'] != expected_dtd_md5:
+            raise ValueError('DBLP DTD checksum mismatch or size limit')
+        manifest, manifest_sha, expected, volumes = catalogue_evidence(
+            catalogue, families, expected_sha, expected_dtd_md5)
+        totals, found = Counter(), Counter()
+        seen_keys, found_volumes = set(), set()
+        with tempfile.TemporaryDirectory(prefix='.dblp-family-', dir=output.parent) as tmp:
+            stage = Path(tmp) / 'recovered'
+            stage.mkdir()
+            target = stage / 'dblp-family-candidates.jsonl'
+            with target.open('w') as stream:
+                for row in records(archive, dtd.read_bytes()):
+                    totals[row['record_type']] += 1
+                    name = family(row['key'])
+                    if name in families:
+                        if row['key'] in seen_keys:
+                            raise ValueError('Duplicate raw bibliographic key')
+                        seen_keys.add(row['key'])
+                        found[(name, row['record_type'])] += 1
+                        if row['record_type'] == 'proceedings':
+                            if volumes.get(row['key']) != row:
+                                raise ValueError('Raw proceedings differs from catalogue evidence')
+                            found_volumes.add(row['key'])
+                        stream.write(json.dumps(row, ensure_ascii=False) + '\n')
+                    scanned = sum(totals.values())
+                    if scanned % 250000 == 0:
+                        print(json.dumps({'status': 'scanning_raw_family_candidates',
+                                          'records_scanned': scanned,
+                                          'recovered_records': sum(found.values())}), flush=True)
+            if (totals != Counter(manifest['record_types'])
+                    or sum(totals.values()) != manifest['records_scanned']):
+                raise ValueError('Complete snapshot totals differ from catalogue')
+            if found != expected or found_volumes != volumes.keys():
+                raise ValueError('Recovered family totals differ from catalogue')
+            report = {
+                'format_version': 1, 'source': 'dblp',
+                'status': 'recovered_raw_candidates_pending_scope_review',
+                'finished_at': timestamp(), 'source_release': RELEASE,
+                'snapshot_sha256': actual['sha256'], 'dtd_md5': expected_dtd_md5,
+                'catalogue_manifest_sha256': manifest_sha, 'families': families,
+                'records_scanned': sum(totals.values()), 'record_types': dict(totals),
+                'recovered_records': sum(found.values()),
+                'recovered_publications': sum(found[(n, 'inproceedings')] for n in families),
+                'recovered_volumes': len(found_volumes),
+                'family_counts': [{'family': name, 'record_type': kind, 'count': count}
+                                  for (name, kind), count in sorted(found.items())],
+                'old_candidate_family_counts': {
+                    name: manifest.get('candidate_family_counts', {}).get(name, 0)
+                    for name in families},
+                'catalogue_volumes': [volumes[k] for k in sorted(volumes)],
+                'file': str(output / target.name),
+                'file_sha256': digest_file(target)['sha256'], 'database_written': False,
+                'note': 'Raw bibliographic candidates, including volume records and possible '
+                        'keynotes/workshops. No venue, track, acceptance or work identity '
+                        'inferred. Requires primary-source scope and identity review; '
+                        'not a normalized import queue. Existing imported sources must '
+                        'not be replayed.',
+            }
+            # Deliberately not an import-records sidecar manifest.
+            (stage / 'recovery-manifest.json').write_text(
+                json.dumps(report, ensure_ascii=False, indent=2) + '\n')
+            if output.exists() or output.is_symlink():
+                raise ValueError('Recovery output appeared during scan')
+            stage.rename(output)
+    return report
+
+
+def main(argv=None, *, prog=None):
+    parser = argparse.ArgumentParser(prog=prog, description=__doc__)
+    parser.add_argument('--archive', required=True, type=Path)
+    parser.add_argument('--dtd', required=True, type=Path)
+    parser.add_argument('--catalogue', required=True, type=Path)
+    parser.add_argument('--family', required=True, action='append', choices=sorted(FAMILIES))
+    parser.add_argument('--out', required=True, type=Path)
+    args = parser.parse_args(argv)
+    print(json.dumps(recover(args.archive, args.dtd, args.catalogue, args.out, args.family),
+                     ensure_ascii=False))

tests/fixtures/acl-t78.xml

@@ -0,0 +1,273 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<collection id="T78">
+  <volume id="1" type="proceedings">
+    <!-- https://aclanthology.org/T78-1/ -->
+    <meta>
+      <booktitle>Theoretical Issues in Natural Language <fixed-case>P</fixed-case>rocessing-2</booktitle>
+      <editor id="david-l-waltz"><first>David L.</first><last>Waltz</last></editor>
+      <venue>tinlap</venue>
+      <year>1978</year>
+    </meta>
+    <frontmatter>
+      <!-- https://aclanthology.org/T78-1000/ -->
+      <pdf hash="fb6164ad"/>
+      <bibkey>tinlap-1978-theoretical</bibkey>
+    </frontmatter>
+    <paper id="1">
+      <!-- https://aclanthology.org/T78-1001/ -->
+      <title>Testing The Psychological Reality of a Representational Model</title>
+      <author><first>Dedre</first><last>Gentner</last></author>
+      <pdf hash="f723a956"/>
+      <bibkey>gentner-1978-testing-psychological</bibkey>
+    </paper>
+    <paper id="2">
+      <!-- https://aclanthology.org/T78-1002/ -->
+      <title>What Makes Something “Ad Hoc”</title>
+      <author id="roger-c-schank"><first>Roger C.</first><last>Schank</last></author>
+      <pdf hash="636853f9"/>
+      <bibkey>schank-1978-makes-something</bibkey>
+    </paper>
+    <paper id="3">
+      <!-- https://aclanthology.org/T78-1003/ -->
+      <title>The Relation of Grammar to Cognition–a Synopsis</title>
+      <author><first>Leonard</first><last>Talmy</last></author>
+      <pdf hash="101c67dc"/>
+      <bibkey>talmy-1978-relation-grammar</bibkey>
+    </paper>
+    <paper id="4">
+      <!-- https://aclanthology.org/T78-1004/ -->
+      <title>On primitives, prototypes, and other semantic anomalies</title>
+      <author><first>Terry</first><last>Winograd</last></author>
+      <pdf hash="f8e47023"/>
+      <bibkey>winograd-1978-primitives-prototypes</bibkey>
+    </paper>
+    <paper id="5">
+      <!-- https://aclanthology.org/T78-1005/ -->
+      <title>Taxonomic Lattice Structures for Situation Recognition</title>
+      <author id="william-a-woods"><first>William A.</first><last>Woods</last></author>
+      <pdf hash="92957ddf"/>
+      <bibkey>woods-1978-taxonomic-lattice</bibkey>
+    </paper>
+    <paper id="6">
+      <!-- https://aclanthology.org/T78-1006/ -->
+      <title>Description Formation and Discourse Model Synthesis</title>
+      <author id="bonnie-webber"><first>Bonnie Lynn</first><last>Webber</last></author>
+      <pdf hash="ffded1d2"/>
+      <bibkey>webber-1978-description-formation</bibkey>
+    </paper>
+    <paper id="7">
+      <!-- https://aclanthology.org/T78-1007/ -->
+      <title>The Processing of Referring Expressions within a Semantic Network</title>
+      <author><first>John R.</first><last>Anderson</last></author>
+      <pdf hash="8bcc083a"/>
+      <bibkey>anderson-1978-processing-referring</bibkey>
+    </paper>
+    <paper id="8">
+      <!-- https://aclanthology.org/T78-1008/ -->
+      <title>Reference Diaries</title>
+      <author><first>Herbert H.</first><last>Clark</last></author>
+      <author><first>Catherine</first><last>Marshall</last></author>
+      <pdf hash="d06b7ea2"/>
+      <bibkey>clark-marshall-1978-reference-diaries</bibkey>
+    </paper>
+    <paper id="9">
+      <!-- https://aclanthology.org/T78-1009/ -->
+      <title>Subsequent Reference: Syntactic and Rhetorical Constraints</title>
+      <author id="david-d-mcdonald"><first>David D.</first><last>McDonald</last></author>
+      <pdf hash="2d424559"/>
+      <bibkey>mcdonald-1978-subsequent-reference</bibkey>
+    </paper>
+    <paper id="10">
+      <!-- https://aclanthology.org/T78-1010/ -->
+      <title>Some Psycholinguistic Constraints on the Construction and Interpretation of Definite Descriptions</title>
+      <author><first>Andrew</first><last>Ortony</last></author>
+      <pdf hash="69807727"/>
+      <bibkey>ortony-1978-psycholinguistic-constraints</bibkey>
+    </paper>
+    <paper id="11">
+      <!-- https://aclanthology.org/T78-1011/ -->
+      <title>Bound Variables and Other Anaphors</title>
+      <author id="barbara-h-partee"><first>Barbara H.</first><last>Partee</last></author>
+      <pdf hash="e8e6a510"/>
+      <bibkey>partee-1978-bound-variables</bibkey>
+    </paper>
+    <paper id="12">
+      <!-- https://aclanthology.org/T78-1012/ -->
+      <title>The Use of Focus as a Tool for Disambiguation of Definite Noun Phrases</title>
+      <author id="candace-l-sidner"><first>Candace L.</first><last>Sidner</last></author>
+      <pdf hash="7b96d2b7"/>
+      <bibkey>sidner-1978-use-focus</bibkey>
+    </paper>
+    <paper id="13">
+      <!-- https://aclanthology.org/T78-1013/ -->
+      <title>Focusing in Dialog</title>
+      <author id="barbara-j-grosz"><first>Barbara J.</first><last>Grosz</last></author>
+      <pdf hash="8c8d4c2e"/>
+      <bibkey>grosz-1978-focusing-dialog</bibkey>
+    </paper>
+    <paper id="14">
+      <!-- https://aclanthology.org/T78-1014/ -->
+      <title>Topic Levels</title>
+      <author><first>Joseph E.</first><last>Grimes</last></author>
+      <pdf hash="f485b2ba"/>
+      <bibkey>grimes-1978-topic-levels</bibkey>
+    </paper>
+    <paper id="15">
+      <!-- https://aclanthology.org/T78-1015/ -->
+      <title>Toward a Rational Model of Discourse Comprehension</title>
+      <author><first>Jerry L.</first><last>Morgan</last></author>
+      <pdf hash="5e12a597"/>
+      <bibkey>morgan-1978-toward-rational</bibkey>
+    </paper>
+    <paper id="16">
+      <!-- https://aclanthology.org/T78-1016/ -->
+      <title>Assent and Compliance in Children’s Language Comprehension</title>
+      <author><first>David R.</first><last>Olson</last></author>
+      <pdf hash="708bc4a0"/>
+      <bibkey>olson-1978-assent-compliance</bibkey>
+    </paper>
+    <paper id="17">
+      <!-- https://aclanthology.org/T78-1017/ -->
+      <title>Speech Acts as a Basis for Understanding Dialogue Coherence</title>
+      <author id="c-raymond-perrault"><first>C. Raymond</first><last>Perrault</last></author>
+      <author id="james-allen"><first>James F.</first><last>Allen</last></author>
+      <pdf hash="d111402b"/>
+      <bibkey>perrault-allen-1978-speech</bibkey>
+    </paper>
+    <paper id="18">
+      <!-- https://aclanthology.org/T78-1018/ -->
+      <title>A Framework for Comparing Language Experiences (with particular emphasis on: The Effect of Audience on Discourse Models)</title>
+      <author><first>Andee</first><last>Rubin</last></author>
+      <pdf hash="5d9a6150"/>
+      <bibkey>rubin-1978-framework-comparing</bibkey>
+    </paper>
+    <paper id="19">
+      <!-- https://aclanthology.org/T78-1019/ -->
+      <title>Intentlonallty and Human Conversations</title>
+      <author id="jaime-g-carbonell"><first>Jaime G.</first><last>Carbonell Jr</last></author>
+      <pdf hash="c50fc619"/>
+      <bibkey>carbonell-jr-1978-intentlonallty-human</bibkey>
+    </paper>
+    <paper id="20">
+      <!-- https://aclanthology.org/T78-1020/ -->
+      <title>On the Interdependence of Language and Perception</title>
+      <author id="david-l-waltz"><first>David L.</first><last>Waltz</last></author>
+      <pdf hash="3c708191"/>
+      <bibkey>waltz-1978-interdependence-language</bibkey>
+    </paper>
+    <paper id="21">
+      <!-- https://aclanthology.org/T78-1021/ -->
+      <title>The Problem of Naming Shapes: Vision-Language Interface</title>
+      <author id="ruzena-bajcsy"><first>R.</first><last>Bajcsy</last></author>
+      <author id="aravind-joshi"><first>A. K.</first><last>Joshi</last></author>
+      <pdf hash="50331b4e"/>
+      <bibkey>bajcsy-joshi-1978-problem-naming</bibkey>
+    </paper>
+    <paper id="22">
+      <!-- https://aclanthology.org/T78-1022/ -->
+      <title>An Argument about the Composition of Conceptual Structure</title>
+      <author><first>Ray</first><last>Jackendoff</last></author>
+      <pdf hash="57d001f8"/>
+      <bibkey>jackendoff-1978-argument-composition</bibkey>
+    </paper>
+    <paper id="23">
+      <!-- https://aclanthology.org/T78-1023/ -->
+      <title>On the Ontological Status of Visual Mental Images</title>
+      <author><first>Stephen Michael</first><last>Kosslyn</last></author>
+      <pdf hash="fd06e265"/>
+      <bibkey>kosslyn-1978-ontological-status</bibkey>
+    </paper>
+    <paper id="24">
+      <!-- https://aclanthology.org/T78-1024/ -->
+      <title>What has language to do with perception? Some speculations on the <i>Lingua Mentis</i></title>
+      <author><first>Zenon W.</first><last>Pylyshyn</last></author>
+      <pdf hash="a2f3a044"/>
+      <bibkey>pylyshyn-1978-language-perception</bibkey>
+    </paper>
+    <paper id="25">
+      <!-- https://aclanthology.org/T78-1025/ -->
+      <title>Semantic Primitives in Language and Vision</title>
+      <author id="yorick-wilks"><first>Yorick</first><last>Wilks</last></author>
+      <pdf hash="b4e13896"/>
+      <bibkey>wilks-1978-semantic-primitives</bibkey>
+    </paper>
+    <paper id="26">
+      <!-- https://aclanthology.org/T78-1026/ -->
+      <title>A Note on Partial Match of Descriptions. Can One Simultaneously Question (Retrieve) and Inform (Update)?</title>
+      <author id="aravind-joshi"><first>Aravind K.</first><last>Joshi</last></author>
+      <pdf hash="74969243"/>
+      <bibkey>joshi-1978-note-partial</bibkey>
+    </paper>
+    <paper id="27">
+      <!-- https://aclanthology.org/T78-1027/ -->
+      <title>With a Spoon in Hand This Must Be the Eating Frame</title>
+      <author><first>Eugene</first><last>Charniak</last></author>
+      <pdf hash="f1be68bb"/>
+      <bibkey>charniak-1978-spoon-hand</bibkey>
+    </paper>
+    <paper id="28">
+      <!-- https://aclanthology.org/T78-1028/ -->
+      <title>Fragments of a Theory of Human Plausible Reasoning</title>
+      <author><first>Allan</first><last>Collins</last></author>
+      <pdf hash="67b4bca2"/>
+      <bibkey>collins-1978-fragments-theory</bibkey>
+    </paper>
+    <paper id="29">
+      <!-- https://aclanthology.org/T78-1029/ -->
+      <title>Indirect Responses to Loaded Questions</title>
+      <author><first>S. Jerrold</first><last>Kaplan</last></author>
+      <pdf hash="135037f9"/>
+      <bibkey>kaplan-1978-indirect-responses</bibkey>
+    </paper>
+    <paper id="30">
+      <!-- https://aclanthology.org/T78-1030/ -->
+      <title>On Reasoning by Default</title>
+      <author><first>Raymond</first><last>Reiter</last></author>
+      <pdf hash="0de53d33"/>
+      <bibkey>reiter-1978-reasoning-default</bibkey>
+    </paper>
+    <paper id="31">
+      <!-- https://aclanthology.org/T78-1031/ -->
+      <title>Path-Based and Node-Based Inference in Semantic Networks</title>
+      <author id="stuart-c-shapiro"><first>Stuart C.</first><last>Shapiro</last></author>
+      <pdf hash="2ddc5535"/>
+      <bibkey>shapiro-1978-path-based</bibkey>
+    </paper>
+    <paper id="32">
+      <!-- https://aclanthology.org/T78-1032/ -->
+      <title>The Representation of Derivable Information in Memory: When What Might Have Been Left Unsaid Is Said</title>
+      <author><first>Rand J.</first><last>Spiro</last></author>
+      <author><first>Joseph</first><last>Esposito</last></author>
+      <author><first>Richard J.</first><last>Vondruska</last></author>
+      <pdf hash="7758fb45"/>
+      <bibkey>spiro-etal-1978-representation-derivable</bibkey>
+    </paper>
+    <paper id="33">
+      <!-- https://aclanthology.org/T78-1033/ -->
+      <title>A Heuristic for Paradigms</title>
+      <author><first>Joseph E.</first><last>Grimes</last></author>
+      <pdf hash="53b42c16"/>
+      <bibkey>grimes-1978-heuristic-paradigms</bibkey>
+    </paper>
+    <paper id="34">
+      <!-- https://aclanthology.org/T78-1034/ -->
+      <title>A Computational Account of Some Constraints on Language</title>
+      <author id="mitch-marcus"><first>Mitchell</first><last>Marcus</last></author>
+      <pdf hash="c9724c43"/>
+      <bibkey>marcus-1978-computational-account</bibkey>
+    </paper>
+    <paper id="35">
+      <!-- https://aclanthology.org/T78-1035/ -->
+      <title>Remarks on Processing, Constraints, and the Lexicon</title>
+      <author id="thomas-wasow"><first>Thomas</first><last>Wasow</last></author>
+      <pdf hash="8beed275"/>
+      <bibkey>wasow-1978-remarks-processing</bibkey>
+    </paper>
+    <paper id="36">
+      <!-- https://aclanthology.org/T78-1036/ -->
+      <title>List of questions suggested for consideration in each session</title>
+      <pdf hash="1950b908"/>
+      <bibkey>nn-1978-list-questions</bibkey>
+    </paper>
+  </volume>
+</collection>

tests/test_acl_historical.py

@@ -0,0 +1,114 @@
+"""Pinned T78 evidence, edition-local aliases and an explicitly identified material."""
+import copy
+import hashlib
+import json
+from pathlib import Path
+import sqlite3
+import tempfile
+import unittest
+import xml.etree.ElementTree as ET
+
+from paper_atlas.collectors.acl import collect_xml, volume_scope
+from paper_atlas.collectors.acl_historical import event_scope, material_reason
+from paper_atlas.db.store import Store
+from paper_atlas.ingestion.records import ingest_file
+
+FIXTURE = Path(__file__).parent/'fixtures/acl-t78.xml'
+COMMIT = 'a7be623d01c48c61405dec31c871121cdbed5a01'
+
+
+class AclHistoricalTests(unittest.TestCase):
+    def setUp(self):
+        self.root = ET.parse(FIXTURE).getroot()
+        self.volume = self.root.find('volume')
+        self.meta = self.volume.find('meta')
+
+    def test_frozen_volume_matches_official_snapshot(self):
+        self.assertEqual(hashlib.sha256(FIXTURE.read_bytes()).hexdigest(), 'f712f94cff0cba05f786b7ac31dad2ea9408fde2a17a04531130df623f57dd84')
+        self.assertEqual(len(self.volume.findall('paper')), 36)
+        self.assertEqual(volume_scope(self.meta, '1', 'T78'), ('ACL+TINLAP', ['ACL', 'TINLAP'], 'proceedings_unspecified'))
+
+    def test_unknown_year_collection_or_volume_is_not_globally_aliased(self):
+        for collection, volume in [('T75', '1'), ('T87', '1'), ('T78', '2'), ('C65', '1')]:
+            with self.subTest(collection=collection, volume=volume):
+                self.assertIsNone(event_scope(self.meta, volume, collection))
+                self.assertIsNone(volume_scope(self.meta, volume, collection))
+        self.assertIsNone(volume_scope(self.meta, '1'))
+
+    def test_known_volume_scope_rejects_critical_metadata_drift(self):
+        for field, replacement in [('venue', 'acl'), ('year', '1979'), ('booktitle', 'Another TINLAP'), ('editor/first', 'Another')]:
+            changed = copy.deepcopy(self.meta)
+            changed.find(field).text = replacement
+            with self.subTest(field=field), self.assertRaises(ValueError):
+                event_scope(changed, '1', 'T78')
+        changed = copy.deepcopy(self.meta)
+        ET.SubElement(changed, 'year').text = '1978'
+        with self.assertRaises(ValueError):
+            event_scope(changed, '1', 'T78')
+
+    def test_evidence_retains_original_series_dates_and_role_boundary(self):
+        scope = event_scope(self.meta, '1', 'T78')
+        self.assertEqual(scope['original_venue'], 'TINLAP')
+        self.assertEqual(scope['event_dates'], ['1978-07-25', '1978-07-27'])
+        self.assertEqual([e['pdf_pages'] for e in scope['evidence']], [[9, 10], [3]])
+        self.assertIn('not identified', scope['track_scope_note'])
+        self.assertIn('pending', scope['evidence'][0]['scope'])
+
+    def test_only_exact_session_questions_material_is_excluded(self):
+        paper = copy.deepcopy(self.volume.findall('paper')[-1])
+        self.assertIn('Session question list', material_reason('T78', '1', paper))
+        self.assertIsNone(material_reason('C65', '1', paper))
+        self.assertIsNone(material_reason('T78', '2', paper))
+        paper.set('id', '37')
+        self.assertIsNone(material_reason('T78', '1', paper))
+        paper.set('id', '36')
+        ET.SubElement(paper, 'author')
+        with self.assertRaises(ValueError):
+            material_reason('T78', '1', paper)
+
+    def test_all_authored_source_fields_and_material_receipt_survive_collection(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            manifest, = collect_xml(FIXTURE, temporary, COMMIT)
+            rows = [json.loads(raw) for raw in Path(manifest['file']).read_text().splitlines()]
+        self.assertEqual(manifest['count'], 35)
+        self.assertEqual(manifest['xml_paper_elements'], 36)
+        self.assertEqual(manifest['missing_abstract_count'], 35)
+        self.assertEqual(manifest['excluded_materials'][0]['external_id'], 'T78-1036')
+        for number, row in enumerate(rows, 1):
+            paper = self.volume.findall('paper')[number-1]
+            self.assertEqual(row['external_id'], f'T78-1{number:03d}')
+            self.assertEqual(row['title'], ' '.join(''.join(paper.find('title').itertext()).split()))
+            self.assertEqual(row['metadata']['venue_codes'], ['tinlap'])
+            self.assertEqual(row['metadata']['historical_event_scope']['original_venue'], 'TINLAP')
+            self.assertEqual(row['metadata']['metadata_commit'], COMMIT)
+            self.assertEqual((row['status'], row['track'], row['abstract']), ('accepted', 'proceedings_unspecified', None))
+
+    def test_old_coling_scope_is_unchanged_by_calendar_replacement_evidence(self):
+        meta = ET.fromstring('<meta><venue>coling</venue><year>1965</year><booktitle>COLING 1965</booktitle></meta>')
+        self.assertEqual(volume_scope(meta, '1', 'C65'), ('COLING', ['COLING'], 'main'))
+        self.assertIsNone(event_scope(meta, '1', 'C65'))
+
+    def test_native_ingestion_does_not_alias_other_tinlap_years_or_infer_negative_status(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            root = Path(temporary)
+            manifest, = collect_xml(FIXTURE, root/'source', COMMIT)
+            db = root/'sample.sqlite'
+            store = Store(db)
+            pid, rid = store.record('test_control', 'tinlap1975', 'https://example.org/tinlap1975',
+                                    'An earlier TINLAP contribution', None, ['Jane Doe'])
+            store.decision(pid, rid, 'TINLAP', 1975, 'main', 'tinlap1975', 'https://example.org/tinlap1975', 'accepted', 'test control')
+            store.conn.commit()
+            store.conn.close()
+            ingest_file(db, manifest['file'])
+            ingest_file(db, manifest['file'])
+            with sqlite3.connect(db) as c:
+                self.assertEqual(c.execute('SELECT count(*) FROM papers').fetchone()[0], 36)
+                self.assertEqual(c.execute('SELECT count(*) FROM submissions').fetchone()[0], 36)
+                self.assertEqual(c.execute('SELECT count(*) FROM decisions').fetchone()[0], 36)
+                self.assertEqual(c.execute("SELECT count(*) FROM decisions WHERE status!='accepted'").fetchone()[0], 0)
+                self.assertIsNone(c.execute("SELECT 1 FROM source_records WHERE external_id='T78-1036'").fetchone())
+                aliases = c.execute('SELECT v.acronym,a.alias FROM venue_aliases a JOIN venues v ON v.id=a.venue_id ORDER BY a.alias').fetchall()
+                self.assertEqual(aliases, [('ACL+TINLAP', 'ACL'), ('ACL+TINLAP', 'TINLAP')])
+                metadata = json.loads(c.execute("SELECT metadata_json FROM source_records WHERE external_id='T78-1001'").fetchone()[0])
+                self.assertEqual(metadata['historical_event_scope']['event_year'], 1978)
+                self.assertEqual(c.execute('PRAGMA foreign_key_check').fetchall(), [])

tests/test_dblp_recovery.py

@@ -0,0 +1,141 @@
+import gzip
+import hashlib
+import json
+from pathlib import Path
+import sqlite3
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from paper_atlas.collectors.dblp import catalogue
+from paper_atlas.collectors.dblp_recovery import recover
+from paper_atlas.collectors.dblp_scope import scope
+from paper_atlas.ingestion.records import ingest_file
+
+
+class DblpRecoveryTests(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.addCleanup(self.tmp.cleanup)
+        self.root = Path(self.tmp.name)
+        self.dtd = self.root / 'dblp.dtd'
+        self.dtd.write_bytes(b'<!ENTITY eacute "&#233;">')
+        self.archive = self.root / 'dblp.xml.gz'
+        # These include a keynote and a workshop, neither a main-paper inference.
+        body = '''<proceedings key="conf/ifaamas/2025"><title>AAMAS 2025</title><year>2025</year></proceedings>
+<inproceedings key="conf/ifaamas/keynote"><author>Ren&eacute;</author><title>Keynote: Agents</title><year>2025</year><booktitle>AAMAS</booktitle><crossref>conf/ifaamas/2025</crossref></inproceedings>
+<inproceedings key="conf/ifaamas/paper"><author>Alice Doe</author><title>Research <i>agents</i></title><year>2025</year><booktitle>AAMAS</booktitle><crossref>conf/ifaamas/2025</crossref><ee>https://dl.acm.org/doi/10.5555/3709347.123</ee></inproceedings>
+<proceedings key="conf/ifaamas/2025w"><title>AAMAS Workshop</title><year>2025</year></proceedings>
+<inproceedings key="conf/ifaamas/workshop"><title>Workshop paper</title><year>2025</year><booktitle>AAMAS Workshop</booktitle><crossref>conf/ifaamas/2025w</crossref></inproceedings>
+<inproceedings key="conf/unrelated/other"><title>Other family</title><year>2025</year></inproceedings>'''
+        with gzip.open(self.archive, 'wb') as stream:
+            stream.write(('<?xml version="1.0"?><!DOCTYPE dblp SYSTEM "dblp.dtd"><dblp>'
+                          + body + '</dblp>').encode())
+        self.sha = hashlib.sha256(self.archive.read_bytes()).hexdigest()
+        self.md5 = hashlib.md5(self.dtd.read_bytes()).hexdigest()
+        self.manifest = catalogue(self.archive, self.dtd, self.root / 'catalogue',
+                                  self.sha, self.md5)
+        self.catalogue = Path(self.manifest['catalogue'])
+        self.manifest_path = self.catalogue.with_name('dblp-ai-candidates.manifest.json')
+
+    def run_recovery(self, **kwargs):
+        options = dict(expected_sha=self.sha, expected_dtd_md5=self.md5)
+        options.update(kwargs)
+        return recover(self.archive, self.dtd, self.catalogue, self.root / 'recovered',
+                       ['ifaamas'], **options)
+
+    def test_fresh_discovery_and_recovery_from_old_candidate_omission(self):
+        self.assertEqual(self.manifest['candidate_family_counts']['ifaamas'], 3)
+        # Simulate an older filter without altering its complete catalogue.
+        manifest = dict(self.manifest)
+        manifest['candidate_family_counts'] = {}
+        self.manifest_path.write_text(json.dumps(manifest))
+        candidate = Path(manifest['file'])
+        candidate.write_text('old incomplete candidate file\n')
+        before = {p: p.read_bytes() for p in [self.archive, self.dtd, self.catalogue,
+                                             self.manifest_path, candidate]}
+        with patch('urllib.request.urlopen', side_effect=AssertionError('No network')):
+            report = self.run_recovery()
+        rows = [json.loads(line) for line in Path(report['file']).read_text().splitlines()]
+        self.assertEqual(report['recovered_publications'], 3)
+        self.assertEqual(report['recovered_volumes'], 2)
+        self.assertEqual(report['recovered_records'], 5)
+        self.assertEqual(report['records_scanned'], 6)
+        self.assertEqual(report['old_candidate_family_counts'], {'ifaamas': 0})
+        self.assertFalse(report['database_written'])
+        self.assertEqual(next(r for r in rows if r['key'].endswith('/keynote'))['fields']['author'], ['René'])
+        self.assertTrue(any(r['key'].endswith('/workshop') for r in rows))
+        self.assertTrue(all(not ({'source', 'status', 'venue', 'track'} & r.keys()) for r in rows))
+        publication = next(r for r in rows if r['key'].endswith('/paper'))
+        volume = next(r for r in rows if r['key'] == 'conf/ifaamas/2025')
+        self.assertEqual(scope(publication, volume), (None, 'outside_target_family'))
+        self.assertEqual({p: p.read_bytes() for p in before}, before)
+        self.assertEqual(hashlib.sha256(Path(report['file']).read_bytes()).hexdigest(),
+                         report['file_sha256'])
+        # Raw recovery files cannot accidentally enter the normalized writer.
+        with patch('paper_atlas.ingestion.records.open_store') as writer:
+            with self.assertRaisesRegex(ValueError, 'source manifest'):
+                ingest_file(self.root / 'never.sqlite', report['file'])
+            writer.assert_not_called()
+        self.assertFalse((self.root / 'never.sqlite').exists())
+
+    def test_checksum_mismatch_does_not_publish(self):
+        for options in [{'expected_sha': '0' * 64}, {'expected_dtd_md5': '0' * 32}]:
+            with self.subTest(options=options), self.assertRaisesRegex(ValueError, 'checksum'):
+                self.run_recovery(**options)
+            self.assertFalse((self.root / 'recovered').exists())
+
+    def test_wrong_catalogue_snapshot_and_incomplete_totals_are_rejected(self):
+        for changes in [{'snapshot_sha256': '0' * 64}, {'dtd_md5': '0' * 32},
+                        {'records_scanned': 99}, {'record_types': {'proceedings': 2}}]:
+            manifest = {**self.manifest, **changes}
+            self.manifest_path.write_text(json.dumps(manifest))
+            with self.subTest(changes=changes), self.assertRaises(ValueError):
+                self.run_recovery()
+            self.assertFalse((self.root / 'recovered').exists())
+
+    def test_changed_family_count_does_not_publish_partial_result(self):
+        with sqlite3.connect(self.catalogue) as c:
+            c.execute("UPDATE family_counts SET count=count+1 WHERE family='ifaamas' AND record_type='inproceedings'")
+        with self.assertRaisesRegex(ValueError, 'family totals'):
+            self.run_recovery()
+        self.assertFalse((self.root / 'recovered').exists())
+
+    def test_changed_volume_metadata_is_detected(self):
+        with sqlite3.connect(self.catalogue) as c:
+            row = json.loads(c.execute("SELECT metadata_json FROM proceedings WHERE key='conf/ifaamas/2025'").fetchone()[0])
+            row['fields']['title'] = ['Wrong volume']
+            c.execute("UPDATE proceedings SET metadata_json=? WHERE key='conf/ifaamas/2025'", (json.dumps(row),))
+        with self.assertRaisesRegex(ValueError, 'proceedings differs'):
+            self.run_recovery()
+        self.assertFalse((self.root / 'recovered').exists())
+
+    def test_truncated_gzip_never_publishes(self):
+        self.archive.write_bytes(self.archive.read_bytes()[:-8])
+        revised_sha = hashlib.sha256(self.archive.read_bytes()).hexdigest()
+        self.manifest_path.write_text(json.dumps({**self.manifest, 'snapshot_sha256': revised_sha}))
+        with self.assertRaises((EOFError, OSError)):
+            self.run_recovery(expected_sha=revised_sha)
+        self.assertFalse((self.root / 'recovered').exists())
+
+    def test_existing_output_is_preserved_and_unknown_family_is_rejected(self):
+        out = self.root / 'recovered'
+        out.mkdir()
+        sentinel = out / 'keep'
+        sentinel.write_text('existing')
+        with self.assertRaisesRegex(ValueError, 'already exist'):
+            self.run_recovery()
+        self.assertEqual(sentinel.read_text(), 'existing')
+        for names in [[], ['../ifaamas'], ['not-a-supported-family'], [None]]:
+            with self.subTest(names=names), self.assertRaises(ValueError):
+                recover(self.archive, self.dtd, self.catalogue, self.root / 'other', names)
+
+    def test_requested_family_missing_from_catalogue_is_explicit(self):
+        with self.assertRaisesRegex(ValueError, 'absent from complete catalogue'):
+            recover(self.archive, self.dtd, self.catalogue, self.root / 'recovered',
+                    ['ifaamas', 'icml'], expected_sha=self.sha, expected_dtd_md5=self.md5)
+        self.assertFalse((self.root / 'recovered').exists())
+
+
+if __name__ == '__main__':
+    unittest.main()