Preserve edition-specific ACL and TINLAP 1978 source evidence

Reviewed-on: #41

.gitea/workflows/check.yml

@@ -50,6 +50,8 @@ jobs:
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','cvf','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','www-archive','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--help'],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','iclr-archive','--help'],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','acl','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','postgres','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','edition-tracks','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','joint-editions','--help'],cwd=outside,check=True)

README.md

@@ -6,8 +6,12 @@ Paper Atlas 将 arXiv、AI 会议论文集及公开投稿决定整理到可追
 
 项目仍在建设中。候选范围包含 43 个会议（含 ACM MM、MLSys），尚未完成全部历史数据和最终去重复核。实际进度以数据库、来源清单和覆盖报告为准，而不是仓库文件数量。
 
+历史 ICLR 2014–2016 官方录用页由 `collect iclr-archive` 适配，保留主会/workshop及重复展示证据；运行和身份核验边界见 [历史档案说明](docs/coverage/iclr-archive-2014-2016.md)。
+
 ## 项目结构
 
+ACL 1978 的年会与 TINLAP-2 来源关系按精确卷适配，保留原系列和不明逐篇角色；1965 的替代关系不转成全局别名。来源、排除条目和恢复边界见 [ACL 早期来源核查](docs/coverage/acl-early-sources-2026-11-09.md)。
+
 ```text
 paper_atlas/
   cli.py                 统一命令入口

docs/coverage/acl-early-sources-2026-11-09.md

@@ -0,0 +1,52 @@
+# ACL 早期年份：会议存在、出版目录与逐篇资料分开核验
+
+固定普查中 ACL 1963–1978 仍有年份缺口。不能因为会议年表连续，就假设每年都有独立的正式论文集；也不能因为当前目录始于 1979，就断言此前没有摘要、通知或其他公开资料。本次不修改固定的 1,266 个目标年份基准。
+
+## 1978 年：限定 T78-1 的来源关系
+
+[同期年会通知](https://aclanthology.org/J77-4001.pdf)的 PDF 第 9–10 页（印刷页码 9–10）明确说明，1978 年 ACL 年会采用第二届 Theoretical Issues in Natural Language Processing 的形式，地点为 Urbana/Champaign，日期为 7 月 25–27 日。通知中的 ACM 联合赞助确认仍处于待确认状态，不能将该预告改写为已完成的行政决定。
+
+[1982 年 Donald E. Walker 的历史回顾](https://aclanthology.org/www.mt-archive.info/70/ACL-1982-Walker-1.pdf)第 3 页（印刷页码 91）的年会表及缩写说明，独立对应了 1978 年的时间、地点与 TNLP/TINLAP-2 关系。两份扫描件相关页已渲染并逐页目视核对，不仅依靠 OCR。
+
+[官方卷目录](https://aclanthology.org/volumes/T78-1/)与固定 XML `T78.xml` 保留原系列名 TINLAP、完整卷名、1978 年及编辑 David L. Waltz。适配仅接受精确的 collection `T78`、volume `1` 和上述字段组合；关键元数据变化会拒绝处理，等待重新核查。
+
+- 规范事件名使用 `ACL+TINLAP`，原系列名 `TINLAP`、原始 XML 的 `venue_codes: [tinlap]` 及证据均保留。
+- ACL/TINLAP 别名只挂在这一规范事件系列上，不把所有 TINLAP 年份都算成 ACL。T75、T87 及 T78 其他卷没有此例外。
+- 通知描述了邀请与提交论文两种参与方式，XML 没有逐篇角色。统一暂存为 `proceedings_unspecified`，不猜测每篇属于 invited、main 或 workshop。
+- XML 共 36 条条目，其中 T78-1036 是无作者的场次讨论问题列表。精确识别并在 manifest 的 `excluded_materials` 保留其标识、标题与原因；其余 35 条作为具名贡献整理。该排除不推广到所有无作者条目，已识别条目发生变化也须重新核查。
+- 全部 35 条 XML 均无摘要。出版目录中的 accepted 是来源观察；通知提及录用／不录用通知时间，不提供任何可识别的拒稿决定，不能制造 rejected 或 withdrawn。
+
+采集器仅生成元数据文件，不写业务数据库。真实来源逐条验收、PostgreSQL 身份预检、串行导入与独立验收仍是后续门槛。35 条来源不等于 35 个新增论文实体，更不等于完成全库去重。
+
+## 1965 年：替代关系不改写为别名
+
+Walker 回顾第 1 页（印刷页码 89）说明，1965 年 ICCL（后来称 COLING）替代了当年的年会。[官方 C65-1 目录](https://aclanthology.org/volumes/C65-1/)及 XML 仍以 COLING 标记该卷。这是限定年份的历史关系，不证明 COLING 与 ACL 是两个可全局互换的名字。
+
+本次保留 C65 原有 `COLING` 归属，不添加 ACL 全局别名、不重放已导入记录、不将某一年的日历关系推广到整个系列，也不据此悄悄减少固定普查缺口。其他 1963–1977 年资料仍须逐年核查；本次不能证明这些年份整体属于非归档例外。
+
+## 固定来源与恢复
+
+服务器证据目录：`/data/paper-index/collected/acl-early-source-evidence-v1`。五份官方响应均 HTTP 200，另有两份来自既有官方 XML checkout 的文件，已与 Git 对象逐字节核对。未下载论文全文集合；两份 PDF 是本次历史／行政取证资料，不进入论文全文库。
+
+| 文件 | SHA256 |
+|---|---|
+| `acl-venue.html` | `c60f64cb4d72e677aeb84a5844cde28a8d50579e1995f08e8c7801eed1e3fac5` |
+| `acl-history-1982.pdf` | `35c21bd4983975bcaccbb94bd8ab91c9019fc190feec1791255407cf18a7e49a` |
+| `acl-1978-notice.pdf` | `14b5deb471c55afb07d789529b95d0462a2a1a77df3dfba7fa1f5d3ab75eb411` |
+| `tinlap1978-volume.html` | `7e20d709f72f415819e4d17e3674b19dc7d871eabcf0cf8b4935f0392d775729` |
+| `coling1965-volume.html` | `0d2f2f49f2a311f9c0b80a8832f6476eb32f94f35abf36f6d2f2e10063ac880f` |
+| `T78.xml` | `f712f94cff0cba05f786b7ac31dad2ea9408fde2a17a04531130df623f57dd84` |
+| `C65.xml` | `572791610e20b7c6045d984d968e73a18a144bcb5c3da64112001c781826808e` |
+
+XML commit：`a7be623d01c48c61405dec31c871121cdbed5a01`。测试 fixture `tests/fixtures/acl-t78.xml` 是该原始文件，不是手工模拟论文；不包含 PDF。PDF 哈希建立完整性，不单独证明语义，语义依据是上述具体页面。
+
+在服务器使用既有命令，固定 checkout 并输出到新目录：
+
+```sh
+paper-atlas collect acl \
+  --checkout /data/paper-index/raw/metadata-checkouts/acl-anthology \
+  --collection T78 \
+  --out /data/paper-index/collected/acl-t78-source-v1
+```
+
+该命令会校验／更新官方 metadata checkout，实际 commit 必须以生成清单为准；严格重现本次取证时，使用已冻结的 XML 和上述 commit，不把更新后的内容冒充旧快照。不要将无 collection 限制的全部历史输出直接作为增量队列。采集完成不代表导入完成，恢复前先检查当前单元和报告，不因日志安静重启已执行的写入。

docs/coverage/iclr-archive-2014-2016.md

@@ -0,0 +1,44 @@
+# ICLR 2014–2016 官方历史录用档案
+
+此适配器是长期维护工具的一部分，按需运行。只识别已经独立核对的三种页面布局；后续页面发生变化必须重新核查，不能放松计数或轨道边界以使采集成功。
+
+| 官方页面 | 页面链接次数 | 整理后的不同记录 |
+|---|---:|---:|
+| [2014 Conference Proceedings](https://iclr.cc/archive/2014/conference-proceedings/) | 35 | main 35 |
+| [2015 accepted-main](https://iclr.cc/archive/www/doku.php%3Fid=iclr2015:accepted-main.html) | 116 | main 31、workshop 74 |
+| [2016 accepted-main](https://iclr.cc/archive/www/doku.php%3Fid=iclr2016:accepted-main.html) | 80 | main 80 |
+
+2015 的 11 篇 oral 全部再次出现在 poster 段落中。相同年份、轨道及 arXiv 标识的重复展示只生成一条来源记录，但每次展示的标题、作者显示字符串、链接、版本和段落均保留在 `source_variants`。两处标题或解析作者冲突时停止采集、等待人工核查。不同年份或不同轨道不合并投稿。
+
+2014 的作者以分号列出，一条有末尾分号，一条标题和作者斜体块不在普通段落中，部分作者段落拆成多个格式片段；必须读取完整作者段落或独立斜体块，不能只读首个格式片段。2016 的代码链接不属于作者。保留原文，解析结果不替代来源证据。oral、poster 作为 `iclr_archive:presentation` 标签，不改成独立投稿轨道。arXiv 版本留在来源变体中，稳定标识使用去版本号的编号。
+
+## 来源快照与运行
+
+2026-11-09 已冻结的官方响应 SHA256：
+
+- 2014：`a33858176fbffae0066ed3c10310f7dd98a66d12a36334b6339ef1f915ef07af`
+- 2015：`131491663742cb084662a4804009b8318e0fa7bc184d4a9044d16dd28f05b888`
+- 2016：`7320b8652431793b6ea0743f871ee446bf08b6361ccd4e421b6b53449e71f4f6`
+
+在服务器运行，输出新目录，不下载论文 PDF：
+
+```sh
+paper-atlas collect iclr-archive --year 2015 \
+  --snapshots /data/paper-index/collected/historical-next-source-evidence-v1 \
+  --out /data/paper-index/collected/iclr-archive-source-v1/2015
+
+# 或重新下载官方 HTML，并保留 HTTP 缓存与收据
+paper-atlas collect iclr-archive --year 2016 \
+  --cache /data/paper-index/cache/iclr-archive \
+  --out /data/paper-index/collected/iclr-archive-source-v1/2016
+```
+
+冻结输入目录必须包含 `receipts.json`，每项有精确 `url`、HTTP `status: 200`、相对 `file`、`bytes`、`sha256` 和带时区的 `finished_at`。读取时重新核对尺寸、哈希和路径；拒绝越界及指向目录外的软链接。已完成输出不可覆盖，应检查报告后另建明确版本目录。解析或计数失败时不生成完成报告，也不导入数据库。
+
+输出分轨道 JSONL、manifest、原始 HTML 证据和年度采集报告。仅支持这三年已审查的页面。2014 workshop、其他未列轨道、逐篇摘要及公开拒稿范围尚未通过这些页面核验，不能据此声称完整。缺席不代表 rejected、withdrawn 或从未被录用。
+
+## 导入前后的边界
+
+这三页合计 220 条不同来源记录，不等于新增 220 篇论文；同一工作可能已存在于 arXiv 或其他会议。先核验来源字段与原始页面，再执行 PostgreSQL 只读预检，比较稳定标识、标题、作者和原始来源。标题/作者差异、版本关联及冲突必须保留证据并明确评估；不能仅靠标题猜测身份，也不能覆盖原有摘要。
+
+采集器不连接数据库。正式导入使用既有 `import records`，必须有新的生产保护基线和独立验收；全部导入结束后仍需全库去重和历史覆盖审计。来自官方录用清单的 accepted 是清单观察，不是原始审稿决定签名或决定日期。

paper_atlas/cli.py

@@ -13,6 +13,7 @@ COMMANDS = {
     ('download',): ('paper_atlas.io.snapshots', {}, '下载并校验固定版本快照'),
     ('collect','pmlr'): ('paper_atlas.collectors.pmlr', {}, 'PMLR 官方元数据'),
     ('collect','acl'): ('paper_atlas.collectors.acl', {}, 'ACL Anthology XML'),
+    ('collect','iclr-archive'): ('paper_atlas.collectors.iclr_archive', {}, '核对ICLR 2014–2016官方历史录用档案'),
     ('collect','iclr-program'): ('paper_atlas.collectors.iclr', {}, 'ICLR 官方公开程序'),
     ('collect','iclr-snapshot'): ('paper_atlas.collectors.openreview', {}, 'ICLR 公开投稿与决定快照'),
     ('collect','public-openreview'): ('paper_atlas.collectors.public_openreview', {}, '核验其他会议的公开拒稿及撤稿标签'),

paper_atlas/collectors/acl.py

@@ -6,6 +6,7 @@ import xml.etree.ElementTree as ET
 from pathlib import Path
 
 from paper_atlas.io.git_metadata import checkout_metadata
+from paper_atlas.collectors.acl_historical import event_scope, material_reason
 
 VENUES={'acl':'ACL','emnlp':'EMNLP','naacl':'NAACL','eacl':'EACL','coling':'COLING','conll':'CoNLL'}
 
@@ -29,7 +30,10 @@ def paper_id(collection,volume,number):
     return f'{collection}-{int(volume):0{width}d}{int(number):0{digits}d}'
 
 
-def volume_scope(meta,volume):
+def volume_scope(meta,volume,collection=None):
+    historical=event_scope(meta,volume,collection)
+    if historical:
+        return historical['canonical_venue'],historical['venue_aliases'],historical['track']
     codes=[text(n).lower() for n in meta.findall('venue')]
     book=text(meta.find('booktitle'))
     lower=book.lower()
@@ -59,7 +63,7 @@ def collect_xml(path,out,commit):
         vid=volume.attrib['id']
         meta=volume.find('meta')
         if meta is None:raise ValueError('Volume metadata missing')
-        scope=volume_scope(meta,vid)
+        scope=volume_scope(meta,vid,collection)
         key=f'{collection}-{vid}'
         if scope is None:
             results.append({'volume':key,'status':'excluded_by_venue','venue_codes':[text(n) for n in meta.findall('venue')],'booktitle':text(meta.find('booktitle')),'year':text(meta.find('year'))})
@@ -70,12 +74,18 @@ def collect_xml(path,out,commit):
         year=int(year_text)
         records=[]
         frontmatter=0
+        excluded_materials=[]
+        historical=event_scope(meta,vid,collection)
         for paper in volume.findall('paper'):
             pid=paper.attrib['id']
             if int(pid)==0:
                 frontmatter+=1
                 continue
             identifier=paper_id(collection,vid,pid)
+            reason=material_reason(collection,vid,paper)
+            if reason:
+                excluded_materials.append({'external_id':identifier,'title':text(paper.find('title')),'reason':reason})
+                continue
             doi=text(paper.find('doi'))
             if doi.startswith('10.18653/v1/') and doi.split('10.18653/v1/',1)[1].casefold()!=identifier.casefold():
                 raise ValueError('Official DOI disagrees with constructed Anthology ID: '+identifier)
@@ -83,6 +93,8 @@ def collect_xml(path,out,commit):
             if not title:raise ValueError('Missing paper title: '+identifier)
             authors=[{'given':text(a.find('first')),'family':text(a.find('last'))} for a in paper.findall('author')]
             records.append({'source':'acl_anthology','external_id':identifier,'url':f'https://aclanthology.org/{identifier}/','title':title,'abstract':text(paper.find('abstract')) or None,'authors':authors,'identifiers':{'acl':identifier,'doi':doi},'venue':venue,'venue_aliases':aliases if len(aliases)>1 else [],'year':year,'track':track,'status':'accepted','raw_decision':'listed in authoritative ACL Anthology proceedings XML','metadata':{'collection':collection,'volume':vid,'booktitle':text(meta.find('booktitle')),'venue_codes':[text(n) for n in meta.findall('venue')],'pages':text(paper.find('pages')),'publisher':text(meta.find('publisher')),'metadata_commit':commit,'source_xml':f'https://github.com/acl-org/acl-anthology/blob/{commit}/data/xml/{collection}.xml'}})
+            if historical:
+                records[-1]['metadata']['historical_event_scope']=historical
         if not records:
             results.append({'volume':key,'status':'unavailable','reason':'no research paper elements after excluding front matter'})
             continue
@@ -93,7 +105,7 @@ def collect_xml(path,out,commit):
             for record in records:output.write(json.dumps(record,ensure_ascii=False)+'\n')
         part.replace(target)
         missing=sum(not row['abstract'] for row in records)
-        manifest={'source':'acl_anthology','venue':venue,'year':year,'track':track,'count':len(records),'xml_paper_elements':len(volume.findall('paper')),'excluded_frontmatter':frontmatter,'missing_abstract_count':missing,'list_url':f'https://aclanthology.org/{key}/','metadata_commit':commit,'file':str(target),'status':'partial' if missing else 'collected'}
+        manifest={'source':'acl_anthology','venue':venue,'year':year,'track':track,'count':len(records),'xml_paper_elements':len(volume.findall('paper')),'excluded_frontmatter':frontmatter,'excluded_materials':excluded_materials,'missing_abstract_count':missing,'list_url':f'https://aclanthology.org/{key}/','metadata_commit':commit,'file':str(target),'status':'partial' if missing else 'collected'}
         target.with_suffix('.manifest.json').write_text(json.dumps(manifest,indent=2))
         results.append(manifest)
         print(json.dumps(manifest),flush=True)

paper_atlas/collectors/acl_historical.py

@@ -0,0 +1,45 @@
+"""Edition-specific historical scope; never alias every TINLAP year to ACL."""
+
+HISTORY_URL = 'https://aclanthology.org/www.mt-archive.info/70/ACL-1982-Walker-1.pdf'
+NOTICE_URL = 'https://aclanthology.org/J77-4001.pdf'
+BOOK = 'Theoretical Issues in Natural Language Processing-2'
+MATERIAL_TITLE = 'List of questions suggested for consideration in each session'
+
+
+def text(node):
+    return ' '.join(''.join(node.itertext()).split()) if node is not None else ''
+
+
+def event_scope(meta, volume, collection):
+    if (collection, volume) != ('T78', '1'):
+        return None
+    if ([text(n) for n in meta.findall('venue')] != ['tinlap']
+            or [text(n) for n in meta.findall('year')] != ['1978']
+            or [text(n) for n in meta.findall('booktitle')] != [BOOK]
+            or [(text(n.find('first')), text(n.find('last'))) for n in meta.findall('editor')]
+               != [('David L.', 'Waltz')]):
+        raise ValueError('Reviewed T78-1 scope metadata changed; independent review required')
+    return {
+        'collection': 'T78', 'volume': '1', 'original_venue': 'TINLAP',
+        'event_year': 1978, 'event_dates': ['1978-07-25', '1978-07-27'],
+        'relationship': 'ACL annual meeting took the form of TINLAP-2',
+        'canonical_venue': 'ACL+TINLAP', 'venue_aliases': ['ACL', 'TINLAP'],
+        'track': 'proceedings_unspecified',
+        'track_scope_note': 'Invited and submitted contributions are described at volume level; individual roles are not identified in XML.',
+        'evidence': [
+            {'url': NOTICE_URL, 'sha256': '14b5deb471c55afb07d789529b95d0462a2a1a77df3dfba7fa1f5d3ab75eb411',
+             'pdf_pages': [9, 10], 'printed_pages': ['9', '10'],
+             'scope': 'Contemporary annual meeting notice, not individual decision notes; ACM confirmation was pending in this notice.'},
+            {'url': HISTORY_URL, 'sha256': '35c21bd4983975bcaccbb94bd8ab91c9019fc190feec1791255407cf18a7e49a',
+             'pdf_pages': [3], 'printed_pages': ['91'],
+             'scope': 'Retrospective annual meeting table and TNLP abbreviation confirm the 1978 event association.'},
+        ],
+    }
+
+
+def material_reason(collection, volume, paper):
+    if (collection, volume, paper.attrib.get('id')) != ('T78', '1', '36'):
+        return None
+    if text(paper.find('title')) != MATERIAL_TITLE or paper.findall('author'):
+        raise ValueError('Reviewed T78-1036 material changed; independent review required')
+    return 'Session question list, not an authored research contribution; no decision inferred.'

paper_atlas/collectors/iclr_archive.py

@@ -0,0 +1,225 @@
+"""Frozen official ICLR 2014–2016 lists; presentation evidence is not a track."""
+import argparse
+from collections import Counter, defaultdict
+import fcntl
+import hashlib
+import json
+from pathlib import Path
+import re
+import tempfile
+from urllib.parse import urlsplit
+
+from paper_atlas.db.submissions import timestamp
+from paper_atlas.io.html import Document, Node, clean
+from paper_atlas.io.http import fetch
+
+SOURCE = 'iclr_archive'
+PAGES = {
+    2014: ('https://iclr.cc/archive/2014/conference-proceedings/', {'main': 35}),
+    2015: ('https://iclr.cc/archive/www/doku.php%3Fid=iclr2015:accepted-main.html', {'main': 31, 'workshop': 74}),
+    2016: ('https://iclr.cc/archive/www/doku.php%3Fid=iclr2016:accepted-main.html', {'main': 80}),
+}
+SECTIONS = {
+    2015: {'Main Conference - Oral Presentations': ('main', 'oral'),
+           'Main Conference - Poster Presentations': ('main', 'poster'),
+           'Workshop Papers': ('workshop', 'workshop')},
+    2016: {'Oral Presentations': ('main', 'oral'), 'Poster Presentations': ('main', 'poster')},
+}
+
+
+def arxiv_link(url):
+    parsed = urlsplit(url)
+    if parsed.hostname != 'arxiv.org':
+        return None
+    match = re.fullmatch(r'/(?:abs|pdf)/(\d{4}\.\d{4,5})(v[1-9]\d*)?(?:\.pdf)?', parsed.path)
+    if (parsed.scheme not in ('http', 'https') or parsed.netloc != 'arxiv.org'
+            or parsed.query or parsed.fragment or not match):
+        raise ValueError('Unexpected official arXiv link')
+    return match[1], match[2] or None
+
+
+def without_links(node):
+    if node.tag in ('a', 'script', 'style'):
+        return ''
+    value = ''.join(without_links(n) if isinstance(n, Node) else n for n in node.children)
+    return ' '+value+' ' if node.tag in ('div', 'li', 'p', 'br') else value
+
+
+def authors(display, year):
+    parts = display.rstrip(';').split(';') if year == 2014 else re.split(r',\s*(?:and\s+)?|\s+and\s+', display)
+    names = [clean(p) for p in parts]
+    if not names or any(not name for name in names):
+        raise ValueError('Missing author or unsupported author delimiter')
+    return names
+
+
+def parse_page(html, year, evidence):
+    if year not in PAGES:
+        raise ValueError('Unreviewed ICLR archive year')
+    document = Document(html)
+    nodes = list(document.root.walk())
+    headings = [clean(n.text()) for n in nodes if n.tag in ('h1', 'h2', 'h3')]
+    if year == 2014:
+        if 'Conference Proceedings' not in headings or 'conference papers accepted' not in clean(document.root.text()):
+            raise ValueError('Unexpected ICLR 2014 accepted list')
+    elif (set(SECTIONS[year]) - set(headings)
+          or (year == 2016 and 'Accepted Papers (Conference Track)' not in headings)):
+        raise ValueError('Unexpected ICLR accepted sections')
+    entries = []
+    section = None
+    for index, node in enumerate(nodes):
+        if node.tag in ('h1', 'h2', 'h3'):
+            section = clean(node.text())
+        if year == 2014:
+            if node.tag != 'a' or not arxiv_link(node.attrs.get('href', '')):
+                continue
+            link = node
+            # This archived Google Sites page has an entry outside a paragraph.
+            # Authors can span several differently formatted italic spans.
+            # Read their whole paragraph, never only the first styled fragment.
+            display = None
+            for following in nodes[index+sum(1 for _ in link.walk()):]:
+                if following.tag == 'a' and arxiv_link(following.attrs.get('href', '')):
+                    break
+                if following.tag in ('i', 'em') or (following.tag == 'p' and any(
+                        n.tag in ('i', 'em') or 'font-style:italic' in n.attrs.get('style', '').replace(' ', '')
+                        for n in following.walk())):
+                    text = clean(following.text())
+                    if text:
+                        display = text
+                        break
+            if not display:
+                raise ValueError('Missing explicit ICLR 2014 author span')
+            track, presentation = 'main', 'conference'
+        else:
+            if node.tag != 'li':
+                continue
+            links = [n for n in node.walk() if n.tag == 'a' and arxiv_link(n.attrs.get('href', ''))]
+            if not links:
+                continue
+            if len(links) != 1 or section not in SECTIONS[year]:
+                raise ValueError('Ambiguous paper link or unknown accepted section')
+            link, = links
+            display = clean(without_links(node)).lstrip(',').strip()
+            track, presentation = SECTIONS[year][section]
+        arxiv, version = arxiv_link(link.attrs['href'])
+        title = clean(link.text())
+        if not title:
+            raise ValueError('Empty official title')
+        entries.append({'arxiv': arxiv, 'version': version, 'title': title,
+                        'authors': authors(display, year), 'authors_display': display,
+                        'paper_url': link.attrs['href'], 'track': track,
+                        'section': section, 'presentation': presentation})
+    # Check every arXiv occurrence, including links outside the expected layout.
+    occurrences = sum(n.tag == 'a' and bool(arxiv_link(n.attrs.get('href', ''))) for n in nodes)
+    if len(entries) != occurrences or not entries:
+        raise ValueError('Some official paper links were not parsed')
+    grouped = defaultdict(list)
+    for entry in entries:
+        grouped[(entry['track'], entry['arxiv'])].append(entry)
+    result = []
+    for (track, arxiv), variants in grouped.items():
+        first = variants[0]
+        if any((v['title'], v['authors']) != (first['title'], first['authors']) for v in variants):
+            raise ValueError('Repeated presentation has conflicting title or authors; review required')
+        presentations = sorted({v['presentation'] for v in variants})
+        result.append({'source': SOURCE, 'external_id': f'{year}/{track}/{arxiv}',
+                       'url': PAGES[year][0], 'title': first['title'], 'abstract': None,
+                       'authors': first['authors'], 'identifiers': {'arxiv': arxiv},
+                       'venue': 'ICLR', 'venue_aliases': [], 'year': year, 'track': track,
+                       'status': 'accepted', 'observed_at': evidence['observed_at'],
+                       'raw_decision': 'listed in official ICLR accepted '+track+' list',
+                       'metadata': {'source_evidence': evidence, 'source_variants': variants,
+                                    'presentations': presentations, 'decision_date_precision': 'not_supplied',
+                                    'identity_note': 'Official arXiv link, not title matching; validate canonical identity before import.',
+                                    'decision_evidence': 'Archived accepted list, not an original decision note.'},
+                       'tags': [{'namespace': SOURCE+':presentation', 'value': value} for value in presentations]})
+    return result
+
+
+def read_page(year, *, cache=None, snapshots=None):
+    if bool(cache) == bool(snapshots):
+        raise ValueError('Choose exactly one cache or frozen snapshot directory')
+    url = PAGES[year][0]
+    if snapshots:
+        root = Path(snapshots).resolve()
+        receipts = json.loads((root/'receipts.json').read_bytes())['receipts']
+        matches = [r for r in receipts if r['url'] == url]
+        if len(matches) != 1:
+            raise ValueError('Expected one exact official response receipt')
+        receipt, = matches
+        filename = Path(receipt['file'])
+        path = root/filename
+        if (filename.is_absolute() or '..' in filename.parts or not path.resolve().is_relative_to(root)
+                or receipt['status'] != 200):
+            raise ValueError('Unsafe or unsuccessful archive snapshot')
+        data = path.read_bytes()
+        if (len(data) > 8*1024**2 or len(data) != receipt['bytes']
+                or hashlib.sha256(data).hexdigest() != receipt['sha256']):
+            raise ValueError('Archive snapshot checksum or size differs')
+        observed = timestamp(receipt['finished_at'])
+    else:
+        data = fetch(url, cache, 8*1024**2, delay=1)
+        receipt = json.loads((Path(cache)/hashlib.sha256(url.encode()).hexdigest()).with_suffix('.json').read_bytes())
+        observed = timestamp(receipt['retrieved_at'])
+    evidence = {'url': url, 'sha256': hashlib.sha256(data).hexdigest(), 'bytes': len(data),
+                'file': 'evidence/'+hashlib.sha256(data).hexdigest()+'.html',
+                'observed_at': observed, 'http_status': 200, 'response_receipt': receipt}
+    return data, evidence
+
+
+def collect(year, out, *, cache=None, snapshots=None):
+    if year not in PAGES:
+        raise ValueError('Only independently reviewed ICLR archive years are supported')
+    out = Path(out)
+    out.mkdir(parents=True, exist_ok=True)
+    with (out/'iclr-archive.collect.lock').open('a') as lock:
+        fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        report = out/f'iclr-archive-{year}.json'
+        if report.exists():
+            raise FileExistsError('Existing completed archive output must be inspected, not overwritten')
+        data, evidence = read_page(year, cache=cache, snapshots=snapshots)
+        records = parse_page(data.decode('utf-8'), year, evidence)
+        counts = Counter(r['track'] for r in records)
+        if dict(counts) != PAGES[year][1]:
+            raise ValueError('Distinct archive entries differ from reviewed inventory')
+        files = []
+        with tempfile.TemporaryDirectory(prefix='.iclr-archive-', dir=out) as temporary:
+            stage = Path(temporary)
+            (stage/'evidence').mkdir()
+            (stage/evidence['file']).write_bytes(data)
+            for track in sorted(counts):
+                group = [r for r in records if r['track'] == track]
+                path = stage/f'iclr-archive-{year}-{track}.jsonl'
+                path.write_text(''.join(json.dumps(r, ensure_ascii=False)+'\n' for r in group))
+                manifest = {'source': SOURCE, 'parser_version': 1, 'venue': 'ICLR', 'year': year,
+                            'track': track, 'count': len(group), 'file': str(out/path.name),
+                            'file_sha256': hashlib.sha256(path.read_bytes()).hexdigest(),
+                            'list_url': PAGES[year][0], 'status': 'partial', 'missing_abstract_count': len(group)}
+                path.with_suffix('.manifest.json').write_text(json.dumps(manifest, ensure_ascii=False, indent=2)+'\n')
+                files.append(manifest)
+            result = {'source': SOURCE, 'year': year, 'records': len(records), 'files': files,
+                      'source_evidence': evidence, 'database_written': False, 'pdfs_downloaded': False,
+                      'status': 'collected_pending_identity_review',
+                      'note': 'Accepted list only. Oral/poster are presentation evidence; absent papers are not rejected. '
+                              'ICLR2014 workshop and other unlisted tracks remain outside these reviewed pages.'}
+            (stage/report.name).write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+            (out/'evidence').mkdir(exist_ok=True)
+            for path in (stage/'evidence').iterdir():
+                path.replace(out/'evidence'/path.name)
+            for path in stage.iterdir():
+                if path.is_file() and path.name != report.name:
+                    path.replace(out/path.name)
+            (stage/report.name).replace(report)
+        return result
+
+
+def main(argv=None, *, prog=None):
+    parser = argparse.ArgumentParser(prog=prog)
+    parser.add_argument('--year', type=int, choices=sorted(PAGES), required=True)
+    parser.add_argument('--out', required=True)
+    source = parser.add_mutually_exclusive_group(required=True)
+    source.add_argument('--cache')
+    source.add_argument('--snapshots')
+    args = parser.parse_args(argv)
+    print(json.dumps(collect(args.year, args.out, cache=args.cache, snapshots=args.snapshots), ensure_ascii=False))

tests/fixtures/acl-t78.xml

@@ -0,0 +1,273 @@
+<?xml version='1.0' encoding='UTF-8'?>
+<collection id="T78">
+  <volume id="1" type="proceedings">
+    <!-- https://aclanthology.org/T78-1/ -->
+    <meta>
+      <booktitle>Theoretical Issues in Natural Language <fixed-case>P</fixed-case>rocessing-2</booktitle>
+      <editor id="david-l-waltz"><first>David L.</first><last>Waltz</last></editor>
+      <venue>tinlap</venue>
+      <year>1978</year>
+    </meta>
+    <frontmatter>
+      <!-- https://aclanthology.org/T78-1000/ -->
+      <pdf hash="fb6164ad"/>
+      <bibkey>tinlap-1978-theoretical</bibkey>
+    </frontmatter>
+    <paper id="1">
+      <!-- https://aclanthology.org/T78-1001/ -->
+      <title>Testing The Psychological Reality of a Representational Model</title>
+      <author><first>Dedre</first><last>Gentner</last></author>
+      <pdf hash="f723a956"/>
+      <bibkey>gentner-1978-testing-psychological</bibkey>
+    </paper>
+    <paper id="2">
+      <!-- https://aclanthology.org/T78-1002/ -->
+      <title>What Makes Something “Ad Hoc”</title>
+      <author id="roger-c-schank"><first>Roger C.</first><last>Schank</last></author>
+      <pdf hash="636853f9"/>
+      <bibkey>schank-1978-makes-something</bibkey>
+    </paper>
+    <paper id="3">
+      <!-- https://aclanthology.org/T78-1003/ -->
+      <title>The Relation of Grammar to Cognition–a Synopsis</title>
+      <author><first>Leonard</first><last>Talmy</last></author>
+      <pdf hash="101c67dc"/>
+      <bibkey>talmy-1978-relation-grammar</bibkey>
+    </paper>
+    <paper id="4">
+      <!-- https://aclanthology.org/T78-1004/ -->
+      <title>On primitives, prototypes, and other semantic anomalies</title>
+      <author><first>Terry</first><last>Winograd</last></author>
+      <pdf hash="f8e47023"/>
+      <bibkey>winograd-1978-primitives-prototypes</bibkey>
+    </paper>
+    <paper id="5">
+      <!-- https://aclanthology.org/T78-1005/ -->
+      <title>Taxonomic Lattice Structures for Situation Recognition</title>
+      <author id="william-a-woods"><first>William A.</first><last>Woods</last></author>
+      <pdf hash="92957ddf"/>
+      <bibkey>woods-1978-taxonomic-lattice</bibkey>
+    </paper>
+    <paper id="6">
+      <!-- https://aclanthology.org/T78-1006/ -->
+      <title>Description Formation and Discourse Model Synthesis</title>
+      <author id="bonnie-webber"><first>Bonnie Lynn</first><last>Webber</last></author>
+      <pdf hash="ffded1d2"/>
+      <bibkey>webber-1978-description-formation</bibkey>
+    </paper>
+    <paper id="7">
+      <!-- https://aclanthology.org/T78-1007/ -->
+      <title>The Processing of Referring Expressions within a Semantic Network</title>
+      <author><first>John R.</first><last>Anderson</last></author>
+      <pdf hash="8bcc083a"/>
+      <bibkey>anderson-1978-processing-referring</bibkey>
+    </paper>
+    <paper id="8">
+      <!-- https://aclanthology.org/T78-1008/ -->
+      <title>Reference Diaries</title>
+      <author><first>Herbert H.</first><last>Clark</last></author>
+      <author><first>Catherine</first><last>Marshall</last></author>
+      <pdf hash="d06b7ea2"/>
+      <bibkey>clark-marshall-1978-reference-diaries</bibkey>
+    </paper>
+    <paper id="9">
+      <!-- https://aclanthology.org/T78-1009/ -->
+      <title>Subsequent Reference: Syntactic and Rhetorical Constraints</title>
+      <author id="david-d-mcdonald"><first>David D.</first><last>McDonald</last></author>
+      <pdf hash="2d424559"/>
+      <bibkey>mcdonald-1978-subsequent-reference</bibkey>
+    </paper>
+    <paper id="10">
+      <!-- https://aclanthology.org/T78-1010/ -->
+      <title>Some Psycholinguistic Constraints on the Construction and Interpretation of Definite Descriptions</title>
+      <author><first>Andrew</first><last>Ortony</last></author>
+      <pdf hash="69807727"/>
+      <bibkey>ortony-1978-psycholinguistic-constraints</bibkey>
+    </paper>
+    <paper id="11">
+      <!-- https://aclanthology.org/T78-1011/ -->
+      <title>Bound Variables and Other Anaphors</title>
+      <author id="barbara-h-partee"><first>Barbara H.</first><last>Partee</last></author>
+      <pdf hash="e8e6a510"/>
+      <bibkey>partee-1978-bound-variables</bibkey>
+    </paper>
+    <paper id="12">
+      <!-- https://aclanthology.org/T78-1012/ -->
+      <title>The Use of Focus as a Tool for Disambiguation of Definite Noun Phrases</title>
+      <author id="candace-l-sidner"><first>Candace L.</first><last>Sidner</last></author>
+      <pdf hash="7b96d2b7"/>
+      <bibkey>sidner-1978-use-focus</bibkey>
+    </paper>
+    <paper id="13">
+      <!-- https://aclanthology.org/T78-1013/ -->
+      <title>Focusing in Dialog</title>
+      <author id="barbara-j-grosz"><first>Barbara J.</first><last>Grosz</last></author>
+      <pdf hash="8c8d4c2e"/>
+      <bibkey>grosz-1978-focusing-dialog</bibkey>
+    </paper>
+    <paper id="14">
+      <!-- https://aclanthology.org/T78-1014/ -->
+      <title>Topic Levels</title>
+      <author><first>Joseph E.</first><last>Grimes</last></author>
+      <pdf hash="f485b2ba"/>
+      <bibkey>grimes-1978-topic-levels</bibkey>
+    </paper>
+    <paper id="15">
+      <!-- https://aclanthology.org/T78-1015/ -->
+      <title>Toward a Rational Model of Discourse Comprehension</title>
+      <author><first>Jerry L.</first><last>Morgan</last></author>
+      <pdf hash="5e12a597"/>
+      <bibkey>morgan-1978-toward-rational</bibkey>
+    </paper>
+    <paper id="16">
+      <!-- https://aclanthology.org/T78-1016/ -->
+      <title>Assent and Compliance in Children’s Language Comprehension</title>
+      <author><first>David R.</first><last>Olson</last></author>
+      <pdf hash="708bc4a0"/>
+      <bibkey>olson-1978-assent-compliance</bibkey>
+    </paper>
+    <paper id="17">
+      <!-- https://aclanthology.org/T78-1017/ -->
+      <title>Speech Acts as a Basis for Understanding Dialogue Coherence</title>
+      <author id="c-raymond-perrault"><first>C. Raymond</first><last>Perrault</last></author>
+      <author id="james-allen"><first>James F.</first><last>Allen</last></author>
+      <pdf hash="d111402b"/>
+      <bibkey>perrault-allen-1978-speech</bibkey>
+    </paper>
+    <paper id="18">
+      <!-- https://aclanthology.org/T78-1018/ -->
+      <title>A Framework for Comparing Language Experiences (with particular emphasis on: The Effect of Audience on Discourse Models)</title>
+      <author><first>Andee</first><last>Rubin</last></author>
+      <pdf hash="5d9a6150"/>
+      <bibkey>rubin-1978-framework-comparing</bibkey>
+    </paper>
+    <paper id="19">
+      <!-- https://aclanthology.org/T78-1019/ -->
+      <title>Intentlonallty and Human Conversations</title>
+      <author id="jaime-g-carbonell"><first>Jaime G.</first><last>Carbonell Jr</last></author>
+      <pdf hash="c50fc619"/>
+      <bibkey>carbonell-jr-1978-intentlonallty-human</bibkey>
+    </paper>
+    <paper id="20">
+      <!-- https://aclanthology.org/T78-1020/ -->
+      <title>On the Interdependence of Language and Perception</title>
+      <author id="david-l-waltz"><first>David L.</first><last>Waltz</last></author>
+      <pdf hash="3c708191"/>
+      <bibkey>waltz-1978-interdependence-language</bibkey>
+    </paper>
+    <paper id="21">
+      <!-- https://aclanthology.org/T78-1021/ -->
+      <title>The Problem of Naming Shapes: Vision-Language Interface</title>
+      <author id="ruzena-bajcsy"><first>R.</first><last>Bajcsy</last></author>
+      <author id="aravind-joshi"><first>A. K.</first><last>Joshi</last></author>
+      <pdf hash="50331b4e"/>
+      <bibkey>bajcsy-joshi-1978-problem-naming</bibkey>
+    </paper>
+    <paper id="22">
+      <!-- https://aclanthology.org/T78-1022/ -->
+      <title>An Argument about the Composition of Conceptual Structure</title>
+      <author><first>Ray</first><last>Jackendoff</last></author>
+      <pdf hash="57d001f8"/>
+      <bibkey>jackendoff-1978-argument-composition</bibkey>
+    </paper>
+    <paper id="23">
+      <!-- https://aclanthology.org/T78-1023/ -->
+      <title>On the Ontological Status of Visual Mental Images</title>
+      <author><first>Stephen Michael</first><last>Kosslyn</last></author>
+      <pdf hash="fd06e265"/>
+      <bibkey>kosslyn-1978-ontological-status</bibkey>
+    </paper>
+    <paper id="24">
+      <!-- https://aclanthology.org/T78-1024/ -->
+      <title>What has language to do with perception? Some speculations on the <i>Lingua Mentis</i></title>
+      <author><first>Zenon W.</first><last>Pylyshyn</last></author>
+      <pdf hash="a2f3a044"/>
+      <bibkey>pylyshyn-1978-language-perception</bibkey>
+    </paper>
+    <paper id="25">
+      <!-- https://aclanthology.org/T78-1025/ -->
+      <title>Semantic Primitives in Language and Vision</title>
+      <author id="yorick-wilks"><first>Yorick</first><last>Wilks</last></author>
+      <pdf hash="b4e13896"/>
+      <bibkey>wilks-1978-semantic-primitives</bibkey>
+    </paper>
+    <paper id="26">
+      <!-- https://aclanthology.org/T78-1026/ -->
+      <title>A Note on Partial Match of Descriptions. Can One Simultaneously Question (Retrieve) and Inform (Update)?</title>
+      <author id="aravind-joshi"><first>Aravind K.</first><last>Joshi</last></author>
+      <pdf hash="74969243"/>
+      <bibkey>joshi-1978-note-partial</bibkey>
+    </paper>
+    <paper id="27">
+      <!-- https://aclanthology.org/T78-1027/ -->
+      <title>With a Spoon in Hand This Must Be the Eating Frame</title>
+      <author><first>Eugene</first><last>Charniak</last></author>
+      <pdf hash="f1be68bb"/>
+      <bibkey>charniak-1978-spoon-hand</bibkey>
+    </paper>
+    <paper id="28">
+      <!-- https://aclanthology.org/T78-1028/ -->
+      <title>Fragments of a Theory of Human Plausible Reasoning</title>
+      <author><first>Allan</first><last>Collins</last></author>
+      <pdf hash="67b4bca2"/>
+      <bibkey>collins-1978-fragments-theory</bibkey>
+    </paper>
+    <paper id="29">
+      <!-- https://aclanthology.org/T78-1029/ -->
+      <title>Indirect Responses to Loaded Questions</title>
+      <author><first>S. Jerrold</first><last>Kaplan</last></author>
+      <pdf hash="135037f9"/>
+      <bibkey>kaplan-1978-indirect-responses</bibkey>
+    </paper>
+    <paper id="30">
+      <!-- https://aclanthology.org/T78-1030/ -->
+      <title>On Reasoning by Default</title>
+      <author><first>Raymond</first><last>Reiter</last></author>
+      <pdf hash="0de53d33"/>
+      <bibkey>reiter-1978-reasoning-default</bibkey>
+    </paper>
+    <paper id="31">
+      <!-- https://aclanthology.org/T78-1031/ -->
+      <title>Path-Based and Node-Based Inference in Semantic Networks</title>
+      <author id="stuart-c-shapiro"><first>Stuart C.</first><last>Shapiro</last></author>
+      <pdf hash="2ddc5535"/>
+      <bibkey>shapiro-1978-path-based</bibkey>
+    </paper>
+    <paper id="32">
+      <!-- https://aclanthology.org/T78-1032/ -->
+      <title>The Representation of Derivable Information in Memory: When What Might Have Been Left Unsaid Is Said</title>
+      <author><first>Rand J.</first><last>Spiro</last></author>
+      <author><first>Joseph</first><last>Esposito</last></author>
+      <author><first>Richard J.</first><last>Vondruska</last></author>
+      <pdf hash="7758fb45"/>
+      <bibkey>spiro-etal-1978-representation-derivable</bibkey>
+    </paper>
+    <paper id="33">
+      <!-- https://aclanthology.org/T78-1033/ -->
+      <title>A Heuristic for Paradigms</title>
+      <author><first>Joseph E.</first><last>Grimes</last></author>
+      <pdf hash="53b42c16"/>
+      <bibkey>grimes-1978-heuristic-paradigms</bibkey>
+    </paper>
+    <paper id="34">
+      <!-- https://aclanthology.org/T78-1034/ -->
+      <title>A Computational Account of Some Constraints on Language</title>
+      <author id="mitch-marcus"><first>Mitchell</first><last>Marcus</last></author>
+      <pdf hash="c9724c43"/>
+      <bibkey>marcus-1978-computational-account</bibkey>
+    </paper>
+    <paper id="35">
+      <!-- https://aclanthology.org/T78-1035/ -->
+      <title>Remarks on Processing, Constraints, and the Lexicon</title>
+      <author id="thomas-wasow"><first>Thomas</first><last>Wasow</last></author>
+      <pdf hash="8beed275"/>
+      <bibkey>wasow-1978-remarks-processing</bibkey>
+    </paper>
+    <paper id="36">
+      <!-- https://aclanthology.org/T78-1036/ -->
+      <title>List of questions suggested for consideration in each session</title>
+      <pdf hash="1950b908"/>
+      <bibkey>nn-1978-list-questions</bibkey>
+    </paper>
+  </volume>
+</collection>

tests/test_acl_historical.py

@@ -0,0 +1,114 @@
+"""Pinned T78 evidence, edition-local aliases and an explicitly identified material."""
+import copy
+import hashlib
+import json
+from pathlib import Path
+import sqlite3
+import tempfile
+import unittest
+import xml.etree.ElementTree as ET
+
+from paper_atlas.collectors.acl import collect_xml, volume_scope
+from paper_atlas.collectors.acl_historical import event_scope, material_reason
+from paper_atlas.db.store import Store
+from paper_atlas.ingestion.records import ingest_file
+
+FIXTURE = Path(__file__).parent/'fixtures/acl-t78.xml'
+COMMIT = 'a7be623d01c48c61405dec31c871121cdbed5a01'
+
+
+class AclHistoricalTests(unittest.TestCase):
+    def setUp(self):
+        self.root = ET.parse(FIXTURE).getroot()
+        self.volume = self.root.find('volume')
+        self.meta = self.volume.find('meta')
+
+    def test_frozen_volume_matches_official_snapshot(self):
+        self.assertEqual(hashlib.sha256(FIXTURE.read_bytes()).hexdigest(), 'f712f94cff0cba05f786b7ac31dad2ea9408fde2a17a04531130df623f57dd84')
+        self.assertEqual(len(self.volume.findall('paper')), 36)
+        self.assertEqual(volume_scope(self.meta, '1', 'T78'), ('ACL+TINLAP', ['ACL', 'TINLAP'], 'proceedings_unspecified'))
+
+    def test_unknown_year_collection_or_volume_is_not_globally_aliased(self):
+        for collection, volume in [('T75', '1'), ('T87', '1'), ('T78', '2'), ('C65', '1')]:
+            with self.subTest(collection=collection, volume=volume):
+                self.assertIsNone(event_scope(self.meta, volume, collection))
+                self.assertIsNone(volume_scope(self.meta, volume, collection))
+        self.assertIsNone(volume_scope(self.meta, '1'))
+
+    def test_known_volume_scope_rejects_critical_metadata_drift(self):
+        for field, replacement in [('venue', 'acl'), ('year', '1979'), ('booktitle', 'Another TINLAP'), ('editor/first', 'Another')]:
+            changed = copy.deepcopy(self.meta)
+            changed.find(field).text = replacement
+            with self.subTest(field=field), self.assertRaises(ValueError):
+                event_scope(changed, '1', 'T78')
+        changed = copy.deepcopy(self.meta)
+        ET.SubElement(changed, 'year').text = '1978'
+        with self.assertRaises(ValueError):
+            event_scope(changed, '1', 'T78')
+
+    def test_evidence_retains_original_series_dates_and_role_boundary(self):
+        scope = event_scope(self.meta, '1', 'T78')
+        self.assertEqual(scope['original_venue'], 'TINLAP')
+        self.assertEqual(scope['event_dates'], ['1978-07-25', '1978-07-27'])
+        self.assertEqual([e['pdf_pages'] for e in scope['evidence']], [[9, 10], [3]])
+        self.assertIn('not identified', scope['track_scope_note'])
+        self.assertIn('pending', scope['evidence'][0]['scope'])
+
+    def test_only_exact_session_questions_material_is_excluded(self):
+        paper = copy.deepcopy(self.volume.findall('paper')[-1])
+        self.assertIn('Session question list', material_reason('T78', '1', paper))
+        self.assertIsNone(material_reason('C65', '1', paper))
+        self.assertIsNone(material_reason('T78', '2', paper))
+        paper.set('id', '37')
+        self.assertIsNone(material_reason('T78', '1', paper))
+        paper.set('id', '36')
+        ET.SubElement(paper, 'author')
+        with self.assertRaises(ValueError):
+            material_reason('T78', '1', paper)
+
+    def test_all_authored_source_fields_and_material_receipt_survive_collection(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            manifest, = collect_xml(FIXTURE, temporary, COMMIT)
+            rows = [json.loads(raw) for raw in Path(manifest['file']).read_text().splitlines()]
+        self.assertEqual(manifest['count'], 35)
+        self.assertEqual(manifest['xml_paper_elements'], 36)
+        self.assertEqual(manifest['missing_abstract_count'], 35)
+        self.assertEqual(manifest['excluded_materials'][0]['external_id'], 'T78-1036')
+        for number, row in enumerate(rows, 1):
+            paper = self.volume.findall('paper')[number-1]
+            self.assertEqual(row['external_id'], f'T78-1{number:03d}')
+            self.assertEqual(row['title'], ' '.join(''.join(paper.find('title').itertext()).split()))
+            self.assertEqual(row['metadata']['venue_codes'], ['tinlap'])
+            self.assertEqual(row['metadata']['historical_event_scope']['original_venue'], 'TINLAP')
+            self.assertEqual(row['metadata']['metadata_commit'], COMMIT)
+            self.assertEqual((row['status'], row['track'], row['abstract']), ('accepted', 'proceedings_unspecified', None))
+
+    def test_old_coling_scope_is_unchanged_by_calendar_replacement_evidence(self):
+        meta = ET.fromstring('<meta><venue>coling</venue><year>1965</year><booktitle>COLING 1965</booktitle></meta>')
+        self.assertEqual(volume_scope(meta, '1', 'C65'), ('COLING', ['COLING'], 'main'))
+        self.assertIsNone(event_scope(meta, '1', 'C65'))
+
+    def test_native_ingestion_does_not_alias_other_tinlap_years_or_infer_negative_status(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            root = Path(temporary)
+            manifest, = collect_xml(FIXTURE, root/'source', COMMIT)
+            db = root/'sample.sqlite'
+            store = Store(db)
+            pid, rid = store.record('test_control', 'tinlap1975', 'https://example.org/tinlap1975',
+                                    'An earlier TINLAP contribution', None, ['Jane Doe'])
+            store.decision(pid, rid, 'TINLAP', 1975, 'main', 'tinlap1975', 'https://example.org/tinlap1975', 'accepted', 'test control')
+            store.conn.commit()
+            store.conn.close()
+            ingest_file(db, manifest['file'])
+            ingest_file(db, manifest['file'])
+            with sqlite3.connect(db) as c:
+                self.assertEqual(c.execute('SELECT count(*) FROM papers').fetchone()[0], 36)
+                self.assertEqual(c.execute('SELECT count(*) FROM submissions').fetchone()[0], 36)
+                self.assertEqual(c.execute('SELECT count(*) FROM decisions').fetchone()[0], 36)
+                self.assertEqual(c.execute("SELECT count(*) FROM decisions WHERE status!='accepted'").fetchone()[0], 0)
+                self.assertIsNone(c.execute("SELECT 1 FROM source_records WHERE external_id='T78-1036'").fetchone())
+                aliases = c.execute('SELECT v.acronym,a.alias FROM venue_aliases a JOIN venues v ON v.id=a.venue_id ORDER BY a.alias').fetchall()
+                self.assertEqual(aliases, [('ACL+TINLAP', 'ACL'), ('ACL+TINLAP', 'TINLAP')])
+                metadata = json.loads(c.execute("SELECT metadata_json FROM source_records WHERE external_id='T78-1001'").fetchone()[0])
+                self.assertEqual(metadata['historical_event_scope']['event_year'], 1978)
+                self.assertEqual(c.execute('PRAGMA foreign_key_check').fetchall(), [])

tests/test_iclr_archive.py

@@ -0,0 +1,148 @@
+"""Historical accepted lists, repeated presentations and preserved work histories."""
+import hashlib
+import json
+from pathlib import Path
+import sqlite3
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from paper_atlas.collectors import iclr_archive as archive
+from paper_atlas.db.store import Store
+from paper_atlas.ingestion.records import ingest_file
+
+OBSERVED = '2026-11-09T00:00:00Z'
+
+
+def entry(identifier, title='A shared research paper', display='Jane Doe and Bob Smith'):
+    return f'<li><div><a href="http://arxiv.org/abs/{identifier}">{title}</a>, {display}</div></li>'
+
+
+def page2015():
+    return ('<h3>Main Conference - Oral Presentations</h3><ol>'+entry('1412.0001v2')+'</ol>'
+            '<h3>Main Conference - Poster Presentations</h3><ol>'+entry('1412.0001v2')+'</ol>'
+            '<h3>Workshop Papers</h3><ol>'+entry('1412.0002', 'A workshop paper')+'</ol>')
+
+
+class IclrArchiveTests(unittest.TestCase):
+    def frozen(self, root, year, html):
+        root.mkdir()
+        data = html.encode()
+        (root/'page.html').write_bytes(data)
+        receipt = {'url': archive.PAGES[year][0], 'file': 'page.html', 'status': 200,
+                   'bytes': len(data), 'sha256': hashlib.sha256(data).hexdigest(), 'finished_at': OBSERVED}
+        (root/'receipts.json').write_text(json.dumps({'receipts': [receipt]}))
+        return root
+
+    def test_oral_poster_repeat_is_one_attempt_with_all_source_variants(self):
+        rows = archive.parse_page(page2015(), 2015, {'observed_at': OBSERVED})
+        main, workshop = rows
+        self.assertEqual((main['track'], workshop['track']), ('main', 'workshop'))
+        self.assertEqual(main['identifiers'], {'arxiv': '1412.0001'})
+        self.assertEqual(main['external_id'], '2015/main/1412.0001')
+        self.assertEqual(main['metadata']['presentations'], ['oral', 'poster'])
+        self.assertEqual(len(main['metadata']['source_variants']), 2)
+        self.assertEqual(main['metadata']['source_variants'][0]['version'], 'v2')
+        self.assertEqual(main['authors'], ['Jane Doe', 'Bob Smith'])
+        self.assertTrue(all(r['status'] == 'accepted' and r['abstract'] is None for r in rows))
+
+    def test_2014_author_span_outside_paragraph_and_trailing_separator(self):
+        html = ('<h3>Conference Proceedings</h3><p>conference papers accepted</p>'
+                '<div><b><a href="http://arxiv.org/abs/1312.0001">A <i>research</i> title</a></b></div>'
+                '<p><span style="font-style: italic">Jane Doe; </span>'
+                '<span style="font-style:italic">Bob Smith; Carol Wu;</span></p>'
+                '<div><a href="http://arxiv.org/abs/1312.0002">Another title</a>'
+                '<div><i><span>First Author; </span><span>Second Author</span></i></div></div>')
+        row, unwrapped = archive.parse_page(html, 2014, {'observed_at': OBSERVED})
+        self.assertEqual(row['title'], 'A research title')
+        self.assertEqual(row['authors'], ['Jane Doe', 'Bob Smith', 'Carol Wu'])
+        self.assertEqual(row['metadata']['source_variants'][0]['authors_display'], 'Jane Doe; Bob Smith; Carol Wu;')
+        self.assertEqual(row['track'], 'main')
+        self.assertEqual(unwrapped['authors'], ['First Author', 'Second Author'])
+
+    def test_same_arxiv_in_different_tracks_keeps_separate_source_attempts(self):
+        html = page2015().replace('1412.0002', '1412.0001v3')
+        rows = archive.parse_page(html, 2015, {'observed_at': OBSERVED})
+        self.assertEqual(len(rows), 2)
+        self.assertEqual({r['external_id'] for r in rows}, {'2015/main/1412.0001', '2015/workshop/1412.0001'})
+        self.assertEqual(rows[1]['metadata']['source_variants'][0]['version'], 'v3')
+
+    def test_2016_code_links_and_breaks_do_not_become_authors(self):
+        html = ('<h1>Accepted Papers (Conference Track)</h1><h3>Oral Presentations</h3><ol>'
+                '<li><a href="http://arxiv.org/abs/1511.00001">A title</a>'
+                '<a href="https://example.org/code">[code]</a><br>Jane Doe, Bob Smith</li></ol>'
+                '<h3>Poster Presentations</h3>')
+        row, = archive.parse_page(html, 2016, {'observed_at': OBSERVED})
+        self.assertEqual(row['authors'], ['Jane Doe', 'Bob Smith'])
+        self.assertEqual(row['metadata']['presentations'], ['oral'])
+
+    def test_conflicting_repeat_unknown_sections_and_unparsed_links_fail(self):
+        variants = [page2015().replace('Poster Presentations', 'Other Presentations'),
+                    page2015().replace('</ol>', '</ol><a href="http://arxiv.org/abs/1412.0003">stray</a>', 1),
+                    page2015().replace('Jane Doe and Bob Smith', 'Different Author', 1)]
+        for html in variants:
+            with self.subTest(html=html), self.assertRaises(ValueError):
+                archive.parse_page(html, 2015, {'observed_at': OBSERVED})
+
+    def test_arxiv_versions_are_preserved_and_unsafe_links_rejected(self):
+        self.assertEqual(archive.arxiv_link('https://arxiv.org/pdf/1511.00001v3.pdf'), ('1511.00001', 'v3'))
+        self.assertIsNone(archive.arxiv_link('https://arxiv.org.example.org/abs/1511.00001'))
+        for url in ['https://arxiv.org/abs/1511.00001?other=1', 'ftp://arxiv.org/abs/1511.00001',
+                    'https://user@arxiv.org/abs/1511.00001', 'https://arxiv.org/abs/not-a-paper']:
+            with self.subTest(url=url), self.assertRaises(ValueError):
+                archive.arxiv_link(url)
+
+    def test_receipt_hash_status_path_and_symlink_cannot_escape_snapshot(self):
+        for failure in ['hash', 'status', 'path', 'symlink']:
+            with self.subTest(failure=failure), tempfile.TemporaryDirectory() as temporary:
+                root = Path(temporary)
+                source = self.frozen(root/'source', 2015, page2015())
+                file = source/'receipts.json'
+                data = json.loads(file.read_text())
+                receipt = data['receipts'][0]
+                if failure == 'hash': receipt['sha256'] = '0'*64
+                elif failure == 'status': receipt['status'] = 404
+                elif failure == 'path': receipt['file'] = '../outside.html'
+                else:
+                    (root/'outside.html').write_text(page2015())
+                    (source/'page.html').unlink()
+                    (source/'page.html').symlink_to(root/'outside.html')
+                file.write_text(json.dumps(data))
+                with self.assertRaises(ValueError):
+                    archive.read_page(2015, snapshots=source)
+
+    def test_changed_inventory_leaves_no_completed_output(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            root = Path(temporary)
+            source = self.frozen(root/'source', 2015, page2015())
+            with self.assertRaises(ValueError):
+                archive.collect(2015, root/'out', snapshots=source)
+            self.assertFalse((root/'out/iclr-archive-2015.json').exists())
+
+    def test_native_import_keeps_rejected_to_accepted_history_and_abstract(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            root = Path(temporary)
+            source = self.frozen(root/'source', 2015, page2015())
+            with patch.dict(archive.PAGES, {2015: (archive.PAGES[2015][0], {'main': 1, 'workshop': 1})}):
+                result = archive.collect(2015, root/'out', snapshots=source)
+                with self.assertRaises(FileExistsError):
+                    archive.collect(2015, root/'out', snapshots=source)
+            db = root/'sample.sqlite'
+            store = Store(db)
+            pid, rid = store.record('arxiv', '1412.0001', 'https://arxiv.org/abs/1412.0001',
+                                    'A shared research paper', 'Keep this canonical abstract.',
+                                    ['Jane Doe', 'Bob Smith'], {'arxiv': '1412.0001'}, {})
+            store.decision(pid, rid, 'Earlier Conference', 2014, 'main', 'original-attempt',
+                           'https://example.org/rejected', 'rejected', 'explicit earlier decision', observed_at=OBSERVED)
+            store.conn.commit()
+            store.conn.close()
+            for manifest in result['files']:
+                ingest_file(db, manifest['file'])
+                ingest_file(db, manifest['file'])
+            with sqlite3.connect(db) as c:
+                self.assertEqual(c.execute('SELECT count(*) FROM papers').fetchone()[0], 2)
+                self.assertEqual(c.execute('SELECT abstract FROM papers WHERE id=?', (pid,)).fetchone()[0], 'Keep this canonical abstract.')
+                history = c.execute('SELECT v.acronym,e.year,d.status FROM decisions d JOIN submissions s ON s.id=d.submission_id JOIN editions e ON e.id=s.edition_id JOIN venues v ON v.id=e.venue_id WHERE s.paper_id=? ORDER BY e.year', (pid,)).fetchall()
+                self.assertEqual(history, [('Earlier Conference', 2014, 'rejected'), ('ICLR', 2015, 'accepted')])
+                self.assertEqual(c.execute('SELECT count(*) FROM submissions').fetchone()[0], 3)
+                self.assertEqual(c.execute('PRAGMA foreign_key_check').fetchall(), [])