Add reviewed AAMAS 2025 publisher metadata adapter

Reviewed-on: #41

.gitea/workflows/check.yml

@@ -50,6 +50,8 @@ jobs:
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','cvf','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','www-archive','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--help'],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--year','2025','--snapshots',str(src/'tests/fixtures/aamas2025'),'--out',str(root/'installed-aamas2025')],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','iclr-archive','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','postgres','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','edition-tracks','--help'],cwd=outside,check=True)
               subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','joint-editions','--help'],cwd=outside,check=True)

README.md

@@ -6,6 +6,8 @@ Paper Atlas 将 arXiv、AI 会议论文集及公开投稿决定整理到可追
 
 项目仍在建设中。候选范围包含 43 个会议（含 ACM MM、MLSys），尚未完成全部历史数据和最终去重复核。实际进度以数据库、来源清单和覆盖报告为准，而不是仓库文件数量。
 
+历史 ICLR 2014–2016 官方录用页由 `collect iclr-archive` 适配，保留主会/workshop及重复展示证据；运行和身份核验边界见 [历史档案说明](docs/coverage/iclr-archive-2014-2016.md)。
+
 ## 项目结构
 
 ```text
@@ -64,7 +66,7 @@ paper-atlas collect colm --cache /data/paper-index/raw/http-cache \
 
 EWSL、EPS、ECP、EWSP 的[具体前身卷与完整快照要求](docs/coverage/historical-predecessors-2026-11-09.md)单独核验，保留原始系列及事件/出版年份；旧候选文件缺少 ECP/EWSP 时会明确提示，避免静默遗漏。
 
-AAMAS 的[官方论文集采集器](docs/aamas-proceedings.md)目前核验了 2012 版式，可从缓存或已校验快照采集标题、作者及页面提供的摘要；发表形式单独保存，全文/扩展摘要中未明确的投稿 track 保持待明确，不根据会话主题推定主会轨道。
+AAMAS 的[官方论文集采集器](docs/aamas-proceedings.md)支持已核验的 2012、2025 版式，可从缓存或已校验快照采集元数据。2025 年目录含 476 条非 keynote 出版物，保留完整作者和机构，HTML 未提供摘要时留空；发表形式与投稿 track 分开，不根据主题推定主会轨道或研究类型。
 
 `paper-atlas collect --help` 查看来源；`paper-atlas import --help` 查看导入方式。完整批次步骤见[使用指南](docs/usage.md)。
 

docs/aamas-proceedings.md

@@ -1,6 +1,14 @@
 # AAMAS 官方论文集元数据
 
-`paper_atlas.collectors.aamas` 是按需运行的采集适配器，通过统一入口 `collect aamas-proceedings` 使用。目前只支持已独立核验的 **AAMAS 2012** HTML 版式；其他年份需先核对官方目录和逐篇结构，再增加适配，不能仅替换年份。
+`paper_atlas.collectors.aamas` 是按需运行的采集适配器，通过统一入口 `collect aamas-proceedings` 使用。支持已独立核验的 **AAMAS 2012、2025** HTML 版式；其他年份需先核对官方目录和逐篇结构，再增加适配，不能仅替换年份。2025 解析单独封装在 `paper_atlas.collectors.aamas2025`。
+
+2025 版式使用[官方首页](https://www.ifaamas.org/Proceedings/aamas2025/)和[出版目录](https://www.ifaamas.org/Proceedings/aamas2025/forms/contents.htm)。原始目录共 479 条：3 keynote、260 research paper、148 extended abstract、12 Blue Sky、25 doctoral、22 Demo 栏目、9 JAAMAS。排除 keynote，输出 476 条；Demo 栏目中的 p3000 是明确标题的 COMP24 竞赛报告，发表年份仍为 2025，格式记 `competition_report`、投稿 track 留作 `proceedings_unspecified`，保留原 Demo 栏目证据。其余输出按 track 分组为 409 unspecified、12 blue_sky、25 doctoral、21 demo、9 jaamas。
+
+作者按逐篇 BR 行解析，斜体机构原文另存，保留 Unicode、顺序及完整名单。HTML 没有提供摘要或逐篇 DOI，故 `abstract=null`、标识字典为空；只保存 PDF 链接，不下载研究 PDF。不能把 Research Paper Track 的所有条目推成 main，不能把目录缺席推为 rejected/withdrawn，也不能把出版格式标签当研究类型。
+
+2025 用法：`paper-atlas collect aamas-proceedings --year 2025 --snapshots /data/review/aamas2025 --out /data/paper-index/collected/aamas-2025`；在线模式改用 `--cache /data/paper-index/cache/aamas2025`，只请求上述两个 HTML 页面。快照目录包含原始 `index.html`、`contents.html` 和 `receipts.json`，凭据必须列出各原始 URL、文件、字节数、SHA256、`http_status=200` 和带时区 `at`。已恢复的凭据需保留 `at_precision=filesystem_write_time_recovered` 等说明，不能伪造 HTTP 头或请求时间。2026-11-09 保存的两个页面通过哈希校验；独立录用清单 URL 返回 403，失败保留，不绕过或依赖其完成采集。
+
+2025 命令同样使用目录锁、临时目录、最后写入采集报告。五个 JSONL/manifest 均保持 `partial`，记录摘要缺失数量；它不连接数据库。首次采集需独立核验全部标题/作者/栏目和 source IDs，随后做 PostgreSQL 身份预检、必要的逐条版本审核、原生导入与独立验收。若失败且没有完成报告，应先检查原始响应和错误，再使用新输出目录重跑，不重放已提交批次。代码验证、原始目录盘点和采集成功均不代表已导入生产或最终去重完成。
 
 ## 来源与范围
 

docs/coverage/iclr-archive-2014-2016.md

@@ -0,0 +1,44 @@
+# ICLR 2014–2016 官方历史录用档案
+
+此适配器是长期维护工具的一部分，按需运行。只识别已经独立核对的三种页面布局；后续页面发生变化必须重新核查，不能放松计数或轨道边界以使采集成功。
+
+| 官方页面 | 页面链接次数 | 整理后的不同记录 |
+|---|---:|---:|
+| [2014 Conference Proceedings](https://iclr.cc/archive/2014/conference-proceedings/) | 35 | main 35 |
+| [2015 accepted-main](https://iclr.cc/archive/www/doku.php%3Fid=iclr2015:accepted-main.html) | 116 | main 31、workshop 74 |
+| [2016 accepted-main](https://iclr.cc/archive/www/doku.php%3Fid=iclr2016:accepted-main.html) | 80 | main 80 |
+
+2015 的 11 篇 oral 全部再次出现在 poster 段落中。相同年份、轨道及 arXiv 标识的重复展示只生成一条来源记录，但每次展示的标题、作者显示字符串、链接、版本和段落均保留在 `source_variants`。两处标题或解析作者冲突时停止采集、等待人工核查。不同年份或不同轨道不合并投稿。
+
+2014 的作者以分号列出，一条有末尾分号，一条标题和作者斜体块不在普通段落中，部分作者段落拆成多个格式片段；必须读取完整作者段落或独立斜体块，不能只读首个格式片段。2016 的代码链接不属于作者。保留原文，解析结果不替代来源证据。oral、poster 作为 `iclr_archive:presentation` 标签，不改成独立投稿轨道。arXiv 版本留在来源变体中，稳定标识使用去版本号的编号。
+
+## 来源快照与运行
+
+2026-11-09 已冻结的官方响应 SHA256：
+
+- 2014：`a33858176fbffae0066ed3c10310f7dd98a66d12a36334b6339ef1f915ef07af`
+- 2015：`131491663742cb084662a4804009b8318e0fa7bc184d4a9044d16dd28f05b888`
+- 2016：`7320b8652431793b6ea0743f871ee446bf08b6361ccd4e421b6b53449e71f4f6`
+
+在服务器运行，输出新目录，不下载论文 PDF：
+
+```sh
+paper-atlas collect iclr-archive --year 2015 \
+  --snapshots /data/paper-index/collected/historical-next-source-evidence-v1 \
+  --out /data/paper-index/collected/iclr-archive-source-v1/2015
+
+# 或重新下载官方 HTML，并保留 HTTP 缓存与收据
+paper-atlas collect iclr-archive --year 2016 \
+  --cache /data/paper-index/cache/iclr-archive \
+  --out /data/paper-index/collected/iclr-archive-source-v1/2016
+```
+
+冻结输入目录必须包含 `receipts.json`，每项有精确 `url`、HTTP `status: 200`、相对 `file`、`bytes`、`sha256` 和带时区的 `finished_at`。读取时重新核对尺寸、哈希和路径；拒绝越界及指向目录外的软链接。已完成输出不可覆盖，应检查报告后另建明确版本目录。解析或计数失败时不生成完成报告，也不导入数据库。
+
+输出分轨道 JSONL、manifest、原始 HTML 证据和年度采集报告。仅支持这三年已审查的页面。2014 workshop、其他未列轨道、逐篇摘要及公开拒稿范围尚未通过这些页面核验，不能据此声称完整。缺席不代表 rejected、withdrawn 或从未被录用。
+
+## 导入前后的边界
+
+这三页合计 220 条不同来源记录，不等于新增 220 篇论文；同一工作可能已存在于 arXiv 或其他会议。先核验来源字段与原始页面，再执行 PostgreSQL 只读预检，比较稳定标识、标题、作者和原始来源。标题/作者差异、版本关联及冲突必须保留证据并明确评估；不能仅靠标题猜测身份，也不能覆盖原有摘要。
+
+采集器不连接数据库。正式导入使用既有 `import records`，必须有新的生产保护基线和独立验收；全部导入结束后仍需全库去重和历史覆盖审计。来自官方录用清单的 accepted 是清单观察，不是原始审稿决定签名或决定日期。

paper_atlas/cli.py

@@ -13,6 +13,7 @@ COMMANDS = {
     ('download',): ('paper_atlas.io.snapshots', {}, '下载并校验固定版本快照'),
     ('collect','pmlr'): ('paper_atlas.collectors.pmlr', {}, 'PMLR 官方元数据'),
     ('collect','acl'): ('paper_atlas.collectors.acl', {}, 'ACL Anthology XML'),
+    ('collect','iclr-archive'): ('paper_atlas.collectors.iclr_archive', {}, '核对ICLR 2014–2016官方历史录用档案'),
     ('collect','iclr-program'): ('paper_atlas.collectors.iclr', {}, 'ICLR 官方公开程序'),
     ('collect','iclr-snapshot'): ('paper_atlas.collectors.openreview', {}, 'ICLR 公开投稿与决定快照'),
     ('collect','public-openreview'): ('paper_atlas.collectors.public_openreview', {}, '核验其他会议的公开拒稿及撤稿标签'),

paper_atlas/collectors/aamas.py

@@ -136,6 +136,9 @@ def read_pages(cache=None, snapshots=None):
 
 
 def _collect(year, out, *, cache=None, snapshots=None):
+    if year == 2025:
+        from paper_atlas.collectors.aamas2025 import collect as collect_2025
+        return collect_2025(out, cache=cache, snapshots=snapshots)
     if year != 2012:
         raise ValueError('Only the independently reviewed AAMAS 2012 layout is supported')
     pages, receipts = read_pages(cache, snapshots)
@@ -190,8 +193,8 @@ def _collect(year, out, *, cache=None, snapshots=None):
 
 
 def collect(year, out, *, cache=None, snapshots=None):
-    if year != 2012:
-        raise ValueError('Only the independently reviewed AAMAS 2012 layout is supported')
+    if year not in (2012, 2025):
+        raise ValueError('Only independently reviewed AAMAS 2012/2025 layouts are supported')
     out = Path(out)
     out.mkdir(parents=True, exist_ok=True)
     with (out/'aamas.collect.lock').open('a') as lock:
@@ -201,7 +204,7 @@ def collect(year, out, *, cache=None, snapshots=None):
 
 def main(argv=None, *, prog=None):
     parser = argparse.ArgumentParser(prog=prog)
-    parser.add_argument('--year', type=int, choices=[2012], required=True)
+    parser.add_argument('--year', type=int, choices=[2012, 2025], required=True)
     parser.add_argument('--out', required=True)
     source = parser.add_mutually_exclusive_group(required=True)
     source.add_argument('--cache')

paper_atlas/collectors/aamas2025.py

@@ -0,0 +1,183 @@
+"""Reviewed AAMAS 2025 publisher TOC; HTML metadata only, never paper PDFs."""
+from collections import Counter, defaultdict
+import hashlib
+import json
+from pathlib import Path
+import re
+import tempfile
+
+from paper_atlas.db.submissions import timestamp
+from paper_atlas.io.html import Document, Node, clean
+from paper_atlas.io.http import fetch
+
+ROOT = 'https://www.ifaamas.org/Proceedings/aamas2025/'
+URLS = {'index': ROOT, 'contents': ROOT+'forms/contents.htm'}
+TITLE = 'AAMAS 2025: Proceedings of the 2025 International Conference on Autonomous Agents and Multiagent Systems'
+SECTIONS = {
+    '1': ('Keynote Talks', 'keynote', 'keynote', 3),
+    '2': ('Research Paper Track', 'full_paper', 'proceedings_unspecified', 260),
+    '3': ('Extended Abstracts', 'extended_abstract', 'proceedings_unspecified', 148),
+    '4': ('Blue Sky Ideas', 'blue_sky_paper', 'blue_sky', 12),
+    '5': ('Doctoral Consortium', 'doctoral_abstract', 'doctoral', 25),
+    '6': ('Demo Track', 'demo', 'demo', 22),
+    '7': ('JAAMAS Track', 'journal_presentation', 'jaamas', 9),
+}
+COMPETITION_TITLE = '[COMP24] The Automated Negotiating Agents Competition (ANAC) 2024 Challenges and Results'
+PAPER = re.compile(r'\.\./pdfs/p([0-9]+)\.pdf')
+
+
+def one(items, label):
+    if len(items) != 1:
+        raise ValueError('Expected one '+label)
+    return items[0]
+
+
+def parse_contents(html, evidence):
+    doc = Document(html)
+    if [clean(n.text()) for n in doc.root.walk() if n.tag == 'title'] != [TITLE]:
+        raise ValueError('Unexpected AAMAS proceedings title/year')
+    for paragraph in doc.root.walk():
+        if paragraph.tag != 'p' or any(n is not paragraph and n.tag == 'p' for n in paragraph.walk()):
+            continue
+        anchors = [n.attrs['name'] for n in paragraph.walk() if n.tag == 'a' and n.attrs.get('name') in SECTIONS]
+        if anchors and (len(anchors) != 1 or clean(paragraph.text()) != SECTIONS[anchors[0]][0]):
+            raise ValueError('Publisher section heading differs from reviewed layout')
+    section = None
+    headers = []
+    records, excluded, counts, seen = [], [], Counter(), set()
+    for node in doc.root.walk():
+        if node.tag == 'a' and node.attrs.get('name') in SECTIONS:
+            section = node.attrs['name']
+            headers.append(section)
+        if node.tag != 'p' or any(n is not node and n.tag == 'p' for n in node.walk()):
+            continue
+        links = [n for n in node.walk() if n.tag == 'a' and PAPER.fullmatch(n.attrs.get('href', ''))]
+        if not links:
+            continue
+        link = one(links, 'paper link per paragraph')
+        if section not in SECTIONS:
+            raise ValueError('Paper outside a reviewed publisher section')
+        page = int(PAPER.fullmatch(link.attrs['href'])[1])
+        if page in seen:
+            raise ValueError('Repeated publisher paper/page ID')
+        seen.add(page)
+        title = clean(link.text())
+        if not title:
+            raise ValueError('Empty paper title')
+        # Names are outside italicized affiliations and separated by explicit BRs.
+        font = one([n for n in node.walk() if n.tag == 'font' and n.attrs.get('color', '').lower() == '#222222'], 'ordered author block')
+        authors, affiliations = [], []
+        name, affiliation = '', ''
+        def flush():
+            nonlocal name, affiliation
+            if clean(name) or clean(affiliation):
+                if not clean(name):
+                    raise ValueError('Affiliation without author')
+                authors.append(clean(name)); affiliations.append(clean(affiliation))
+            name, affiliation = '', ''
+        for child in font.children:
+            if isinstance(child, str):
+                name += child
+            elif child.tag == 'br':
+                flush()
+            elif child.tag == 'i':
+                affiliation += child.text()
+            else:
+                raise ValueError('Unreviewed author markup: '+child.tag)
+        flush()
+        if not authors:
+            raise ValueError('Empty author list')
+        heading, publication_format, track, _ = SECTIONS[section]
+        counts[section] += 1
+        if section == '1':
+            excluded.append({'publisher_page': page, 'title': title, 'authors': authors, 'reason': 'publisher keynote talk, not a research-paper submission'})
+            continue
+        note = 'Explicit publisher section; section topics are not research types.'
+        if track == 'proceedings_unspecified':
+            note = 'Published format is known; individual main/special submission track is not provided.'
+        if page == 3000:
+            if section != '6' or title != COMPETITION_TITLE:
+                raise ValueError('Competition report differs from reviewed publisher entry')
+            publication_format, track = 'competition_report', 'proceedings_unspecified'
+            note = 'Publisher places this competition report in Demo Track; no individual demonstration submission is inferred.'
+        records.append({
+            'source': 'aamas_proceedings', 'external_id': '2025/p'+str(page),
+            'url': URLS['contents']+'#'+section, 'title': title, 'abstract': None,
+            'authors': authors, 'identifiers': {}, 'venue': 'AAMAS', 'venue_aliases': [],
+            'year': 2025, 'track': track, 'status': 'accepted',
+            'observed_at': evidence['observed_at'],
+            'raw_decision': 'listed in official AAMAS 2025 published proceedings '+heading,
+            'tags': [{'namespace': 'aamas_proceedings:publication_format', 'value': publication_format}],
+            'metadata': {'publisher_page': page, 'paper_url': ROOT+'pdfs/p'+str(page)+'.pdf',
+                         'section_url': URLS['contents'], 'section_anchor': section,
+                         'section_heading': heading, 'publication_format': publication_format,
+                         'author_affiliations': affiliations, 'source_evidence': evidence,
+                         'track_scope_note': note, 'decision_date_precision': 'not_supplied',
+                         'missing_abstract_reason': 'Publisher HTML supplies no abstract; research PDF not downloaded.',
+                         'publication_evidence': 'Published proceedings entry, not an original decision note.'}})
+    if headers != list(SECTIONS) or dict(counts) != {k: v[3] for k, v in SECTIONS.items()}:
+        raise ValueError('Publisher section inventory differs from reviewed 2025 layout')
+    if len(records) != 476 or len(excluded) != 3 or sum(r['metadata']['publication_format'] == 'competition_report' for r in records) != 1:
+        raise ValueError('Unexpected publication partition')
+    return records, excluded
+
+
+def read_pages(cache, snapshots):
+    if bool(cache) == bool(snapshots):
+        raise ValueError('Choose exactly one cache or frozen snapshot directory')
+    frozen = json.loads((Path(snapshots)/'receipts.json').read_text())['receipts'] if snapshots else []
+    pages = {}
+    for key, url in URLS.items():
+        if snapshots:
+            r = one([r for r in frozen if r['url'] == url], 'frozen response receipt')
+            filename = Path(r['file']); base = Path(snapshots).resolve()
+            if filename.is_absolute() or '..' in filename.parts or not (base/filename).resolve().is_relative_to(base) or r.get('http_status') != 200:
+                raise ValueError('Unsafe or unsuccessful publisher response')
+            data = (base/filename).read_bytes()
+            if len(data) > 8*1024**2 or len(data) != r['bytes'] or hashlib.sha256(data).hexdigest() != r['sha256']:
+                raise ValueError('Frozen publisher response checksum/size mismatch')
+            observed = r['at']
+        else:
+            data = fetch(url, cache, 8*1024**2, delay=1)
+            r = json.loads((Path(cache)/hashlib.sha256(url.encode()).hexdigest()).with_suffix('.json').read_text())
+            observed = r['retrieved_at']
+        timestamp(observed)
+        pages[key] = data, {'url': url, 'sha256': hashlib.sha256(data).hexdigest(), 'bytes': len(data),
+                            'file': 'evidence/'+hashlib.sha256(data).hexdigest()+'.html',
+                            'observed_at': observed, 'http_status': 200, 'response_receipt': r}
+    return pages
+
+
+def collect(out, *, cache=None, snapshots=None):
+    pages = read_pages(cache, snapshots)
+    if [clean(n.text()) for n in Document(pages['index'][0].decode('utf-8')).root.walk() if n.tag == 'title'] != [TITLE]:
+        raise ValueError('Unexpected AAMAS 2025 introduction')
+    records, excluded = parse_contents(pages['contents'][0].decode('utf-8'), pages['contents'][1])
+    grouped = defaultdict(list)
+    for row in records: grouped[row['track']].append(row)
+    out = Path(out); out.mkdir(parents=True, exist_ok=True); files = []
+    with tempfile.TemporaryDirectory(prefix='.aamas2025-', dir=out) as temporary:
+        stage = Path(temporary); (stage/'evidence').mkdir()
+        for data, e in pages.values(): (stage/e['file']).write_bytes(data)
+        for track, group in sorted(grouped.items()):
+            p = stage/f'aamas-proceedings-2025-{track}.jsonl'
+            p.write_text(''.join(json.dumps(r, ensure_ascii=False)+'\n' for r in group))
+            m = {'source': 'aamas_proceedings', 'parser_version': 2, 'venue': 'AAMAS', 'year': 2025,
+                 'track': track, 'count': len(group), 'file': str(out/p.name), 'file_sha256': hashlib.sha256(p.read_bytes()).hexdigest(),
+                 'list_url': URLS['contents'], 'status': 'partial', 'missing_abstract_count': len(group),
+                 'publication_format_counts': dict(Counter(r['metadata']['publication_format'] for r in group)),
+                 'note': 'Complete published TOC partition; HTML abstracts absent, identities and submission-track scope require independent review.'}
+            p.with_suffix('.manifest.json').write_text(json.dumps(m, ensure_ascii=False, indent=2)+'\n'); files.append(m)
+        result = {'source': 'aamas_proceedings', 'year': 2025, 'finished_at': timestamp(),
+                  'status': 'collected_pending_identity_review', 'records': len(records), 'files': files,
+                  'excluded_publications': excluded, 'source_receipts': [e for _, e in pages.values()],
+                  'publication_format_counts': dict(Counter(r['metadata']['publication_format'] for r in records)),
+                  'database_written': False, 'pdfs_downloaded': False,
+                  'note': 'Neither absence from proceedings nor aggregate rejection/withdrawal counts identify an individual negative decision.'}
+        report = 'aamas-proceedings-2025.json'; (stage/report).write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+        (out/'evidence').mkdir(exist_ok=True)
+        for p in (stage/'evidence').iterdir(): p.replace(out/'evidence'/p.name)
+        for p in stage.iterdir():
+            if p.is_file() and p.name != report: p.replace(out/p.name)
+        (stage/report).replace(out/report)
+    return result

paper_atlas/collectors/iclr_archive.py

@@ -0,0 +1,225 @@
+"""Frozen official ICLR 2014–2016 lists; presentation evidence is not a track."""
+import argparse
+from collections import Counter, defaultdict
+import fcntl
+import hashlib
+import json
+from pathlib import Path
+import re
+import tempfile
+from urllib.parse import urlsplit
+
+from paper_atlas.db.submissions import timestamp
+from paper_atlas.io.html import Document, Node, clean
+from paper_atlas.io.http import fetch
+
+SOURCE = 'iclr_archive'
+PAGES = {
+    2014: ('https://iclr.cc/archive/2014/conference-proceedings/', {'main': 35}),
+    2015: ('https://iclr.cc/archive/www/doku.php%3Fid=iclr2015:accepted-main.html', {'main': 31, 'workshop': 74}),
+    2016: ('https://iclr.cc/archive/www/doku.php%3Fid=iclr2016:accepted-main.html', {'main': 80}),
+}
+SECTIONS = {
+    2015: {'Main Conference - Oral Presentations': ('main', 'oral'),
+           'Main Conference - Poster Presentations': ('main', 'poster'),
+           'Workshop Papers': ('workshop', 'workshop')},
+    2016: {'Oral Presentations': ('main', 'oral'), 'Poster Presentations': ('main', 'poster')},
+}
+
+
+def arxiv_link(url):
+    parsed = urlsplit(url)
+    if parsed.hostname != 'arxiv.org':
+        return None
+    match = re.fullmatch(r'/(?:abs|pdf)/(\d{4}\.\d{4,5})(v[1-9]\d*)?(?:\.pdf)?', parsed.path)
+    if (parsed.scheme not in ('http', 'https') or parsed.netloc != 'arxiv.org'
+            or parsed.query or parsed.fragment or not match):
+        raise ValueError('Unexpected official arXiv link')
+    return match[1], match[2] or None
+
+
+def without_links(node):
+    if node.tag in ('a', 'script', 'style'):
+        return ''
+    value = ''.join(without_links(n) if isinstance(n, Node) else n for n in node.children)
+    return ' '+value+' ' if node.tag in ('div', 'li', 'p', 'br') else value
+
+
+def authors(display, year):
+    parts = display.rstrip(';').split(';') if year == 2014 else re.split(r',\s*(?:and\s+)?|\s+and\s+', display)
+    names = [clean(p) for p in parts]
+    if not names or any(not name for name in names):
+        raise ValueError('Missing author or unsupported author delimiter')
+    return names
+
+
+def parse_page(html, year, evidence):
+    if year not in PAGES:
+        raise ValueError('Unreviewed ICLR archive year')
+    document = Document(html)
+    nodes = list(document.root.walk())
+    headings = [clean(n.text()) for n in nodes if n.tag in ('h1', 'h2', 'h3')]
+    if year == 2014:
+        if 'Conference Proceedings' not in headings or 'conference papers accepted' not in clean(document.root.text()):
+            raise ValueError('Unexpected ICLR 2014 accepted list')
+    elif (set(SECTIONS[year]) - set(headings)
+          or (year == 2016 and 'Accepted Papers (Conference Track)' not in headings)):
+        raise ValueError('Unexpected ICLR accepted sections')
+    entries = []
+    section = None
+    for index, node in enumerate(nodes):
+        if node.tag in ('h1', 'h2', 'h3'):
+            section = clean(node.text())
+        if year == 2014:
+            if node.tag != 'a' or not arxiv_link(node.attrs.get('href', '')):
+                continue
+            link = node
+            # This archived Google Sites page has an entry outside a paragraph.
+            # Authors can span several differently formatted italic spans.
+            # Read their whole paragraph, never only the first styled fragment.
+            display = None
+            for following in nodes[index+sum(1 for _ in link.walk()):]:
+                if following.tag == 'a' and arxiv_link(following.attrs.get('href', '')):
+                    break
+                if following.tag in ('i', 'em') or (following.tag == 'p' and any(
+                        n.tag in ('i', 'em') or 'font-style:italic' in n.attrs.get('style', '').replace(' ', '')
+                        for n in following.walk())):
+                    text = clean(following.text())
+                    if text:
+                        display = text
+                        break
+            if not display:
+                raise ValueError('Missing explicit ICLR 2014 author span')
+            track, presentation = 'main', 'conference'
+        else:
+            if node.tag != 'li':
+                continue
+            links = [n for n in node.walk() if n.tag == 'a' and arxiv_link(n.attrs.get('href', ''))]
+            if not links:
+                continue
+            if len(links) != 1 or section not in SECTIONS[year]:
+                raise ValueError('Ambiguous paper link or unknown accepted section')
+            link, = links
+            display = clean(without_links(node)).lstrip(',').strip()
+            track, presentation = SECTIONS[year][section]
+        arxiv, version = arxiv_link(link.attrs['href'])
+        title = clean(link.text())
+        if not title:
+            raise ValueError('Empty official title')
+        entries.append({'arxiv': arxiv, 'version': version, 'title': title,
+                        'authors': authors(display, year), 'authors_display': display,
+                        'paper_url': link.attrs['href'], 'track': track,
+                        'section': section, 'presentation': presentation})
+    # Check every arXiv occurrence, including links outside the expected layout.
+    occurrences = sum(n.tag == 'a' and bool(arxiv_link(n.attrs.get('href', ''))) for n in nodes)
+    if len(entries) != occurrences or not entries:
+        raise ValueError('Some official paper links were not parsed')
+    grouped = defaultdict(list)
+    for entry in entries:
+        grouped[(entry['track'], entry['arxiv'])].append(entry)
+    result = []
+    for (track, arxiv), variants in grouped.items():
+        first = variants[0]
+        if any((v['title'], v['authors']) != (first['title'], first['authors']) for v in variants):
+            raise ValueError('Repeated presentation has conflicting title or authors; review required')
+        presentations = sorted({v['presentation'] for v in variants})
+        result.append({'source': SOURCE, 'external_id': f'{year}/{track}/{arxiv}',
+                       'url': PAGES[year][0], 'title': first['title'], 'abstract': None,
+                       'authors': first['authors'], 'identifiers': {'arxiv': arxiv},
+                       'venue': 'ICLR', 'venue_aliases': [], 'year': year, 'track': track,
+                       'status': 'accepted', 'observed_at': evidence['observed_at'],
+                       'raw_decision': 'listed in official ICLR accepted '+track+' list',
+                       'metadata': {'source_evidence': evidence, 'source_variants': variants,
+                                    'presentations': presentations, 'decision_date_precision': 'not_supplied',
+                                    'identity_note': 'Official arXiv link, not title matching; validate canonical identity before import.',
+                                    'decision_evidence': 'Archived accepted list, not an original decision note.'},
+                       'tags': [{'namespace': SOURCE+':presentation', 'value': value} for value in presentations]})
+    return result
+
+
+def read_page(year, *, cache=None, snapshots=None):
+    if bool(cache) == bool(snapshots):
+        raise ValueError('Choose exactly one cache or frozen snapshot directory')
+    url = PAGES[year][0]
+    if snapshots:
+        root = Path(snapshots).resolve()
+        receipts = json.loads((root/'receipts.json').read_bytes())['receipts']
+        matches = [r for r in receipts if r['url'] == url]
+        if len(matches) != 1:
+            raise ValueError('Expected one exact official response receipt')
+        receipt, = matches
+        filename = Path(receipt['file'])
+        path = root/filename
+        if (filename.is_absolute() or '..' in filename.parts or not path.resolve().is_relative_to(root)
+                or receipt['status'] != 200):
+            raise ValueError('Unsafe or unsuccessful archive snapshot')
+        data = path.read_bytes()
+        if (len(data) > 8*1024**2 or len(data) != receipt['bytes']
+                or hashlib.sha256(data).hexdigest() != receipt['sha256']):
+            raise ValueError('Archive snapshot checksum or size differs')
+        observed = timestamp(receipt['finished_at'])
+    else:
+        data = fetch(url, cache, 8*1024**2, delay=1)
+        receipt = json.loads((Path(cache)/hashlib.sha256(url.encode()).hexdigest()).with_suffix('.json').read_bytes())
+        observed = timestamp(receipt['retrieved_at'])
+    evidence = {'url': url, 'sha256': hashlib.sha256(data).hexdigest(), 'bytes': len(data),
+                'file': 'evidence/'+hashlib.sha256(data).hexdigest()+'.html',
+                'observed_at': observed, 'http_status': 200, 'response_receipt': receipt}
+    return data, evidence
+
+
+def collect(year, out, *, cache=None, snapshots=None):
+    if year not in PAGES:
+        raise ValueError('Only independently reviewed ICLR archive years are supported')
+    out = Path(out)
+    out.mkdir(parents=True, exist_ok=True)
+    with (out/'iclr-archive.collect.lock').open('a') as lock:
+        fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        report = out/f'iclr-archive-{year}.json'
+        if report.exists():
+            raise FileExistsError('Existing completed archive output must be inspected, not overwritten')
+        data, evidence = read_page(year, cache=cache, snapshots=snapshots)
+        records = parse_page(data.decode('utf-8'), year, evidence)
+        counts = Counter(r['track'] for r in records)
+        if dict(counts) != PAGES[year][1]:
+            raise ValueError('Distinct archive entries differ from reviewed inventory')
+        files = []
+        with tempfile.TemporaryDirectory(prefix='.iclr-archive-', dir=out) as temporary:
+            stage = Path(temporary)
+            (stage/'evidence').mkdir()
+            (stage/evidence['file']).write_bytes(data)
+            for track in sorted(counts):
+                group = [r for r in records if r['track'] == track]
+                path = stage/f'iclr-archive-{year}-{track}.jsonl'
+                path.write_text(''.join(json.dumps(r, ensure_ascii=False)+'\n' for r in group))
+                manifest = {'source': SOURCE, 'parser_version': 1, 'venue': 'ICLR', 'year': year,
+                            'track': track, 'count': len(group), 'file': str(out/path.name),
+                            'file_sha256': hashlib.sha256(path.read_bytes()).hexdigest(),
+                            'list_url': PAGES[year][0], 'status': 'partial', 'missing_abstract_count': len(group)}
+                path.with_suffix('.manifest.json').write_text(json.dumps(manifest, ensure_ascii=False, indent=2)+'\n')
+                files.append(manifest)
+            result = {'source': SOURCE, 'year': year, 'records': len(records), 'files': files,
+                      'source_evidence': evidence, 'database_written': False, 'pdfs_downloaded': False,
+                      'status': 'collected_pending_identity_review',
+                      'note': 'Accepted list only. Oral/poster are presentation evidence; absent papers are not rejected. '
+                              'ICLR2014 workshop and other unlisted tracks remain outside these reviewed pages.'}
+            (stage/report.name).write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+            (out/'evidence').mkdir(exist_ok=True)
+            for path in (stage/'evidence').iterdir():
+                path.replace(out/'evidence'/path.name)
+            for path in stage.iterdir():
+                if path.is_file() and path.name != report.name:
+                    path.replace(out/path.name)
+            (stage/report.name).replace(report)
+        return result
+
+
+def main(argv=None, *, prog=None):
+    parser = argparse.ArgumentParser(prog=prog)
+    parser.add_argument('--year', type=int, choices=sorted(PAGES), required=True)
+    parser.add_argument('--out', required=True)
+    source = parser.add_mutually_exclusive_group(required=True)
+    source.add_argument('--cache')
+    source.add_argument('--snapshots')
+    args = parser.parse_args(argv)
+    print(json.dumps(collect(args.year, args.out, cache=args.cache, snapshots=args.snapshots), ensure_ascii=False))

tests/fixtures/aamas2025/index.html

@@ -0,0 +1,54 @@
+<HTML><HEAD><TITLE>AAMAS 2025: Proceedings of the 2025 International Conference on Autonomous Agents and Multiagent Systems</TITLE>
+<META http-equiv=Content-Type content="text/html; charset=utf-8"><LINK 
+href="forms/index_files/factemplate.css" type=text/css rel=stylesheet>
+<META content="MSHTML 6.00.2900.3562" name=GENERATOR></HEAD>
+<BODY bgcolor="#010101" bgproperties="fixed"><DIV style="TEXT-ALIGN: center" >
+<table width="724" border="0" align="center" cellpadding="0" cellspacing="0" valign="top">
+  <td colspan="2" align="center" valign="top" bgcolor="#FFFFFF"><img src="forms/index_files/aamas25.png" width="750" height="300" alt=""/></td>
+  </tr>
+  <tr bgcolor="#FFFFFF"> 
+    <td width="125" align="center" valign="top" bgcolor="#FFFFFF"><table width="125" border="1" align="center" cellpadding="1" cellspacing="1" bordercolor="#444444" style="margin-bottom: 2">
+        <tr> 
+          <td bgcolor="#FFFFFF"><p style="text-indent: 0; margin-left: 6; margin-right: 6; margin-top: 5; margin-bottom: 4" align="left"><A href="forms/index.htm">Main 
+              Page</A></p></td>
+        </tr>
+        <tr> 
+          <td bgcolor="#FFFFFF"><p style="text-indent: 0; margin-left: 6; margin-right: 6; margin-top: 5; margin-bottom: 4" align="left"><A href="forms/contents.htm">Table 
+              of Contents</A></p></td>
+        </tr>
+      </table></td>
+    <td width="599"><p style="text-indent: 0; margin-left: 12; margin-right: 14; margin-top: 12; margin-bottom: 12" align="left"><font color="#010101" size="4" face="Arial, Helvetica, sans-serif"><strong>Welcome 
+        to the Proceedings of AAMAS-2025</strong></font></p>
+      <p style="text-indent: 0; margin-left: 12; margin-right: 14; margin-top: 12; margin-bottom: 12;" align="left"><font color="#333333" size="2" face="Times New Roman, Times, serif">These 
+        are the proceedings of the 24th International Conference on Autonomous 
+        Agents and Multiagent Systems (AAMAS-2025). They are published by the 
+        International Foundation for Autonomous Agents and Multiagent Systems 
+    (IFAAMAS).</font></p></td>
+  </tr>
+  <tr bgcolor="#FFFFFF"> 
+    <td align="center" valign="middle" bgcolor="#FFFFFF">&nbsp;</td>
+    <td><table width="96%" border="0" cellspacing="2" cellpadding="2">
+        <tr align="left" valign="top"> 
+          <td> <p style="text-indent: 0; margin-left: 12; margin-right: 6; margin-top: 0; margin-bottom: 6;" align="left"><font color="#333333" face="Times New Roman, Times, serif"><font size="2"><strong>ISBN: 979-8-4007-1426-9</strong></font></font></p></td>
+          <td> <p style="text-indent: 0; margin-left: 12; margin-right: 6; margin-top: 0; margin-bottom: 6;" align="left"><font color="#333333" face="Times New Roman, Times, serif"></font></p></td>
+        </tr>
+        <tr> 
+          <td width="42%"><div align="left"><img src="forms/index_files/ifaamas.jpg" width="256" height="104"></div></td>
+          <td width="58%"><div align="center"><img src="forms/index_files/ACM-In-Cooperation_medium.jpg" width="285" height="86"></div></td>
+        </tr>
+      </table>
+      <p style="text-indent: 0; margin-left: 12; margin-right: 14; margin-top: 12; margin-bottom: 12;" align="left"><font color="#333333" size="2" face="Times New Roman, Times, serif">Copyright 
+        &copy; 2025 by International Foundation for Autonomous Agents and Multiagent 
+        Systems (IFAAMAS). Permission to make digital or hard copies of portions 
+        of this work for personal or classroom use is granted without fee provided 
+        that copies are not made or distributed for profit or commercial advantage 
+        and that copies bear this notice and the full citation on the first page. 
+        Copyright for components of this work owned by others than IFAAMAS must 
+        be honored. Abstracting with credit is permitted. To copy otherwise, to 
+        republish, to post on servers or to redistribute to lists, requires prior 
+    specific permission and/or a fee. </font></p></td>
+  </tr>
+</table>
+  <p>&nbsp;</p>
+
+</BODY></HTML>

tests/fixtures/aamas2025/receipts.json

@@ -0,0 +1,38 @@
+{
+  "status": "two_successful_official_metadata_pages_recovered_third403_not_retried",
+  "receipts": [
+    {
+      "file": "index.html",
+      "url": "https://www.ifaamas.org/Proceedings/aamas2025/",
+      "bytes": 4107,
+      "sha256": "850b7825c66d0147ac8a8bbf03a554590e4a9e8f4b5c576e043999bbd43317f4",
+      "at": "2026-11-09T12:31:49.347577+00:00",
+      "at_precision": "filesystem_write_time_recovered",
+      "http_status": 200,
+      "content_type": "text/html",
+      "receipt_recovered": true,
+      "scope": "Collectorasserted200/htmlbeforewrite;request3acceptedprogram403;noheaders/finalredirectURLpersisted;no refetch or rawbytesmutation."
+    },
+    {
+      "file": "contents.html",
+      "url": "https://www.ifaamas.org/Proceedings/aamas2025/forms/contents.htm",
+      "bytes": 370650,
+      "sha256": "732847b51972f34c721a49d8bd963a80a029a48f3ec08022c9a0d5ddc2945549",
+      "at": "2026-11-09T12:31:52.006642+00:00",
+      "at_precision": "filesystem_write_time_recovered",
+      "http_status": 200,
+      "content_type": "text/html",
+      "receipt_recovered": true,
+      "scope": "Collectorasserted200/htmlbeforewrite;request3acceptedprogram403;noheaders/finalredirectURLpersisted;no refetch or rawbytesmutation."
+    }
+  ],
+  "failed_url": "https://aamas2025.org/index.php/conference/program/accepted-papers/",
+  "not_attempted_url": "https://aamas2025.org/index.php/conference/calls/submission-instructions-main-technical-track/",
+  "original_error": {
+    "error": "HTTP Error 403: Forbidden",
+    "type": "HTTPError",
+    "downloaded": 2
+  },
+  "research_fulltext_downloaded": false,
+  "database_written": false
+}

tests/test_aamas2025.py

@@ -0,0 +1,85 @@
+import hashlib
+import json
+from pathlib import Path
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from paper_atlas.collectors import aamas, aamas2025
+
+
+FIXTURE = Path(__file__).parent/'fixtures/aamas2025'
+
+
+class Aamas2025Tests(unittest.TestCase):
+    def parse(self, html=None):
+        return aamas2025.parse_contents(html or (FIXTURE/'contents.html').read_text(),
+            {'observed_at': '2026-11-09T12:31:50+00:00', 'url': aamas2025.URLS['contents']})
+
+    def test_real_toc_preserves_every_title_author_and_affiliation(self):
+        expected = json.loads((FIXTURE/'expected-authors.json').read_text())
+        self.assertEqual(hashlib.sha256((FIXTURE/'contents.html').read_bytes()).hexdigest(), expected['source_sha256'])
+        rows, excluded = self.parse()
+        self.assertEqual((len(rows), len(excluded)), (476, 3))
+        by_url = {r['paper_url']: r for r in expected['entries']}
+        for row in rows:
+            original = by_url[row['metadata']['paper_url']]
+            self.assertEqual(row['title'], original['title'])
+            self.assertEqual(row['authors'], [a['name'] for a in original['authors']])
+            self.assertEqual(row['metadata']['author_affiliations'], [a['affiliation'] for a in original['authors']])
+            self.assertIsNone(row['abstract'])
+            self.assertEqual(row['identifiers'], {})
+            self.assertEqual(row['status'], 'accepted')
+            self.assertNotIn('decision_date', row)
+            self.assertTrue(all(t['namespace'] == 'aamas_proceedings:publication_format' for t in row['tags']))
+
+    def test_publication_format_and_track_are_separate(self):
+        rows, excluded = self.parse()
+        groups = {k: sum(r['track'] == k for r in rows) for k in {r['track'] for r in rows}}
+        self.assertEqual(groups, {'proceedings_unspecified': 409, 'blue_sky': 12, 'doctoral': 25, 'demo': 21, 'jaamas': 9})
+        competition = next(r for r in rows if r['external_id'] == '2025/p3000')
+        self.assertEqual(competition['year'], 2025)
+        self.assertEqual(competition['metadata']['publication_format'], 'competition_report')
+        self.assertEqual(competition['metadata']['section_heading'], 'Demo Track')
+        self.assertEqual(competition['track'], 'proceedings_unspecified')
+        self.assertEqual(len({r['external_id'] for r in rows}), 476)
+        self.assertEqual([r['publisher_page'] for r in excluded], [1, 3, 4])
+
+    def test_refuses_changed_edition_inventory_or_author_markup(self):
+        original = (FIXTURE/'contents.html').read_text()
+        for html in [original.replace('AAMAS 2025:', 'AAMAS 2026:', 1),
+                     original.replace('../pdfs/p5.pdf', '../pdfs/p14.pdf', 1),
+                     original.replace('Paper Track</font>', 'Workshop Track</font>'),
+                     original.replace('Amir Abolfazli', '', 1),
+                     original.replace('[COMP24]', '[CHANGED]', 1)]:
+            with self.subTest(change=html[:60]):
+                with self.assertRaises(ValueError): self.parse(html)
+
+    def test_frozen_collection_is_partial_and_never_fetches_pdfs(self):
+        with tempfile.TemporaryDirectory() as tmp, patch.object(aamas2025, 'fetch', side_effect=AssertionError('No HTTP in frozen mode')):
+            out = Path(tmp)/'out'
+            report = aamas.collect(2025, out, snapshots=FIXTURE)
+            self.assertEqual(report['records'], 476)
+            self.assertFalse(report['database_written'])
+            self.assertFalse(report['pdfs_downloaded'])
+            self.assertEqual(len(report['files']), 5)
+            for m in report['files']:
+                self.assertEqual(m['status'], 'partial')
+                self.assertEqual(m['missing_abstract_count'], m['count'])
+                self.assertEqual(hashlib.sha256(Path(m['file']).read_bytes()).hexdigest(), m['file_sha256'])
+            receipt = report['source_receipts'][0]['response_receipt']
+            self.assertEqual(receipt['at_precision'], 'filesystem_write_time_recovered')
+            self.assertTrue((out/'aamas-proceedings-2025.json').is_file())
+
+    def test_rejects_tampered_or_unsafe_frozen_response(self):
+        import shutil
+        for mode in ['tamper', 'path', 'status']:
+            with self.subTest(mode=mode), tempfile.TemporaryDirectory() as tmp:
+                root = Path(tmp); source = root/'source'; shutil.copytree(FIXTURE, source)
+                p = source/'receipts.json'; r = json.loads(p.read_text())
+                if mode == 'tamper': (source/'contents.html').write_text('bad')
+                elif mode == 'path': r['receipts'][0]['file'] = '../index.html'
+                else: r['receipts'][0]['http_status'] = 403
+                p.write_text(json.dumps(r))
+                with self.assertRaises(ValueError): aamas.collect(2025, root/'out', snapshots=source)
+                self.assertFalse((root/'out/aamas-proceedings-2025.json').exists())

tests/test_aamas_proceedings.py

@@ -99,7 +99,7 @@ class AamasProceedingsTests(unittest.TestCase):
                 aamas.collect(2012, root/'out', snapshots=source)
             self.assertFalse((root/'out/aamas-proceedings-2012.json').exists())
             with self.assertRaises(ValueError):
-                aamas.collect(2025, root/'future', snapshots=source)
+                aamas.collect(2026, root/'future', snapshots=source)
             self.assertFalse((root/'future').exists())
 
     def test_ids_colliding_across_sections_do_not_produce_a_completed_collection(self):

tests/test_iclr_archive.py

@@ -0,0 +1,148 @@
+"""Historical accepted lists, repeated presentations and preserved work histories."""
+import hashlib
+import json
+from pathlib import Path
+import sqlite3
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from paper_atlas.collectors import iclr_archive as archive
+from paper_atlas.db.store import Store
+from paper_atlas.ingestion.records import ingest_file
+
+OBSERVED = '2026-11-09T00:00:00Z'
+
+
+def entry(identifier, title='A shared research paper', display='Jane Doe and Bob Smith'):
+    return f'<li><div><a href="http://arxiv.org/abs/{identifier}">{title}</a>, {display}</div></li>'
+
+
+def page2015():
+    return ('<h3>Main Conference - Oral Presentations</h3><ol>'+entry('1412.0001v2')+'</ol>'
+            '<h3>Main Conference - Poster Presentations</h3><ol>'+entry('1412.0001v2')+'</ol>'
+            '<h3>Workshop Papers</h3><ol>'+entry('1412.0002', 'A workshop paper')+'</ol>')
+
+
+class IclrArchiveTests(unittest.TestCase):
+    def frozen(self, root, year, html):
+        root.mkdir()
+        data = html.encode()
+        (root/'page.html').write_bytes(data)
+        receipt = {'url': archive.PAGES[year][0], 'file': 'page.html', 'status': 200,
+                   'bytes': len(data), 'sha256': hashlib.sha256(data).hexdigest(), 'finished_at': OBSERVED}
+        (root/'receipts.json').write_text(json.dumps({'receipts': [receipt]}))
+        return root
+
+    def test_oral_poster_repeat_is_one_attempt_with_all_source_variants(self):
+        rows = archive.parse_page(page2015(), 2015, {'observed_at': OBSERVED})
+        main, workshop = rows
+        self.assertEqual((main['track'], workshop['track']), ('main', 'workshop'))
+        self.assertEqual(main['identifiers'], {'arxiv': '1412.0001'})
+        self.assertEqual(main['external_id'], '2015/main/1412.0001')
+        self.assertEqual(main['metadata']['presentations'], ['oral', 'poster'])
+        self.assertEqual(len(main['metadata']['source_variants']), 2)
+        self.assertEqual(main['metadata']['source_variants'][0]['version'], 'v2')
+        self.assertEqual(main['authors'], ['Jane Doe', 'Bob Smith'])
+        self.assertTrue(all(r['status'] == 'accepted' and r['abstract'] is None for r in rows))
+
+    def test_2014_author_span_outside_paragraph_and_trailing_separator(self):
+        html = ('<h3>Conference Proceedings</h3><p>conference papers accepted</p>'
+                '<div><b><a href="http://arxiv.org/abs/1312.0001">A <i>research</i> title</a></b></div>'
+                '<p><span style="font-style: italic">Jane Doe; </span>'
+                '<span style="font-style:italic">Bob Smith; Carol Wu;</span></p>'
+                '<div><a href="http://arxiv.org/abs/1312.0002">Another title</a>'
+                '<div><i><span>First Author; </span><span>Second Author</span></i></div></div>')
+        row, unwrapped = archive.parse_page(html, 2014, {'observed_at': OBSERVED})
+        self.assertEqual(row['title'], 'A research title')
+        self.assertEqual(row['authors'], ['Jane Doe', 'Bob Smith', 'Carol Wu'])
+        self.assertEqual(row['metadata']['source_variants'][0]['authors_display'], 'Jane Doe; Bob Smith; Carol Wu;')
+        self.assertEqual(row['track'], 'main')
+        self.assertEqual(unwrapped['authors'], ['First Author', 'Second Author'])
+
+    def test_same_arxiv_in_different_tracks_keeps_separate_source_attempts(self):
+        html = page2015().replace('1412.0002', '1412.0001v3')
+        rows = archive.parse_page(html, 2015, {'observed_at': OBSERVED})
+        self.assertEqual(len(rows), 2)
+        self.assertEqual({r['external_id'] for r in rows}, {'2015/main/1412.0001', '2015/workshop/1412.0001'})
+        self.assertEqual(rows[1]['metadata']['source_variants'][0]['version'], 'v3')
+
+    def test_2016_code_links_and_breaks_do_not_become_authors(self):
+        html = ('<h1>Accepted Papers (Conference Track)</h1><h3>Oral Presentations</h3><ol>'
+                '<li><a href="http://arxiv.org/abs/1511.00001">A title</a>'
+                '<a href="https://example.org/code">[code]</a><br>Jane Doe, Bob Smith</li></ol>'
+                '<h3>Poster Presentations</h3>')
+        row, = archive.parse_page(html, 2016, {'observed_at': OBSERVED})
+        self.assertEqual(row['authors'], ['Jane Doe', 'Bob Smith'])
+        self.assertEqual(row['metadata']['presentations'], ['oral'])
+
+    def test_conflicting_repeat_unknown_sections_and_unparsed_links_fail(self):
+        variants = [page2015().replace('Poster Presentations', 'Other Presentations'),
+                    page2015().replace('</ol>', '</ol><a href="http://arxiv.org/abs/1412.0003">stray</a>', 1),
+                    page2015().replace('Jane Doe and Bob Smith', 'Different Author', 1)]
+        for html in variants:
+            with self.subTest(html=html), self.assertRaises(ValueError):
+                archive.parse_page(html, 2015, {'observed_at': OBSERVED})
+
+    def test_arxiv_versions_are_preserved_and_unsafe_links_rejected(self):
+        self.assertEqual(archive.arxiv_link('https://arxiv.org/pdf/1511.00001v3.pdf'), ('1511.00001', 'v3'))
+        self.assertIsNone(archive.arxiv_link('https://arxiv.org.example.org/abs/1511.00001'))
+        for url in ['https://arxiv.org/abs/1511.00001?other=1', 'ftp://arxiv.org/abs/1511.00001',
+                    'https://user@arxiv.org/abs/1511.00001', 'https://arxiv.org/abs/not-a-paper']:
+            with self.subTest(url=url), self.assertRaises(ValueError):
+                archive.arxiv_link(url)
+
+    def test_receipt_hash_status_path_and_symlink_cannot_escape_snapshot(self):
+        for failure in ['hash', 'status', 'path', 'symlink']:
+            with self.subTest(failure=failure), tempfile.TemporaryDirectory() as temporary:
+                root = Path(temporary)
+                source = self.frozen(root/'source', 2015, page2015())
+                file = source/'receipts.json'
+                data = json.loads(file.read_text())
+                receipt = data['receipts'][0]
+                if failure == 'hash': receipt['sha256'] = '0'*64
+                elif failure == 'status': receipt['status'] = 404
+                elif failure == 'path': receipt['file'] = '../outside.html'
+                else:
+                    (root/'outside.html').write_text(page2015())
+                    (source/'page.html').unlink()
+                    (source/'page.html').symlink_to(root/'outside.html')
+                file.write_text(json.dumps(data))
+                with self.assertRaises(ValueError):
+                    archive.read_page(2015, snapshots=source)
+
+    def test_changed_inventory_leaves_no_completed_output(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            root = Path(temporary)
+            source = self.frozen(root/'source', 2015, page2015())
+            with self.assertRaises(ValueError):
+                archive.collect(2015, root/'out', snapshots=source)
+            self.assertFalse((root/'out/iclr-archive-2015.json').exists())
+
+    def test_native_import_keeps_rejected_to_accepted_history_and_abstract(self):
+        with tempfile.TemporaryDirectory() as temporary:
+            root = Path(temporary)
+            source = self.frozen(root/'source', 2015, page2015())
+            with patch.dict(archive.PAGES, {2015: (archive.PAGES[2015][0], {'main': 1, 'workshop': 1})}):
+                result = archive.collect(2015, root/'out', snapshots=source)
+                with self.assertRaises(FileExistsError):
+                    archive.collect(2015, root/'out', snapshots=source)
+            db = root/'sample.sqlite'
+            store = Store(db)
+            pid, rid = store.record('arxiv', '1412.0001', 'https://arxiv.org/abs/1412.0001',
+                                    'A shared research paper', 'Keep this canonical abstract.',
+                                    ['Jane Doe', 'Bob Smith'], {'arxiv': '1412.0001'}, {})
+            store.decision(pid, rid, 'Earlier Conference', 2014, 'main', 'original-attempt',
+                           'https://example.org/rejected', 'rejected', 'explicit earlier decision', observed_at=OBSERVED)
+            store.conn.commit()
+            store.conn.close()
+            for manifest in result['files']:
+                ingest_file(db, manifest['file'])
+                ingest_file(db, manifest['file'])
+            with sqlite3.connect(db) as c:
+                self.assertEqual(c.execute('SELECT count(*) FROM papers').fetchone()[0], 2)
+                self.assertEqual(c.execute('SELECT abstract FROM papers WHERE id=?', (pid,)).fetchone()[0], 'Keep this canonical abstract.')
+                history = c.execute('SELECT v.acronym,e.year,d.status FROM decisions d JOIN submissions s ON s.id=d.submission_id JOIN editions e ON e.id=s.edition_id JOIN venues v ON v.id=e.venue_id WHERE s.paper_id=? ORDER BY e.year', (pid,)).fetchall()
+                self.assertEqual(history, [('Earlier Conference', 2014, 'rejected'), ('ICLR', 2015, 'accepted')])
+                self.assertEqual(c.execute('SELECT count(*) FROM submissions').fetchone()[0], 3)
+                self.assertEqual(c.execute('PRAGMA foreign_key_check').fetchall(), [])