docs: set 2020 backfill scope and preserve recovery inventory since 2000

Merge pull request '限定ACL1978与TINLAP卷的来源关系及非论文条目' (#42 ) from codex/acl-early into main
Reviewed-on: #42
2026-11-10 00:50:15 +08:00 · 2026-11-09 13:42:18 +00:00 · 2026-11-09 13:42:00 +00:00 · 2026-11-09 20:50:29 +08:00
14 changed files with 16202 additions and 6 deletions
@@ -50,6 +50,7 @@ jobs:
              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','cvf','--help'],cwd=outside,check=True)
              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','www-archive','--help'],cwd=outside,check=True)
              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--help'],cwd=outside,check=True)
+              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','aamas-proceedings','--year','2025','--snapshots',str(src/'tests/fixtures/aamas2025'),'--out',str(root/'installed-aamas2025')],cwd=outside,check=True)
              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','iclr-archive','--help'],cwd=outside,check=True)
              subprocess.run([python,'-I','-B','-m','paper_atlas','collect','acl','--help'],cwd=outside,check=True)
              subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','postgres','--help'],cwd=outside,check=True)
@@ -6,6 +6,8 @@ Paper Atlas 将 arXiv、AI 会议论文集及公开投稿决定整理到可追

 项目仍在建设中。候选范围包含 43 个会议（含 ACM MM、MLSys），尚未完成全部历史数据和最终去重复核。实际进度以数据库、来源清单和覆盖报告为准，而不是仓库文件数量。

+当前主动补漏从 **2020 年**起；2000 年起已发现的名单、缺口与待办保存在[补漏恢复清单](docs/coverage/backfill-handoff-2026-11-10.md)，下次从这里接续。已有旧数据保留，范围排除不算补齐。
+
 历史 ICLR 2014–2016 官方录用页由 `collect iclr-archive` 适配，保留主会/workshop及重复展示证据；运行和身份核验边界见 [历史档案说明](docs/coverage/iclr-archive-2014-2016.md)。

 ## 项目结构
@@ -68,7 +70,7 @@ paper-atlas collect colm --cache /data/paper-index/raw/http-cache \

 EWSL、EPS、ECP、EWSP 的[具体前身卷与完整快照要求](docs/coverage/historical-predecessors-2026-11-09.md)单独核验，保留原始系列及事件/出版年份；旧候选文件缺少 ECP/EWSP 时会明确提示，避免静默遗漏。

-AAMAS 的[官方论文集采集器](docs/aamas-proceedings.md)目前核验了 2012 版式，可从缓存或已校验快照采集标题、作者及页面提供的摘要；发表形式单独保存，全文/扩展摘要中未明确的投稿 track 保持待明确，不根据会话主题推定主会轨道。
+AAMAS 的[官方论文集采集器](docs/aamas-proceedings.md)支持已核验的 2012、2025 版式，可从缓存或已校验快照采集元数据。2025 年目录含 476 条非 keynote 出版物，保留完整作者和机构，HTML 未提供摘要时留空；发表形式与投稿 track 分开，不根据主题推定主会轨道或研究类型。

 `paper-atlas collect --help` 查看来源；`paper-atlas import --help` 查看导入方式。完整批次步骤见[使用指南](docs/usage.md)。

@@ -1,6 +1,14 @@
 # AAMAS 官方论文集元数据

-`paper_atlas.collectors.aamas` 是按需运行的采集适配器，通过统一入口 `collect aamas-proceedings` 使用。目前只支持已独立核验的 **AAMAS 2012** HTML 版式；其他年份需先核对官方目录和逐篇结构，再增加适配，不能仅替换年份。
+`paper_atlas.collectors.aamas` 是按需运行的采集适配器，通过统一入口 `collect aamas-proceedings` 使用。支持已独立核验的 **AAMAS 2012、2025** HTML 版式；其他年份需先核对官方目录和逐篇结构，再增加适配，不能仅替换年份。2025 解析单独封装在 `paper_atlas.collectors.aamas2025`。
+
+2025 版式使用[官方首页](https://www.ifaamas.org/Proceedings/aamas2025/)和[出版目录](https://www.ifaamas.org/Proceedings/aamas2025/forms/contents.htm)。原始目录共 479 条：3 keynote、260 research paper、148 extended abstract、12 Blue Sky、25 doctoral、22 Demo 栏目、9 JAAMAS。排除 keynote，输出 476 条；Demo 栏目中的 p3000 是明确标题的 COMP24 竞赛报告，发表年份仍为 2025，格式记 `competition_report`、投稿 track 留作 `proceedings_unspecified`，保留原 Demo 栏目证据。其余输出按 track 分组为 409 unspecified、12 blue_sky、25 doctoral、21 demo、9 jaamas。
+
+作者按逐篇 BR 行解析，斜体机构原文另存，保留 Unicode、顺序及完整名单。HTML 没有提供摘要或逐篇 DOI，故 `abstract=null`、标识字典为空；只保存 PDF 链接，不下载研究 PDF。不能把 Research Paper Track 的所有条目推成 main，不能把目录缺席推为 rejected/withdrawn，也不能把出版格式标签当研究类型。
+
+2025 用法：`paper-atlas collect aamas-proceedings --year 2025 --snapshots /data/review/aamas2025 --out /data/paper-index/collected/aamas-2025`；在线模式改用 `--cache /data/paper-index/cache/aamas2025`，只请求上述两个 HTML 页面。快照目录包含原始 `index.html`、`contents.html` 和 `receipts.json`，凭据必须列出各原始 URL、文件、字节数、SHA256、`http_status=200` 和带时区 `at`。已恢复的凭据需保留 `at_precision=filesystem_write_time_recovered` 等说明，不能伪造 HTTP 头或请求时间。2026-11-09 保存的两个页面通过哈希校验；独立录用清单 URL 返回 403，失败保留，不绕过或依赖其完成采集。
+
+2025 命令同样使用目录锁、临时目录、最后写入采集报告。五个 JSONL/manifest 均保持 `partial`，记录摘要缺失数量；它不连接数据库。首次采集需独立核验全部标题/作者/栏目和 source IDs，随后做 PostgreSQL 身份预检、必要的逐条版本审核、原生导入与独立验收。若失败且没有完成报告，应先检查原始响应和错误，再使用新输出目录重跑，不重放已提交批次。代码验证、原始目录盘点和采集成功均不代表已导入生产或最终去重完成。

 ## 来源与范围

@@ -1,5 +1,7 @@
 # 会议覆盖与公开决定核查

+**当前工作范围：2020 年起主动补漏。** 2000 年起已发现资料保留，恢复工作从 [补漏范围与恢复清单（2026-11-10）](backfill-handoff-2026-11-10.md) 开始；下文旧日期快照不改写，较早年份排除不算补齐。
+
 这是 2026-11-08 的首份核查基线，**完整逐年政策普查和最终去重仍未完成**。收录 43 个目标，不能把“有记录”当作“完整覆盖”。结构化清单见 [inventory-2026-11-08.json](inventory-2026-11-08.json)。清单中的 `policy_checks` 均限定所列年份和轨道；空数组表示尚未核实，绝不表示没有公开拒稿。

 ## 快照与复核
@@ -0,0 +1,52 @@
+# 补漏范围与恢复清单（2026-11-10）
+
+用户确定：**后续主动补漏从 2020 年起**。2000 年起已经发现的名单、缺口、资料组合和身份核查记录保留在项目中，供下次接着做；已有旧数据继续保留。停止补采的年份标为范围外或延期，不算作已补齐。
+
+本页是下一次工作的入口，结构化记录见 [backfill-handoff-2026-11-10.json](backfill-handoff-2026-11-10.json)。它保存 43 个会议的 2000–2026 已观察年份、缺年、官方目录文件入口、近期待办与恢复证据。旧的日期快照不改写，也不能当实时数据库。
+
+## 当前范围和年份差集
+
+依据生产只读快照 `2026-11-09 16:26:41.688509+00:00`，原全历史口径为 1266 个目标年份槽位：1208 已观察、43 个历史缺年、14 个 2026 缺年，以及 1 个 SysML 2018 非存档例外。原 135 个历史缺口中有 92 个通过数据补入解决；其余 43 个现在停止补采，不能因此宣称全部解决。
+
+- **2020–2025**：没有整年完全缺记录的目标槽位，但逐篇、track、摘要、公开负面决定仍不完整。
+- **2026**：NeurIPS、ECCV、RecSys、CIKM、ICDM、SDM、IROS、RSS、CoRL、ICAPS、AAMAS、ICASSP、ISMIR、ACM MM，共 14 个槽位。须逐个核查是否已经公开出版；不能将尚未发布的资料说成漏采。
+- **2000–2019 延期线索**：WWW 2000、ICAPS 2001 两个整年缺口；官方历届记录分别见 [WWW](www-editions-2026-11-08.json)、[ICAPS](icaps-editions-2026-11-08.json)。SysML 2018 是非存档首届，见 [MLSys](mlsys-editions-2026-11-08.json)，不能制造正式出版记录。
+- **2000 年前**：剩余 41 个历史缺年停止补采。旧来源文件继续保留作证据，不再作为当前工作队列。
+
+以上只判断目标会议年份是否至少有一条投稿记录，绝不等于目录完整、独立论文数量或最终去重验收。
+
+## 近期资料与恢复位置
+
+服务器采集根目录为 `/data/paper-index/collected`，本地证据备份为 `/Users/zzyang/Documents/Codex/2026-11-06/paper-index`。这些目录提供大体积原始文件，仓库 JSON 提供路径、状态和可用哈希；恢复前先验证文件存在、哈希和生产实际状态。
+
+| 资料 | 当前状态 | 下一步与证据入口 |
+| --- | --- | --- |
+| AAMAS 2026 | 官方目录 643 条，3 个 keynote 排除；640 个非 keynote 中含 1 个明确撤稿。尚未导入 | `aamas2026-primary-evidence-v1/` 保存官方 HTML、回执、逐项目录、审计和校验清单。八个标题差异/撤稿前缀条目需核查。先扩展正式 collector 并远程验证，再做新 PostgreSQL 身份预检。 |
+| CVPR 2026 Findings | 941 条摘要已核验，尚未导入 | `cvf-2026-verified/recovery-audit.json`；原 404 与 RVLF/FVLF 路径更正证据保留。先核对 manifests、track 和实体关系。 |
+| AAMAS 2025 | 476 个非 keynote 来源中，240 已验收、236 待身份核查 | `aamas2025-import225-v1/` 与 `aamas2025-import15-v1/` 是已消费批次，禁止重放。15 条最近验收报告哈希见 JSON。余下资料从当前身份候选与新只读基线继续。 |
+| RSS 2025 | 163 条目录中 162 已验收，`2025/p117` 待核查 | 保留 spinal/RMCC 关系问题；查看工作状态键 `rss2025_held7_primary_evidence_v1`，不能重跑已消费的 6/88/54/14 批次。 |
+| 近期公开负面决定首批 | 1625 条已验收：1624 rejected、1 withdrawn | `public-negative-postgresql-import-postcheck-v1.json`；仅该批次验收，不是公开拒稿全集，不要重放。 |
+
+AAMAS 2026 官方目录分区：research 338、extended abstracts 185、AAAI 19、Blue Sky 17、doctoral 34、demo 44、JAAMAS 3，另有 keynote 3。JAAMAS 是期刊展示关系，不能凭目录再造一次独立审稿录用。官方撤稿条目在 research 分区，不能把全目录都标为当前 accepted。
+
+AAAI 转投政策有年份冲突：[详细征稿页](https://cyprusconferences.org/aamas2026/call-for-papers-aaai-track/) 指 AAAI-26 rejected，[总览](https://cyprusconferences.org/aamas2026/calls/) 写 AAAI-25。保留两份证据；19 个 AAAI track 条目不能直接转成 19 条有原始 note 的拒稿决定。原投稿 ID、逐篇决定和版本关系仍需核实。
+
+## 2000–2019 已发现但延期的身份与名单工作
+
+- **AAMAS 2012**：官方 334 条，319 已独立验收、15 个身份条目保留待查。验收报告 `aamas2012-import-v1/postgresql-postcheck-v1.json`，哈希在 JSON；已验收 319 条及被替代的 316 条计划均不得重放。来源布局与用法见 [AAMAS 适配说明](../aamas-proceedings.md)。
+- **UAI 2008**：官方 programme 与归档资料确认了四篇独立论文，但部分原始 DBLP 键的标题、作者、页码、出版链接对应仍有冲突。JSON 保留四组候选、arXiv ID、官方题名及归档报告哈希。之前仅根据页码推断题名/作者错误的结论已撤回，未应用纠正；恢复时不得据此覆盖或合并。
+- **其他名单**：本目录全部 `*-editions-2026-11-08.json` 保留各会议官方年份及来源；[历史缺口](historical-gaps-2026-11-09.md)、[来源候选](historical-source-candidates-2026-11-09.json)、[来源范围](historical-source-scope-2026-11-09.md)、[主系列](historical-primary-series-2026-11-09.md)、[前身会议](historical-predecessors-2026-11-09.md)、[ICLR 档案](iclr-archive-2014-2016.md)继续留档。这些旧报告的“缺失”可能已被后续导入解决，恢复时必须对照当前状态，不能直接当可执行队列。
+
+## 公开拒稿资料不能遗漏
+
+结构化清单保留最近已查的政策 URL、限定年份和可用性结论。优先继续 UAI 2024–2026 的 opt-in 公开拒稿来源、EMNLP 2023 的公开集合范围及其他目标的逐年核查；平台使用 OpenReview 不能证明 rejected 公开。访问受限如实记录，不绕过 Challenge。
+
+保留 rejected、desk rejected、withdrawn、retracted、resubmit 的区别；评审建议不等于最终决定。COLM 2026 的不公开政策只适用于已核实年份，不能推广到其他会议。PaperCopilot 与原 OpenReview、镜像间有重叠，候选计数不能相加。论文先被 A 拒绝再被 B 录用时，两次投稿、决定、来源与证据全部保留。
+
+## 下次启动顺序
+
+1. 读取本页和 JSON，再核实服务器工作状态、明确的 systemd invocation 与终态报告。权威工作状态 `/Users/zzyang/Documents/Codex/2026-11-06/paper-index/work-state.json` 中部分旧字段会过期；以最新验收和实际查询为准。
+2. 当前主动范围为 2020 起，2000–2019 只恢复线索，不自动启动其采集。先检查新来源是否已导入，验收过的队列禁止重放。
+3. 使用现有 `foundry-postgres` 的 `foundry_data.research`。当前已验证 writer 为提交 `714f745bd5c3fbc742be1efae0b31ab37a55e21f` 的固定 release；恢复时重新确认，不使用旧 `/data/paper-index/code` 替代。服务配置仅由服务器受保护文件读取，不复制密码。
+4. 对待导入资料做新只读身份预检，处理稳定标识、短标题、作者/版本差异、track 与跨会历史，再生成小范围增量队列。每批独立验收，SQLite 只读备份保持保护。
+5. 编译、测试或 wheel 构建仅在已确认 `ces-control:host` runner 执行。代码按工作分支交付 PR，等待用户手动合并。范围内导入结束后仍须全库去重、近似标题/作者冲突、来源-年份-track、摘要、约束、实际全文查询验收；不能用单批导入或 CI 成功替代。
@@ -136,6 +136,9 @@ def read_pages(cache=None, snapshots=None):


 def _collect(year, out, *, cache=None, snapshots=None):
+    if year == 2025:
+        from paper_atlas.collectors.aamas2025 import collect as collect_2025
+        return collect_2025(out, cache=cache, snapshots=snapshots)
    if year != 2012:
        raise ValueError('Only the independently reviewed AAMAS 2012 layout is supported')
    pages, receipts = read_pages(cache, snapshots)
@@ -190,8 +193,8 @@ def _collect(year, out, *, cache=None, snapshots=None):


 def collect(year, out, *, cache=None, snapshots=None):
-    if year != 2012:
-        raise ValueError('Only the independently reviewed AAMAS 2012 layout is supported')
+    if year not in (2012, 2025):
+        raise ValueError('Only independently reviewed AAMAS 2012/2025 layouts are supported')
    out = Path(out)
    out.mkdir(parents=True, exist_ok=True)
    with (out/'aamas.collect.lock').open('a') as lock:
@@ -201,7 +204,7 @@ def collect(year, out, *, cache=None, snapshots=None):

 def main(argv=None, *, prog=None):
    parser = argparse.ArgumentParser(prog=prog)
-    parser.add_argument('--year', type=int, choices=[2012], required=True)
+    parser.add_argument('--year', type=int, choices=[2012, 2025], required=True)
    parser.add_argument('--out', required=True)
    source = parser.add_mutually_exclusive_group(required=True)
    source.add_argument('--cache')
@@ -0,0 +1,183 @@
+"""Reviewed AAMAS 2025 publisher TOC; HTML metadata only, never paper PDFs."""
+from collections import Counter, defaultdict
+import hashlib
+import json
+from pathlib import Path
+import re
+import tempfile
+
+from paper_atlas.db.submissions import timestamp
+from paper_atlas.io.html import Document, Node, clean
+from paper_atlas.io.http import fetch
+
+ROOT = 'https://www.ifaamas.org/Proceedings/aamas2025/'
+URLS = {'index': ROOT, 'contents': ROOT+'forms/contents.htm'}
+TITLE = 'AAMAS 2025: Proceedings of the 2025 International Conference on Autonomous Agents and Multiagent Systems'
+SECTIONS = {
+    '1': ('Keynote Talks', 'keynote', 'keynote', 3),
+    '2': ('Research Paper Track', 'full_paper', 'proceedings_unspecified', 260),
+    '3': ('Extended Abstracts', 'extended_abstract', 'proceedings_unspecified', 148),
+    '4': ('Blue Sky Ideas', 'blue_sky_paper', 'blue_sky', 12),
+    '5': ('Doctoral Consortium', 'doctoral_abstract', 'doctoral', 25),
+    '6': ('Demo Track', 'demo', 'demo', 22),
+    '7': ('JAAMAS Track', 'journal_presentation', 'jaamas', 9),
+}
+COMPETITION_TITLE = '[COMP24] The Automated Negotiating Agents Competition (ANAC) 2024 Challenges and Results'
+PAPER = re.compile(r'\.\./pdfs/p([0-9]+)\.pdf')
+
+
+def one(items, label):
+    if len(items) != 1:
+        raise ValueError('Expected one '+label)
+    return items[0]
+
+
+def parse_contents(html, evidence):
+    doc = Document(html)
+    if [clean(n.text()) for n in doc.root.walk() if n.tag == 'title'] != [TITLE]:
+        raise ValueError('Unexpected AAMAS proceedings title/year')
+    for paragraph in doc.root.walk():
+        if paragraph.tag != 'p' or any(n is not paragraph and n.tag == 'p' for n in paragraph.walk()):
+            continue
+        anchors = [n.attrs['name'] for n in paragraph.walk() if n.tag == 'a' and n.attrs.get('name') in SECTIONS]
+        if anchors and (len(anchors) != 1 or clean(paragraph.text()) != SECTIONS[anchors[0]][0]):
+            raise ValueError('Publisher section heading differs from reviewed layout')
+    section = None
+    headers = []
+    records, excluded, counts, seen = [], [], Counter(), set()
+    for node in doc.root.walk():
+        if node.tag == 'a' and node.attrs.get('name') in SECTIONS:
+            section = node.attrs['name']
+            headers.append(section)
+        if node.tag != 'p' or any(n is not node and n.tag == 'p' for n in node.walk()):
+            continue
+        links = [n for n in node.walk() if n.tag == 'a' and PAPER.fullmatch(n.attrs.get('href', ''))]
+        if not links:
+            continue
+        link = one(links, 'paper link per paragraph')
+        if section not in SECTIONS:
+            raise ValueError('Paper outside a reviewed publisher section')
+        page = int(PAPER.fullmatch(link.attrs['href'])[1])
+        if page in seen:
+            raise ValueError('Repeated publisher paper/page ID')
+        seen.add(page)
+        title = clean(link.text())
+        if not title:
+            raise ValueError('Empty paper title')
+        # Names are outside italicized affiliations and separated by explicit BRs.
+        font = one([n for n in node.walk() if n.tag == 'font' and n.attrs.get('color', '').lower() == '#222222'], 'ordered author block')
+        authors, affiliations = [], []
+        name, affiliation = '', ''
+        def flush():
+            nonlocal name, affiliation
+            if clean(name) or clean(affiliation):
+                if not clean(name):
+                    raise ValueError('Affiliation without author')
+                authors.append(clean(name)); affiliations.append(clean(affiliation))
+            name, affiliation = '', ''
+        for child in font.children:
+            if isinstance(child, str):
+                name += child
+            elif child.tag == 'br':
+                flush()
+            elif child.tag == 'i':
+                affiliation += child.text()
+            else:
+                raise ValueError('Unreviewed author markup: '+child.tag)
+        flush()
+        if not authors:
+            raise ValueError('Empty author list')
+        heading, publication_format, track, _ = SECTIONS[section]
+        counts[section] += 1
+        if section == '1':
+            excluded.append({'publisher_page': page, 'title': title, 'authors': authors, 'reason': 'publisher keynote talk, not a research-paper submission'})
+            continue
+        note = 'Explicit publisher section; section topics are not research types.'
+        if track == 'proceedings_unspecified':
+            note = 'Published format is known; individual main/special submission track is not provided.'
+        if page == 3000:
+            if section != '6' or title != COMPETITION_TITLE:
+                raise ValueError('Competition report differs from reviewed publisher entry')
+            publication_format, track = 'competition_report', 'proceedings_unspecified'
+            note = 'Publisher places this competition report in Demo Track; no individual demonstration submission is inferred.'
+        records.append({
+            'source': 'aamas_proceedings', 'external_id': '2025/p'+str(page),
+            'url': URLS['contents']+'#'+section, 'title': title, 'abstract': None,
+            'authors': authors, 'identifiers': {}, 'venue': 'AAMAS', 'venue_aliases': [],
+            'year': 2025, 'track': track, 'status': 'accepted',
+            'observed_at': evidence['observed_at'],
+            'raw_decision': 'listed in official AAMAS 2025 published proceedings '+heading,
+            'tags': [{'namespace': 'aamas_proceedings:publication_format', 'value': publication_format}],
+            'metadata': {'publisher_page': page, 'paper_url': ROOT+'pdfs/p'+str(page)+'.pdf',
+                         'section_url': URLS['contents'], 'section_anchor': section,
+                         'section_heading': heading, 'publication_format': publication_format,
+                         'author_affiliations': affiliations, 'source_evidence': evidence,
+                         'track_scope_note': note, 'decision_date_precision': 'not_supplied',
+                         'missing_abstract_reason': 'Publisher HTML supplies no abstract; research PDF not downloaded.',
+                         'publication_evidence': 'Published proceedings entry, not an original decision note.'}})
+    if headers != list(SECTIONS) or dict(counts) != {k: v[3] for k, v in SECTIONS.items()}:
+        raise ValueError('Publisher section inventory differs from reviewed 2025 layout')
+    if len(records) != 476 or len(excluded) != 3 or sum(r['metadata']['publication_format'] == 'competition_report' for r in records) != 1:
+        raise ValueError('Unexpected publication partition')
+    return records, excluded
+
+
+def read_pages(cache, snapshots):
+    if bool(cache) == bool(snapshots):
+        raise ValueError('Choose exactly one cache or frozen snapshot directory')
+    frozen = json.loads((Path(snapshots)/'receipts.json').read_text())['receipts'] if snapshots else []
+    pages = {}
+    for key, url in URLS.items():
+        if snapshots:
+            r = one([r for r in frozen if r['url'] == url], 'frozen response receipt')
+            filename = Path(r['file']); base = Path(snapshots).resolve()
+            if filename.is_absolute() or '..' in filename.parts or not (base/filename).resolve().is_relative_to(base) or r.get('http_status') != 200:
+                raise ValueError('Unsafe or unsuccessful publisher response')
+            data = (base/filename).read_bytes()
+            if len(data) > 8*1024**2 or len(data) != r['bytes'] or hashlib.sha256(data).hexdigest() != r['sha256']:
+                raise ValueError('Frozen publisher response checksum/size mismatch')
+            observed = r['at']
+        else:
+            data = fetch(url, cache, 8*1024**2, delay=1)
+            r = json.loads((Path(cache)/hashlib.sha256(url.encode()).hexdigest()).with_suffix('.json').read_text())
+            observed = r['retrieved_at']
+        timestamp(observed)
+        pages[key] = data, {'url': url, 'sha256': hashlib.sha256(data).hexdigest(), 'bytes': len(data),
+                            'file': 'evidence/'+hashlib.sha256(data).hexdigest()+'.html',
+                            'observed_at': observed, 'http_status': 200, 'response_receipt': r}
+    return pages
+
+
+def collect(out, *, cache=None, snapshots=None):
+    pages = read_pages(cache, snapshots)
+    if [clean(n.text()) for n in Document(pages['index'][0].decode('utf-8')).root.walk() if n.tag == 'title'] != [TITLE]:
+        raise ValueError('Unexpected AAMAS 2025 introduction')
+    records, excluded = parse_contents(pages['contents'][0].decode('utf-8'), pages['contents'][1])
+    grouped = defaultdict(list)
+    for row in records: grouped[row['track']].append(row)
+    out = Path(out); out.mkdir(parents=True, exist_ok=True); files = []
+    with tempfile.TemporaryDirectory(prefix='.aamas2025-', dir=out) as temporary:
+        stage = Path(temporary); (stage/'evidence').mkdir()
+        for data, e in pages.values(): (stage/e['file']).write_bytes(data)
+        for track, group in sorted(grouped.items()):
+            p = stage/f'aamas-proceedings-2025-{track}.jsonl'
+            p.write_text(''.join(json.dumps(r, ensure_ascii=False)+'\n' for r in group))
+            m = {'source': 'aamas_proceedings', 'parser_version': 2, 'venue': 'AAMAS', 'year': 2025,
+                 'track': track, 'count': len(group), 'file': str(out/p.name), 'file_sha256': hashlib.sha256(p.read_bytes()).hexdigest(),
+                 'list_url': URLS['contents'], 'status': 'partial', 'missing_abstract_count': len(group),
+                 'publication_format_counts': dict(Counter(r['metadata']['publication_format'] for r in group)),
+                 'note': 'Complete published TOC partition; HTML abstracts absent, identities and submission-track scope require independent review.'}
+            p.with_suffix('.manifest.json').write_text(json.dumps(m, ensure_ascii=False, indent=2)+'\n'); files.append(m)
+        result = {'source': 'aamas_proceedings', 'year': 2025, 'finished_at': timestamp(),
+                  'status': 'collected_pending_identity_review', 'records': len(records), 'files': files,
+                  'excluded_publications': excluded, 'source_receipts': [e for _, e in pages.values()],
+                  'publication_format_counts': dict(Counter(r['metadata']['publication_format'] for r in records)),
+                  'database_written': False, 'pdfs_downloaded': False,
+                  'note': 'Neither absence from proceedings nor aggregate rejection/withdrawal counts identify an individual negative decision.'}
+        report = 'aamas-proceedings-2025.json'; (stage/report).write_text(json.dumps(result, ensure_ascii=False, indent=2)+'\n')
+        (out/'evidence').mkdir(exist_ok=True)
+        for p in (stage/'evidence').iterdir(): p.replace(out/'evidence'/p.name)
+        for p in stage.iterdir():
+            if p.is_file() and p.name != report: p.replace(out/p.name)
+        (stage/report).replace(out/report)
+    return result
@@ -0,0 +1,54 @@
+<HTML><HEAD><TITLE>AAMAS 2025: Proceedings of the 2025 International Conference on Autonomous Agents and Multiagent Systems</TITLE>
+<META http-equiv=Content-Type content="text/html; charset=utf-8"><LINK 
+href="forms/index_files/factemplate.css" type=text/css rel=stylesheet>
+<META content="MSHTML 6.00.2900.3562" name=GENERATOR></HEAD>
+<BODY bgcolor="#010101" bgproperties="fixed"><DIV style="TEXT-ALIGN: center" >
+<table width="724" border="0" align="center" cellpadding="0" cellspacing="0" valign="top">
+  <td colspan="2" align="center" valign="top" bgcolor="#FFFFFF"><img src="forms/index_files/aamas25.png" width="750" height="300" alt=""/></td>
+  </tr>
+  <tr bgcolor="#FFFFFF"> 
+    <td width="125" align="center" valign="top" bgcolor="#FFFFFF"><table width="125" border="1" align="center" cellpadding="1" cellspacing="1" bordercolor="#444444" style="margin-bottom: 2">
+        <tr> 
+          <td bgcolor="#FFFFFF"><p style="text-indent: 0; margin-left: 6; margin-right: 6; margin-top: 5; margin-bottom: 4" align="left"><A href="forms/index.htm">Main 
+              Page</A></p></td>
+        </tr>
+        <tr> 
+          <td bgcolor="#FFFFFF"><p style="text-indent: 0; margin-left: 6; margin-right: 6; margin-top: 5; margin-bottom: 4" align="left"><A href="forms/contents.htm">Table 
+              of Contents</A></p></td>
+        </tr>
+      </table></td>
+    <td width="599"><p style="text-indent: 0; margin-left: 12; margin-right: 14; margin-top: 12; margin-bottom: 12" align="left"><font color="#010101" size="4" face="Arial, Helvetica, sans-serif"><strong>Welcome 
+        to the Proceedings of AAMAS-2025</strong></font></p>
+      <p style="text-indent: 0; margin-left: 12; margin-right: 14; margin-top: 12; margin-bottom: 12;" align="left"><font color="#333333" size="2" face="Times New Roman, Times, serif">These 
+        are the proceedings of the 24th International Conference on Autonomous 
+        Agents and Multiagent Systems (AAMAS-2025). They are published by the 
+        International Foundation for Autonomous Agents and Multiagent Systems 
+    (IFAAMAS).</font></p></td>
+  </tr>
+  <tr bgcolor="#FFFFFF"> 
+    <td align="center" valign="middle" bgcolor="#FFFFFF">&nbsp;</td>
+    <td><table width="96%" border="0" cellspacing="2" cellpadding="2">
+        <tr align="left" valign="top"> 
+          <td> <p style="text-indent: 0; margin-left: 12; margin-right: 6; margin-top: 0; margin-bottom: 6;" align="left"><font color="#333333" face="Times New Roman, Times, serif"><font size="2"><strong>ISBN: 979-8-4007-1426-9</strong></font></font></p></td>
+          <td> <p style="text-indent: 0; margin-left: 12; margin-right: 6; margin-top: 0; margin-bottom: 6;" align="left"><font color="#333333" face="Times New Roman, Times, serif"></font></p></td>
+        </tr>
+        <tr> 
+          <td width="42%"><div align="left"><img src="forms/index_files/ifaamas.jpg" width="256" height="104"></div></td>
+          <td width="58%"><div align="center"><img src="forms/index_files/ACM-In-Cooperation_medium.jpg" width="285" height="86"></div></td>
+        </tr>
+      </table>
+      <p style="text-indent: 0; margin-left: 12; margin-right: 14; margin-top: 12; margin-bottom: 12;" align="left"><font color="#333333" size="2" face="Times New Roman, Times, serif">Copyright 
+        &copy; 2025 by International Foundation for Autonomous Agents and Multiagent 
+        Systems (IFAAMAS). Permission to make digital or hard copies of portions 
+        of this work for personal or classroom use is granted without fee provided 
+        that copies are not made or distributed for profit or commercial advantage 
+        and that copies bear this notice and the full citation on the first page. 
+        Copyright for components of this work owned by others than IFAAMAS must 
+        be honored. Abstracting with credit is permitted. To copy otherwise, to 
+        republish, to post on servers or to redistribute to lists, requires prior 
+    specific permission and/or a fee. </font></p></td>
+  </tr>
+</table>
+  <p>&nbsp;</p>
+
+</BODY></HTML>
@@ -0,0 +1,38 @@
+{
+  "status": "two_successful_official_metadata_pages_recovered_third403_not_retried",
+  "receipts": [
+    {
+      "file": "index.html",
+      "url": "https://www.ifaamas.org/Proceedings/aamas2025/",
+      "bytes": 4107,
+      "sha256": "850b7825c66d0147ac8a8bbf03a554590e4a9e8f4b5c576e043999bbd43317f4",
+      "at": "2026-11-09T12:31:49.347577+00:00",
+      "at_precision": "filesystem_write_time_recovered",
+      "http_status": 200,
+      "content_type": "text/html",
+      "receipt_recovered": true,
+      "scope": "Collectorasserted200/htmlbeforewrite;request3acceptedprogram403;noheaders/finalredirectURLpersisted;no refetch or rawbytesmutation."
+    },
+    {
+      "file": "contents.html",
+      "url": "https://www.ifaamas.org/Proceedings/aamas2025/forms/contents.htm",
+      "bytes": 370650,
+      "sha256": "732847b51972f34c721a49d8bd963a80a029a48f3ec08022c9a0d5ddc2945549",
+      "at": "2026-11-09T12:31:52.006642+00:00",
+      "at_precision": "filesystem_write_time_recovered",
+      "http_status": 200,
+      "content_type": "text/html",
+      "receipt_recovered": true,
+      "scope": "Collectorasserted200/htmlbeforewrite;request3acceptedprogram403;noheaders/finalredirectURLpersisted;no refetch or rawbytesmutation."
+    }
+  ],
+  "failed_url": "https://aamas2025.org/index.php/conference/program/accepted-papers/",
+  "not_attempted_url": "https://aamas2025.org/index.php/conference/calls/submission-instructions-main-technical-track/",
+  "original_error": {
+    "error": "HTTP Error 403: Forbidden",
+    "type": "HTTPError",
+    "downloaded": 2
+  },
+  "research_fulltext_downloaded": false,
+  "database_written": false
+}
@@ -0,0 +1,85 @@
+import hashlib
+import json
+from pathlib import Path
+import tempfile
+import unittest
+from unittest.mock import patch
+
+from paper_atlas.collectors import aamas, aamas2025
+
+
+FIXTURE = Path(__file__).parent/'fixtures/aamas2025'
+
+
+class Aamas2025Tests(unittest.TestCase):
+    def parse(self, html=None):
+        return aamas2025.parse_contents(html or (FIXTURE/'contents.html').read_text(),
+            {'observed_at': '2026-11-09T12:31:50+00:00', 'url': aamas2025.URLS['contents']})
+
+    def test_real_toc_preserves_every_title_author_and_affiliation(self):
+        expected = json.loads((FIXTURE/'expected-authors.json').read_text())
+        self.assertEqual(hashlib.sha256((FIXTURE/'contents.html').read_bytes()).hexdigest(), expected['source_sha256'])
+        rows, excluded = self.parse()
+        self.assertEqual((len(rows), len(excluded)), (476, 3))
+        by_url = {r['paper_url']: r for r in expected['entries']}
+        for row in rows:
+            original = by_url[row['metadata']['paper_url']]
+            self.assertEqual(row['title'], original['title'])
+            self.assertEqual(row['authors'], [a['name'] for a in original['authors']])
+            self.assertEqual(row['metadata']['author_affiliations'], [a['affiliation'] for a in original['authors']])
+            self.assertIsNone(row['abstract'])
+            self.assertEqual(row['identifiers'], {})
+            self.assertEqual(row['status'], 'accepted')
+            self.assertNotIn('decision_date', row)
+            self.assertTrue(all(t['namespace'] == 'aamas_proceedings:publication_format' for t in row['tags']))
+
+    def test_publication_format_and_track_are_separate(self):
+        rows, excluded = self.parse()
+        groups = {k: sum(r['track'] == k for r in rows) for k in {r['track'] for r in rows}}
+        self.assertEqual(groups, {'proceedings_unspecified': 409, 'blue_sky': 12, 'doctoral': 25, 'demo': 21, 'jaamas': 9})
+        competition = next(r for r in rows if r['external_id'] == '2025/p3000')
+        self.assertEqual(competition['year'], 2025)
+        self.assertEqual(competition['metadata']['publication_format'], 'competition_report')
+        self.assertEqual(competition['metadata']['section_heading'], 'Demo Track')
+        self.assertEqual(competition['track'], 'proceedings_unspecified')
+        self.assertEqual(len({r['external_id'] for r in rows}), 476)
+        self.assertEqual([r['publisher_page'] for r in excluded], [1, 3, 4])
+
+    def test_refuses_changed_edition_inventory_or_author_markup(self):
+        original = (FIXTURE/'contents.html').read_text()
+        for html in [original.replace('AAMAS 2025:', 'AAMAS 2026:', 1),
+                     original.replace('../pdfs/p5.pdf', '../pdfs/p14.pdf', 1),
+                     original.replace('Paper Track</font>', 'Workshop Track</font>'),
+                     original.replace('Amir Abolfazli', '', 1),
+                     original.replace('[COMP24]', '[CHANGED]', 1)]:
+            with self.subTest(change=html[:60]):
+                with self.assertRaises(ValueError): self.parse(html)
+
+    def test_frozen_collection_is_partial_and_never_fetches_pdfs(self):
+        with tempfile.TemporaryDirectory() as tmp, patch.object(aamas2025, 'fetch', side_effect=AssertionError('No HTTP in frozen mode')):
+            out = Path(tmp)/'out'
+            report = aamas.collect(2025, out, snapshots=FIXTURE)
+            self.assertEqual(report['records'], 476)
+            self.assertFalse(report['database_written'])
+            self.assertFalse(report['pdfs_downloaded'])
+            self.assertEqual(len(report['files']), 5)
+            for m in report['files']:
+                self.assertEqual(m['status'], 'partial')
+                self.assertEqual(m['missing_abstract_count'], m['count'])
+                self.assertEqual(hashlib.sha256(Path(m['file']).read_bytes()).hexdigest(), m['file_sha256'])
+            receipt = report['source_receipts'][0]['response_receipt']
+            self.assertEqual(receipt['at_precision'], 'filesystem_write_time_recovered')
+            self.assertTrue((out/'aamas-proceedings-2025.json').is_file())
+
+    def test_rejects_tampered_or_unsafe_frozen_response(self):
+        import shutil
+        for mode in ['tamper', 'path', 'status']:
+            with self.subTest(mode=mode), tempfile.TemporaryDirectory() as tmp:
+                root = Path(tmp); source = root/'source'; shutil.copytree(FIXTURE, source)
+                p = source/'receipts.json'; r = json.loads(p.read_text())
+                if mode == 'tamper': (source/'contents.html').write_text('bad')
+                elif mode == 'path': r['receipts'][0]['file'] = '../index.html'
+                else: r['receipts'][0]['http_status'] = 403
+                p.write_text(json.dumps(r))
+                with self.assertRaises(ValueError): aamas.collect(2025, root/'out', snapshots=source)
+                self.assertFalse((root/'out/aamas-proceedings-2025.json').exists())
@@ -99,7 +99,7 @@ class AamasProceedingsTests(unittest.TestCase):
                aamas.collect(2012, root/'out', snapshots=source)
            self.assertFalse((root/'out/aamas-proceedings-2012.json').exists())
            with self.assertRaises(ValueError):
-                aamas.collect(2025, root/'future', snapshots=source)
+                aamas.collect(2026, root/'future', snapshots=source)
            self.assertFalse((root/'future').exists())

    def test_ids_colliding_across_sections_do_not_produce_a_completed_collection(self):
Author	SHA1	Message	Date
codex-bot	25ed4559ca	docs: set 2020 backfill scope and preserve recovery inventory since 2000 Metadata integrity checks / metadata (push) Successful in 2m9s Details	2026-11-10 00:50:15 +08:00
Zayden-Jung	b226b3b98b	Merge pull request '限定ACL1978与TINLAP卷的来源关系及非论文条目' (#42 ) from codex/acl-early into main Metadata integrity checks / metadata (push) Successful in 2m18s Details Reviewed-on: #42	2026-11-09 13:42:18 +00:00
Zayden-Jung	05ada8daf8	Merge pull request '支持 AAMAS 2025 官方出版目录元数据采集' (#44 ) from codex/aamas2025 into main Metadata integrity checks / metadata (push) Successful in 3m20s Details Reviewed-on: #44	2026-11-09 13:42:00 +00:00
codex-bot	13408869c2	Add reviewed AAMAS 2025 publisher metadata adapter Metadata integrity checks / metadata (push) Successful in 2m11s Details	2026-11-09 20:50:29 +08:00