paper-atlas/docs/www-archive.md

Early WWW archive collector

paper-atlas collect www-archive is a maintained metadata adapter. It reads official IW3C2 archive HTML, keeps exact source hashes, and separates importable rows from program entries that still need review. It fetches no PDF, follows no paper-body or reference link, and makes no database connection.

For a fresh source snapshot on the server:

paper-atlas collect www-archive \
  --http-cache /data/paper-index/cache/www-archive \
  --source-cache /data/paper-index/collected/www-archive-source-v2 \
  --out /data/paper-index/collected/www-archive-normalized-v2

For reproducible offline replay of the already verified snapshot:

paper-atlas collect www-archive \
  --source-cache /data/paper-index/collected/www-early-archive-source-v1 \
  --source-cache-sha256 c9388ccef44486e0684c9ee9e8620a465cdc4574e6bc9ea9391d743e01a3312c \
  --out /data/paper-index/collected/www-archive-normalized-v1

Use a new output directory for another batch. Existing snapshots and normalized batches are never overwritten. The manifest hash binds every source URL, byte count, response hash and cache filename. Plain files must remain directly within the cache; symlinks, escaping paths, duplicate URLs, non-200 responses and altered bytes fail. The command uses UTF-8 where valid and explicitly records an ISO-8859-1 fallback for older pages; original bytes remain authoritative for encoding corrections.

Evidence and outputs

• WWW3 Spring 1995 papers use event_key=www-1995-spring, track=main. The author, session and abstract indexes must have the same nonempty entry-ID set. The ID is local to this exact event and proceedings section, never a DOI or OpenReview ID.
• Posters remain track=poster. An abstract without a linked author-index entry stays in review even when another index has a placeholder # link for it.
• Paper URLs must agree between author and session indexes. A malformed whitespace URL in the abstract list may use the exact event/entry-ID match only when both independent indexes corroborate the same clean URL. The original href is retained.
• Canonical names come from the explicitly labeled author index. The source lists them alphabetically; this is not original byline order. Both the raw index names and the full abstract-page byline/affiliation text remain in metadata. No person ID or cross-source identity is inferred from a name. Conflicting titles or missing abstracts stay in review. Topic keywords are not treated as an abstract.
• WWW1 1994 formal presentations and WWW2 1994 mixed categories remain review candidates. The first list contains uncertain business-track annotations and a count discrepancy; the second contains panels, tutorials, products and other activities. Neither presentation listing nor a missing full-text link establishes a regular-paper acceptance or a rejection. Raw category/title/name claims are kept.

The main and poster JSONL files have independent manifests with explicit event key, track, count and SHA256. source_listing_count records the full recognized abstract listing, while held_entry_count records excluded entries; neither is a ready-row count. Their status stays partial: a verified linked subset does not prove complete event coverage. review-candidates.jsonl contains nested candidates and reasons, with no importable decision status. It is excluded from import-queue.json. report.json accounts for ready/held records and every source used. Newlines delimit JSONL; Unicode U+2028/U+2029 inside metadata are not line breaks. The output is published only after all required source and scope checks pass.

Before importing

This independent collector branch starts from main; it does not include or silently install the event-aware writer. Its output requires the reviewed integrated runtime from PR31 (or a main revision containing that implementation). The production event schema migration has been independently accepted, preserving existing edition1212 and its 59 Fall submissions; that is not an import of the Spring proceedings.

Inspect held entries and original author names, preflight every ready row against current PostgreSQL stable identifiers and candidate identities, and verify the source/year/track/event coverage before running import records. When a ready subset excludes held entries, preserve the independently known listing total in the existing incremental-coverage protocol: bind a freshly checked parent count and expected count, and set the full expected count from this manifest. A new source uses previous count 0 and previous expected count null; an existing source requires its actual current values. These live parent values are not guessed by the offline collector. Do not automatically run this queue against an older writer. Preserve extended versions and independent submission/decision evidence, and keep unresolved identities held. A normalization report or passing tests alone do not establish production import, complete historical coverage or final global deduplication.

Initial fixed-cache replay (64ced7a, CI1057): 38 main papers and38 posters ready; 372 entries held (50 first-conference fields,307 mixed program entries,15 posters). Independent report44dfd070 verified source hashes,counts and author evidence, finding one existing exact-title candidate. That publication is explicitly linked to arXiv1811.12161 by its WWW1995/journal-reference declaration; arXiv1810.07232 is a different1997 invited chapter and remains separate despite the overlap note. This is a preparation observation, not evidence of completed production import or final global deduplication.

以单篇官方页面核实待处理 poster

作者索引漏链接、摘要目录使用短标题或标题存在拼写差异时，不放宽全局标题匹配。可以先取得单篇官方 HTML，人工核对 H1、署名区与原目录同一 entry URL，再提供带 SHA256 的逐条核查文件：

paper-atlas collect www-archive \
  --source-cache /data/paper-index/collected/www-poster-reviewed-source-v2 \
  --source-cache-sha256 8a65562f3490497cada2919ec77827134beabd5ec9c3690dec3625e2318aa54e \
  --reviewed-posters docs/coverage/www-1995-posters-reviewed-2026-11-09.json \
  --reviewed-posters-sha256 <核查文件的SHA256> \
  --out <新的输出目录>

1995 年15条poster的核查输入 绑定61页来源快照：原46页和补取的15页单篇HTML。程序验证输入文件及快照哈希、单篇页面哈希、完全一致的 canonical H1、原摘要目录中的同一URL/entry ID及作者证据，不通过相似标题猜身份。规范标题采用单篇H1，摘要目录标题及作者索引标题均在 metadata 中保留。

author_source=canonical_byline 表示名字出现在H1之后、摘要/关键词或下一节标题之前的署名区域，由逐条核查明确选出，不从全文引用或机构猜作者；保留原始缩写、顺序、重音和拼写。只有 author_source=author_index 时使用原作者索引顺序并逐项核对；05的单篇页面写的是联系人，作者身份使用原作者索引证据，不由联系人角色推断。核查输入是审阅后的数据声明，程序的字符串/哈希检查不代替人工识别署名语义。

38和05的来源没有可明确引用的摘要。保留 abstract=null 和逐条缺失原因，manifest准确报告 missing_abstract_count，继续标partial；正文、联系方式及PostScript链接不当作摘要，也不下载正文。缺摘要不等于拒稿或未发表。本批输出仍包含原先已验收记录，正式导入前必须只取新增external ID，检查实时覆盖父记录并按incremental模式准备，不能直接重导入全部输出。