Compare commits

..

2 Commits

Author SHA1 Message Date
codex-bot 8e82742900 Recognize explicit ICLR workshop decisions with scope guards
Metadata integrity checks / metadata (push) Successful in 2m40s
2026-11-08 04:18:09 +08:00
codex-bot f327861a00 Hold untitled public assertions and retain explicit withdrawals
Metadata integrity checks / metadata (push) Successful in 1m17s
2026-11-08 03:23:25 +08:00
13 changed files with 167 additions and 496 deletions
-2
View File
@@ -53,10 +53,8 @@ jobs:
subprocess.run([python,'-I','-B','-m','paper_atlas','import','arxiv','--help'],cwd=outside,check=True)
subprocess.run([python,'-I','-B','-m','paper_atlas','maintain','backfill','--help'],cwd=outside,check=True)
subprocess.run([python,'-I','-B','-m','paper_atlas','audit','public-openreview','--help'],cwd=outside,check=True)
subprocess.run([python,'-I','-B','-m','paper_atlas','audit','histories','--help'],cwd=outside,check=True)
subprocess.run([python,'-I','-B','-c','from paper_atlas.postgres.writer import PostgresStore, WriteConnection'],cwd=outside,check=True)
smoke = "from paper_atlas.db.store import Store; from paper_atlas.config import DEFAULT_REGISTRY; import json; assert {'COLM','MM','MLSys'} <= {v['acronym'] for v in json.loads(DEFAULT_REGISTRY.read_text())['venues']}; s=Store('installed.sqlite'); s.conn.close()"
subprocess.run([python,'-I','-B','-c',smoke],cwd=outside,check=True)
subprocess.run([python,'-I','-B','-m','paper_atlas','query','--db','installed.sqlite','--stats'],cwd=outside,check=True)
subprocess.run([python,'-I','-B','-m','paper_atlas','audit','histories','--db','installed.sqlite','--examples','0'],cwd=outside,check=True)
PY
+6 -4
View File
@@ -93,6 +93,12 @@ paper-atlas audit public-openreview --db /data/paper-index/papers.sqlite \
输出按会议、届次及 main、position、datasets_benchmarks 分文件。workshop 默认进入单独的 held 清单;有明确 workshop 名称时可通过 `--include-workshops` 单列采集,不合并到主会。Retraction、未映射的轨道或负面标签也保留在 `held-records.jsonl`,不强改为 Reject 或 Withdrawn。当前决定表没有独立 retracted 状态,因此撤回发表记录仍需后续专门建模。
`parser_version=2` 还会将明确负面标签但标题缺失、空白或 null 的记录放入 `held-records.jsonl`,保留原始状态、forum ID、固定来源版本和行号;不制造标题或导入不可识别的论文,也不让这一条缺失阻断其他可用记录。held 文件哈希与各原因数量保存在 inventory;非文本标题、错误身份、冲突届次和哈希不符仍使整批失败。旧年 ICLR 镜像的明确 `Withdraw` 标签归为 `withdrawn`,`Workshop`、`Poster Workshop`、`Oral Workshop` 均不作为拒稿。采集文件与导入主库仍是两个独立步骤。
`parser_version=3` 增加五组已核实身份的 ICLR workshop 标签:2024 ME-FoMo / GenAI4DM 的 Withdrawn Submission、2025 AgenticAI 的 Reject / LMRL 的 Withdrawn Submission,以及 2026 LLM Reasoning 的 Withdrawn Submission。仅在 `--include-workshops` 下输出,并分别保存 `workshop_me_fomo`、`workshop_genai4dm`、`workshop_agenticai`、`workshop_lmrl`、`workshop_llm_reasoning`;默认仍暂存。标签必须与原文件会议、年份和原始 workshop track 一致,错年/错会议使采集失败,错轨道单独暂存,未知标签继续待核查,不用宽泛的 Reject/Withdraw 正则猜测。
[OpenReview 官方 ICLR 活动目录](https://openreview.net/venue?id=ICLR.cc) 确认上述独立 workshop 身份;[ME-FoMo 2024](https://sites.google.com/view/me-fomo2024/call-for-papers)、[GenAI4DM 2024](https://sites.google.com/view/genai4dm-iclr2024)、[AgenticAI 2025](https://iclragenticai.github.io/) 和 [LLM Reasoning 2026](https://sites.google.com/view/iclr-2026-llmreasoning) 的活动网站也提供对应投稿入口。输出在 metadata 保存 `workshop_venueid` 和身份目录证据,决定仍只标为固定镜像的公开分类断言,保留原始标签、forum、来源行号与哈希,不声称核验原始决定签名。2026-11-08 核查时部分原站记录页要求浏览器验证;身份目录和主办方入口可读,不代表逐篇原始决定已可访问,也不外推其他 workshop 的公开政策。采集后仍须经过 PostgreSQL 身份/决定冲突预检、串行导入和独立验收。
该来源明确标记为 `curated_public_status_assertion`:保留源 commit、文件哈希、行号、原标签和 forum ID,不声称已核验原始 PC 签名。只保存标题、摘要、公开作者名、关键词、领域、索引与决定来源,不缓存镜像中的个人资料、邮箱、性别、评审内容或 PDF。所有输出清单均为 partial;盘点中的零条记录仅描述这个固定镜像。
入库前审计核验文件哈希、数量、轨道和来源声明,并用真实 forum ID 对比同届同轨道已有投稿。存在不同决定时阻止整批导入,交由来源和时序核查;另一会议的后续录用属于独立投稿历史,不构成该次拒稿的冲突。同一 forum 的镜像不新增投稿事件,但论文实体的跨平台去重仍须在全量结束后核验。审计通过的报告包含带 SHA-256 的导入队列;应在单写入维护锁下及时串行导入,主库变化后重新审计。
@@ -178,10 +184,6 @@ paper-atlas collect dblp \
输出按会议/年份/track 分文件,附 manifest、`dblp-normalization.json` 范围审查统计和带哈希的 `dblp-import-queue.json`。不能识别的书目、年份或归属会逐类登记,不默认算作主会。DBLP 不含摘要,因此输出摘要为空、清单为 partial;需要通过有证据的跨来源关联补齐论文实体摘要。书目发表证据与原始审稿决定分别保留,不依据书目缺席推断拒稿。已存在其他出版来源的届次,还须先回填其权威身份并核验镜像记录,不能直接累加投稿数。
早期 AISTATS 1995/1997/1999/2001/2003/2005 和 COLT 1988–1991 的主系列名称本身含 Workshop,不能当作附属 workshop 漏掉。识别要求已核对的完整 proceedings key、会议书目名、原始标题前缀、届数和事件年一致;轨道名为 `main` 表示主系列,保留原始 Workshop 名称及官方目录证据,不改造其历史名称或出版形式。参见 [AISTATS 官方历史目录](https://aistats.org/other.html)、[COLT 协会目录](https://www.learningtheory.org/)。1995 AISTATS 的会议年与1996图书出版年分开;[PMLR R0](https://proceedings.mlr.press/r0/) 是后来重发的 pre-proceedings,不与 DBLP 图书条目盲目逐篇等同。
真正附属的 AISTATS 2010 Active Learning and Experimental Design workshop 仍单列。COLT 2003 与 Kernel Workshop 的 [混合论文集](https://doi.org/10.1007/b12006) 缺少逐篇章节信息,使用 `proceedings_unspecified`,不能整卷改成 main 或附属 workshop。输出保存 `track_scope_note` 与 `track_scope_evidence_url`,导入后仍可追查分类理由;这些出版证据不是真实拒稿或原始决定签名,也不自动修正既有主库记录。
部分早期 SIGIR、CIKM、IJCAI 出版卷使用两位年份键;只有论文集标题明确给出唯一完整年份且与短键相符时才采用,不推测世纪。WWW 特殊兴趣、Alternate track 单列;同卷混合 Applied Data Science、Demo、Industrial 且不能逐篇区分时使用 `proceedings_unspecified`,不把整卷归为某一个具体 track。
书目中的前后页、目录组合和明确会务标题不产生论文录用记录;缺作者的赞助、委员会、活动汇总等标题单独排除。WWW 2019 companion 的匿名合成活动条目归入待核查材料,原始快照仍保留。范围报告分别统计这些原因;单纯缺作者并不排除真实研究论文,也不补造作者。
+5 -21
View File
@@ -79,10 +79,10 @@ paper-atlas maintain merge --db /data/paper-index/samples/example.sqlite \
## 4. 查询与分析前检查
```sh
paper-atlas query --pg-service foundry-research --query '"language model"' --limit 20
paper-atlas query --pg-service foundry-research --tag arxiv:category=cs.AI
paper-atlas query --pg-service foundry-research --venue ICLR --year 2025 --status rejected
paper-atlas query --pg-service foundry-research --paper 123
paper-atlas query --db /data/paper-index/papers.sqlite --query '"language model"' --limit 20
paper-atlas query --db /data/paper-index/papers.sqlite --tag arxiv:category=cs.AI
paper-atlas query --db /data/paper-index/papers.sqlite --venue ICLR --year 2025 --status rejected
paper-atlas query --db /data/paper-index/papers.sqlite --paper 123
paper-atlas coverage --db /data/paper-index/papers.sqlite --out /data/paper-index/collected/coverage.json
```
@@ -91,26 +91,10 @@ paper-atlas coverage --db /data/paper-index/papers.sqlite --out /data/paper-inde
例如,查找在 ICLR 有拒稿记录、但在其他已知投稿中已有录用的论文:
```sh
paper-atlas query --pg-service foundry-research \
paper-atlas query --db /data/paper-index/papers.sqlite \\
--venue ICLR --status rejected --outcome accepted
```
`known_acceptances` 列出目前仍为录用状态的各次投稿及会议、年份、track 和决定证据,`publication_outcome` 汇总为 accepted 或 no_known_acceptance。汇总会检查所有会议和 track,不把某次拒稿覆盖到整篇论文;同一次投稿的后续撤稿也会更新该次状态并保留原决定。no_known_acceptance 只表示现有来源中没有查到录用,不能称为“从未录用”。
`--stats` 中 papers 是当前规则下的论文实体数;source_records 是来源记录数;decisions 是决定观察数。一个投稿可能有多个来源和多条观察,不能相加当作论文数。
## 5. 投稿历史关系审计
```sh
PGSERVICEFILE=/etc/paper-atlas/pg_service.conf paper-atlas audit histories \
--pg-service foundry-research --pg-schema research --examples 20 \
--out /data/paper-index/collected/history-audit.json
```
使用一个只读一致性快照,区分论文实体、投稿和决定观察;分别统计同次投稿曾出现负面与录用观察、不同投稿共存负面与当前已知录用、不同会议之间的共存。workshop/main 是不同 track,属于同一会议时不算跨会议;会议别名也不会增加一个会议。当前已知录用使用与 `query --outcome` 相同的“每次投稿最新观察”规则,后续撤稿的该次投稿不再算当前录用,但旧录用决定仍保留。
报告不凭观察时间、届次年份或行顺序断言“拒稿发生在录用之前”。样例保留真实 `decision_date`、`observed_at`、原始标签、证据 URL 和决定来源,供进一步核查;日期缺失仍保持未知。`negative` 统计包含 rejected、desk_rejected 和 withdrawn,其原始状态不互相替换。
总数覆盖快照内全部投稿,样例由 `--examples` 限定为 0–100 篇;每篇最多展示 20 次投稿、每次最多 50 条决定、每条决定最多 100 个来源,并明确标记截断。样例优先保留负面/录用的不同投稿对。四项来源关系检查报告缺失关联和来源论文归属不一致,不自动修复。
这些统计依赖现有论文身份关联,不证明实体合并正确,也不证明原始决定签名、公开拒稿全集、标签类型或最终全库去重已完成。SQLite 独立样本可使用 `--db`;冻结备份不能代表当前 PostgreSQL 的历史情况。
-196
View File
@@ -1,196 +0,0 @@
"""Audit linked submission histories without inferring event chronology."""
import argparse
from collections import Counter
import json
from pathlib import Path
import sqlite3
from paper_atlas.db.backend import database_arguments, open_read_connection
from paper_atlas.db.submissions import timestamp
from paper_atlas.querying import LATEST_DECISION
ATTEMPTS = '''SELECT s.paper_id,s.id AS submission_id,e.venue_id,v.acronym AS venue,e.year,e.track,
count(d.id) AS decision_observations,
MAX(CASE WHEN d.status IN ('rejected','desk_rejected','withdrawn') THEN 1 ELSE 0 END)
AS historical_negative,
MAX(CASE WHEN d.status='accepted' THEN 1 ELSE 0 END) AS historical_acceptance,
COALESCE(MAX(CASE WHEN ''' + LATEST_DECISION + ''' THEN d.status END),'unknown')
AS latest_observed_status
FROM submissions s JOIN editions e ON e.id=s.edition_id JOIN venues v ON v.id=e.venue_id
LEFT JOIN decisions d ON d.submission_id=s.id
GROUP BY s.paper_id,s.id,e.venue_id,v.acronym,e.year,e.track
ORDER BY s.paper_id,e.venue_id,s.id'''
def _paper_counts(attempts):
negative = sum(row['historical_negative'] for row in attempts)
accepted = sum(row['latest_observed_status'] == 'accepted' for row in attempts)
overlap = sum(row['historical_negative'] and row['latest_observed_status'] == 'accepted'
for row in attempts)
venues = {}
for row in attempts:
counts = venues.setdefault(row['venue_id'], [0, 0])
counts[0] += row['historical_negative']
counts[1] += row['latest_observed_status'] == 'accepted'
return {
'papers_with_submissions': 1,
'papers_with_multiple_submissions': int(len(attempts) > 1),
'papers_with_historical_negative': int(negative > 0),
'papers_with_current_known_acceptance': int(accepted > 0),
'papers_with_negative_and_acceptance_in_distinct_attempts': int(negative * accepted > overlap),
'papers_with_negative_and_acceptance_in_distinct_venues':
int(negative * accepted > sum(n * a for n, a in venues.values())),
}
def _example(connection, attempts):
# Keep a negative/accepted pair visible even when a paper has many attempts.
accepted = [row for row in attempts if row['latest_observed_status'] == 'accepted']
first = accepted[0] if accepted else None
other_venue = next((row for row in accepted if row['venue_id'] != first['venue_id']), None)
pair = None
for negative in attempts:
if negative['historical_negative'] and first:
candidate = first if first['venue_id'] != negative['venue_id'] else other_venue
if candidate:
pair = (negative, candidate)
break
if pair is None:
for negative in attempts:
if negative['historical_negative'] and first:
candidate = first if first['submission_id'] != negative['submission_id'] else (
accepted[1] if len(accepted) > 1 else None)
if candidate:
pair = (negative, candidate)
break
selected = {}
for row in list(pair or ()) + attempts:
if len(selected) < 20:
selected.setdefault(row['submission_id'], dict(row))
for row in selected.values():
row['decisions'] = []
for decision in connection.execute('''SELECT id,status,raw_value,evidence_url,
decision_date,observed_at FROM decisions WHERE submission_id=?
ORDER BY observed_at DESC,COALESCE(decision_date,observed_at) DESC,id DESC LIMIT 50''',
(row['submission_id'],)):
entry = dict(decision)
sources = [dict(source) for source in connection.execute('''
SELECT so.name,r.external_id,ds.source_decision_id FROM decision_sources ds
JOIN source_records r ON r.id=ds.source_record_id JOIN sources so ON so.id=r.source_id
WHERE ds.decision_id=? ORDER BY so.name,r.external_id LIMIT 101''', (entry['id'],))]
entry['sources'] = sources[:100]
entry['sources_truncated'] = len(sources) > 100
row['decisions'].append(entry)
row['decisions_truncated'] = row['decision_observations'] > len(row['decisions'])
paper_id = attempts[0]['paper_id']
return {'paper_id': paper_id,
'title': connection.execute('SELECT title FROM papers WHERE id=?', (paper_id,)).fetchone()[0],
'counts': _paper_counts(attempts), 'attempt_count': len(attempts),
'attempts': list(selected.values()), 'attempts_truncated': len(attempts) > len(selected),
'chronology': 'Not inferred from observed_at, edition year, or row order.'}
def report(db=None, *, pg_service=None, pg_schema=None, examples=20):
if not 0 <= examples <= 100:
raise ValueError('Examples must be between 0 and 100')
started_at = timestamp()
connection = open_read_connection(db, pg_service=pg_service, pg_schema=pg_schema)
if not pg_service:
connection.row_factory = sqlite3.Row
try:
if pg_service:
identity = connection.execute('SELECT current_database(),current_schema()').fetchone()
database, schema = identity[0], identity[1]
else:
database, schema = str(Path(db).resolve()), None
names = ['papers_with_submissions', 'papers_with_multiple_submissions',
'papers_with_historical_negative', 'papers_with_current_known_acceptance',
'papers_with_negative_and_acceptance_in_distinct_attempts',
'papers_with_negative_and_acceptance_in_distinct_venues']
totals = Counter({key: 0 for key in names})
totals.update(submissions=0, decision_observations=0,
attempts_with_historical_negative=0,
attempts_with_historical_negative_and_acceptance=0)
latest = Counter()
sample_attempts = []
def consume(rows):
if not rows:
return
counts = _paper_counts(rows)
totals.update(counts)
if counts['papers_with_negative_and_acceptance_in_distinct_attempts'] and len(sample_attempts) < examples:
sample_attempts.append(rows)
current = []
for raw in connection.execute(ATTEMPTS):
row = dict(raw)
if current and row['paper_id'] != current[0]['paper_id']:
consume(current)
current = []
current.append(row)
totals['submissions'] += 1
totals['decision_observations'] += row['decision_observations']
totals['attempts_with_historical_negative'] += row['historical_negative']
totals['attempts_with_historical_negative_and_acceptance'] += int(
row['historical_negative'] and row['historical_acceptance'])
latest[row['latest_observed_status']] += 1
consume(current)
# These are relationship checks, not a substitute for the final entity/FK audit.
integrity = {}
checks = {
'submissions_without_source_links': '''SELECT count(*) FROM submissions s
WHERE NOT EXISTS(SELECT 1 FROM submission_sources ss WHERE ss.submission_id=s.id)''',
'decisions_without_source_links': '''SELECT count(*) FROM decisions d
WHERE NOT EXISTS(SELECT 1 FROM decision_sources ds WHERE ds.decision_id=d.id)''',
'submission_source_paper_mismatches': '''SELECT count(*) FROM submission_sources ss
JOIN submissions s ON s.id=ss.submission_id JOIN source_records r ON r.id=ss.source_record_id
WHERE s.paper_id<>r.paper_id''',
'decision_source_paper_mismatches': '''SELECT count(*) FROM decision_sources ds
JOIN decisions d ON d.id=ds.decision_id JOIN submissions s ON s.id=d.submission_id
JOIN source_records r ON r.id=ds.source_record_id WHERE s.paper_id<>r.paper_id''',
}
for name, query in checks.items():
integrity[name] = connection.execute(query).fetchone()[0]
return {'report_started_at': started_at, 'checked_at': timestamp(),
'engine': 'postgresql' if pg_service else 'sqlite', 'database': database,
'schema': schema, 'pg_service': pg_service,
'status': 'observed_history_audit_not_final_dedup', 'counts': dict(totals),
'latest_attempt_status_counts': dict(sorted(latest.items())),
'evidence_relationship_checks': integrity,
'examples': [_example(connection, rows) for rows in sample_attempts],
'example_limit': examples,
'interpretation': [
'Counts use existing canonical paper links, not independently verified real-world identity.',
'Negative includes rejected, desk_rejected and withdrawn observations; these remain distinct decisions.',
'Known acceptance follows the latest observed decision per attempt, matching query --outcome.',
'Distinct attempts/venues require separate submission/venue IDs; aliases do not create new venues.',
'Coexisting evidence does not prove rejection happened before acceptance. observed_at is collection time.',
'No known acceptance is not proof of never accepted. Source signatures and completeness are not established here.',
'Examples are bounded; counts include all attempts and observations in one read-only snapshot.',
'This does not perform entity merging, final deduplication, or label namespace/type validation.',
]}
finally:
connection.close()
def main(argv=None, *, prog=None):
parser = argparse.ArgumentParser(prog=prog)
database_arguments(parser)
parser.add_argument('--examples', type=int, default=20)
parser.add_argument('--out')
args = parser.parse_args(argv)
result = report(args.db, pg_service=args.pg_service, pg_schema=args.pg_schema, examples=args.examples)
text = json.dumps(result, ensure_ascii=False, indent=2)
if args.out:
path = Path(args.out)
stage = path.with_suffix('.part')
stage.write_text(text + '\n')
stage.replace(path)
else:
print(text)
if __name__ == '__main__':
main()
+1 -3
View File
@@ -39,9 +39,7 @@ def audit(db, inventory, out, *, pg_service=None, pg_schema=None):
if json.loads(path.with_suffix('.manifest.json').read_text()) != manifest:
raise ValueError('Collected manifest differs from inventory')
count = 0
# Unicode paragraph/line separators are valid inside JSON strings;
# split physical byte lines, not Unicode text boundaries.
for line in path.read_bytes().splitlines():
for line in path.read_text().splitlines():
row = json.loads(line)
if any(row[key] != manifest[key] for key in ('source', 'venue', 'year', 'track')):
raise ValueError('Record edition differs from inventory')
-1
View File
@@ -35,7 +35,6 @@ COMMANDS = {
('audit','arxiv'): ('paper_atlas.audits.arxiv', {}, '核验 arXiv 原始编号、重复行与 ZIP 完整性'),
('audit','arxiv-tags'): ('paper_atlas.audits.arxiv_tags', {}, '逐记录核验 arXiv 类别及出处完整性'),
('audit','public-openreview'): ('paper_atlas.audits.public_openreview', {}, '核对公开负面决定与已有投稿历史'),
('audit','histories'): ('paper_atlas.audits.histories', {}, '只读核对跨会议投稿历史及决定来源关系'),
}
-2
View File
@@ -90,8 +90,6 @@ def normalize_row(row,volumes,targets,include_workshops=False,through_year=2026)
'ambiguous_identifiers':ambiguous,'publication_evidence':'inproceedings bibliography; not an original decision note'}
if edition.get('track_scope_note'):
metadata['track_scope_note']=edition['track_scope_note']
if edition.get('track_scope_evidence_url'):
metadata['track_scope_evidence_url']=edition['track_scope_evidence_url']
flagged=bool(re.search(r'\b(?:withdrawn|retracted)\b',row.get('publtype') or '',re.I))
result={'source':'dblp','external_id':row['key'],'url':'https://dblp.org/rec/'+row['key']+'.html',
'title':title,'abstract':None,'authors':authors,'identifiers':own,
+1 -39
View File
@@ -38,39 +38,6 @@ BOOKS = {
}
# These are the historical primary series themselves, not co-located workshops.
# Exact proceedings keys, original names, ordinals and event years must agree.
HISTORICAL_WORKSHOP_TITLES = {
**{('aistats', year): prefix + ordinal + ' International Workshop on Artificial Intelligence and Statistics'
for year, ordinal, prefix in [
(1995, 'Fifth', 'Learning from Data - '),
(1997, 'Sixth', 'Proceedings of the '),
(1999, 'Seventh', 'Proceedings of the '),
(2001, 'Eighth', 'Proceedings of the '),
(2003, 'Ninth', 'Proceedings of the '),
(2005, 'Tenth', 'Proceedings of the ')]},
**{('colt', year): 'Proceedings of the ' + ordinal + ' Annual Workshop on Computational Learning Theory'
for year, ordinal in [(1988, 'First'), (1989, 'Second'), (1990, 'Third'), (1991, 'Fourth')]},
}
def historical_track(family, base, suffix, volume_key, year, title):
if suffix or base.casefold() != family or volume_key != f'conf/{family}/{year}':
return None
prefix = HISTORICAL_WORKSHOP_TITLES.get((family, year))
if (prefix and title.casefold().startswith(prefix.casefold() + ',')
and re.search(r'\b' + str(year) + r'\b', title)):
url = 'https://aistats.org/other.html' if family == 'aistats' else 'https://www.learningtheory.org/'
return ('main', 'Historical primary series named Workshop; exact proceedings key, ordinal, title and event year verified.', url)
if (family == 'colt' and year == 2003
and title.startswith('Computational Learning Theory and Kernel Machines, 16th Annual Conference on Computational Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003,')
and title.endswith('Proceedings')):
return ('proceedings_unspecified',
'Combined COLT 2003 and Kernel Workshop volume; per-paper section absent from DBLP XML. Do not infer a main or workshop track.',
'https://doi.org/10.1007/b12006')
return None
def scope(row,volume,through_year=2026):
fields=row['fields'];books=fields.get('booktitle',[])
if len(books)!=1:return None,'ambiguous_booktitle'
@@ -108,9 +75,7 @@ def scope(row,volume,through_year=2026):
if len(bibliographic)!=1 or not re.fullmatch(r'\d{4}',bibliographic[0]):return None,'ambiguous_bibliographic_year'
if int(bibliographic[0])!=year and not re.search(r'\b'+str(year)+r'\b',title):return None,'unverified_year_difference'
text=(suffix+' '+title).casefold();track='main'
historical = historical_track(family, base, suffix, volume['key'], year, title)
if historical:track=historical[0]
elif 'findings' in text:track='findings'
if 'findings' in text:track='findings'
elif 'shared task' in text:track='shared_task'
elif 'student research' in text or 'workshop' in text or 'wokshop' in text or base in ('ICCVW','EWSL'):track='workshop'
elif ('demo track' in text and ('industrial track' in text or 'applied data science track' in text)):track='proceedings_unspecified'
@@ -133,9 +98,6 @@ def scope(row,volume,through_year=2026):
if venue=='AAMAS' and year<2002:return None,'predecessor_not_aamas'
result = {'venue':venue,'venue_aliases':aliases,'year':year,'track':track,'bibliographic_year':int(bibliographic[0]),
'rule':family+':'+base,'proceedings_title':title,'edition_year_basis':year_basis}
if historical:
result['track_scope_note'] = historical[1]
result['track_scope_evidence_url'] = historical[2]
if venue=='MM' and track=='main':
# A main MM volume also includes doctoral, tutorials, companion papers
# and other sections. DBLP XML omits the per-paper TOC section.
+35 -4
View File
@@ -27,11 +27,25 @@ RAW_ROOT = 'https://raw.githubusercontent.com/papercopilot/openreview/' + REVISI
TREE_URL = 'https://api.github.com/repos/papercopilot/openreview/git/trees/' + REVISION + '?recursive=1'
NEGATIVE = {'reject': 'rejected', 'rejected': 'rejected',
'desk reject': 'desk_rejected', 'desk rejected': 'desk_rejected',
'withdrawn': 'withdrawn', 'withdrawal': 'withdrawn'}
'withdraw': 'withdrawn', 'withdrawn': 'withdrawn', 'withdrawal': 'withdrawn'}
TRACKS = {'main': 'main', 'Position': 'position',
'Datasets & Benchmarks': 'datasets_benchmarks'}
WORKSHOP_REJECT_LABELS = {"aim-fm workshop @ neurips'24 reject",
'coginterp @ neurips 2025 reject'}
# Exact curated labels, tied to the year and raw workshop track. The official
# venue directory establishes workshop identity, not an original decision signature.
ICLR_WORKSHOP_LABELS = {
'iclr 2024 workshop mefomo withdrawn submission':
(2024, 'workshop_me_fomo', 'withdrawn', 'ME-FoMo'),
'iclr 2024 workshop genai4dm withdrawn submission':
(2024, 'workshop_genai4dm', 'withdrawn', 'GenAI4DM'),
'iclr 2025 workshop agenticai reject':
(2025, 'workshop_agenticai', 'rejected', 'AgenticAI'),
'iclr 2025 workshop lmrl withdrawn submission':
(2025, 'workshop_lmrl', 'withdrawn', 'LMRL'),
'iclr 2026 workshop llm reasoning withdrawn submission':
(2026, 'workshop_llm_reasoning', 'withdrawn', 'LLM_Reasoning'),
}
def download(url, limit):
@@ -63,6 +77,20 @@ def normalize(row, family, year, provenance, include_workshops=False):
"""Return a normalized assertion or a reason requiring separate review."""
raw_status = row.get('status', '')
status = negative_status(raw_status)
workshop = ICLR_WORKSHOP_LABELS.get(' '.join(raw_status.casefold().split()))
workshop_metadata = {}
if workshop:
expected_year, expected_track, status, group_name = workshop
if family != 'iclr' or year != expected_year:
raise ValueError('Workshop status label conflicts with source conference year')
if row.get('track') != expected_track:
return None, 'conflicting_workshop_track'
if not include_workshops:
return None, 'separate_workshop'
workshop_metadata = {
'workshop_venueid': f'ICLR.cc/{year}/Workshop/{group_name}',
'workshop_identity_evidence_url': 'https://openreview.net/venue?id=ICLR.cc',
}
if status is None:
if re.search(r'\bretract\w*\b', raw_status, re.I):
return None, 'retracted'
@@ -90,7 +118,9 @@ def normalize(row, family, year, provenance, include_workshops=False):
return None, 'unmapped_venue'
title = row.get('title')
abstract = row.get('abstract')
if not isinstance(title, str) or not title.strip():
if title is None or (isinstance(title, str) and not title.strip()):
return None, 'missing_title'
if not isinstance(title, str):
raise ValueError('Public submission lacks a title')
if abstract is not None and not isinstance(abstract, str):
raise ValueError('Abstract must be text or null')
@@ -116,7 +146,8 @@ def normalize(row, family, year, provenance, include_workshops=False):
'venue': venue, 'year': year, 'track': normalized_track,
'status': status, 'raw_decision': raw_status, 'evidence_url': evidence,
'observed_at': AS_OF, 'tags': tags,
'metadata': {**provenance, 'raw_track': track, 'raw_status': raw_status,
'metadata': {**provenance, **workshop_metadata,
'raw_track': track, 'raw_status': raw_status,
'evidence_kind': 'curated_public_status_assertion',
'original_decision_signature_verified': False,
'source_snapshot_upper_bound': AS_OF}}, None
@@ -215,7 +246,7 @@ def _collect_files(out, selected, include_workshops, fetcher):
name = f'public-openreview-{family}-{year}-{track}.jsonl'
path = stage / name
path.write_text(''.join(json.dumps(row, ensure_ascii=False) + '\n' for row in records))
manifest = {'source': SOURCE, 'parser_version': 1, 'venue': records[0]['venue'],
manifest = {'source': SOURCE, 'parser_version': 3, 'venue': records[0]['venue'],
'year': year, 'track': track, 'count': len(records), 'status': 'partial',
'file': str(out / name), 'file_sha256': hashlib.sha256(path.read_bytes()).hexdigest(),
'list_url': REPOSITORY + '/tree/' + REVISION, 'source_revision': REVISION,
+1 -64
View File
@@ -10,7 +10,7 @@ from unittest.mock import patch
from paper_atlas.collectors.dblp import XML_SHA
from paper_atlas.collectors.dblp_records import collect,identifiers,normalize_row,material_reason
from paper_atlas.collectors.dblp_scope import scope, HISTORICAL_WORKSHOP_TITLES
from paper_atlas.collectors.dblp_scope import scope
from paper_atlas.db.store import Store
from paper_atlas.ingestion.records import ingest_file
@@ -70,69 +70,6 @@ class DblpRecordTests(unittest.TestCase):
self.assertEqual(scope(self.row(),self.volume('kdd',2024))[1],'crossref_venue_conflict')
self.assertEqual(scope(self.row(year=2027),self.volume('aaai',2027))[1],'future_edition')
def test_historical_primary_series_workshop_names_require_exact_editions(self):
self.assertEqual(set(HISTORICAL_WORKSHOP_TITLES),
{('aistats',year) for year in [1995,1997,1999,2001,2003,2005]} |
{('colt',year) for year in [1988,1989,1990,1991]})
for (family,year),prefix in HISTORICAL_WORKSHOP_TITLES.items():
with self.subTest(family=family,year=year):
title=prefix+f', {family.upper()} {year}, Proceedings.'
volume=self.volume(family,year,title)
row=self.row(family,family.upper(),year)
if year==1995:
row['fields']['year']=['1996'];volume['fields']['year']=['1996']
edition,reason=scope(row,volume)
self.assertIsNone(reason);self.assertEqual(edition['track'],'main')
self.assertEqual(edition['year'],year)
self.assertIn('Historical primary series',edition['track_scope_note'])
record,reason=normalize_row(row,{volume['key']:volume},{family.upper()})
self.assertIsNone(reason);self.assertEqual(record['metadata']['track_scope_evidence_url'],edition['track_scope_evidence_url'])
self.assertEqual(record['metadata']['proceedings_title'],title)
if year==1995:self.assertEqual(record['metadata']['bibliographic_year'],1996)
def test_historical_exception_does_not_promote_affiliated_or_wrong_scope_workshops(self):
title=HISTORICAL_WORKSHOP_TITLES[('aistats',1995)]+', AISTATS 1995, Proceedings.'
for book,key,changed in [
('AISTATS','conf/aistats/1995w',title),
('AISTATS (Workshops)','conf/aistats/1995',title),
('AISTATS','conf/aistats/1995',title.replace('Fifth','Sixth')),
('AISTATS','conf/aistats/1995',title.replace('1995','1997'))]:
volume=self.volume('aistats',1995,changed);volume['key']=key
row=self.row('aistats',book,1995);row['fields']['crossref']=[key]
self.assertEqual(scope(row,volume)[0]['track'],'workshop')
volume=self.volume('aistats',2010,'Active Learning and Experimental Design workshop, In conjunction with AISTATS 2010')
volume['key']='conf/aistats/2010w'
row=self.row('aistats','AISTATS',2011);row['fields']['crossref']=[volume['key']]
self.assertEqual(scope(row,volume)[0]['track'],'workshop')
def test_combined_colt_kernel_volume_keeps_unknown_per_paper_track(self):
title='Computational Learning Theory and Kernel Machines, 16th Annual Conference on Computational Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003, Proceedings'
volume=self.volume('colt',2003,title);row=self.row('colt','COLT',2003)
record,reason=normalize_row(row,{volume['key']:volume},{'COLT'})
self.assertIsNone(reason);self.assertEqual(record['track'],'proceedings_unspecified')
self.assertEqual(record['status'],'accepted')
self.assertIn('per-paper section absent',record['metadata']['track_scope_note'])
self.assertEqual(record['metadata']['track_scope_evidence_url'],'https://doi.org/10.1007/b12006')
volume['key']='conf/colt/2003w';row['fields']['crossref']=[volume['key']]
self.assertEqual(scope(row,volume)[0]['track'],'workshop')
def test_historical_scope_provenance_survives_file_ingestion(self):
title=HISTORICAL_WORKSHOP_TITLES[('colt',1988)]+', COLT 1988, Proceedings.'
row=self.row('colt','COLT',1988);volume=self.volume('colt',1988,title)
record,reason=normalize_row(row,{volume['key']:volume},{'COLT'})
self.assertIsNone(reason)
with tempfile.TemporaryDirectory() as tmp:
root=Path(tmp);path=root/'history.jsonl'
path.write_text(json.dumps(record)+'\n')
path.with_suffix('.manifest.json').write_text(json.dumps({'source':'dblp','count':1,'list_url':'https://doi.org/10.4230/dblp.xml.2026-11-01','file_sha256':hashlib.sha256(path.read_bytes()).hexdigest()}))
ingest_file(root/'sample.sqlite',path)
with closing(sqlite3.connect(root/'sample.sqlite')) as connection:
track=connection.execute('SELECT track FROM editions').fetchone()[0]
metadata=json.loads(connection.execute('SELECT metadata_json FROM source_records').fetchone()[0])
self.assertEqual(track,'main')
self.assertEqual(metadata['track_scope_evidence_url'],'https://www.learningtheory.org/')
self.assertEqual(metadata['proceedings_title'],title)
def test_only_own_identifier_links_are_used_and_ambiguity_is_preserved(self):
own,ambiguous=identifiers({'ee':['https://doi.org/10.1234/published','https://doi.org/10.48550/arXiv.2401.00001','https://openreview.net/forum?id=realForum'],
'cite':['https://doi.org/10.9999/reference']})
-114
View File
@@ -1,114 +0,0 @@
from pathlib import Path
import tempfile
import unittest
from paper_atlas.audits.histories import report
from paper_atlas.db.store import Store
def add_history_fixture(store):
def event(work, forum, venue, year, track, status, observed):
paper, record = store.record('history_fixture', work + '/' + forum,
'https://example.org/' + forum, work + ' title\x00 preserved', 'Abstract', ['Jane Doe'],
{'arxiv': work})
store.decision(paper, record, venue, year, track, forum,
'https://example.org/' + forum + '/' + str(status), status, status or '',
identities=[('openreview', forum)], observed_at=observed)
return paper
# The negative observation was collected later: observation order is not chronology.
event('2501.00001', 'HistoryA01', 'ICLR', 2024, 'main', 'rejected', '2026-02-01T00:00:00Z')
event('2501.00001', 'HistoryA02', 'ICML', 2025, 'main', 'accepted', '2026-01-01T00:00:00Z')
event('2501.00002', 'HistoryB01', 'ICLR', 2024, 'main', 'rejected', '2026-01-01T00:00:00Z')
event('2501.00002', 'HistoryB01', 'ICLR', 2024, 'main', 'accepted', '2026-02-01T00:00:00Z')
event('2501.00003', 'HistoryC01', 'ICLR', 2024, 'main', 'desk_rejected', '2026-01-01T00:00:00Z')
event('2501.00003', 'HistoryC02', 'ICLR', 2024, 'workshop_example', 'accepted', '2026-02-01T00:00:00Z')
event('2501.00004', 'HistoryD01', 'ICML', 2025, 'main', 'accepted', '2026-01-01T00:00:00Z')
event('2501.00004', 'HistoryD01', 'ICML', 2025, 'main', 'withdrawn', '2026-02-01T00:00:00Z')
event('2501.00004', 'HistoryD02', 'ICLR', 2024, 'main', 'rejected', '2026-01-01T00:00:00Z')
event('2501.00005', 'HistoryE01', 'COLM', 2026, 'main', None, '2026-01-01T00:00:00Z')
store.conn.commit()
class HistoryTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
self.db = Path(self.tmp.name) / 'sample.sqlite'
self.store = Store(self.db)
self.addCleanup(self.store.conn.close)
def test_distinct_attempts_venues_and_latest_acceptances_are_not_observation_counts(self):
add_history_fixture(self.store)
result = report(self.db)
self.assertEqual(result['counts'], {
'papers_with_submissions': 5, 'papers_with_multiple_submissions': 3,
'papers_with_historical_negative': 4, 'papers_with_current_known_acceptance': 3,
'papers_with_negative_and_acceptance_in_distinct_attempts': 2,
'papers_with_negative_and_acceptance_in_distinct_venues': 1,
'submissions': 8, 'decision_observations': 9,
'attempts_with_historical_negative': 5,
'attempts_with_historical_negative_and_acceptance': 2,
})
self.assertEqual(result['latest_attempt_status_counts'],
{'accepted': 3, 'desk_rejected': 1, 'rejected': 2, 'unknown': 1, 'withdrawn': 1})
self.assertEqual(len(result['examples']), 2)
first = result['examples'][0]
self.assertIn('\x00', first['title'])
self.assertEqual({r['venue'] for r in first['attempts']}, {'ICLR', 'ICML'})
self.assertTrue(all(r['decisions'][0]['sources'] for r in first['attempts']))
self.assertTrue(all(r['decisions'][0]['decision_date'] is None for r in first['attempts']))
self.assertIn('Not inferred', first['chronology'])
self.assertEqual(result['status'], 'observed_history_audit_not_final_dedup')
self.assertTrue(all(v == 0 for v in result['evidence_relationship_checks'].values()))
def test_missing_and_mislinked_source_evidence_is_reported_without_repair(self):
add_history_fixture(self.store)
self.store.conn.execute('DELETE FROM decision_sources WHERE decision_id=1')
self.store.conn.execute('DELETE FROM submission_sources WHERE submission_id=1')
# Record3 belongs to paper2; move it to paper1 to create a real mismatch.
self.store.conn.execute('UPDATE source_records SET paper_id=1 WHERE id=3')
self.store.conn.commit()
result = report(self.db, examples=0)
checks = result['evidence_relationship_checks']
self.assertEqual(checks['decisions_without_source_links'], 1)
self.assertEqual(checks['submissions_without_source_links'], 1)
self.assertEqual(checks['submission_source_paper_mismatches'], 1)
self.assertEqual(checks['decision_source_paper_mismatches'], 2)
self.assertEqual(self.store.conn.execute('SELECT paper_id FROM source_records WHERE id=3').fetchone()[0], 1)
def test_example_limit_does_not_change_global_counts(self):
add_history_fixture(self.store)
none, one = report(self.db, examples=0), report(self.db, examples=1)
self.assertEqual(none['counts'], one['counts'])
self.assertEqual(none['examples'], [])
self.assertEqual(len(one['examples']), 1)
def test_empty_snapshot_and_invalid_limit_do_not_create_missing_database(self):
empty = report(self.db)
self.assertTrue(all(v == 0 for v in empty['counts'].values()))
self.assertEqual(empty['latest_attempt_status_counts'], {})
absent = self.db.parent / 'absent.sqlite'
with self.assertRaisesRegex(ValueError, 'Examples'):
report(absent, examples=-1)
with self.assertRaisesRegex(ValueError, 'Existing database'):
report(absent)
self.assertFalse(absent.exists())
def test_truncated_example_keeps_the_cross_venue_acceptance_visible(self):
paper, record = self.store.record('fixture', 'many', 'https://example.org/many',
'Many attempts', 'Abstract', [])
for index in range(26):
accepted = index == 25
forum = f'BoundedForum{index}'
self.store.decision(paper, record, 'ICML' if accepted else 'ICLR', 2025, 'main', forum,
'https://example.org/' + forum, 'accepted' if accepted else 'rejected',
'Accept' if accepted else 'Reject', identities=[('openreview', forum)])
self.store.conn.commit()
result = report(self.db)
example, = result['examples']
self.assertEqual((example['attempt_count'], len(example['attempts'])), (26, 20))
self.assertTrue(example['attempts_truncated'])
self.assertEqual({row['latest_observed_status'] for row in example['attempts']},
{'accepted', 'rejected'})
self.assertEqual(result['counts']['submissions'], 26)
-16
View File
@@ -175,22 +175,6 @@ class PostgreSQLIntegrationTests(unittest.TestCase):
finally:
reader.close()
def test_history_audit_matches_sqlite_for_actual_postgresql_snapshot(self):
from paper_atlas.audits.histories import report
from tests.test_histories import add_history_fixture
store = Store(self.db)
try:
add_history_fixture(store)
finally:
store.conn.close()
expected = report(self.db)
self.assertEqual(self.migrate()['status'], 'complete')
actual = report(pg_service=self.service, pg_schema=self.target)
self.assertEqual(actual['engine'], 'postgresql')
self.assertEqual(actual['schema'], self.target)
for key in ('counts', 'latest_attempt_status_counts', 'evidence_relationship_checks', 'examples'):
self.assertEqual(actual[key], expected[key], key)
def interrupt_after_first_table(self):
calls = 0
def interrupt(*args):
+118 -30
View File
@@ -45,10 +45,12 @@ class PublicOpenReviewTests(unittest.TestCase):
def test_only_explicit_decisions_and_verified_scope_become_events(self):
for value in ('Pending', 'Active', 'Submitted to NeurIPS 2025', 'TBD', '',
'Low score', 'ICML 2025 Retracted by Authors', 'Do not reject'):
'Low score', 'ICML 2025 Retracted by Authors', 'Do not reject',
'Workshop', 'Poster Workshop', 'Oral Workshop'):
self.assertIsNone(negative_status(value), value)
self.assertEqual(negative_status('Reject'), 'rejected')
self.assertEqual(negative_status('Desk rejected'), 'desk_rejected')
self.assertEqual(negative_status('Withdraw'), 'withdrawn')
self.assertEqual(negative_status('NeurIPS 2023 Conference Withdrawn Submission'), 'withdrawn')
for raw, expected in [('main', 'main'), ('Position', 'position'),
('Datasets & Benchmarks', 'datasets_benchmarks')]:
@@ -98,6 +100,121 @@ class PublicOpenReviewTests(unittest.TestCase):
self.assertEqual(list(Path(tmp).glob('*.jsonl')), [])
self.assertFalse((Path(tmp) / 'public-openreview-inventory.json').exists())
def test_missing_title_is_held_without_losing_valid_negative_records(self):
with tempfile.TemporaryDirectory() as tmp:
result = self.collect(tmp, {'venues/iclr/iclr2017.jsonl': [
self.row(id='SJUdkecgx', title=''),
self.row(id='NoTitle001', title=None),
self.row(id='Whitespace', title=' '),
self.row(id='ValidReject'),
self.row(id='Withdraw01', status='Withdraw'),
self.row(id='Workshop01', status='Workshop'),
]}, families=['iclr'], years=[2017])
self.assertEqual(result['input_rows'], 6)
self.assertEqual(result['normalized_records'], 2)
self.assertEqual(result['source_files'][0]['excluded_or_held'],
{'missing_title': 3, 'not_explicit_negative': 1})
held_path = Path(result['held_file'])
self.assertEqual(hashlib.sha256(held_path.read_bytes()).hexdigest(),
result['held_file_sha256'])
held = [json.loads(line) for line in held_path.read_text().splitlines()]
self.assertEqual([row['id'] for row in held], ['SJUdkecgx', 'NoTitle001', 'Whitespace'])
self.assertTrue(all(row['reason'] == 'missing_title' and row['status'] == 'Reject'
for row in held))
self.assertEqual([row['provenance']['source_line'] for row in held], [1, 2, 3])
self.assertNotIn('private@example.org', held_path.read_text())
self.assertNotIn('private review', held_path.read_text())
manifest = result['files'][0]
self.assertEqual((manifest['parser_version'], manifest['status']), (3, 'partial'))
rows = [json.loads(line) for line in Path(manifest['file']).read_text().splitlines()]
self.assertEqual([(row['identifiers']['openreview'], row['status']) for row in rows],
[('ValidReject', 'rejected'), ('Withdraw01', 'withdrawn')])
for row in held:
self.assertEqual(row['provenance']['source_revision'], REVISION)
def test_nontext_title_still_fails_collection(self):
with tempfile.TemporaryDirectory() as tmp:
with self.assertRaisesRegex(ValueError, 'lacks a title'):
self.collect(tmp, {'venues/iclr/iclr2017.jsonl': [self.row(title=['invalid'])]},
families=['iclr'], years=[2017])
self.assertFalse((Path(tmp) / 'public-openreview-inventory.json').exists())
self.assertEqual(list(Path(tmp).glob('*.jsonl')), [])
def workshop_cases(self):
return [
(2024, 'workshop_me_fomo', 'ICLR 2024 Workshop MEFoMo Withdrawn Submission', 'withdrawn', 'ME-FoMo'),
(2024, 'workshop_genai4dm', 'ICLR 2024 Workshop GenAI4DM Withdrawn Submission', 'withdrawn', 'GenAI4DM'),
(2025, 'workshop_agenticai', 'ICLR 2025 Workshop AgenticAI Reject', 'rejected', 'AgenticAI'),
(2025, 'workshop_lmrl', 'ICLR 2025 Workshop LMRL Withdrawn Submission', 'withdrawn', 'LMRL'),
(2026, 'workshop_llm_reasoning', 'ICLR 2026 Workshop LLM Reasoning Withdrawn Submission', 'withdrawn', 'LLM_Reasoning'),
]
def test_iclr_workshop_labels_require_opt_in_and_retain_identity_evidence(self):
for year, track, label, status, group in self.workshop_cases():
with self.subTest(label=label):
raw = self.row(track=track, status=label)
self.assertEqual(normalize(raw, 'iclr', year, self.provenance()),
(None, 'separate_workshop'))
row, reason = normalize(raw, 'iclr', year, self.provenance(), True)
self.assertIsNone(reason)
self.assertEqual((row['venue'], row['year'], row['track'], row['status']),
('ICLR', year, track, status))
self.assertEqual(row['raw_decision'], label)
self.assertEqual(row['metadata']['workshop_venueid'],
f'ICLR.cc/{year}/Workshop/{group}')
self.assertFalse(row['metadata']['original_decision_signature_verified'])
self.assertEqual(row['evidence_url'], 'https://example.org/pinned.jsonl#L7')
self.assertNotIn('private review', json.dumps(row))
def test_workshop_label_scope_conflicts_never_become_main_events(self):
for year, track, label, _, _ in self.workshop_cases():
for enabled in (False, True):
for wrong_track in ('main', 'workshop_other'):
self.assertEqual(normalize(self.row(track=wrong_track, status=label),
'iclr', year, self.provenance(), enabled),
(None, 'conflicting_workshop_track'))
for family, wrong_year in [('icml', year), ('iclr', year + 1)]:
with self.assertRaisesRegex(ValueError, 'conference year'):
normalize(self.row(track=track, status=label), family,
wrong_year, self.provenance(), enabled)
row, reason = normalize(self.row(track='workshop_unknown',
status='ICLR 2026 Workshop Unknown Reject'),
'iclr', 2026, self.provenance(), True)
self.assertIsNone(row)
self.assertEqual(reason, 'unmapped_negative_label')
def test_workshop_collection_and_import_keep_five_tracks_separate_from_main(self):
files = {}
for index, (year, track, label, _, _) in enumerate(self.workshop_cases()):
files.setdefault(f'venues/iclr/iclr{year}.jsonl', []).append(
self.row(id=f'Workshop{index:02}', track=track, status=label))
files['venues/iclr/iclr2024.jsonl'].append(self.row())
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
default = self.collect(root / 'default', files, families=['iclr'])
self.assertEqual(default['normalized_records'], 1)
held = [json.loads(line) for line in Path(default['held_file']).read_bytes().split(b'\n') if line]
self.assertEqual(len(held), 5)
self.assertTrue(all(row['reason'] == 'separate_workshop' for row in held))
selected = self.collect(root / 'selected', files, families=['iclr'], include_workshops=True)
self.assertEqual(selected['normalized_records'], 6)
self.assertEqual(len(selected['files']), 6)
self.assertEqual(Path(selected['held_file']).read_bytes(), b'')
db = root / 'db.sqlite'
for manifest in selected['files']:
self.assertEqual(manifest['parser_version'], 3)
ingest_file(db, manifest['file'])
store = Store(db)
try:
scopes = set(store.conn.execute('SELECT e.year,e.track,d.status FROM submissions s '
'JOIN editions e ON e.id=s.edition_id '
'JOIN decisions d ON d.submission_id=s.id'))
expected = {(year, track, status) for year, track, _, status, _ in self.workshop_cases()}
self.assertEqual(scopes, expected | {(2024, 'main', 'rejected')})
self.assertEqual(store.conn.execute('PRAGMA foreign_key_check').fetchall(), [])
finally:
store.conn.close()
def test_mirror_reimport_preserves_one_attempt_and_later_acceptance_elsewhere(self):
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
@@ -152,35 +269,6 @@ class PublicOpenReviewTests(unittest.TestCase):
finally:
store.conn.close()
def test_unicode_separators_in_abstract_survive_audit_and_import(self):
for separator in ('\u0085', '\u2028', '\u2029'):
with self.subTest(separator=repr(separator)), tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
abstract = 'First paragraph' + separator + 'Second paragraph'
result = self.collect(root / 'source', {
'venues/icml/icml2025.jsonl': [self.row(abstract=abstract)]})
path = Path(result['files'][0]['file'])
data = path.read_bytes()
self.assertIn(separator.encode(), data)
self.assertEqual(data.count(b'\n'), 1)
db = root / 'db.sqlite'
store = Store(db)
store.conn.close()
inventory = root / 'source/public-openreview-inventory.json'
before = audit(db, inventory, root / 'before.json')
self.assertEqual(before['status'], 'ready_for_serial_import')
self.assertEqual(before['records_checked'], 1)
ingest_file(db, path)
after = audit(db, inventory, root / 'after.json')
self.assertEqual(after['status'], 'ready_for_serial_import')
self.assertEqual(after['existing_same_edition_attempts'], 1)
store = Store(db)
try:
self.assertEqual(store.conn.execute(
'SELECT abstract FROM source_records').fetchone()[0], abstract)
finally:
store.conn.close()
if __name__ == '__main__':
unittest.main()