Preserve complete fragmented ICLR 2014 author paragraphs
Metadata integrity checks / metadata (push) Successful in 2m9s
Metadata integrity checks / metadata (push) Successful in 2m9s
This commit is contained in:
@@ -10,7 +10,7 @@
|
||||
|
||||
2015 的 11 篇 oral 全部再次出现在 poster 段落中。相同年份、轨道及 arXiv 标识的重复展示只生成一条来源记录,但每次展示的标题、作者显示字符串、链接、版本和段落均保留在 `source_variants`。两处标题或解析作者冲突时停止采集、等待人工核查。不同年份或不同轨道不合并投稿。
|
||||
|
||||
2014 的作者以分号列出,一条有末尾分号,一条标题不在普通段落中;2016 的代码链接不属于作者。保留原文,解析结果不替代来源证据。oral、poster 作为 `iclr_archive:presentation` 标签,不改成独立投稿轨道。arXiv 版本留在来源变体中,稳定标识使用去版本号的编号。
|
||||
2014 的作者以分号列出,一条有末尾分号,一条标题不在普通段落中,部分作者段落拆成多个格式片段;必须读取完整作者段落,不能只读首个斜体片段。2016 的代码链接不属于作者。保留原文,解析结果不替代来源证据。oral、poster 作为 `iclr_archive:presentation` 标签,不改成独立投稿轨道。arXiv 版本留在来源变体中,稳定标识使用去版本号的编号。
|
||||
|
||||
## 来源快照与运行
|
||||
|
||||
|
||||
@@ -75,12 +75,15 @@ def parse_page(html, year, evidence):
|
||||
continue
|
||||
link = node
|
||||
# This archived Google Sites page has an entry outside a paragraph.
|
||||
# Author spans are explicit italic text, before the next paper link.
|
||||
# Authors can span several differently formatted italic spans.
|
||||
# Read their whole paragraph, never only the first styled fragment.
|
||||
display = None
|
||||
for following in nodes[index+sum(1 for _ in link.walk()):]:
|
||||
if following.tag == 'a' and arxiv_link(following.attrs.get('href', '')):
|
||||
break
|
||||
if (following.tag in ('i', 'em') or 'font-style:italic' in following.attrs.get('style', '').replace(' ', '')):
|
||||
if following.tag == 'p' and any(
|
||||
n.tag in ('i', 'em') or 'font-style:italic' in n.attrs.get('style', '').replace(' ', '')
|
||||
for n in following.walk()):
|
||||
text = clean(following.text())
|
||||
if text:
|
||||
display = text
|
||||
|
||||
@@ -49,11 +49,12 @@ class IclrArchiveTests(unittest.TestCase):
|
||||
def test_2014_author_span_outside_paragraph_and_trailing_separator(self):
|
||||
html = ('<h3>Conference Proceedings</h3><p>conference papers accepted</p>'
|
||||
'<div><b><a href="http://arxiv.org/abs/1312.0001">A <i>research</i> title</a></b></div>'
|
||||
'<p><span style="font-style: italic">Jane Doe; Bob Smith;</span></p>')
|
||||
'<p><span style="font-style: italic">Jane Doe; </span>'
|
||||
'<span style="font-style:italic">Bob Smith; Carol Wu;</span></p>')
|
||||
row, = archive.parse_page(html, 2014, {'observed_at': OBSERVED})
|
||||
self.assertEqual(row['title'], 'A research title')
|
||||
self.assertEqual(row['authors'], ['Jane Doe', 'Bob Smith'])
|
||||
self.assertEqual(row['metadata']['source_variants'][0]['authors_display'], 'Jane Doe; Bob Smith;')
|
||||
self.assertEqual(row['authors'], ['Jane Doe', 'Bob Smith', 'Carol Wu'])
|
||||
self.assertEqual(row['metadata']['source_variants'][0]['authors_display'], 'Jane Doe; Bob Smith; Carol Wu;')
|
||||
self.assertEqual(row['track'], 'main')
|
||||
|
||||
def test_same_arxiv_in_different_tracks_keeps_separate_source_attempts(self):
|
||||
|
||||
Reference in New Issue
Block a user