Compare commits

...

68 Commits

Author SHA1 Message Date
codex-bot d9d675ce56 Complete industry tag review and distinguish service revenue run rates 2026-10-31 02:17:19 +08:00
codex-bot b8a82359e3 Review 20 industry tag candidates and Litmus historical ARR 2026-10-31 02:05:45 +08:00
codex-bot 4682e3513e Review 20 tag-classified industry candidates and current AI relevance 2026-10-31 02:00:16 +08:00
codex-bot c0ff99e640 Complete remaining 39 industry classifications and revenue boundary checks 2026-10-31 01:53:42 +08:00
codex-bot b5e4f817fb Review 20 industry companies and separate services from model claims 2026-10-31 01:44:23 +08:00
codex-bot df227df18c Review 30 industry companies and distinguish revenue commitments 2026-10-31 01:37:48 +08:00
codex-bot 1e81887682 research: review regulated services and separate client assets from income 2026-10-31 01:23:41 +08:00
codex-bot c25fb79eee research: separate industry software from brokerage and operations services 2026-10-31 01:18:08 +08:00
codex-bot a8b054381f research: distinguish industry services and model training inputs 2026-10-31 01:12:51 +08:00
codex-bot 8e5df06fdd research: review industry workflows and historical ARR boundaries 2026-10-31 01:07:22 +08:00
Zayden-Jung ba042ed5ae Merge pull request '扩展 AI 公司研究并修复收入统计表布局' (#31) from codex/ai-research-cn-sme into main
Verify and deploy Tabularium / verify-and-deploy (push) Successful in 9m0s
Reviewed-on: #31
2026-10-30 16:11:48 +00:00
codex-bot 0cc3815bd8 Review sales delivery models and Native founding conflict 2026-10-31 00:11:28 +08:00
codex-bot f49aac2778 Review creative and consumer classifications and historical income 2026-10-31 00:00:28 +08:00
codex-bot d588f2e1fb Review remaining robotics classifications and pilot contract boundaries 2026-10-30 23:49:14 +08:00
codex-bot ad1f351a6b Review software development classifications and delivery models 2026-10-30 23:40:12 +08:00
codex-bot f73d2631fb Review remaining model infrastructure classifications 2026-10-30 23:31:15 +08:00
codex-bot 6dca32dc66 Review compute infrastructure category boundaries 2026-10-30 23:21:43 +08:00
codex-bot 992275560f Resolve candidate years and preserve seller and legacy business boundaries 2026-10-30 23:12:15 +08:00
codex-bot 74c74ddda9 Expand platform income samples and derive report conclusions from current data 2026-10-30 22:58:47 +08:00
codex-bot 61201c1bb0 Review agent infrastructure classifications and financial service boundaries 2026-10-30 22:47:04 +08:00
codex-bot 4eb08c0857 Research 2025 Speedrun companies and separate pilot and client revenue 2026-10-30 22:35:50 +08:00
codex-bot 3e7792191e Audit biotech candidates and separate research milestones from revenue 2026-10-30 22:19:52 +08:00
codex-bot fa09a01421 Audit financial and real-estate candidates without treating asset values as ARR 2026-10-30 22:12:07 +08:00
Zayden-Jung 87987f6cc6 Merge pull request '支持基于内部摘要的章节级文章编辑' (#30) from codex/document-sections into main
Verify and deploy Tabularium / verify-and-deploy (push) Successful in 8m39s
Reviewed-on: #30
2026-10-30 14:05:53 +00:00
codex-bot c9497ccc6b Audit visible government and legal startup candidates with source gaps 2026-10-30 22:04:11 +08:00
codex-bot 05b500e911 Refine physical industry classifications and preserve pivot revenue limits 2026-10-30 21:54:16 +08:00
codex-bot ea09b414a2 Merge income statistics layout fix into AI research delivery 2026-10-30 21:40:21 +08:00
codex-bot 945e7eadb2 Keep income statistics columns readable with grouped rows 2026-10-30 21:36:38 +08:00
codex-bot f86a4a75a0 Expand Speedrun companies and separate contract revenue claims 2026-10-30 21:20:12 +08:00
codex-bot ccba2c546a Expand Chinese SME AI delivery models and physical-AI candidates 2026-10-30 21:11:14 +08:00
Zayden-Jung 56b0ee60c0 Merge pull request '扩展中英文 AI 公司分类与收入核查' (#29) from codex/ai-research-source-lineage into main
Verify and deploy Tabularium / verify-and-deploy (push) Successful in 6m33s
Reviewed-on: #29
2026-10-30 13:05:46 +00:00
codex-bot f7cdf16933 Audit four further TechCrunch candidates and bundled-service boundaries 2026-10-30 20:58:01 +08:00
codex-bot deadd24dce Verify AiMOGA founding and preserve interim revenue disclosure 2026-10-30 20:50:58 +08:00
codex-bot 4f1d3caafd Expand Chinese candidates and audit data-vendor revenue evidence 2026-10-30 20:40:17 +08:00
codex-bot 28ced713dc Preserve unspecified period of the AfterQuery run-rate claim 2026-10-30 20:22:00 +08:00
codex-bot c8c8bad563 Separate revenue run-rate claims from recurring contract evidence 2026-10-30 20:19:03 +08:00
codex-bot 13f2b31e8a Review model-category business boundaries and reconcile Eka history 2026-10-30 20:08:28 +08:00
codex-bot 9ee2bd219a Add chapter editing API with internal summaries and revision control 2026-10-30 19:58:40 +08:00
codex-bot 20e5097783 Record Nirva founder-reported team size and planned price 2026-10-30 19:55:45 +08:00
codex-bot ae4bd203a1 Add three model and wearable companies from archive discovery 2026-10-30 19:50:05 +08:00
codex-bot 5b6d3d9bfd Reconcile four pre-2025 businesses with original source evidence 2026-10-30 19:36:51 +08:00
Zayden-Jung 6b4fa711c1 Merge pull request '补核六项既有业务并保留 Sponstar 年份冲突' (#28) from codex/ai-research-year-proofs into main
Verify and deploy Tabularium / verify-and-deploy (push) Successful in 8m34s
Reviewed-on: #28
2026-10-30 11:32:34 +00:00
codex-bot 76e32490e9 Verify six company histories and retain conflicting founding dates 2026-10-30 19:27:55 +08:00
Zayden-Jung fa0bcb15fb Merge pull request '补齐32项企业竞赛登记并核查19项既有业务' (#26) from codex/ai-research-2025-coverage into main
Verify and deploy Tabularium / verify-and-deploy (push) Has been cancelled
Reviewed-on: #26
2026-10-30 11:20:47 +00:00
codex-bot 12123c278f Resolve fourteen older Battlefield businesses with source evidence 2026-10-30 19:17:43 +08:00
codex-bot 4273dd77e0 Register all 32 enterprise Battlefield discovery entries 2026-10-30 19:09:13 +08:00
Zayden-Jung 3ab2f8e122 Merge pull request '汇总 AI 公司核查更新与文档上传 API' (#25) from codex/research-api-integration into main
Verify and deploy Tabularium / verify-and-deploy (push) Successful in 7m33s
Reviewed-on: #25
2026-10-30 10:28:45 +00:00
codex-bot 15024c3908 merge: include latest company research updates from PR #23 2026-10-30 18:20:34 +08:00
codex-bot 770be83692 Backfill 2025 discovery ledger and verify public archive scope 2026-10-30 18:19:29 +08:00
codex-bot 083f532609 merge: integrate document upload API from PR #24 2026-10-30 18:17:53 +08:00
codex-bot 2b3706c90d merge: integrate company categorization and deployment fixes from PR #23 2026-10-30 18:17:53 +08:00
codex-bot 6bfeea4e7e feat: add scoped document API and resumable batch uploader 2026-10-30 18:14:33 +08:00
codex-bot f156406265 Classify Thrive services rollup and exclude Brila existing operator 2026-10-30 18:12:52 +08:00
codex-bot e65d90c78b Resolve Utopai founding history and preserve Fundraisly entity gaps 2026-10-30 18:10:35 +08:00
codex-bot 96f1417cda Add Goldfish with investor-backed founding-year evidence 2026-10-30 18:08:01 +08:00
codex-bot 347b9587c6 Reconcile Tapflow history and Atoms company-year evidence 2026-10-30 18:04:11 +08:00
codex-bot 677cd73945 Fix Peec team bounds regression assertion 2026-10-30 17:54:08 +08:00
codex-bot e075eee8a5 Separate legacy launches from unresolved Happycapy entity lineage 2026-10-30 17:41:15 +08:00
codex-bot d971362a97 Resolve four launch candidates through operator and app history evidence 2026-10-30 17:31:44 +08:00
codex-bot 0e5ccafa30 Classify remaining directory profiles using official business evidence 2026-10-30 17:21:06 +08:00
Zayden-Jung 0c2a909521 Merge pull request '汇总 README 整理与 Product Hunt 候选历史核查' (#22) from codex/readme-research-integration into main
Verify and deploy Tabularium / verify-and-deploy (push) Failing after 3m51s
Reviewed-on: #22
2026-10-30 09:17:21 +00:00
codex-bot a58a409c38 merge: combine README cleanup and Product Hunt history research 2026-10-30 17:13:48 +08:00
codex-bot c4ab5caac0 merge: integrate Product Hunt candidate history from PR #21 2026-10-30 17:13:13 +08:00
codex-bot 4cbb88181a Exclude four pre-2025 businesses from launch candidates 2026-10-30 17:10:51 +08:00
codex-bot ce87abd1cb docs: streamline README around content management and operations 2026-10-30 17:10:00 +08:00
Zayden-Jung dc7ec078e1 Merge pull request '补充行业基准适用性与收费研究,适配数据库文档' (#19) from codex/ai-research-context into main
Verify and deploy Tabularium / verify-and-deploy (push) Waiting to run
Reviewed-on: #19
2026-10-30 09:05:44 +00:00
codex-bot bba9fddef1 Add benchmark methodology and pricing research after document migration 2026-10-30 17:01:09 +08:00
Zayden-Jung 1302287205 Merge pull request '汇总研究更新、奇思妙想与文档数据库迁移(main-only CI)' (#18) from codex/docs-integration into main
Verify and deploy Tabularium / verify-and-deploy (push) Successful in 7m37s
Reviewed-on: #18
2026-10-30 08:48:28 +00:00
28 changed files with 26273 additions and 4173 deletions
+2 -1
View File
@@ -3,7 +3,8 @@
- 本仓库专门维护 Endor 文档与制图脚本。支持本地开发;CI/CD 仅在 main 使用现有 ces-control Runner,验证通过后自动部署;工作分支与 PR 不运行 CI,确认 Runner 不构成扩大触发范围的授权。不得修改其他项目、共享 Runner 配置或清理共享 Docker 缓存。
- Web 技术栈为 TypeScript、React、Tailwind CSS、Next.js Pages Router;不要切换到 App Router。
- 使用 pnpm 管理 JavaScript workspace,使用 uv 管理 `tools/figures` 的 Python 项目,保留两套锁文件。
- 文档正文、目录元数据与修订记录以站点现有 SQLite 数据库为唯一权威源,使用 docs 管理工具直接更新;不得把新正文、目录 JSON、数据库文件或日常内容快照提交到仓库。历史 Git 文档只用于显式的一次性迁移,不作为运行时种子。Python 图形源码放在 `tools/figures/src/endor_figures`,可复现的 SVG 输出放在 `apps/docs/public/figures`。
- 文档正文、目录元数据与修订记录以站点现有 SQLite 数据库为唯一权威源,使用文档 API 或 docs 管理工具直接更新;不得把新正文、目录 JSON、数据库文件或日常内容快照提交到仓库。历史 Git 文档只用于显式的一次性迁移,不作为运行时种子。Python 图形源码放在 `tools/figures/src/endor_figures`,可复现的 SVG 输出放在 `apps/docs/public/figures`。
- 编辑已有文章时优先使用章节 API:先读取章节目录与内部摘要,据此选择需要读取正文的章节,再提交单章或多章操作。摘要缺失时先读对应正文并补充;修改观点前必须核对该章原文,检查读取版本与提交 revision 一致。摘要只写入内部章节数据,不放入网页正文、目录摘要或 Git;改写正文时同步更新摘要。完整操作与 Python 客户端用法见 README 的「先读摘要,再按章节编辑」。
- 未确认的架构提议明确标注为草案,不能表述为现有实现。
- 本地开发与预览只监听 `127.0.0.1`。部署容器监听 `0.0.0.0:3000`,服务器入口默认 18135,可通过独立 production.env 配置。
- 对 gitea.entroia.com 仅使用 codex-bot,提交身份为 Codex Bot <codex-bot@gitea.entroia.local>。SSH 使用隔离配置:`ssh -F /dev/null -o IdentitiesOnly=yes -i /Users/zzyang/.codex/credentials/gitea-entroia/codex_entroia_ed25519 -p 22222`。
+209 -136
View File
@@ -1,18 +1,38 @@
# Endor · Tabularium
Endor 文档站、研究工作台与制图程序。应用代码在 Git 中评审;全部文档正文、目录元数据和内容修订以站点 SQLite 数据库为唯一权威源。日常写作不需要 commit、构建或部署。
Endor 的文档站与 AI 公司研究工作台,包含文档阅读、全文搜索、研究数据展示和制图工具。使用 Next.js Pages Router、React、TypeScript 与 Tailwind CSS;JavaScript workspace 使用 pnpm,制图 Python 项目使用 uv。
## 存储与读取
## 代码与内容放在哪里
沿用 `AUTH_CONFIG_FILE` 同目录的 `login-state.sqlite` 和 `/auth` 持久挂载。`documents` 存正文及分类、合集、分组、摘要、草案与推荐状态;`document_aliases` 保留旧地址;`document_revisions` 保存每次修改;`document_imports` 标记一次性迁移。没有第二套文档数据库。
文档正文、目录元数据和修订记录由站点 SQLite 数据库管理,日常写作直接更新数据库,无需提交代码、构建或部署。研究业务数据仍沿用版本化 JSON 导入机制,后续单独迁移。
首页、阅读页、目录和全文搜索按请求查询数据库,使用 `private, no-store`。旧文件式地址与改名前的语义地址通过数据库别名跳转,保留查询参数;不再在构建阶段枚举文档。Markdown 原文通过受认证保护的 `/api/documents/source?slug=...` 获取。
| 内容 | 权威来源与维护方式 |
| --- | --- |
| 文档正文、分类、合集、分组、摘要、草案和推荐状态 | 数据库;通过文档 API 或服务器上的 `docs` 管理工具读写 |
| 文档旧地址与修订历史 | 数据库;改名保留别名,修改追加修订 |
| AI 公司、观测、证据、候选与核查记录 | `content/research` 中的版本化快照;服务端显式按版本导入同库后查询 |
| 页面、组件、管理工具和部署逻辑 | 本仓库,通过工作分支与 PR 评审 |
| 图形源码与静态资源 | `tools/figures` 与 `apps/docs/public`;生成步骤在远程 Runner 执行 |
仓库不保存日常正文、目录 JSON、数据库文件或内容快照;历史 Git 记录仅用于迁移。页面不再读取 `content/docs` 或 `content/catalog.json`。Markdown 与 GFM 表格仍可用,不解析原始 HTML。图片在写入时检查压缩格式;图形程序与版本化静态资源仍在代码仓库,本轮不迁移媒体存储。研究业务数据留待下一步。
以上数据库数据共用 `AUTH_CONFIG_FILE` 同目录下的 `login-state.sqlite`,该文件名沿用历史命名。生产环境挂载到容器 `/auth`,不随镜像替换。文档使用 `documents`、`document_aliases`、`document_revisions`、`document_imports` 表,研究数据使用 `research_*` 表,登录状态使用 `login_state` 表。
## 在服务器上管理文档
仓库不接收新的文档正文 Markdown、目录 JSON、数据库文件或日常正文快照。历史 Git 正文仅用于一次性迁移;运行时不读取 `content/docs` 或 `content/catalog.json`。README 维护程序使用与运维约定,研究个案、统计数量与内容更新日志在工作台和数据库文档中查看。
部署安装 `/opt/endorphin-tabularium-docs/docs`,通过当前容器执行管理 CLI。需要服务器的运维权限;不增加可公开写入的 API,不改变现有动态码登录。
## 阅读与访问
首页、阅读页、目录及全文搜索按请求读取数据库,内容更新后刷新即可看到。旧文件式地址和改名前的语义地址继续跳转到当前地址,并保留查询参数。
- 文档原文:`GET /api/documents/source?slug=<文档地址>`。
- 全文搜索:`GET /api/search`。
- 研究工作台:`/docs/other-research/ai-company-research/overview`。
- 收入样本:`/docs/other-research/ai-company-research/income-snapshot`。
- 覆盖与缺项:`/docs/other-research/ai-company-research/coverage`。
正文、搜索、研究 API 与静态资料均经过站点登录认证,私有响应使用 `private, no-store`。Markdown 支持 GFM 表格,不解析原始 HTML;正文图片写入时检查压缩格式。草案标记用于展示状态,不构成单独的访问权限。
## 管理文档
部署后,管理员可在服务器上使用 `/opt/endorphin-tabularium-docs/docs`。该工具在当前容器中操作数据库,需要服务器运维权限。自动化上传使用下文的文档 API,日常读写无需登录服务器。
```sh
sudo /opt/endorphin-tabularium-docs/docs list
@@ -20,174 +40,227 @@ sudo /opt/endorphin-tabularium-docs/docs get endor/endor-purpose-draft
sudo /opt/endorphin-tabularium-docs/docs history 04-endor-purpose-draft
```
新建或修改使用标准输入传入完整 JSON。必填字段:`slug`、`sourceSlug`、`title`、`category`、`collection`、`group`、`description`、`draft`、`featured`、`content` 和 `revision`。新建时 `revision: 0`;修改时先 get 并使用读到的 revision,避免覆盖别人刚写入的版本。`sourceSlug` 是稳定标识,修改公共 slug 时保持它不变,旧地址继续可用。调整目录直接修改 category、collection、group 等字段。
`save` 从标准输入接收完整 JSON。必填字段如下:
| 字段 | 用途 |
| --- | --- |
| `slug` | 语义化公开地址,不含数字章节前缀 |
| `sourceSlug` | 稳定标识;修改公开地址时保持不变 |
| `title`、`content` | 标题与 Markdown 正文 |
| `category`、`collection`、`group` | 分类、合集与目录分组 |
| `description`、`draft`、`featured` | 摘要、草案状态与推荐状态 |
| `revision` | 新建传 `0`;修改传最近一次读取到的版本号 |
修改现有文档的流程:
```sh
sudo /opt/endorphin-tabularium-docs/docs get endor/endor-purpose-draft > /private/work/purpose.json
# 在私有文件中修改正文或目录字段,保留读到的 revision。
sudo /opt/endorphin-tabularium-docs/docs save < /private/work/purpose.json
umask 077
doc_work_dir=$(mktemp -d)
sudo /opt/endorphin-tabularium-docs/docs get endor/endor-purpose-draft > "$doc_work_dir/document.json"
# 编辑该私有 JSON 文件,保留读取到的 revision 和 sourceSlug。
sudo /opt/endorphin-tabularium-docs/docs save < "$doc_work_dir/document.json"
```
路径仅为操作示例,应选择实际存在且受保护的目录。运行成功后刷新网页即可看到正文、导航和搜索更新。版本冲突时重新读取并合并修改;不要把旧 revision 改成新数字强行覆盖。草案是现有展示标记,不是按分支或用户隔离的访问控制。
保存成功后,正文、目录和搜索通过后续请求读取新内容。发生版本冲突时重新读取并合并修改,不能仅改大 `revision` 覆盖并发编辑。调整分类或分组也走同一保存流程;`history` 使用稳定的 `sourceSlug` 查询修订。
新增正文不提交回 Git。奇思妙想这类内容同样通过数据库写入;本次用户明确要求汇总四个 PR,因此该草案已纳入首次迁移清单,保留草案状态。代码分支与数据发布是不同动作,其他分支的内容不会自动发布。
## 文档上传 API
## 首次迁移与备份
文档 API 直接读写同一个数据库,供自动化整理和大量文档上传使用。它只接受独立的文档 API Key,不接受网页 Cookie 或通用网页会话令牌;Key 不能用于研究、登录或其他站点 API。
本次汇总先合入 PR #14、#15、#16,再应用 PR #17 的数据库迁移。最新研究正文及奇思妙想草案合计 51 篇,连同目录可从固定历史提交 `cffca6942b1bc0005321e1685f38593fb0b853d5` 完整导出;其中 AI 公司研究三篇采用汇总期间追加的 `74af205b2959e1bb6a1fe9f567bc87069ac6d12c` 历史版本,保留 Wabi 更新:
| 方法与地址 | 用途 |
| --- | --- |
| `GET /api/documents?limit=50&offset=0` | 分页读取目录元数据;不返回全部正文 |
| `GET /api/documents/item/<slug>` | 读取单篇完整正文、目录字段和当前 revision;支持旧地址 |
| `GET /api/documents/item/<slug>?history=1&limit=50&offset=0` | 分页读取修订元数据与时间,按新到旧排列 |
| `GET /api/documents/item/<slug>?revision=1` | 读取指定修订的完整正文 |
| `POST /api/documents` | 保存一个完整文档 JSON;字段与 `docs save` 相同 |
| `POST /api/documents/batch` | 保存 `{ "documents": [...] }`,每个请求 1—100 篇 |
| `GET /api/documents/sections/<slug>` | 读取章节 ID、标题和内部摘要,不返回正文 |
| `GET /api/documents/sections/<slug>?section=<id>` | 读取选中章节正文;重复 `section` 可选多章 |
| `GET /api/documents/sections/<slug>?revision=2` | 读取对应修订的章节目录;也可搭配 `section` |
| `POST /api/documents/sections/<slug>` | 原子执行章节新增、更新、删除和移动 |
写入使用 `Authorization: Bearer <文档 API Key>`。新建传 `revision: 0`;更新先读取当前全文,再使用读到的 revision。每篇正文最多 2,000,000 个 JavaScript 字符单位;每次请求最多 8 MiB。目录和修订分页的 `limit` 范围为 1—100,响应包含 `total` 和 `nextOffset`。
单个批量请求在同一事务提交,任意一篇格式错误、版本冲突或地址冲突都会整体回滚。成功返回文档地址、稳定标识和新修订号;版本或幂等键冲突返回 `409`,格式错误返回 `400`,超出请求大小返回 `413`。缺少、过期或撤销的 Key 返回 `401`,只读 Key 写入返回 `403`。
建议写入时提供 `Idempotency-Key`,由 1—128 个字母、数字或 `._:-` 组成。同一个 Key、相同请求 ID 和相同输入重试时,返回原回执并标记 `replayed: true`,不追加修订;相同 ID 改换输入会冲突。回执描述原请求,不代表之后没有其他修改;需要当前内容时重新读取文档。改过正文或 revision 后,应使用新的请求 ID。
### 先读摘要,再按章节编辑
章节接口使用相同的文档 Key 和读写权限。编辑流程为:先读章节目录及摘要,选择需要深入的章节,再读取这些章节的正文,最后提交一组操作。不要只凭摘要修改未读正文;目录与正文响应的 `revision` 应一致,否则重新读取。接口不调用模型生成摘要:由编辑 Agent 阅读正文、撰写摘要,并在编辑请求中保存。
章节按行首 Markdown ATX 二级标题(`##`)划分,三级至六级标题随所属章节一起编辑;代码围栏内的标题不会切章。二级标题之前的文档标题及导言属于固定 ID `preamble`,可以修改但不能删除或移动。Setext 标题、带缩进或引用、列表标记的标题不作为章节边界。正文无行首二级标题时,全文作为导言处理。拆分及组合保持换行和未编辑内容原样,不自动格式化 Markdown。
目录中的每章包含 `id`、`title`、`position`、`summary`、`summaryStatus` 和 `contentHash`。摘要为编辑内部数据,单独保存在 SQLite 的 `document_sections` 和 `document_section_revisions` 中,不加入普通文档响应、网页 props、搜索索引或页面正文。完整数据库备份包含这些表;`docs export` 仍只导出当前文档,不包含内部摘要和修订。
旧文章无需重写,首次读取即可按原文返回目录,初始摘要为空,`summaryStatus` 为 `missing`。Agent 应按需读取这些章节,生成摘要并通过 `update` 保存;保存后状态为 `current`。这里的 `current` 仅表示摘要与该版正文一起保存,不保证模型的总结正确。全文接口仍兼容;未变化章节保留摘要与 ID,正文变化的章节清空摘要,等待更新。全文改名且改写章节时可能产生新 ID,因此全文保存后应重新读取目录。
章节编辑请求示例(示例 ID 需要替换为目录实际返回的值):
```json
{
"revision": 3,
"operations": [
{
"op": "update",
"id": "s_existing",
"content": "## 原有章节\n\n修改后的正文。\n\n",
"summary": "本章说明修改后的核心观点与边界。"
},
{
"op": "insert",
"after": "s_existing",
"content": "## 新章节\n\n新增论证。\n\n",
"summary": "补充上一章尚未覆盖的论证。"
}
]
}
```
`update` 可只改 `summary`,也可改正文;正文改变时若不提供新摘要,旧摘要会清空。`insert` 必须提供正文和非空摘要;摘要最多 8000 个 JavaScript 字符单位。`delete` 使用 `{ "op": "delete", "id": "s_existing" }`;`move` 使用 `{ "op": "move", "id": "s_existing", "after": "preamble" }`。按数组顺序执行,最多 100 个操作;新章 ID 由服务器生成,在回执目录中返回。章正文应从自己的 `##` 标题开始,只包含一个二级标题,并保留与下一章之间的换行;跨章代码围栏或章节边界改变会被拒绝,拆章请使用多个操作。
章节变更、内部摘要、组合后的全文和历史快照在同一事务内提交。每批操作只增加一次文章 `revision`;摘要单独更新也增加版本号。后续操作失败或全文版本过期时全部回滚。写入沿用幂等规则,回执包含新版本、章节目录和 `replayed`;章节接口与全文接口的幂等记录分开。修改章节不会改变其 ID,移动亦保留 ID;删除后的正文和摘要仍可从对应的旧修订读取。启用前的历史修订只有原文,摘要显示缺失。
Python 客户端对应命令:
```sh
python3 scripts/export-legacy-documents.py --output .artifacts/documents-migration.json
python3 scripts/documents-client.py outline endor/endor-purpose-draft
python3 scripts/documents-client.py sections endor/endor-purpose-draft preamble s_existing
python3 scripts/documents-client.py edit-sections endor/endor-purpose-draft /private/path/edit.json --request-id chapter-edit-1
```
导出文件位于 Git 忽略目录、权限 0600,不进入镜像。脚本也支持显式 `--ref`,但不能用未评审分支替换生产迁移基线。它不会在站点读取时执行。CI 使用历史导出验证全部正文与目录的逐项一致性,构建镜像只含程序,不含文档快照或数据库。
`outline` 和 `sections` 支持 `--revision`;请求文件只包含 `revision` 和 `operations`,不要把内部摘要文件提交到 Git。
首次部署先备份整个共享数据库,再显式事务导入到空文档表。普通部署使用 `import --if-empty`,已有内容时跳过,不以历史数据覆盖数据库编辑;重复同一导入也不会重置内容。导入失败则事务整体回滚,既有登录与研究表保留,部署中止。
### 创建、查看与撤销 Key
管理员只需通过已验证的服务器连接进行 Key 管理:
```sh
sudo /opt/endorphin-tabularium-docs/docs key-create codex-upload /auth/codex-docs-key.json read-write 90
sudo /opt/endorphin-tabularium-docs/docs key-list
sudo /opt/endorphin-tabularium-docs/docs key-revoke <key-id>
```
创建命令将完整 Key 写入认证私有目录中的指定新文件,权限 `0600`,终端仅输出 ID、权限和到期信息。文件已存在时拒绝覆盖。权限可选 `read` 或 `read-write`;有效期默认 90 天,可设 1—365 天。数据库仅保存 Key 摘要,列表不返回原文;撤销后不能继续认证。独立 Key 文件和认证 config.json 都不包含在 SQLite 整库备份中。
### 使用上传客户端
`scripts/documents-client.py` 仅使用 Python 标准库。通过 `TABULARIUM_DOCS_URL` 指定站点 origin,通过 `TABULARIUM_DOCS_KEY_FILE` 指定本机私有 Key 文件(支持创建命令输出的 JSON),不把 Key 放入命令行参数。
客户端要求 HTTPS;尚未配置 HTTPS 时,可通过已验证的 SSH 连接建立本机端口转发,连接 `http://127.0.0.1:18136`。以下 `$DOCS_SSH_TARGET` 应事先设置为受信任的 SSH 别名或 `user@host`:
```sh
ssh -N -L 18136:127.0.0.1:18135 "$DOCS_SSH_TARGET"
```
将服务器私有目录中的 Key 文件复制到受保护的本机位置后,在另一个终端配置客户端。`$TABULARIUM_DOCS_KEY_FILE` 文件须仅本人可读写,不能提交到仓库:
```sh
export TABULARIUM_DOCS_URL=http://127.0.0.1:18136
export TABULARIUM_DOCS_KEY_FILE="$HOME/.config/tabularium/codex-docs-key.json"
python3 scripts/documents-client.py list --limit 50
python3 scripts/documents-client.py get endor/endor-purpose-draft
python3 scripts/documents-client.py get endor/endor-purpose-draft --history
python3 scripts/documents-client.py get endor/endor-purpose-draft --revision 1
```
修改流程与 CLI 相同:
```sh
umask 077
api_work_dir=$(mktemp -d)
python3 scripts/documents-client.py get endor/endor-purpose-draft > "$api_work_dir/document.json"
# 编辑正文或目录字段,保留 sourceSlug 和读取到的 revision。
python3 scripts/documents-client.py save "$api_work_dir/document.json"
```
大量文档可使用仅含 `documents` 数组的私有 JSON 文件。客户端按 100 篇和 8 MiB 请求上限自动分批,并为各批派生固定的幂等请求 ID。每批独立提交;后续批次失败时,前面的成功批次保留。客户端会打印根请求 ID 和各批进度,使用相同 ID 与未修改的输入重试可跳过已完成的批次;可通过 `--request-id` 显式指定根 ID(客户端最多 100 个字符)。若后续批次因业务冲突失败,按进度中已提交的文档数量提取尚未成功的部分,修正后使用新的根 ID 上传。
客户端只自动重试一次临时网络或 `502/503/504` 错误,保持正文和请求 ID 不变;`409` 等业务错误不会自动覆盖。它不跟随 HTTP 重定向,避免将 Bearer 凭据转发到其他地址。正文和 Key 继续保存在数据库或私有文件中,不因使用 API 而进入 Git。
## 研究数据与检查
研究工作台提供公司筛选、经营指标、原始证据、收费与交付模式、技术材料及核查记录。它区分公司披露、支付平台指标、第三方估算、融资和客户收益;未知收入不填零,材料可访问也不代表性能或商业效果已经验证。研究范围、样本限制和个案结论以工作台记录为准。
| 仓库资料 | 用途 |
| --- | --- |
| `content/research/ai-companies.seed.json` | 研究数据导入快照;更新必须更换 `version` |
| `content/research/ai-companies.sources.json` | 来源范围、用途、覆盖边界与后续缺项 |
| `content/research/ai-companies.coverage.json` | 候选池逐条筛选台账 |
| `content/research/ai-income-analysis.json` | 独立复算的统计参考快照 |
服务端以事务导入未载入的版本,保留旧研究版本与登录、文档数据;同一版本标识下内容变化会拒绝导入。页面和 `GET /api/research/ai-companies` 从数据库读取对应研究版本。回退程序可能选择旧研究版本,不能等同于撤销数据库中的正文编辑。
以下 Python 检查不触发编译或构建,可在工作分支执行:
```sh
PYTHONDONTWRITEBYTECODE=1 python3 scripts/audit-ai-research.py --check
PYTHONDONTWRITEBYTECODE=1 python3 scripts/summarize-ai-income.py --check
```
若需对照研究数据检查报告正文,先用 `docs export` 导出当前文档,再给对应脚本传入 `--document-export <导出文件>`。默认 `--check` 核对研究数据的统计与审计不变量,不将历史迁移快照当作当前正文。脚本不带参数时仅在 `.artifacts/reports` 生成候选正文,需审阅后通过 `docs save` 写入数据库。
## 备份与首次迁移
```sh
sudo /opt/endorphin-tabularium-docs/docs backup /auth/manual-backup.sqlite
sudo /opt/endorphin-tabularium-docs/docs export /auth/current-documents.json
```
backup 是包含登录、研究、正文、别名和修订的完整 SQLite 一致性备份;export 只导出当前正文与目录,作为空库初始化/外部迁移快照,不代替完整备份或恢复修订历史。输出路径均在容器内,文件已存在时拒绝覆盖。备份含认证状态,应留在私有挂载或受控备份位置,不能放到公开静态目录、仓库或日志。
路径在容器内,输出文件已存在时拒绝覆盖。`backup` 是整个共享 SQLite 数据库的一致性备份,包含登录状态、研究数据、正文、别名和修订;`export` 只保存当前正文与目录,供空库初始化或外部迁移使用,不包含完整修订历史。认证配置单独保存在 `/auth/config.json`,整库备份不包含该配置文件。备份与导出应保存在私有挂载或受控备份位置。
代码回退不会撤销数据库编辑,但回退到旧文件式站点会显示旧正文。恢复完整数据库会同时影响登录和研究状态,必须先停止写入、保留当前备份并由管理员明确选择恢复点;部署脚本不会自动恢复数据库或删除新内容。
首次迁移由 `scripts/export-legacy-documents.py` 从固定历史版本导出正文和目录,基线与研究正文覆盖版本在脚本中明确记录。导出写入 Git 忽略目录,权限为 `0600`,不进入生产镜像:
## 代码验证
```sh
python3 scripts/export-legacy-documents.py --output .artifacts/documents-migration.json
```
使用 React、Next.js Pages Router、TypeScript 和 Tailwind CSS。JavaScript 用 pnpm,制图 Python 用 uv。编译、构建、打包和构建产物全部在已确认的 ces-control Runner 执行;`pnpm test:docs` 等测试脚本会调用 esbuild,也只能在 Runner 执行。
该命令用于迁移准备,不是日常写作入口;目标文件存在时拒绝覆盖。部署先备份共享数据库,再用 `import --if-empty` 向空文档表事务导入。已有文档时跳过,不覆盖后续编辑;导入失败整体回滚并中止部署。`--ref` 仅用于明确选择历史导出版本。
工作分支与 PR 不运行 CI,仅做不会触发编译、构建的轻量检查。用户手动合并 main 后,才在 ces-control 执行 lint、类型检查、数据库事务/版本测试、现有认证与研究测试、图形检查、构建与容器冒烟;验证通过才沿用既有部署流程。
程序回退不会撤销数据库编辑;回退到旧文件式站点则会显示旧正文。完整数据库恢复会同时影响登录、研究和文档状态,需先停止写入、保留当前备份,再明确选择恢复点。部署脚本只回退镜像,不自动恢复数据库。
冒烟使用隔离数据库卷:完整导入后逐项核对全部正文与目录,验证新建、更新、重命名、搜索立即变化及容器重启后的持久化,并覆盖全部文档地址、认证、媒体和原文端点。测试卷不会触及生产数据库。
## 登录管理
`tools/figures/src/endor_figures` 与 `apps/docs/public/figures` 仍保存图形源码及可复现的静态资源。图形构建也在远程执行。
## 动态码登录与二维码
与 PROMPT 的参考流程一致:共享验证器的6位TOTP动态码,每30秒更新;成功登录的HttpOnly、SameSite=Lax会话有效7天。此方案是动态码单因素登录,不是密码加动态码的双因素组合。
正文、SVG、文档数据与API都经过认证,仅登录入口、登录所需代码资源和最小健康检查公开。20次/5分钟的共享尝试上限与已用时间步持久保存在SQLite中,跨进程和重启继续生效。缺失或损坏配置时拒绝访问,不自动降为公开站点。
生产认证文件位于服务器 `/opt/endorphin-tabularium-docs/private-auth`,目录0700、文件0600,只挂载进本项目容器。部署只初始化一次,后续更新保留同一验证器密钥与会话签名密钥。生产镜像、Git、CI日志和网页不包含密钥。
管理员通过SSH管理:
站点使用共享验证器的 6 位 TOTP 动态码,每 30 秒更新。会话使用 HttpOnly、SameSite=Lax Cookie,有效期 7 天;共享尝试上限为每 5 分钟 20 次,重放与限流状态在数据库中持久保存。缺少或损坏认证配置时拒绝私有访问。
```sh
sudo /opt/endorphin-tabularium-docs/auth qr
sudo /opt/endorphin-tabularium-docs/auth verify
```
`qr`只允许交互终端显示;`qr-png`用于通过SSH将PNG直接重定向到私有本地文件,不在CI中运行。二维码等同登录凭据,不放入本仓库或网页。只有明确执行 `auth rotate --confirm` 才更换密钥并使旧二维码、会话失效;重新显示或下载二维码不会轮换。
认证配置和数据库位于服务器 `/opt/endorphin-tabularium-docs/private-auth`。正常部署保留验证器与会话签名密钥。`qr` 只在交互终端显示,`qr-png` 可导出到私有文件;只有 `auth rotate --confirm` 会轮换密钥并使旧会话失效。二维码和令牌不进入仓库、网页或 CI 日志。
构建、打包与会触发构建的测试只在用户确认的 ces-control Runner 上执行。本地不执行 `pnpm dev`、`pnpm build`、`pnpm typecheck` 或打包测试。
当前 Compose 配置默认 `AUTH_COOKIE_SECURE=false`;实际采用 HTTPS 时,应在独立的生产配置中设置为 `true`。HTTP 不加密动态码与会话传输。生产部署使用无副作用的 `verify`,不执行会消耗动态码和触发限流的 `exercise`。
当前服务器入口为HTTP,与参考项目一致使用 `AUTH_COOKIE_SECURE=false`;配置HTTPS后应在独立production.env设置 `AUTH_COOKIE_SECURE=true`。HTTP本身不加密动态码和会话传输。
## 开发、验证与发布
`pnpm test:auth`运行算法、会话、重放及限流测试。容器验证使用自己的匿名测试卷完成登录/并发重放/CSRF/限流测试,并在重启后检查状态仍存在。生产部署只运行无副作用的 `verify`,不会消耗用户动态码或触发生产限流。
代码依赖要求见 `package.json` 和锁文件;当前要求 Node.js 22.13+、pnpm 11.25.0,制图使用 Python 与 uv。构建、打包、制图及会隐式编译的检查只在已确认的 `ces-control` Runner 执行,本机不执行这些步骤。
## AI 公司研究
工作分支完成轻量检查后推送并创建 PR,由用户手动合并。**CI 仅在 main 推送或针对 main 的手动运行时执行,工作分支和 PR 不运行 CI。** 不自动合并,也不因确认 Runner 而扩大触发范围。
导航入口为 **其他研究 → AI 公司研究**,页面路径是
`/docs/other-research/ai-company-research/overview`。工作台提供业务分类、公司搜索与筛选、
公司明细、收入观测、原始证据和数据核查记录。所有页面与
`GET /api/research/ai-companies` 均使用文档站现有的登录认证。
主线通过 `.gitea/workflows/ci-deploy.yml` 执行 `scripts/ci.py`:
研究业务数据本轮暂不迁移,仍沿用版本化 JSON 导入机制。研究数据、文档与登录状态**共用文档站现有的 SQLite 数据库文件**:
`AUTH_CONFIG_FILE` 同目录下的 `login-state.sqlite`。该文件名是历史名称,未创建第二个数据库。
`login_state` 保留登录限流与重放状态,新增的 `research_*` 表保存数据版本、分类、
公司、经营指标、证据与核查记录。沿用现有 `/auth` 持久挂载,数据库不会写入镜像或公开目录。
1. 检查研究统计、审计与 Python 测试,生成首次迁移所需的历史导出。
2. 根据程序、研究快照、图形、锁文件和构建脚本计算内容指纹。命中 verified 镜像时复用,否则安装依赖并执行 lint、类型检查、数据库测试、图形检查与构建。
3. 使用隔离数据库卷核对完整迁移、认证、全部文档与资源,以及不重建内容更新和重启持久化;通过后标记镜像为 verified。
4. `scripts/deploy.py` 确认仍为最新 main,备份并初始化文档数据,使用 `--no-build` 部署;失败时回退上一版镜像。
`content/research/ai-companies.seed.json` 是可评审的研究导入快照,
服务端首次读取该版本时将它以事务导入同一数据库,之后从数据库查询记录。
导入使用外键校验,不删除登录数据,也不覆盖旧研究版本。每次更新快照必须修改 `version`;
同一版本内容变化会拒绝导入,避免静默读取旧数据。回退代码时继续读取对应旧版本。
所有修改都经过工作分支和 PR,部署仍等待人工合并主线。
`pnpm check`、`pnpm typecheck`、`pnpm test:auth`、`pnpm test:research`、`pnpm test:docs`、`pnpm figures`、`pnpm build` 和 `pnpm dev` 等命令涉及编译、构建或产物生成,不在本机运行。图形源码与生成资源保持可复现,前端继续使用 Pages Router。
当前资料包含 1,021 条公司/产品记录、10 类业务及待分类组、1,201 条证据记录和 187 条经营与技术观测。
其中保留 93 家个案资料、912 条目录基础资料,并新增 16 个平台产品/业务样本;目录项不等于完成财务和技术核验。
Product Hunt两年档案各首批19张、合计38张发布/推广卡片以`candidateBatches`保存;公司成立年待核者不自动新增公司。其中18条仍待核成立年;排除依据区分正式成立字段与2025前已运营证据,精确注册日期未核者明确标注。候选元数据与其他研究记录同库持久化,覆盖报告逐条展示状态、缺项和公司关联。Kilo Code与KiloClaw两项映射一个已被收购的2025公司个案,采用规模、token吞吐及同名健康公司的收入不作为其ARR。
机器之心首批4条公司线索以独立候选批次记录:直达正文未读,经公司公开主页、官方代码或发起方公告补核PrismShadow与Stargate;SSI用2024官方历史更新排除,Utopai仍待核。不是两年中文来源全量。
TechCrunch栏目首批3条公司候选已追溯原始材料;Wabi据两个公开名录2025年份入表,精确注册日未核。官方条款称当前免费,收入仍未知,不填零。DreamHQ及Thrive Holdings成立年仍待核,后者2025公开发布公告另存。DreamHQ与名录同名摄像头公司区分,ADT拟收购尚待交割,融资、估值和目标公司业绩不当新AI公司的ARR。分类接口与RSS访问403及两年档案缺项保留。
补充ThinkWheel与中国NaiveAI;MiroMind成立年未核,公开研发年份单列。保留灵初智能成立年排除、深度原理日期待核和Naive近名实体复核记录。
新增7条Speedrun个案,保留标题/正文冲突、七位数数量级、币种不明、合同管线与履历归属排除记录;相对日期不充当同期收入时点。核查页优先按人工确认的公司ID关联明细,旧版本保留按证据归属回退;68条核查中52条指向具体公司、4条属于全局方法、12条是未纳入候选。明细反向列出关联核查;币种不明的声明保留原值,不绘制美元收入长度条。
Lightfield按Magical Tome主体与2025前历史业务列为CRM转型对照,成立年未知,不计新成立公司;Fundraisly成立年待核,其客户融资和创始人历史ARR不转移为公司收入。
Bud保留为2025成立且YC标记已收购的案例;Rocket.new按前身2021成立、2025产品上线的转型对照记录,历史ARR不混入新成立公司同期统计。
原候选36088 EveryApp从AI不明经补证入表,`directoryRechecks`保存后续复核,原1,428条台账及928条原始新增计数不改。OpenSEO与EveryApp不重复计公司;定价、星标、客户数和数据成本加价均不转为ARR。
验收按2026-10-30确认的公开来源清单边界逐条核对并标明缺项;YC候选池有完整读取台账,其他来源目前为定向补充,不能把来源出现次数当作覆盖完成。收入披露不足且统计时点不统一,不代表全部 AI 公司。
界面将公司自述、第三方估算、融资、客户收入和履历收入分开,未知收入不记为零。
保留被撤回的声明作为核查记录,并排除收入比较;并购后业务、管理层预测与历史里程碑单列。
团队规模下界在列表中显示比较符,商业成熟度展示公开交付证据,不等于技术能力排名。
Emergent的1亿美元ARR里程碑补足至2026年2月;Peec采用规模和Wonderful团队规模各保留其公告日期,不将较新非收入指标当作ARR更新。Wonderful按2026年9月官方综合平台定位归C05。
公司披露的 ARR 尚未建立同期同口径比较组。新增平台 MRR 单独计算探索性样本统计,
不能称行业平均 ARR、总体离散排名或财务造假概率。
根 README 等非构建输入变更不要求重新构建;数据库正文和目录更新也不触发构建。保留 pnpm、uv、Next.js 与 SWC 缓存,不清理共享 Docker 缓存或卷,不修改共享 Runner 或其他项目。
收入分布:`IncomeDistribution.tsx` 从数据库返回的观测中筛选平台 USD MRR,
每个产品只保留符合条件的最新值,默认数据日期距截止日不超过7天。
按业务类别、支付来源及可选收费模式分组,显示均值、中位数、样本CV(标准差分母n−1)、
均值/中位数和移除最大值后的均值敏感性。n<3标注样本不足;n=1不计算CV。
MRR×12仅作为显示单位转换,不改写为公司ARR;公司ARR、估算与近30日收入均不混入。
`ai-income-analysis.json` 是独立Python复算的7日/90日快照参考,用于检查统计选择和计算,
页面本身从同一SQLite中的公司与观测记录计算,不读取第二个数据库。
## 部署位置与仓库
平台同步时间、连接过期状态、业务描述的可编辑性和产品/法律主体未去重限制均保留。
覆盖审计页位于 `/docs/other-research/ai-company-research/coverage`,分类链接 `overview?category=<id>` 的筛选数据。
`python3 scripts/audit-ai-research.py --check` 检查候选池逐条计数与来源审计一致性;CI在ces-control运行此检查。
收入快照报告位于 `/docs/other-research/ai-company-research/income-snapshot`,样本名称链接 `overview?company=<id>` 的公司明细。
`python3 scripts/summarize-ai-income.py --check` 独立复核统计与参考快照一致性;不触发编译或构建。数据变更后先审查方法,再运行不带参数的命令生成 `.artifacts/reports` 下的候选正文,使用 docs 管理工具显式更新数据库;不再生成受 Git 管理的正文。
默认7日样本14个产品;90日窗口新增Gojiberry的8月历史点,营销样本的均值和CV随之明显变化。
这提示先核查时间与样本构成,不能将高离散度直接解释为造假。两个近名营养应用按不同App Store ID分别建档。
| 项目 | 位置或默认值 |
| --- | --- |
| Compose 项目 | `endorphin-tabularium-docs` |
| 容器监听 | `0.0.0.0:3000` |
| 服务器入口 | 默认端口 `18135`,由生产配置决定 |
| 生产配置 | `/etc/endorphin-tabularium-docs/production.env` |
| 发布记录与当前版本 | `/opt/endorphin-tabularium-docs/releases/<commit>` 与 `current` |
| 健康检查 | `/api/health` |
覆盖核查:从 yc-oss 非官方镜像发现 2025—2026 八个 YC 批次的 1,428 条候选,
逐条读取对应 YC 官方公开页面,以页面中的成立年份、团队字段和业务描述筛选。
962 条初筛匹配,其中 34 条与原表 URL 重合,实际新增 928 条;268 条年份不符或缺失、
192 条 AI 业务待确认、6 条成立年份晚于批次年份未纳入本批新增。
`content/research/ai-companies.sources.json` 保留最初12项来源与后续补充渠道、用途、覆盖边界及下一步;覆盖报告逐渠道复算证据、公司及观测数量。背景报告、入口可读和候选池逐条读取分别标记,零计数不表示没有公司。
`content/research/ai-companies.coverage.json` 保留逐条筛选结果、来源与响应摘要,
数据快照中的 `coverage` 也随版本进入同一 SQLite 数据库。只覆盖该候选池,不能称全球全量。
服务入口使用 `TABULARIUM_BIND`、`TABULARIUM_PORT` 配置。部署前检查端口是否空闲或属于本服务,不停止其他服务抢占端口;本项目不自动修改防火墙、域名或反向代理。
新增业务分类优先匹配主营简介,再参考目录标签;未匹配者进入待分类组。
初分规则位于 `scripts/classify-ai-profiles.py`,只生成待评审标签,不执行抓取或构建。
目录基础项保留 `classificationBasis` 和 `directoryTags` 供复核。
当前目录基础项中160条已人工复核主营与买家,Contrario及14条收入声明/排除案例已深化为个案。`directoryOrigin`保留原发现来源,`profileOnly`只表示当前资料深度;深化不改变原候选池新增928条的审计分母。
分类按主要工作任务和买家判断,例如Paragon归客户与收入运营,Oki归消费者应用,Lucidic归Agent设施。市场支出、采用规模、性能主张及客户收益均不作为公司收入。
其中121条原待分类记录已按YC官方业务介绍人工复核主营业务,分类理由保存在公司字段与对应证据摘要中;
仍有3条因简介不足保留待分类。人工复核分类不代表独立验证技术水平、客户效果或财务数据。
AI 关键词可能出现在业务背景中,收入、买家和交付状态仍须逐家深查,不能把目录项的未知收入当作未披露。
技术专项评审用独立字段记录层级、日期、摘要与证据:未评审、已读技术自述、公开代码/模型材料可访问、有第三方独立评测记录。
当前13条完成该字段的专项记录,其中6条确认材料可访问、1条记录第三方已发表评测(Gradium TTS Sep 2026);本轮未自行运行模型测试。缺字段不表示无技术能力,代码或模型卡可读不证明性能。
界面支持按资料深度、记录团队规模和技术验证证据筛选;证据列表每页 30 条,避免一次渲染全部来源。
`pnpm test:research` 在远程 Runner 上验证同库持久化、重复读取、版本导入与回退、
登录限流保留、引用完整性及收入排除规则。镜像冒烟检查会验证研究页面与 API 的认证、
返回记录数量、只读方法和容器重启后的读取结果。页面筛选交互仍需实际浏览器验证。
## CI/CD 与缓存
文档内容已退出构建输入。若需对照研究数据检查数据库中的报告正文,可先导出当前文档,再给研究脚本传入 `--document-export`;默认 `--check` 只核对研究数据本身的统计和审计不变量,避免把历史迁移快照继续当作日常正文源。
`.gitea/workflows/ci-deploy.yml` 使用已有 ces-control Runner,不修改共享 Runner 配置。
工作流仅在 main 推送或针对 main 的手动运行时执行,工作分支与 PR 不触发 CI。确认 ces-control 是构建位置,不构成新增分支 CI 的授权;报告须区分轻量检查与主线 CI。
1. 对实际构建输入计算 SHA-256:前端源码、图片、Python 制图源码、研究业务快照、依赖锁文件、Dockerfile 和构建脚本。
2. 命中 `verified-<scope>-<摘要>` 镜像时跳过依赖安装和 Next.js build,只做认证、实际路由与资源冒烟检查。
3. 输入变化时运行 lint、类型检查、制图一致性检查与构建。pnpm store、uv 和 `.next/cache` 使用持久 Docker 卷,main 与每个 PR 独立;额外保留 Next.js SWC 下载缓存。
4. 生产镜像只包含 Next.js standalone 产物;Python 留在检查容器,不进入运行镜像。镜像通过动态码认证、重放/限流、重启持久化、健康、全部文档、JS/CSS、SVG 和 404 检查后才标记为 verified。
5. 部署前检查是否仍为最新 main,使用 `--no-build` 启动,失败恢复上一版镜像。
根 README、工作流或部署配置等非构建输入变化不会触发 Next.js 重建;数据库中的正文和目录变化直接通过请求读取,不触发任何构建。首次构建必须安装依赖并完整生成产物。缓存可能因基础镜像、工具链或锁文件变化而失效;不要清理共享 Docker 缓存或卷。
默认入口为服务器的 **18135** 端口。容器监听 3000,使用独立 Compose 项目 `endorphin-tabularium-docs`。部署前先检查端口是否空闲(或属于本服务),不会停止其他服务以抢占端口。服务器配置位于 `/etc/endorphin-tabularium-docs/production.env`:
```dotenv
TABULARIUM_BIND=0.0.0.0
TABULARIUM_PORT=18135
```
发布记录位于 `/opt/endorphin-tabularium-docs/releases/<commit>`,`current` 指向最近成功发布。修改服务器入口配置后手动重跑 main 工作流即可,复用现有镜像。
健康检查:`/api/health`。本项目不修改 Hestia、PROMPT 或其他服务,也不自动修改防火墙、域名和反向代理。
## Git 远程
仓库:[endorphin-tabularium-docs](https://gitea.entroia.com/Zayden-Jung/endorphin-tabularium-docs)。开发者 Git 操作使用 codex-bot 专用 SSH;Actions 检出使用 Gitea 内置任务令牌,不保存检出凭据。
仓库:[endorphin-tabularium-docs](https://gitea.entroia.com/Zayden-Jung/endorphin-tabularium-docs)。Git 操作仅使用 `codex-bot` 专用 SSH 身份与仓库级提交配置;Actions 检出使用 Gitea 内置任务令牌且不保存凭据。
+44 -3
View File
@@ -18,6 +18,7 @@ import {
totp,
isLoginRedirect,
} from "../src/lib/auth";
import { createDocumentKey, revokeDocumentKey } from "../src/lib/document-keys";
import { readDocument, saveDocument } from "../src/lib/docs";
import { consumeCode, initializeStore } from "../src/lib/auth-store";
@@ -181,10 +182,10 @@ async function main() {
assert.equal(researchResponse.status, 200, "authenticated research API");
assert.ok(researchResponse.headers.get("cache-control")?.includes("no-store"));
const research = await researchResponse.json();
assert.equal(research.companies.length, 1021);
assert.equal(research.companies.length, 1051);
assert.equal(research.categories.length, 11);
assert.equal(research.evidence.length, 1201);
assert.equal(research.observations.length, 187);
assert.equal(research.evidence.length, 2235);
assert.equal(research.observations.length, 314);
assert.equal((await request("/api/research/ai-companies", { method: "POST", headers })).status, 405);
const researchPage = await request("/docs/other-research/ai-company-research/overview", { headers });
assert.equal(researchPage.status, 200);
@@ -210,6 +211,7 @@ async function main() {
assert.equal(raw.status, 200);
assert.ok((await raw.text()).startsWith("# Endor"));
if (process.env.DOCUMENT_SMOKE_TEST) {
if (command === "exercise") await exerciseDocumentApi(request, headers);
const sourceSlug = "ci-live-document";
const previous = readDocument(sourceSlug);
if (command === "exercise") {
@@ -356,3 +358,42 @@ main().catch((error) => {
);
process.exitCode = 1;
});
async function exerciseDocumentApi(request: (path: string, init?: RequestInit) => Promise<Response>, session: Record<string, string>) {
const writer = createDocumentKey("isolated API smoke", "read-write", 1);
const reader = createDocumentKey("isolated read smoke", "read", 1);
const headers = { Authorization: "Bearer " + writer.token, "Content-Type": "application/json", "Idempotency-Key": "smoke-batch" };
const readHeaders = { Authorization: "Bearer " + reader.token, "Content-Type": "application/json" };
const make = (name: string) => ({ slug: `endor/${name}`, sourceSlug: name, title: "API smoke",
category: "Endor", collection: "Endor 文档", group: "CI", description: "Isolated API smoke",
draft: true, featured: false, content: "# API smoke", revision: 0 });
const payload = { documents: [make("ci-api-first"), make("ci-api-second")] };
try {
assert.equal((await request("/api/documents")).status, 401);
assert.equal((await request("/api/documents", { method: "POST", headers: { ...session, "Content-Type": "application/json" }, body: JSON.stringify(payload.documents[0]) })).status, 401);
assert.equal((await request("/api/research/ai-companies", { headers })).status, 401, "Document keys cannot read research");
assert.equal((await request("/api/search", { headers })).status, 401, "Document keys cannot authenticate other APIs");
assert.equal((await request("/api/documents/batch", { method: "POST", headers: readHeaders, body: JSON.stringify(payload) })).status, 403);
const saved = await request("/api/documents/batch", { method: "POST", headers, body: JSON.stringify(payload) });
assert.equal(saved.status, 200);
assert.equal((await saved.json()).documents.length, 2);
const replay = await request("/api/documents/batch", { method: "POST", headers, body: JSON.stringify(payload) });
assert.equal(replay.status, 200); assert.equal((await replay.json()).replayed, true);
const current = await request("/api/documents/item/endor/ci-api-first", { headers: readHeaders });
assert.equal(current.status, 200); assert.equal((await current.json()).revision, 1);
const page = await request("/api/documents?limit=1", { headers: readHeaders });
assert.equal(page.status, 200);
const listing = await page.json();
assert.equal(listing.documents.length, 1); assert.equal(listing.documents[0].content, undefined);
const bad = { documents: [{ ...payload.documents[0], revision: 1, content: "Must roll back" }, payload.documents[1]] };
const conflict = await request("/api/documents/batch", { method: "POST", headers: { ...headers, "Idempotency-Key": "smoke-conflict" }, body: JSON.stringify(bad) });
assert.equal(conflict.status, 409);
const unchanged = await request("/api/documents/item/endor/ci-api-first", { headers: readHeaders });
assert.equal((await unchanged.json()).content, "# API smoke");
const history = await request("/api/documents/item/endor/ci-api-first?history=1", { headers });
assert.equal(history.status, 200); assert.equal((await history.json()).total, 1);
revokeDocumentKey(writer.id);
assert.equal((await request("/api/documents", { headers })).status, 401);
} finally { revokeDocumentKey(writer.id); revokeDocumentKey(reader.id); }
console.log("Document API smoke passed: scoped keys, batch rollback, pagination, history, replay and revocation; no secrets displayed");
}
+29 -3
View File
@@ -1,11 +1,37 @@
import { readFileSync, writeFileSync } from "node:fs";
import { readFileSync, writeFileSync, openSync, closeSync, unlinkSync } from "node:fs";
import { resolve, dirname, sep } from "node:path";
import { createDocumentKey, listDocumentKeys, revokeDocumentKey, type DocumentKeyScope } from "../src/lib/document-keys";
import { openSiteDatabase } from "../src/lib/site-database";
import { allDocuments, readDocument, saveDocument, importDocuments, documentHistory } from "../src/lib/docs";
process.umask(0o077);
const [command, argument] = process.argv.slice(2);
const [command, argument, ...extra] = process.argv.slice(2);
try {
switch (command) {
case "key-create": {
const authFile = process.env.AUTH_CONFIG_FILE;
if (!authFile || !argument || !extra[0]) throw Error("Usage: key-create <name> <private-file> [read|read-write] [days]");
const file = resolve(extra[0]);
const directory = resolve(dirname(authFile));
if (!file.startsWith(directory + sep)) throw Error("Key output must be inside the private authentication directory");
const fd = openSync(file, "wx", 0o600);
let created: { id: string } | undefined;
try {
const key = createDocumentKey(argument, (extra[1] ?? "read-write") as DocumentKeyScope, Number(extra[2] ?? 90));
created = key;
writeFileSync(fd, JSON.stringify(key) + "\n");
console.log(JSON.stringify({ id: key.id, scope: key.scope, expiresAt: key.expiresAt, saved: file }));
} catch (error) {
if (created) revokeDocumentKey(created.id);
unlinkSync(file);
throw error;
} finally { closeSync(fd); }
break;
}
case "key-list": console.log(JSON.stringify(listDocumentKeys(), null, 2)); break;
case "key-revoke":
if (!argument) throw Error("Provide key id");
revokeDocumentKey(argument); console.log("Document key revoked"); break;
case "backup": {
if (!argument) throw Error("Provide a private backup filename");
const db = openSiteDatabase();
@@ -36,7 +62,7 @@ try {
{ flag: "wx", mode: 0o600 });
console.log("Document export created; existing files are never overwritten"); break;
}
default: throw Error("Usage: docs backup <private-file> | list | get <slug> | save < input.json | history <sourceSlug> | import [--if-empty] < export.json | export <private-file>");
default: throw Error("Usage: docs key-create <name> <private-file> [read|read-write] [days] | key-list | key-revoke <id> | backup <private-file> | list | get <slug> | save < input.json | history <sourceSlug> | import [--if-empty] < export.json | export <private-file>");
}
} catch (error) {
console.error(error instanceof Error ? error.message : "Document operation failed");
+3 -4
View File
@@ -6,9 +6,8 @@ import { spawnSync } from "node:child_process";
const dir = await mkdtemp(join(tmpdir(), "tabularium-docs-tests-"));
try {
const file = join(dir, "docs.test.cjs");
await build({ entryPoints: ["tests/docs.test.ts"], bundle: true,
platform: "node", target: "node22", format: "cjs", outfile: file });
const result = spawnSync(process.execPath, ["--test", file], { stdio: "inherit" });
await build({ entryPoints: ["tests/docs.test.ts", "tests/documents-api.test.ts"], bundle: true,
platform: "node", target: "node22", format: "cjs", outdir: dir, outExtension: { ".js": ".cjs" } });
const result = spawnSync(process.execPath, ["--test", join(dir, "docs.test.cjs"), join(dir, "documents-api.test.cjs")], { stdio: "inherit" });
process.exitCode = result.status ?? 1;
} finally { await rm(dir, { recursive: true, force: true }); }
@@ -7,9 +7,9 @@ import {
const pageSize = 12;
const evidencePageSize = 30;
const revenueLabels = { self: "公司披露,待核口径", estimate: "仅第三方估算", platform: "仅支付平台记录", unknown: "暂无可用收入数字" };
const revenueLabels = { self: "企业披露,待核口径", estimate: "仅第三方估算", platform: "仅支付平台记录", unknown: "暂无可用收入数字" };
const useLabels: Record<string, string> = {
platform: "支付平台记录", signal: "公司披露信号", estimate: "第三方估算", historical: "历史里程碑",
platform: "支付平台记录", signal: "企业披露信号", estimate: "第三方估算", historical: "历史里程碑",
"historical-estimate": "其他时点估算", excluded: "排除收入比较", scale: "团队规模", adoption: "采用规模",
funding: "融资", technical: "第三方技术评测", crosscheck: "同一里程碑的其他资料",
};
@@ -138,7 +138,7 @@ export default function AICompanyResearch() {
<label>资料深度<select value={depth} onChange={(e) => { setDepth(e.target.value); setPage(0); }}><option value="">全部资料</option><option value="case">原有个案研究</option><option value="profile">目录基础资料</option></select></label>
<label>记录团队规模<select value={teamBand} onChange={(e) => { setTeamBand(e.target.value); setPage(0); }}><option value="">全部规模</option><option value="small">1—5 人</option><option value="medium">6—20 人</option><option value="large">21—50 人</option><option value="xl">至少 51 人</option><option value="unknown">未披露或分组不确定</option></select></label>
<label>技术验证证据<select value={technicalLevel} onChange={(e) => { setTechnicalLevel(e.target.value); setPage(0); }}><option value="">全部验证层级</option>{Object.entries(technicalReviewLabels).map(([value, label]) => <option key={value} value={value}>{label}</option>)}</select></label>
<label>收入资料<select value={income} onChange={(e) => { setIncome(e.target.value); setPage(0); }}><option value="">全部状态</option><option value="self">找到公司披露</option><option value="estimate">仅第三方估算</option><option value="platform">仅支付平台记录</option><option value="unknown">暂无可用数字</option></select></label>
<label>收入资料<select value={income} onChange={(e) => { setIncome(e.target.value); setPage(0); }}><option value="">全部状态</option><option value="self">找到企业披露</option><option value="estimate">仅第三方估算</option><option value="platform">仅支付平台记录</option><option value="unknown">暂无可用数字</option></select></label>
</div>
<p className="research-muted">技术验证层级记录我们查到了什么,不是能力强弱评分。技术专项评审与业务分类复核分别记录。</p>
<div className="research-result-count" role="status">{filtered.length} 条匹配记录<button onClick={() => { setCategory(""); setQuery(""); setCohort(""); setIncome(""); setDepth(""); setTeamBand(""); setTechnicalLevel(""); setPage(0); }}>清除筛选</button></div>
@@ -154,14 +154,14 @@ export default function AICompanyResearch() {
</>}
{tab === "revenue" && <>
<IncomeDistribution data={data} onSelect={setSelected} />
<h3>公司披露与估算的历史里程碑</h3>
<p className="research-notice">这一组包含 {disclosedCompanies} 家公司的披露或估算信号,多为异时点历史里程碑,ARR 与年化收入口径仍待核对,暂不计算其同期领域均值。币种未确认的记录仅显示原值,不绘制美元长度条。上面的平台MRR统计与这一组分开。</p>
<h3>企业披露与估算的历史里程碑</h3>
<p className="research-notice">这一组包含 {disclosedCompanies} 家公司的披露或估算信号;企业披露含公司或交易当事方公开材料,披露主体与审计状态见证据。多为异时点历史里程碑,期间收入不直接年化,ARR 与年化收入口径仍待核对,暂不计算其同期领域均值。币种未确认的记录仅显示原值,不绘制美元长度条。上面的平台MRR统计与这一组分开。</p>
<div className="research-revenue-chart" aria-label="不同时间的公开收入里程碑,线性长度,非同行排名">{observed.map((o) => <div key={o.id}>
<div><button onClick={() => setSelected(o.company)}>{o.name}</button><strong>{o.bound !== "=" && o.bound}{money(o.value, o.unit)}</strong></div>
{o.unit === "USD" && <span className="research-bar" data-estimate={o.selection === "estimate" || undefined} aria-hidden="true"><i style={{width: `${o.value / Math.max(1, ...observed.filter((v) => v.unit === "USD").map((v) => v.value)) * 100}%`}} /></span>}
<small>{o.kind} · {useLabels[o.selection]} · {o.timing}</small>
</div>)}</div>
<div className="research-table-wrap"><table><caption>当前研究数据中的收入可得性,目录项未逐一检索;未建立同期、同龄、同口径比较组</caption><thead><tr><th>业务分类</th><th>公司披露数</th><th>仅估算数</th><th>仅平台数</th><th>暂无可用数字</th><th>同期均值 / 离散度</th></tr></thead><tbody>{summaries.map((c) => <tr key={c.id}><td><button onClick={() => chooseCategory(c.id)}>{c.name}</button></td><td>{c.self}</td><td>{c.estimate}</td><td>{c.platform}</td><td>{c.unknown}</td><td>条件不足</td></tr>)}</tbody></table></div>
<div className="research-table-wrap"><table><caption>当前研究数据中的收入可得性,目录项未逐一检索;未建立同期、同龄、同口径比较组</caption><thead><tr><th>业务分类</th><th>企业披露数</th><th>仅估算数</th><th>仅平台数</th><th>暂无可用数字</th><th>同期均值 / 离散度</th></tr></thead><tbody>{summaries.map((c) => <tr key={c.id}><td><button onClick={() => chooseCategory(c.id)}>{c.name}</button></td><td>{c.self}</td><td>{c.estimate}</td><td>{c.platform}</td><td>{c.unknown}</td><td>条件不足</td></tr>)}</tbody></table></div>
<p className="research-muted">离散度用于筛选值得核查的异常,还需排除公司年龄、口径和披露偏差;不能据此认定财务造假。</p>
</>}
{tab === "sources" && <><p className="research-muted">来源链接保留原始位置。相同 URL 的记录可能属于同一披露,不能按记录数计算独立证据数量。</p><ul className="research-evidence-list">{data.evidence.slice(sourcePage * evidencePageSize, (sourcePage + 1) * evidencePageSize).map((e) => <Evidence key={e.id} item={e} />)}</ul>{data.evidence.length > evidencePageSize && <nav className="research-pagination" aria-label="原始证据分页"><button disabled={sourcePage === 0} onClick={() => setSourcePage((n) => n - 1)}>上一页</button><span>{sourcePage + 1} / {Math.ceil(data.evidence.length / evidencePageSize)}</span><button disabled={(sourcePage + 1) * evidencePageSize >= data.evidence.length} onClick={() => setSourcePage((n) => n + 1)}>下一页</button></nav>}</>}
@@ -75,7 +75,7 @@ export default function IncomeDistribution({ data, onSelect }: {
</svg></div>
<figcaption>圆点:产品观测 竖线:样本均值 菱形:样本中位数。业务类别标题下各行按支付平台{splitModels && "及收费与交付模式"}分别统计;n为该组产品数。点击点或下表产品名可看原始证据。</figcaption>
</figure>
<div className="research-table-wrap"><table><caption>{metric}样本统计;每行是类别下的一个统计组,CV使用样本标准差</caption><thead><tr><th scope="col">业务类别</th><th scope="col">支付平台 / 收费模式</th><th scope="col">n</th><th scope="col">均值</th><th scope="col">中位数</th><th scope="col">均值/中位数</th><th scope="col">CV</th><th scope="col">去除最大值后的均值 / 下降幅度</th><th scope="col">产品与限制</th></tr></thead>{categoryGroups.map(({ category, rows }) => <tbody key={category}>{rows.map((group, index) => <tr key={group.key}>
<div className="research-table-wrap"><table className="income-statistics"><caption>{metric}样本统计;每行是类别下的一个统计组,CV使用样本标准差</caption><thead><tr><th scope="col">业务类别</th><th scope="col">支付平台 / 收费模式</th><th scope="col">n</th><th scope="col">均值</th><th scope="col">中位数</th><th scope="col">均值/中位数</th><th scope="col">CV</th><th scope="col">去除最大值后的均值 / 下降幅度</th><th scope="col">产品与限制</th></tr></thead>{categoryGroups.map(({ category, rows }) => <tbody key={category}>{rows.map((group, index) => <tr key={group.key}>
{index === 0 && <th scope="rowgroup" rowSpan={rows.length}>{categories.get(category)}<small>{rows.length}个统计组</small></th>}
<td>{group.provider}<small>{group.model}</small></td><td>{group.stats.n}</td><td>{usd(group.stats.mean)}</td><td>{usd(group.stats.median)}</td><td>{group.stats.meanMedianRatio?.toFixed(2) ?? "—"}</td><td>{group.stats.cv?.toFixed(2) ?? "—"}</td><td>{usd(group.stats.meanWithoutLargest)}<small>{percent(group.stats.largestRemovalDrop)}</small></td>
<td>{group.points.map((point) => <span key={point.company.id}><button className="research-company-name" onClick={() => onSelect(point.company.id)}>{point.company.name}</button>{" "}</span>)}<small>{group.stats.n < 3 ? "样本少于3,不能据此判断领域典型收入。" : "自选的小样本,须结合产品年龄、成本与留存评估。"}</small></td>
+178 -18
View File
@@ -1,5 +1,9 @@
import { randomUUID, createHash } from "node:crypto";
import { openSiteDatabase } from "./site-database";
import { DocumentValidationError, DocumentConflictError } from "./document-errors";
import { sectionSchema, loadSections, reconcileSections, persistSections, splitSections, outline,
applySectionOperations, parseSectionEdit, type DocumentSection } from "./document-sections";
export { DocumentValidationError, DocumentConflictError } from "./document-errors";
export type DocLink = {
slug: string; sourceSlug: string; category: string; title: string; searchText?: string;
@@ -9,6 +13,7 @@ export type DocLink = {
export type Doc = DocLink & { content: string };
export type DocInput = Omit<Doc, "revision" | "searchText">;
export type DocumentExport = { format: "tabularium-documents-v1"; documents: Doc[] };
export type DocumentReceipt = Pick<Doc, "slug" | "sourceSlug" | "revision">;
export const validSlug = /^[a-z0-9]+(?:-[a-z0-9]+)*(?:\/[a-z0-9]+(?:-[a-z0-9]+)*)*$/;
function database() {
@@ -25,58 +30,131 @@ function database() {
document_id TEXT NOT NULL REFERENCES documents(id), revision INTEGER NOT NULL,
payload TEXT NOT NULL, created_at TEXT NOT NULL, PRIMARY KEY(document_id, revision)
);
CREATE TABLE IF NOT EXISTS document_write_requests (
namespace TEXT NOT NULL, request_id TEXT NOT NULL, digest TEXT NOT NULL,
response TEXT NOT NULL, PRIMARY KEY(namespace, request_id)
);
CREATE TABLE IF NOT EXISTS document_imports (
digest TEXT PRIMARY KEY, document_count INTEGER NOT NULL, created_at TEXT NOT NULL
);
`);
sectionSchema(db);
return db;
}
export function validateDocument(input: DocInput): void {
if (!input || typeof input !== "object") throw Error("Invalid document");
if (!input || typeof input !== "object") throw new DocumentValidationError("Invalid document");
for (const key of ["slug", "sourceSlug"] as const)
if (typeof input[key] !== "string" || !validSlug.test(input[key])) throw Error(`Invalid ${key}`);
if (input.slug.split("/").some((part) => /^\d+-/.test(part))) throw Error("Use a semantic public slug");
if (typeof input[key] !== "string" || !validSlug.test(input[key])) throw new DocumentValidationError(`Invalid ${key}`);
if (input.slug.split("/").some((part) => /^\d+-/.test(part))) throw new DocumentValidationError("Use a semantic public slug");
for (const key of ["title", "category", "collection", "group"] as const)
if (typeof input[key] !== "string" || !input[key].trim()) throw Error(`Missing ${key}`);
if (typeof input[key] !== "string" || !input[key].trim()) throw new DocumentValidationError(`Missing ${key}`);
if (typeof input.content !== "string" || typeof input.description !== "string" ||
typeof input.draft !== "boolean" || typeof input.featured !== "boolean") throw Error("Invalid document fields");
if (input.content.length > 2_000_000) throw Error("Document exceeds content limit");
typeof input.draft !== "boolean" || typeof input.featured !== "boolean") throw new DocumentValidationError("Invalid document fields");
if (input.content.length > 2_000_000) throw new DocumentValidationError("Document exceeds content limit");
const images = [
...input.content.matchAll(/!\[[^\]]*\]\(<?([^\s)>]+)/g),
...input.content.matchAll(/<img\b[^>]*\bsrc=["']([^"']+)/gi),
];
if (images.some(([, source]) => /\.(?:png|jpe?g|tiff?|bmp)(?:[?#]|$)/i.test(source)))
throw Error("Use compressed WebP images instead of original rasters");
throw new DocumentValidationError("Use compressed WebP images instead of original rasters");
}
function write(db: ReturnType<typeof database>, input: DocInput, expectedRevision: number): Doc {
function write(db: ReturnType<typeof database>, input: DocInput, expectedRevision: number, editedSections?: DocumentSection[]): Doc {
validateDocument(input);
const existing = db.prepare("SELECT id, revision FROM documents WHERE source_slug=?")
.get(input.sourceSlug) as { id: string; revision: number } | undefined;
if ((existing?.revision ?? 0) !== expectedRevision) throw Error("Document revision conflict");
const existing = db.prepare("SELECT id, revision, payload FROM documents WHERE source_slug=?")
.get(input.sourceSlug) as { id: string; revision: number; payload: string } | undefined;
if ((existing?.revision ?? 0) !== expectedRevision) throw new DocumentConflictError("Document revision conflict");
const id = existing?.id ?? randomUUID();
const previous = existing ? loadSections(db, id, (JSON.parse(existing.payload) as Doc).content) : [];
const doc: Doc = { slug: input.slug, sourceSlug: input.sourceSlug, category: input.category,
title: input.title, group: input.group, collection: input.collection, description: input.description,
draft: input.draft, featured: input.featured, content: input.content, revision: expectedRevision + 1 };
const payload = JSON.stringify(doc);
// Aliases remain reserved after renaming; collision rolls back the whole write.
db.prepare(`INSERT INTO documents VALUES (?,?,?,?,?) ON CONFLICT(id) DO UPDATE SET
slug=excluded.slug, revision=excluded.revision, payload=excluded.payload`)
.run(id, doc.slug, doc.sourceSlug, doc.revision, payload);
// Reserve both canonical and historical names before updating any rows.
for (const alias of new Set([doc.slug, doc.sourceSlug])) {
const prior = db.prepare("SELECT document_id FROM document_aliases WHERE alias=?")
.get(alias) as { document_id: string } | undefined;
if (prior && prior.document_id !== id) throw Error(`Document alias collision: ${alias}`);
db.prepare("INSERT OR IGNORE INTO document_aliases VALUES (?,?)").run(alias, id);
if (prior && prior.document_id !== id) throw new DocumentConflictError(`Document alias collision: ${alias}`);
}
db.prepare(`INSERT INTO documents VALUES (?,?,?,?,?) ON CONFLICT(id) DO UPDATE SET
slug=excluded.slug, revision=excluded.revision, payload=excluded.payload`)
.run(id, doc.slug, doc.sourceSlug, doc.revision, payload);
for (const alias of new Set([doc.slug, doc.sourceSlug]))
db.prepare("INSERT OR IGNORE INTO document_aliases VALUES (?,?)").run(alias, id);
db.prepare("INSERT INTO document_revisions VALUES (?,?,?,?)")
.run(id, doc.revision, payload, new Date().toISOString());
persistSections(db, id, doc.revision, editedSections ?? reconcileSections(doc.content, previous));
return doc;
}
// Internal summaries never enter Doc, catalog, search or browser page props.
export function readDocumentSections(slug: string, revision?: number) {
if (!validSlug.test(slug)) throw new DocumentValidationError("Invalid document slug");
if (revision !== undefined && (!Number.isSafeInteger(revision) || revision < 1)) throw new DocumentValidationError("Invalid revision");
const db = database();
try {
db.exec("BEGIN");
const row = db.prepare(`SELECT d.id,d.payload FROM documents d JOIN document_aliases a
ON a.document_id=d.id WHERE a.alias=?`).get(slug) as { id: string; payload: string } | undefined;
if (!row) { db.exec("COMMIT"); return null; }
let doc = JSON.parse(row.payload) as Doc;
let sections: DocumentSection[];
if (revision === undefined || revision === doc.revision) sections = loadSections(db, row.id, doc.content);
else {
const prior = db.prepare("SELECT payload FROM document_revisions WHERE document_id=? AND revision=?")
.get(row.id, revision) as { payload: string } | undefined;
if (!prior) { db.exec("COMMIT"); return null; }
doc = JSON.parse(prior.payload) as Doc;
const snapshot = db.prepare("SELECT payload FROM document_section_revisions WHERE document_id=? AND revision=?")
.get(row.id, revision) as { payload: string } | undefined;
sections = snapshot ? JSON.parse(snapshot.payload) as DocumentSection[] : splitSections(doc.content);
}
db.exec("COMMIT");
return { slug: doc.slug, sourceSlug: doc.sourceSlug, title: doc.title, revision: doc.revision, sections };
} finally { db.close(); }
}
export function editDocumentSections(slug: string, value: unknown, request?: { namespace: string; id: string }) {
if (!validSlug.test(slug)) throw new DocumentValidationError("Invalid document slug");
const edit = parseSectionEdit(value);
if (request && (!request.namespace || !/^[A-Za-z0-9._:-]{1,128}$/.test(request.id))) throw new DocumentValidationError("Invalid Idempotency-Key");
const digest = createHash("sha256").update(JSON.stringify({ slug, edit })).digest("hex");
const namespace = request ? request.namespace + ":sections" : undefined;
const db = database();
try {
db.exec("BEGIN IMMEDIATE");
try {
if (request) {
const prior = db.prepare("SELECT digest,response FROM document_write_requests WHERE namespace=? AND request_id=?")
.get(namespace!, request.id) as { digest: string; response: string } | undefined;
if (prior) {
if (prior.digest !== digest) throw new DocumentConflictError("Idempotency-Key was used with different content");
db.exec("COMMIT");
return { ...JSON.parse(prior.response) as { slug: string; sourceSlug: string; revision: number; sections: ReturnType<typeof outline> }, replayed: true };
}
}
const row = db.prepare(`SELECT d.id,d.payload FROM documents d JOIN document_aliases a
ON a.document_id=d.id WHERE a.alias=?`).get(slug) as { id: string; payload: string } | undefined;
if (!row) { db.exec("COMMIT"); return null; }
const doc = JSON.parse(row.payload) as Doc;
if (doc.revision !== edit.revision) throw new DocumentConflictError("Document revision conflict");
const previous = loadSections(db, row.id, doc.content);
const sections = applySectionOperations(previous, edit.operations);
// Preserve the initial outline when upgrading an existing document for the first time.
db.prepare("INSERT OR IGNORE INTO document_section_revisions VALUES (?,?,?)").run(row.id, doc.revision, JSON.stringify(previous));
const saved = write(db, { ...doc, content: sections.map((s) => s.content).join("") }, doc.revision, sections);
const receipt = { slug: saved.slug, sourceSlug: saved.sourceSlug, revision: saved.revision, sections: outline(sections) };
if (request) db.prepare("INSERT INTO document_write_requests VALUES (?,?,?,?)")
.run(namespace!, request.id, digest, JSON.stringify(receipt));
db.exec("COMMIT");
return { ...receipt, replayed: false };
} catch (error) { db.exec("ROLLBACK"); throw error; }
} finally { db.close(); }
}
export function saveDocument(input: DocInput, expectedRevision: number): Doc {
if (!Number.isSafeInteger(expectedRevision) || expectedRevision < 0) throw Error("Provide expected revision");
if (!Number.isSafeInteger(expectedRevision) || expectedRevision < 0) throw new DocumentValidationError("Provide expected revision");
const db = database();
try {
db.exec("BEGIN IMMEDIATE");
@@ -145,3 +223,85 @@ export function documentHistory(sourceSlug: string): Doc[] {
.map((row) => JSON.parse(row.payload) as Doc);
} finally { db.close(); }
}
// API batches commit entirely or not at all; receipts make lost-response retries safe.
export function saveDocumentBatch(documents: Doc[], request?: { namespace: string; id: string }) {
if (!Array.isArray(documents) || !documents.length || documents.length > 100)
throw new DocumentValidationError("Provide 1 to 100 documents");
const seen = new Set<string>();
for (const doc of documents) {
validateDocument(doc);
if (!Number.isSafeInteger(doc.revision) || doc.revision < 0)
throw new DocumentValidationError("Provide expected revision");
if (seen.has(doc.sourceSlug)) throw new DocumentValidationError("Duplicate sourceSlug in batch");
seen.add(doc.sourceSlug);
}
if (request && (!request.namespace || !/^[A-Za-z0-9._:-]{1,128}$/.test(request.id)))
throw new DocumentValidationError("Invalid Idempotency-Key");
const digest = createHash("sha256").update(JSON.stringify(documents)).digest("hex");
const db = database();
try {
db.exec("BEGIN IMMEDIATE");
try {
if (request) {
const prior = db.prepare("SELECT digest,response FROM document_write_requests WHERE namespace=? AND request_id=?")
.get(request.namespace, request.id) as { digest: string; response: string } | undefined;
if (prior) {
if (prior.digest !== digest) throw new DocumentConflictError("Idempotency-Key was used with different content");
db.exec("COMMIT");
return { documents: JSON.parse(prior.response) as DocumentReceipt[], replayed: true };
}
}
const receipts = documents.map((input) => {
const { slug, sourceSlug, revision } = write(db, input, input.revision);
return { slug, sourceSlug, revision };
});
if (request) db.prepare("INSERT INTO document_write_requests VALUES (?,?,?,?)")
.run(request.namespace, request.id, digest, JSON.stringify(receipts));
db.exec("COMMIT");
return { documents: receipts, replayed: false };
} catch (error) { db.exec("ROLLBACK"); throw error; }
} finally { db.close(); }
}
export function documentPage(limit = 50, offset = 0) {
if (!Number.isSafeInteger(limit) || limit < 1 || limit > 100 || !Number.isSafeInteger(offset) || offset < 0)
throw new DocumentValidationError("Invalid pagination");
const db = database();
try {
db.exec("BEGIN");
const total = (db.prepare("SELECT COUNT(*) AS n FROM documents").get() as { n: number }).n;
const documents = (db.prepare("SELECT json_remove(payload,'$.content') AS payload FROM documents ORDER BY source_slug LIMIT ? OFFSET ?")
.all(limit, offset) as { payload: string }[]).map((row) => JSON.parse(row.payload) as DocLink);
db.exec("COMMIT");
return { documents, total, limit, offset, nextOffset: offset + documents.length < total ? offset + documents.length : null };
} finally { db.close(); }
}
export function documentHistoryPage(sourceSlug: string, limit = 50, offset = 0) {
if (!Number.isSafeInteger(limit) || limit < 1 || limit > 100 || !Number.isSafeInteger(offset) || offset < 0)
throw new DocumentValidationError("Invalid pagination");
const db = database();
try {
db.exec("BEGIN");
const total = (db.prepare(`SELECT COUNT(*) AS n FROM document_revisions r JOIN documents d
ON d.id=r.document_id WHERE d.source_slug=?`).get(sourceSlug) as { n: number }).n;
const revisions = (db.prepare(`SELECT json_remove(r.payload,'$.content') AS payload, r.created_at AS createdAt
FROM document_revisions r JOIN documents d ON d.id=r.document_id
WHERE d.source_slug=? ORDER BY r.revision DESC LIMIT ? OFFSET ?`)
.all(sourceSlug, limit, offset) as { payload: string; createdAt: string }[])
.map((row) => ({ ...JSON.parse(row.payload) as DocLink, createdAt: row.createdAt }));
db.exec("COMMIT");
return { revisions, total, limit, offset, nextOffset: offset + revisions.length < total ? offset + revisions.length : null };
} finally { db.close(); }
}
export function documentRevision(sourceSlug: string, revision: number): Doc | null {
if (!Number.isSafeInteger(revision) || revision < 1) throw new DocumentValidationError("Invalid revision");
const db = database();
try {
const row = db.prepare(`SELECT r.payload FROM document_revisions r JOIN documents d
ON d.id=r.document_id WHERE d.source_slug=? AND r.revision=?`).get(sourceSlug, revision) as { payload: string } | undefined;
return row ? JSON.parse(row.payload) as Doc : null;
} finally { db.close(); }
}
+2
View File
@@ -0,0 +1,2 @@
export class DocumentValidationError extends Error {}
export class DocumentConflictError extends Error {}
+56
View File
@@ -0,0 +1,56 @@
import { createHash, randomBytes, randomUUID } from "node:crypto";
import { openSiteDatabase } from "./site-database";
export type DocumentKeyScope = "read" | "read-write";
export type DocumentKey = { id: string; name: string; scope: DocumentKeyScope;
createdAt: number; expiresAt: number; revokedAt: number | null; lastUsedAt: number | null };
function database() {
const db = openSiteDatabase();
db.exec(`CREATE TABLE IF NOT EXISTS document_api_keys (
id TEXT PRIMARY KEY, name TEXT NOT NULL, digest TEXT NOT NULL UNIQUE,
scope TEXT NOT NULL CHECK(scope IN ('read','read-write')), created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL, revoked_at INTEGER, last_used_at INTEGER
);`);
return db;
}
export function createDocumentKey(name: string, scope: DocumentKeyScope = "read-write", days = 90) {
if (!name.trim() || name.length > 100 || !["read", "read-write"].includes(scope) ||
!Number.isSafeInteger(days) || days < 1 || days > 365) throw Error("Invalid key name, scope or lifetime");
const id = randomUUID();
const token = "tdoc_" + randomBytes(32).toString("base64url");
const createdAt = Date.now();
const expiresAt = createdAt + days * 86400000;
const db = database();
try {
db.prepare("INSERT INTO document_api_keys(id,name,digest,scope,created_at,expires_at) VALUES (?,?,?,?,?,?)")
.run(id, name.trim(), createHash("sha256").update(token).digest("hex"), scope, createdAt, expiresAt);
return { id, token, name: name.trim(), scope, createdAt, expiresAt };
} finally { db.close(); }
}
export function authenticateDocumentKey(token: string | undefined, now = Date.now()) {
if (!token || !/^tdoc_[A-Za-z0-9_-]{43}$/.test(token)) return null;
const db = database();
try {
const row = db.prepare("SELECT id,scope,last_used_at AS lastUsedAt FROM document_api_keys WHERE digest=? AND revoked_at IS NULL AND expires_at>?")
.get(createHash("sha256").update(token).digest("hex"), now) as { id: string; scope: DocumentKeyScope; lastUsedAt: number | null } | undefined;
if (!row) return null;
if (row.lastUsedAt === null || row.lastUsedAt < now - 60000)
db.prepare("UPDATE document_api_keys SET last_used_at=? WHERE id=? AND (last_used_at IS NULL OR last_used_at<?)")
.run(now, row.id, now - 60000);
return { id: row.id, scope: row.scope };
} finally { db.close(); }
}
export function listDocumentKeys(): DocumentKey[] {
const db = database();
try {
return db.prepare(`SELECT id,name,scope,created_at AS createdAt,expires_at AS expiresAt,
revoked_at AS revokedAt,last_used_at AS lastUsedAt FROM document_api_keys ORDER BY created_at,id`).all() as DocumentKey[];
} finally { db.close(); }
}
export function revokeDocumentKey(id: string) {
const db = database();
try {
const result = db.prepare("UPDATE document_api_keys SET revoked_at=COALESCE(revoked_at,?) WHERE id=?").run(Date.now(), id);
if (!result.changes) throw Error("Document key not found");
} finally { db.close(); }
}
+144
View File
@@ -0,0 +1,144 @@
import { createHash, randomUUID } from "node:crypto";
import type { DatabaseSync } from "node:sqlite";
import { DocumentValidationError } from "./document-errors";
export type DocumentSection = {
id: string; title: string; content: string; summary: string;
};
export type SectionOperation =
| { op: "update"; id: string; content?: string; summary?: string }
| { op: "insert"; after: string; content: string; summary: string }
| { op: "delete"; id: string }
| { op: "move"; id: string; after: string };
export type SectionEdit = { revision: number; operations: SectionOperation[] };
const hash = (text: string) => createHash("sha256").update(text).digest("hex");
// Preserve every byte, including CRLF and separators. H3-H6 belong to their H2 chapter.
export function splitSections(content: string): DocumentSection[] {
const starts: { offset: number; title: string }[] = [{ offset: 0, title: "导言" }];
let offset = 0;
let fence: { marker: string; length: number } | undefined;
for (const line of content.match(/[^\n]*\n|[^\n]+$/g) ?? []) {
const opening = line.match(/^ {0,3}(`{3,}|~{3,})(.*?)(?:\r?\n)?$/);
if (fence) {
if (new RegExp(`^ {0,3}${fence.marker}{${fence.length},}[ \\t]*\\r?\\n?$`).test(line)) fence = undefined;
} else if (opening && !(opening[1][0] === "`" && opening[2].includes("`"))) {
fence = { marker: opening[1][0], length: opening[1].length };
} else {
const heading = line.match(/^##(?:[ \t]+(.*?))?[ \t]*\r?\n?$/);
if (heading) starts.push({ offset, title: (heading[1] ?? "").replace(/[ \t]+#+[ \t]*$/, "").trim() });
}
offset += line.length;
}
const seen = new Map<string, number>();
return starts.map((start, index) => {
const occurrence = seen.get(start.title) ?? 0;
seen.set(start.title, occurrence + 1);
return { id: index === 0 ? "preamble" : "s_" + hash(`${start.title}\0${occurrence}`).slice(0, 24),
title: start.title, content: content.slice(start.offset, starts[index + 1]?.offset), summary: "" };
});
}
export function sectionSchema(db: DatabaseSync) {
db.exec(`CREATE TABLE IF NOT EXISTS document_sections (
document_id TEXT PRIMARY KEY REFERENCES documents(id), payload TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS document_section_revisions (
document_id TEXT NOT NULL REFERENCES documents(id), revision INTEGER NOT NULL,
payload TEXT NOT NULL, PRIMARY KEY(document_id, revision)
);`);
}
export function loadSections(db: DatabaseSync, documentId: string, content: string): DocumentSection[] {
const row = db.prepare("SELECT payload FROM document_sections WHERE document_id=?")
.get(documentId) as { payload: string } | undefined;
if (!row) return splitSections(content);
const sections = JSON.parse(row.payload) as DocumentSection[];
// A legacy writer must not leave an apparently current outline over different Markdown.
return sections.map((s) => s.content).join("") === content ? sections : reconcileSections(content, sections);
}
export function reconcileSections(content: string, previous: DocumentSection[]): DocumentSection[] {
const unused = new Set(previous);
return splitSections(content).map((section) => {
const prior = section.id === "preamble" ? previous.find((p) => p.id === "preamble") :
previous.find((p) => p.id !== "preamble" && unused.has(p) && p.content === section.content) ??
previous.find((p) => p.id !== "preamble" && unused.has(p) && p.title === section.title);
if (prior) unused.delete(prior);
return { ...section, id: prior?.id ?? (section.id === "preamble" ? "preamble" : "s_" + randomUUID()),
summary: prior?.content === section.content ? prior.summary : "" };
});
}
export function persistSections(db: DatabaseSync, documentId: string, revision: number, sections: DocumentSection[]) {
const payload = JSON.stringify(sections);
db.prepare(`INSERT INTO document_sections VALUES (?,?) ON CONFLICT(document_id)
DO UPDATE SET payload=excluded.payload`).run(documentId, payload);
db.prepare("INSERT INTO document_section_revisions VALUES (?,?,?)").run(documentId, revision, payload);
}
export function outline(sections: DocumentSection[]) {
return sections.map(({ content, ...section }, position) => ({ ...section, position,
contentHash: hash(content), summaryStatus: section.summary.trim() ? "current" : "missing" }));
}
export function parseSectionEdit(value: unknown): SectionEdit {
if (!value || typeof value !== "object" || Array.isArray(value)) throw new DocumentValidationError("Expected section edit object");
const row = value as Record<string, unknown>;
if (Object.keys(row).some((key) => !["revision", "operations"].includes(key)) ||
!Number.isSafeInteger(row.revision) || (row.revision as number) < 1 || !Array.isArray(row.operations) ||
!row.operations.length || row.operations.length > 100) throw new DocumentValidationError("Provide revision and 1 to 100 operations");
for (const raw of row.operations) {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) throw new DocumentValidationError("Invalid operation");
const op = raw as Record<string, unknown>;
const fields: Record<string, string[]> = { update: ["op", "id", "content", "summary"],
insert: ["op", "after", "content", "summary"], delete: ["op", "id"], move: ["op", "id", "after"] };
if (typeof op.op !== "string" || !Object.hasOwn(fields, op.op) || Object.keys(op).some((key) => !fields[op.op as string].includes(key)))
throw new DocumentValidationError("Unknown operation or field");
for (const key of fields[op.op].filter((key) => !["content", "summary", "op"].includes(key)))
if (typeof op[key] !== "string" || !(op[key] as string).length) throw new DocumentValidationError("Missing section identifier");
for (const key of ["content", "summary"])
if (op[key] !== undefined && typeof op[key] !== "string") throw new DocumentValidationError(`Invalid ${key}`);
if (op.op === "update" && op.content === undefined && op.summary === undefined) throw new DocumentValidationError("Empty update");
if (op.op === "insert" && (typeof op.content !== "string" || typeof op.summary !== "string" || !op.summary.trim()))
throw new DocumentValidationError("Insert requires content and summary");
if (typeof op.summary === "string" && (!op.summary.trim() || op.summary.length > 8000))
throw new DocumentValidationError("Summary must contain 1 to 8000 characters");
}
return { revision: row.revision as number, operations: row.operations as SectionOperation[] };
}
export function applySectionOperations(previous: DocumentSection[], operations: SectionOperation[]) {
const sections = previous.map((section) => ({ ...section }));
function index(id: string) {
const result = sections.findIndex((s) => s.id === id);
if (result < 0) throw new DocumentValidationError("Unknown section id");
return result;
}
for (const op of operations) {
if (op.op === "insert") {
sections.splice(index(op.after) + 1, 0, { id: "s_" + randomUUID(), title: "", content: op.content, summary: op.summary });
} else {
const at = index(op.id);
if (op.op === "update") {
if (op.content !== undefined && op.content !== sections[at].content) sections[at].summary = "";
if (op.content !== undefined) sections[at].content = op.content;
if (op.summary !== undefined) sections[at].summary = op.summary;
} else {
if (op.id === "preamble") throw new DocumentValidationError("Cannot delete or move the preamble");
if (op.op === "delete") sections.splice(at, 1);
else {
if (op.id === op.after) throw new DocumentValidationError("Cannot move section after itself");
index(op.after);
const [section] = sections.splice(at, 1);
sections.splice(index(op.after) + 1, 0, section);
}
}
}
}
const content = sections.map((s) => s.content).join("");
const parsed = splitSections(content);
if (parsed.length !== sections.length || parsed.some((s, i) => s.content !== sections[i].content))
throw new DocumentValidationError("Each chapter must start with one H2 heading; preserve newline boundaries and balanced fences");
return sections.map((s, i) => ({ ...s, title: parsed[i].title }));
}
+51
View File
@@ -0,0 +1,51 @@
import type { NextApiRequest, NextApiResponse } from "next";
import { authenticateDocumentKey } from "./document-keys";
import { DocumentConflictError, DocumentValidationError, validateDocument, type Doc } from "./docs";
export function authorizeDocuments(request: NextApiRequest, response: NextApiResponse, write = false) {
response.setHeader("Cache-Control", "private, no-store");
response.setHeader("X-Content-Type-Options", "nosniff");
const header = request.headers.authorization;
const token = typeof header === "string" && header.startsWith("Bearer ") ? header.slice(7) : undefined;
const key = authenticateDocumentKey(token);
if (!key) {
response.status(401).json({ error: "需要有效的文档 API Key", code: "unauthorized" });
return null;
}
if (write && key.scope !== "read-write") {
response.status(403).json({ error: "该 API Key 仅允许读取文档", code: "read_only" });
return null;
}
return key;
}
export function documentApiError(response: NextApiResponse, error: unknown) {
if (error instanceof DocumentConflictError)
return response.status(409).json({ error: error.message, code: "conflict" });
if (error instanceof DocumentValidationError)
return response.status(400).json({ error: error.message, code: "invalid_document" });
return response.status(503).json({ error: "文档数据库暂时不可用", code: "unavailable" });
}
export function parseDocument(value: unknown): Doc {
if (!value || typeof value !== "object" || Array.isArray(value)) throw new DocumentValidationError("Expected a document object");
const row = value as Record<string, unknown>;
const fields = ["slug", "sourceSlug", "title", "category", "collection", "group", "description", "draft", "featured", "content", "revision"];
if (Object.keys(row).some((key) => !fields.includes(key))) throw new DocumentValidationError("Unknown document field");
const doc = Object.fromEntries(fields.map((key) => [key, row[key]])) as Doc;
validateDocument(doc);
if (!Number.isSafeInteger(doc.revision) || doc.revision < 0) throw new DocumentValidationError("Provide expected revision");
return doc;
}
export function pagination(request: NextApiRequest) {
const number = (value: string | string[] | undefined, fallback: number) => {
if (value === undefined) return fallback;
if (typeof value !== "string" || !/^\d+$/.test(value)) throw new DocumentValidationError("Invalid pagination");
return Number(value);
};
return { limit: number(request.query.limit, 50), offset: number(request.query.offset, 0) };
}
export function idempotency(request: NextApiRequest, namespace: string) {
const id = request.headers["idempotency-key"];
if (id === undefined) return undefined;
if (typeof id !== "string" || !/^[A-Za-z0-9._:-]{1,128}$/.test(id)) throw new DocumentValidationError("Invalid Idempotency-Key");
return { namespace, id };
}
@@ -0,0 +1,22 @@
import type { NextApiRequest, NextApiResponse } from "next";
import { DocumentValidationError, saveDocumentBatch } from "@/lib/docs";
import { authorizeDocuments, documentApiError, idempotency, parseDocument } from "@/lib/documents-api";
export const config = { api: { bodyParser: { sizeLimit: "8mb" } } };
export default function handler(request: NextApiRequest, response: NextApiResponse) {
try {
const key = authorizeDocuments(request, response, true);
if (!key) return;
if (request.method !== "POST") {
response.setHeader("Allow", "POST");
return response.status(405).end();
}
const body = request.body;
if (!body || typeof body !== "object" || Array.isArray(body) ||
Object.keys(body).length !== 1 || !Array.isArray(body.documents) ||
!body.documents.length || body.documents.length > 100)
throw new DocumentValidationError("Expected { documents: [...] } with 1 to 100 documents");
const documents = body.documents.map((row: unknown) => parseDocument(row));
return response.status(200).json(saveDocumentBatch(documents, idempotency(request, key.id)));
} catch (error) { return documentApiError(response, error); }
}
@@ -0,0 +1,21 @@
import type { NextApiRequest, NextApiResponse } from "next";
import { documentPage, saveDocumentBatch } from "@/lib/docs";
import { authorizeDocuments, documentApiError, idempotency, pagination, parseDocument } from "@/lib/documents-api";
export const config = { api: { bodyParser: { sizeLimit: "8mb" } } };
export default function handler(request: NextApiRequest, response: NextApiResponse) {
try {
const key = authorizeDocuments(request, response, request.method === "POST");
if (!key) return;
if (request.method === "GET") {
const { limit, offset } = pagination(request);
return response.status(200).json(documentPage(limit, offset));
}
if (request.method === "POST") {
const result = saveDocumentBatch([parseDocument(request.body)], idempotency(request, key.id));
return response.status(200).json({ document: result.documents[0], replayed: result.replayed });
}
response.setHeader("Allow", "GET, POST");
return response.status(405).end();
} catch (error) { return documentApiError(response, error); }
}
@@ -0,0 +1,30 @@
import type { NextApiRequest, NextApiResponse } from "next";
import { DocumentValidationError, documentHistoryPage, documentRevision, readDocument } from "@/lib/docs";
import { authorizeDocuments, documentApiError, pagination } from "@/lib/documents-api";
export default function handler(request: NextApiRequest, response: NextApiResponse) {
try {
if (!authorizeDocuments(request, response)) return;
if (request.method !== "GET") {
response.setHeader("Allow", "GET");
return response.status(405).end();
}
const slug = request.query.slug;
if (!Array.isArray(slug)) throw new DocumentValidationError("Invalid document slug");
const doc = readDocument(slug.join("/"));
if (!doc) return response.status(404).json({ error: "文档不存在", code: "not_found" });
if (request.query.revision !== undefined) {
if (request.query.history !== undefined || typeof request.query.revision !== "string" || !/^\d+$/.test(request.query.revision))
throw new DocumentValidationError("Invalid revision query");
const prior = documentRevision(doc.sourceSlug, Number(request.query.revision));
if (!prior) return response.status(404).json({ error: "修订不存在", code: "not_found" });
return response.status(200).json(prior);
}
if (request.query.history !== undefined) {
if (request.query.history !== "1") throw new DocumentValidationError("Use history=1");
const { limit, offset } = pagination(request);
return response.status(200).json(documentHistoryPage(doc.sourceSlug, limit, offset));
}
return response.status(200).json(doc);
} catch (error) { return documentApiError(response, error); }
}
@@ -0,0 +1,37 @@
import type { NextApiRequest, NextApiResponse } from "next";
import { DocumentValidationError, readDocumentSections, editDocumentSections } from "@/lib/docs";
import { outline } from "@/lib/document-sections";
import { authorizeDocuments, documentApiError, idempotency } from "@/lib/documents-api";
export const config = { api: { bodyParser: { sizeLimit: "8mb" } } };
export default function handler(request: NextApiRequest, response: NextApiResponse) {
try {
const key = authorizeDocuments(request, response, request.method === "POST");
if (!key) return;
if (request.method !== "GET" && request.method !== "POST") {
response.setHeader("Allow", "GET, POST");
return response.status(405).end();
}
if (!Array.isArray(request.query.slug)) throw new DocumentValidationError("Invalid document slug");
const slug = request.query.slug.join("/");
if (Object.keys(request.query).some((k) => !["slug", "section", "revision"].includes(k)))
throw new DocumentValidationError("Unknown section query");
if (request.method === "POST") {
if (request.query.section !== undefined || request.query.revision !== undefined)
throw new DocumentValidationError("POST uses revision and operations in the body");
const saved = editDocumentSections(slug, request.body, idempotency(request, key.id));
return saved ? response.status(200).json(saved) : response.status(404).json({ code: "not_found" });
}
const revision = request.query.revision;
if (revision !== undefined && (typeof revision !== "string" || !/^[1-9]\d*$/.test(revision)))
throw new DocumentValidationError("Invalid revision");
const doc = readDocumentSections(slug, revision === undefined ? undefined : Number(revision));
if (!doc) return response.status(404).json({ code: "not_found" });
const selected = request.query.section;
if (selected === undefined) return response.status(200).json({ ...doc, sections: outline(doc.sections) });
const ids = typeof selected === "string" ? [selected] : selected;
if (!ids.length || ids.length > 100 || ids.some((id) => !id)) throw new DocumentValidationError("Select 1 to 100 sections");
if (ids.some((id) => !doc.sections.some((s) => s.id === id))) return response.status(404).json({ code: "not_found" });
return response.status(200).json({ ...doc, sections: doc.sections.filter((s) => ids.includes(s.id)) });
} catch (error) { return documentApiError(response, error); }
}
+9
View File
@@ -1,5 +1,6 @@
import { NextResponse, type NextRequest } from "next/server";
import { authConfig, SESSION_COOKIE, validSession } from "./lib/auth";
import { authenticateDocumentKey } from "./lib/document-keys";
export function proxy(request: NextRequest) {
const path = request.nextUrl.pathname;
const headers = {
@@ -31,6 +32,14 @@ export function proxy(request: NextRequest) {
? authorization.slice(7)
: undefined;
const token = bearer ?? request.cookies.get(SESSION_COOKIE)?.value;
// Document keys are scoped to the document API; they never authenticate other pages or APIs.
if (bearer && (path === "/api/documents" || path.startsWith("/api/documents/"))) {
try {
if (authenticateDocumentKey(bearer)) return NextResponse.next({ headers });
} catch {
return NextResponse.json({ error: "文档认证暂时不可用" }, { status: 503, headers });
}
}
if (validSession(token, config)) {
if (!["GET", "HEAD", "OPTIONS"].includes(request.method) && !bearer) {
try {
+7
View File
@@ -1473,6 +1473,13 @@ svg { flex-shrink: 0; }
.research-distribution { margin: 24px 0 38px; }
.research-distribution figure { margin: 20px 0; }
.ai-research .income-statistics { min-width: 1200px; }
/* Row-spanning category headings shift cell indexes between rows. */
.ai-research .income-statistics td { min-width: 0; }
.ai-research .income-statistics th[scope="rowgroup"] { min-width: 150px; white-space: nowrap; }
.ai-research .income-statistics th[scope="rowgroup"] small { display: block; margin-top: 5px; }
.ai-research .income-statistics td:not(:last-child) { white-space: nowrap; }
.ai-research .income-statistics td:last-child { min-width: 220px; }
.income-distribution-chart { display: block; width: 100%; min-width: 900px; height: auto; }
.income-distribution-chart [role="button"] { cursor: pointer; outline: none; }
.income-distribution-chart [role="button"]:focus circle:last-child { stroke: #b17725; stroke-width: 3; }
+258
View File
@@ -0,0 +1,258 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { mkdtempSync, rmSync, writeFileSync, readFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { NextApiRequest, NextApiResponse } from "next";
import documents from "../src/pages/api/documents/index";
import batch from "../src/pages/api/documents/batch";
import item from "../src/pages/api/documents/item/[...slug]";
import sections from "../src/pages/api/documents/sections/[...slug]";
import { splitSections } from "../src/lib/document-sections";
import { openSiteDatabase } from "../src/lib/site-database";
import { createDocumentKey, authenticateDocumentKey, revokeDocumentKey, listDocumentKeys } from "../src/lib/document-keys";
import { allDocuments, documentHistory, readDocument, saveDocument, readDocumentSections, type Doc } from "../src/lib/docs";
import { newAuthConfig, signSession } from "../src/lib/auth";
const input: Doc = { slug: "endor/example", sourceSlug: "00-example", title: "Example", category: "Endor",
collection: "Endor 文档", group: "讨论", description: "Description", draft: true, featured: false,
content: "# Example\n\nOriginal body", revision: 0 };
type Options = { method?: string; body?: unknown; headers?: Record<string, string>;
query?: Record<string, string | string[]> };
function call(handler: (req: NextApiRequest, res: NextApiResponse) => unknown, options: Options = {}) {
let status = 200;
let body: unknown;
const headers: Record<string, string> = {};
const response = {
status(value: number) { status = value; return this; },
setHeader(name: string, value: string) { headers[name.toLowerCase()] = value; return this; },
json(value: unknown) { body = value; return this; },
end() { return this; },
};
handler({ method: options.method ?? "GET", body: options.body, headers: options.headers ?? {},
query: options.query ?? {} } as unknown as NextApiRequest, response as unknown as NextApiResponse);
return { status, body, headers };
}
function isolated(run: (context: { directory: string; headers: Record<string, string> }) => void) {
const directory = mkdtempSync(join(tmpdir(), "tabularium-doc-api-"));
const previous = process.env.AUTH_CONFIG_FILE;
process.env.AUTH_CONFIG_FILE = join(directory, "config.json");
writeFileSync(process.env.AUTH_CONFIG_FILE, JSON.stringify(newAuthConfig()), { mode: 0o600 });
try {
const key = createDocumentKey("test uploader");
run({ directory, headers: { authorization: "Bearer " + key.token } });
} finally {
if (previous === undefined) delete process.env.AUTH_CONFIG_FILE;
else process.env.AUTH_CONFIG_FILE = previous;
rmSync(directory, { recursive: true, force: true });
}
}
test("new API requires document keys, rejects browser cookies and broad web sessions", () => isolated(() => {
assert.equal(call(documents).status, 401);
assert.equal(call(documents, { headers: { authorization: "Bearer invalid" } }).status, 401);
const config = JSON.parse(readFileSync(process.env.AUTH_CONFIG_FILE!, "utf8"));
const token = signSession(config);
assert.equal(call(documents, { headers: { authorization: "Bearer " + token } }).status, 401);
assert.equal(call(documents, { headers: { cookie: "tabularium_session=" + token } }).status, 401);
assert.deepEqual(allDocuments(), []);
}));
test("key scopes, revocation, expiry and secret-free listing are persistent", () => isolated(({ directory }) => {
const key = createDocumentKey("reader", "read", 1);
const headers = { authorization: "Bearer " + key.token };
assert.equal(call(documents, { headers }).status, 200);
assert.equal(call(documents, { method: "POST", headers, body: input }).status, 403);
assert.equal(call(batch, { method: "POST", headers, body: { documents: [input] } }).status, 403);
assert.ok(!JSON.stringify(listDocumentKeys()).includes(key.token));
assert.ok(!readFileSync(join(directory, "login-state.sqlite")).includes(Buffer.from(key.token)));
assert.equal(authenticateDocumentKey(key.token, key.expiresAt), null);
revokeDocumentKey(key.id);
assert.equal(call(documents, { headers }).status, 401);
const revoked = listDocumentKeys().find((row) => row.id === key.id);
assert.ok(revoked); assert.equal(typeof revoked.revokedAt, "number");
}));
test("single save, aliases, metadata and historical content are visible without rebuilding", () => isolated(({ headers }) => {
assert.equal(call(documents, { method: "POST", headers, body: input }).status, 200);
assert.equal((call(item, { headers, query: { slug: input.slug.split("/") } }).body as Doc).revision, 1);
const changed = { ...input, revision: 1, slug: "endor/renamed", content: "# Changed", collection: "新合集" };
assert.equal(call(documents, { method: "POST", headers, body: changed }).status, 200);
assert.equal((call(item, { headers, query: { slug: input.slug.split("/") } }).body as Doc).slug, changed.slug);
assert.equal((call(item, { headers, query: { slug: [input.sourceSlug], revision: "1" } }).body as Doc).content, input.content);
const history = call(item, { headers, query: { slug: [input.sourceSlug], history: "1", limit: "1" } });
const page = history.body as { revisions: Doc[]; total: number; nextOffset: number };
assert.equal(page.total, 2);
assert.equal(page.nextOffset, 1);
assert.equal(page.revisions[0].revision, 2);
assert.equal(page.revisions[0].content, undefined);
assert.equal(call(item, { headers, query: { slug: [input.sourceSlug], revision: "99" } }).status, 404);
assert.equal(call(item, { headers, query: { slug: [input.sourceSlug], revision: "0" } }).status, 400);
}));
test("pagination returns metadata only with strict bounds", () => isolated(({ headers }) => {
const second = { ...input, slug: "endor/second", sourceSlug: "01-second" };
assert.equal(call(batch, { method: "POST", headers, body: { documents: [input, second] } }).status, 200);
const result = call(documents, { headers, query: { limit: "1" } });
const page = result.body as { documents: Doc[]; total: number; nextOffset: number };
assert.equal(page.total, 2); assert.equal(page.nextOffset, 1);
assert.equal(page.documents[0].content, undefined);
assert.equal((call(documents, { headers, query: { limit: "1", offset: "1" } }).body as typeof page).documents[0].slug, second.slug);
const invalidQueries: Record<string, string | string[]>[] = [
{ limit: "101" }, { offset: "-1" }, { limit: ["1", "2"] }, { limit: "1e2" },
];
for (const query of invalidQueries) assert.equal(call(documents, { headers, query }).status, 400);
assert.equal(result.headers["cache-control"], "private, no-store");
}));
test("late batch conflict rolls back earlier rows and their revision history", () => isolated(({ headers }) => {
saveDocument({ ...input, slug: "endor/existing", sourceSlug: "existing" }, 0);
const conflict = { ...input, slug: "endor/existing", sourceSlug: "existing", revision: 0 };
assert.equal(call(batch, { method: "POST", headers, body: { documents: [input, conflict] } }).status, 409);
assert.equal(readDocument(input.slug), null);
assert.equal(documentHistory(input.sourceSlug).length, 0);
assert.equal(documentHistory("existing").length, 1);
assert.equal(allDocuments().length, 1);
}));
test("idempotent retries return the original receipt without creating another revision", () => isolated(({ headers }) => {
const keyed = { ...headers, "idempotency-key": "upload-1" };
const first = call(documents, { method: "POST", headers: keyed, body: input });
const second = call(documents, { method: "POST", headers: keyed, body: input });
assert.equal(first.status, 200); assert.equal(second.status, 200);
assert.equal((second.body as { replayed: boolean }).replayed, true);
assert.equal(documentHistory(input.sourceSlug).length, 1);
assert.equal(call(documents, { method: "POST", headers: keyed, body: { ...input, content: "Different" } }).status, 409);
assert.equal(call(documents, { method: "POST", headers, body: input }).status, 409);
}));
test("invalid batch members, duplicate identifiers, raw images and malformed fields never write", () => isolated(({ headers }) => {
for (const body of [{ documents: [] }, { documents: [input, input] },
{ documents: [input], unexpected: true }, { documents: Array.from({ length: 101 }, () => input) }])
assert.equal(call(batch, { method: "POST", headers, body }).status, 400);
for (const body of [{ ...input, revision: undefined }, { ...input, typo: true },
{ ...input, slug: "../private" }, { ...input, content: "![raw](/figures/original.png)" }, null])
assert.equal(call(documents, { method: "POST", headers, body }).status, 400);
assert.deepEqual(allDocuments(), []);
assert.equal(call(documents, { method: "DELETE", headers }).status, 405);
assert.equal(call(item, { headers, query: { slug: ["missing"] } }).status, 404);
}));
const chapterContent = "# Example\n\nIntroduction.\n\n## First\n\nFirst body.\n\n### Detail\n\nChild stays in chapter.\n\n## Second\n\nSecond body.\n";
type SectionResponse = { revision: number; replayed?: boolean;
sections: { id: string; title: string; summary: string; summaryStatus?: string; content?: string }[] };
const chapterQuery = { slug: input.slug.split("/") };
test("chapter parser preserves bytes, fences, duplicate headings and CRLF", () => {
const content = "# Title\r\n\r\n## Repeated\r\n~~~text\r\n## Inside fence\r\n~~~\r\n### Nested\r\n```js\r\n## Also inside\r\n````\r\n\r\n## Repeated\r\nTail";
const parts = splitSections(content);
assert.equal(parts.length, 3);
assert.notEqual(parts[1].id, parts[2].id);
assert.equal(parts.map((p) => p.content).join(""), content);
assert.equal(parts[1].title, "Repeated");
assert.ok(parts[1].content.includes("### Nested"));
assert.equal(splitSections("").length, 1);
assert.equal(splitSections("# Title\n\n- Item\n ## List detail\n\n> ## Quoted\n").length, 1);
});
test("outline reads existing Markdown without changing it and returns only selected bodies", () => isolated(({ headers }) => {
saveDocument({ ...input, content: chapterContent }, 0);
const db = openSiteDatabase();
db.exec("DELETE FROM document_sections; DELETE FROM document_section_revisions;"); db.close();
const result = call(sections, { headers, query: chapterQuery });
assert.equal(result.status, 200);
const outline = result.body as SectionResponse;
assert.equal(outline.revision, 1); assert.equal(outline.sections.length, 3);
assert.deepEqual(outline.sections.map((s) => s.title), ["导言", "First", "Second"]);
assert.ok(outline.sections.every((s) => s.content === undefined && s.summaryStatus === "missing"));
const selected = call(sections, { headers, query: { ...chapterQuery, section: outline.sections[1].id } }).body as SectionResponse;
assert.equal(selected.sections.length, 1);
assert.ok(selected.sections[0].content!.includes("### Detail"));
assert.equal(readDocument(input.slug)!.content, chapterContent);
assert.equal(call(sections, { headers, query: { ...chapterQuery, section: "missing" } }).status, 404);
assert.equal(call(sections, { headers, query: { ...chapterQuery, revision: "0" } }).status, 400);
}));
test("internal summaries persist separately and chapter updates leave other bytes untouched", () => isolated(({ headers }) => {
saveDocument({ ...input, content: chapterContent }, 0);
const before = readDocumentSections(input.slug)!;
const first = before.sections[1]; const second = before.sections[2];
const changed = first.content.replace("First body.", "Revised body.");
const edited = call(sections, { method: "POST", headers, query: chapterQuery, body: { revision: 1, operations: [
{ op: "update", id: first.id, content: changed, summary: "INTERNAL_SUMMARY_FIRST" },
{ op: "update", id: second.id, summary: "INTERNAL_SUMMARY_SECOND" },
] } });
assert.equal(edited.status, 200);
const after = readDocumentSections(input.slug)!;
assert.equal(after.revision, 2); assert.equal(after.sections[1].id, first.id);
assert.equal(after.sections[0].content, before.sections[0].content);
assert.equal(after.sections[2].content, before.sections[2].content);
assert.equal(after.sections[1].summary, "INTERNAL_SUMMARY_FIRST");
assert.equal(readDocument(input.slug)!.content, chapterContent.replace("First body.", "Revised body."));
assert.ok(!JSON.stringify(allDocuments()).includes("INTERNAL_SUMMARY"));
assert.ok(!JSON.stringify(call(item, { headers, query: chapterQuery }).body).includes("INTERNAL_SUMMARY"));
assert.ok(!JSON.stringify(call(documents, { headers }).body).includes("INTERNAL_SUMMARY"));
const history = readDocumentSections(input.slug, 1)!;
assert.equal(history.sections[1].content, first.content); assert.equal(history.sections[1].summary, "");
assert.equal(readDocumentSections(input.slug, 2)!.sections[1].summary, "INTERNAL_SUMMARY_FIRST");
}));
test("insert, move and delete chapters commit together with replay and conflict protection", () => isolated(({ headers }) => {
saveDocument({ ...input, content: chapterContent }, 0);
const before = readDocumentSections(input.slug)!;
const first = before.sections[1]; const second = before.sections[2];
const body = { revision: 1, operations: [
{ op: "move", id: second.id, after: "preamble" },
{ op: "delete", id: first.id },
{ op: "insert", after: second.id, content: "## New chapter\n\nNew body.\n", summary: "New material" },
] };
const options = { method: "POST", headers: { ...headers, "idempotency-key": "chapters-1" }, query: chapterQuery, body };
assert.equal(call(sections, options).status, 200);
const repeated = call(sections, options);
assert.equal(repeated.status, 200); assert.equal((repeated.body as SectionResponse).replayed, true);
const after = readDocumentSections(input.slug)!;
assert.deepEqual(after.sections.map((s) => s.title), ["导言", "Second", "New chapter"]);
assert.equal(after.sections[1].content, second.content);
assert.equal(after.revision, 2); assert.equal(documentHistory(input.sourceSlug).length, 2);
assert.equal(call(sections, { ...options, body: { ...body, revision: 2 } }).status, 409);
assert.equal(call(sections, { ...options, headers }).status, 409);
assert.equal(saveDocument({ ...readDocument(input.slug)!, description: "Changed metadata" }, 2).revision, 3);
assert.equal(readDocumentSections(input.slug)!.sections[2].summary, "New material");
}));
test("invalid late operation rolls back content, summary and history; read-only keys cannot edit", () => isolated(({ headers }) => {
saveDocument({ ...input, content: chapterContent }, 0);
const before = readDocumentSections(input.slug)!;
const invalid = [
[{ op: "update", id: before.sections[1].id, summary: "Should roll back" }, { op: "delete", id: "missing" }],
[{ op: "delete", id: "preamble" }],
[{ op: "move", id: before.sections[1].id, after: before.sections[1].id }],
[{ op: "update", id: before.sections[1].id, content: "## First\n\n## Extra\n" }],
[{ op: "update", id: before.sections[1].id, content: "## First\n![raw](/figures/raw.png)\n" }],
[{ op: "insert", after: "preamble", content: "## Missing summary\n" }],
[{ op: "delete", id: before.sections[1].id, unexpected: true }],
];
for (const operations of invalid) {
assert.equal(call(sections, { method: "POST", headers, query: chapterQuery, body: { revision: 1, operations } }).status, 400);
assert.deepEqual(readDocumentSections(input.slug), before);
assert.equal(documentHistory(input.sourceSlug).length, 1);
}
assert.equal(call(sections, { query: chapterQuery }).status, 401);
const reader = createDocumentKey("chapter reader", "read");
const readHeaders = { authorization: "Bearer " + reader.token };
assert.equal(call(sections, { headers: readHeaders, query: chapterQuery }).status, 200);
assert.equal(call(sections, { method: "POST", headers: readHeaders, query: chapterQuery, body: { revision: 1, operations: [] } }).status, 403);
}));
test("full-document saves invalidate changed chapter summaries and preserve unaffected summaries and ids", () => isolated(({ headers }) => {
saveDocument({ ...input, content: chapterContent }, 0);
const before = readDocumentSections(input.slug)!;
assert.equal(call(sections, { method: "POST", headers, query: chapterQuery, body: { revision: 1, operations:
before.sections.map((s) => ({ op: "update", id: s.id, summary: "Summary of " + s.title })) } }).status, 200);
saveDocument({ ...readDocument(input.slug)!, content: chapterContent.replace("First body.", "Changed externally.") }, 2);
const after = readDocumentSections(input.slug)!;
assert.equal(after.sections[1].id, before.sections[1].id);
assert.equal(after.sections[1].summary, "");
assert.equal(after.sections[2].summary, "Summary of Second");
assert.equal(readDocumentSections(input.slug, 2)!.sections[1].summary, "Summary of First");
}));
+134 -23
View File
@@ -23,9 +23,9 @@ test("research revisions share the authentication database and preserve replay l
for (let i = 0; i < 20; i++) consumeCode(config, "invalid");
assert.equal(consumeCode(config, "invalid"), "limited");
const initial = readResearchSnapshot();
assert.equal(initial.companies.length, 1021);
assert.equal(initial.evidence.length, 1201);
assert.equal(initial.observations.length, 187);
assert.equal(initial.companies.length, 1051);
assert.equal(initial.evidence.length, 2235);
assert.equal(initial.observations.length, 314);
assert.deepEqual(initial.candidateBatches, seed.candidateBatches, "candidate gaps persist in the same database metadata");
assert.deepEqual(initial.directoryRechecks, seed.directoryRechecks, "follow-up classifications persist without rewriting the discovery audit");
assert.deepEqual(readResearchSnapshot(), initial, "reopening does not duplicate records");
@@ -64,9 +64,9 @@ test("research validation rejects broken evidence and unsafe source links", () =
test("reviews link explicit companies without turning global or excluded cases into companies", () => {
const snapshot = structuredClone(seed) as ResearchSnapshot;
assert.equal(snapshot.reviews.filter((r) => r.scope === "companies").length, 52);
assert.equal(snapshot.reviews.filter((r) => r.scope === "dataset").length, 4);
assert.equal(snapshot.reviews.filter((r) => r.scope === "excluded-candidate").length, 12);
assert.equal(snapshot.reviews.filter((r) => r.scope === "companies").length, 140);
assert.equal(snapshot.reviews.filter((r) => r.scope === "dataset").length, 12);
assert.equal(snapshot.reviews.filter((r) => r.scope === "excluded-candidate").length, 44);
const review = snapshot.reviews.find((r) => r.id === "R4")!;
assert.deepEqual(reviewCompanyIds(snapshot, review), ["AI007", "AI013", "AI021", "AI034"]);
assert.deepEqual(reviewCompanyIds(snapshot, snapshot.reviews.find((r) => r.id === "CN-R-PSI")!), []);
@@ -85,20 +85,47 @@ test("reviews link explicit companies without turning global or excluded cases i
test("disclosure counts exclude client income, biographies, targets and duplicate milestones", () => {
const snapshot = seed as ResearchSnapshot;
const summary = categorySummary(snapshot);
assert.equal(summary.reduce((n, c) => n + c.count, 0), 1021);
assert.equal(summary.reduce((n, c) => n + c.self, 0), 26);
assert.equal(summary.reduce((n, c) => n + c.estimate, 0), 1);
assert.equal(summary.reduce((n, c) => n + c.platform, 0), 16);
assert.equal(summary.reduce((n, c) => n + c.unknown, 0), 978);
assert.equal(summary.reduce((n, c) => n + c.count, 0), 1051);
assert.equal(summary.reduce((n, c) => n + c.self, 0), 49);
assert.equal(summary.reduce((n, c) => n + c.estimate, 0), 2);
assert.equal(summary.reduce((n, c) => n + c.platform, 0), 19);
assert.equal(summary.reduce((n, c) => n + c.unknown, 0), 981);
assert.equal(snapshot.companies.find((c) => c.name === "Rork")?.revenueStatus, "unknown");
assert.equal(snapshot.companies.find((c) => c.name === "Result")?.revenueStatus, "unknown");
assert.ok(snapshot.observations.filter((o) => o.selection === "signal").every((o) => o.scope === "公司整体"));
const arena = snapshot.observations.find((o) => o.id === "COARENA-O-CUMULATIVE")!;
assert.equal(arena.selection, "historical", "cumulative launch revenue is not a recurring company-wide signal");
assert.equal(arena.comparisonGroup, null);
assert.ok(arena.metric !== "mrr" && arena.metric !== "arr", "a two-week cumulative figure cannot become MRR or ARR");
});
test("service run rates, legacy freight revenue and paid adoption do not become platform ARR", () => {
const snapshot = seed as ResearchSnapshot;
const runRate = snapshot.observations.find((o) => o.id === "INDUSTRY11-O-LOCKE-RUNRATE")!;
assert.equal(runRate.metric, "annualized-revenue");
assert.equal(runRate.selection, "historical");
assert.equal(runRate.observedAt, undefined);
assert.equal(runRate.comparisonGroup, null);
assert.equal(snapshot.companies.find((c) => c.id === "YC33283")?.revenueStatus, "self");
for (const id of ["TRUEMILE-OLD-REV", "VELTHA-DEMO", "LANTERN-OLD-REV"]) {
const observation = snapshot.observations.find((o) => o.id === `INDUSTRY11-O-${id}`)!;
assert.equal(observation.selection, "excluded");
assert.equal(observation.metric, undefined);
assert.equal(observation.comparisonGroup, null);
}
for (const [id, company] of [["VERDANT-DEPLOY", "YC32189"], ["SUTRIX-PAID", "YC36576"]]) {
const observation = snapshot.observations.find((o) => o.id === `INDUSTRY11-O-${id}`)!;
assert.equal(observation.selection, "adoption");
assert.equal(observation.metric, undefined);
assert.equal(snapshot.companies.find((c) => c.id === company)?.revenueStatus, "unknown");
}
assert.ok(platformIncomeSample(snapshot, { windowDays: 90 }).points.every((point) => !point.observation.id.startsWith("INDUSTRY11-")));
});
test("technical reviews keep accessible materials separate from tested capabilities", () => {
const snapshot = structuredClone(seed) as ResearchSnapshot;
assert.equal(snapshot.companies.filter((c) => c.technicalReview).length, 13);
assert.equal(snapshot.companies.filter((c) => c.technicalReview?.level === "public-artifact").length, 6);
assert.equal(snapshot.companies.filter((c) => c.technicalReview).length, 65);
assert.equal(snapshot.companies.filter((c) => c.technicalReview?.level === "public-artifact").length, 9);
assert.equal(snapshot.companies.filter((c) => c.technicalReview?.level === "independent-test").length, 1);
assert.equal(snapshot.companies.find((c) => c.id === "AI051")?.revenueStatus, "unknown");
assert.match(snapshot.companies.find((c) => c.id === "AI051")!.technicalReview!.testScope!, /Sep 2026/);
@@ -137,6 +164,43 @@ test("ambiguous ARR claims retain currency, precision and attribution limits", (
assert.equal(snapshot.companies.find((c) => c.id === "AI070")?.revenueStatus, "unknown");
assert.equal(snapshot.observations.filter((o) => o.company === "AI070").length, 0,
"unreadable announcement summaries do not become numeric ARR");
const waitlist = snapshot.observations.find((o) => o.id === "SR25-O-OMI-WAITLIST")!;
assert.equal(waitlist.selection, "excluded", "waitlist conversions are not paid recurring revenue");
assert.equal(snapshot.companies.find((c) => c.id === waitlist.company)?.revenueStatus, "unknown");
const bluma = snapshot.observations.find((o) => o.id === "BLUMA-O-HISTORICAL-MRR")!;
assert.equal(bluma.selection, "historical");
assert.ok(bluma.unit !== "USD" && !bluma.observedAt && bluma.comparisonGroup === null);
const native = snapshot.observations.find((o) => o.id === "NATIVE-O-ARR-CLAIM")!;
assert.equal(native.selection, "crosscheck");
assert.ok(!native.metric && !native.observedAt && native.comparisonGroup === null);
const marketVolume = snapshot.observations.find((o) => o.id === "PALISADE-O-GMV")!;
assert.equal(marketVolume.selection, "excluded", "customer GMV must not establish company revenue or ARR");
assert.equal(snapshot.companies.find((c) => c.id === marketVolume.company)?.revenueStatus, "unknown");
const pingo = snapshot.observations.find((o) => o.id === "PINGO-O-MRR-JUL25")!;
assert.equal(pingo.selection, "historical", "a July 2025 self-report is not a current payment-platform sample");
assert.ok(pingo.unit !== "USD" && !pingo.observedAt && pingo.comparisonGroup === null);
const designService = snapshot.observations.find((o) => o.id === "SEMIOTIC-O-LAUNCH-REVENUE")!;
assert.equal(designService.selection, "historical");
assert.ok(!designService.metric && designService.comparisonGroup === null, "four-week service revenue is not MRR or ARR");
const restaurantAdoption = snapshot.observations.filter((o) => o.company === "YC32815");
assert.ok(restaurantAdoption.length > 0 && restaurantAdoption.every((o) => o.selection === "adoption" && !o.metric),
"restaurant count and list price must not produce an invented MRR observation");
const fishPilot = snapshot.observations.find((o) => o.id === "OCTAPULSE-O-CONTRACT")!;
assert.equal(fishPilot.selection, "excluded", "a paid pilot or annual contract does not establish recognized recurring revenue");
assert.equal(snapshot.companies.find((c) => c.id === fishPilot.company)?.revenueStatus, "unknown");
assert.equal(fishPilot.comparisonGroup, null);
assert.notEqual(fishPilot.unit, "USD", "six-figure wording does not establish the currency");
assert.ok(!fishPilot.metric && !fishPilot.observedAt, "the publication date is not the financial reporting date");
const cedar = snapshot.observations.filter((o) => o.company === "AI102");
assert.ok(cedar.every((o) => o.selection !== "signal" && o.comparisonGroup === null),
"closed and contracted amounts without recognition or period do not establish ARR");
assert.equal(snapshot.observations.find((o) => o.id === "SR25-O-LOOPS-CUSTOMER-SALES")?.selection, "excluded");
const portfolio = snapshot.observations.filter((o) => o.company === "AI100");
assert.deepEqual(portfolio.filter((o) => o.selection === "signal").map((o) => o.id), ["SR25-O-DRS-PORTFOLIO"],
"a product milestone must not be added to its company portfolio as another company");
assert.ok(snapshot.observations.filter((o) => o.id.startsWith("SR25-O-")).every((o) =>
o.unit !== "USD" && !o.metric && !o.observedAt && o.comparisonGroup === null),
"access dates and dollar symbols must not invent dated USD income samples");
});
// Retractions and forecasts must stay visible for review without becoming revenue signals.
@@ -151,7 +215,14 @@ test("retracted ARR and projected margins remain excluded", () => {
const forecast = snapshot.observations.find((o) => o.kind === "非GAAP毛利率预期")!;
assert.equal(forecast.selection, "excluded");
const peec = snapshot.companies.find((c) => c.name === "Peec AI")!;
assert.equal(peec.teamBound, ">");
assert.equal(peec.team, 100);
assert.equal(peec.teamBound, "≥");
const currentTeam = snapshot.observations.find((o) => o.id === "PEEC-JOB-TEAM")!;
assert.equal(currentTeam.value, peec.team);
assert.equal(currentTeam.bound, peec.teamBound);
const historicalTeam = snapshot.observations.find((o) => o.id === "O067")!;
assert.equal(historicalTeam.value, 70);
assert.equal(historicalTeam.bound, ">");
assert.equal(snapshot.observations.filter((o) => o.company === peec.id && o.selection === "signal").length, 1);
assert.ok(snapshot.observations.some((o) => o.company === peec.id && o.selection === "historical-estimate"));
});
@@ -172,8 +243,18 @@ test("directory coverage reconciles every candidate and keeps profile-only incom
assert.ok(company.profileOnly || company.directoryOrigin === "yc-2025-2026");
assert.equal(company.url, row.source);
if (company.profileOnly) assert.equal(company.revenueStatus, "unknown");
assert.equal(company.year, row.foundedYear);
assert.ok([2025, 2026].includes(company.year!));
if (company.id === "YC31370") {
assert.equal(row.foundedYear, 2025, "original discovery remains unchanged");
assert.equal(company.year, null, "a pivot year cannot settle the company founding year");
assert.ok(snapshot.reviews.some((review) => review.id === "DAYJOB-R-COHORT" && review.companyIds?.includes(company.id)));
} else if (company.id === "YC36279") {
assert.equal(row.foundedYear, 2026, "original Native discovery remains unchanged");
assert.equal(company.year, null, "a model launch and YC date cannot settle the older business founding year");
assert.ok(snapshot.reviews.some((review) => review.id === "NATIVE-R-COHORT" && review.companyIds?.includes(company.id)));
} else {
assert.equal(company.year, row.foundedYear);
assert.ok([2025, 2026].includes(company.year!));
}
}
const invalid = structuredClone(snapshot);
invalid.coverage!.newProfiles += 1;
@@ -187,6 +268,20 @@ test("directory coverage reconciles every candidate and keeps profile-only incom
assert.throws(() => validateResearch(lostOrigin), /coverage accounting/);
});
test("business pivots and procurement volume do not become comparable startup ARR", () => {
const snapshot = seed as ResearchSnapshot;
const dayjob = snapshot.companies.find((company) => company.id === "YC31370")!;
assert.equal(dayjob.year, null);
assert.equal(dayjob.directoryOrigin, "yc-2025-2026");
const milestones = snapshot.observations.filter((observation) => observation.company === dayjob.id);
assert.deepEqual(milestones.filter((observation) => observation.selection === "crosscheck").map((observation) => observation.value).sort(), [496000, 653000]);
assert.ok(milestones.every((observation) => observation.comparisonGroup === null && !observation.metric && !observation.observedAt));
assert.ok(milestones.filter((observation) => [150000, 800000].includes(observation.value)).every((observation) => observation.selection === "excluded"));
const procurement = snapshot.observations.find((observation) => observation.id === "SAUDARA-O-SOURCING")!;
assert.equal(procurement.selection, "excluded");
assert.equal(procurement.comparisonGroup, null);
});
test("income descriptions report sample dispersion and maximum-value sensitivity", () => {
const stats = describeIncome([100, 200, 300])!;
assert.equal(stats.mean, 200);
@@ -208,14 +303,21 @@ test("income descriptions report sample dispersion and maximum-value sensitivity
test("platform samples exclude stale data and keep ARR, estimates and collections separate", () => {
const snapshot = seed as ResearchSnapshot;
const recent = platformIncomeSample(snapshot, { windowDays: 7 });
assert.equal(recent.points.length, 14);
assert.equal(recent.excluded.length, 3);
assert.equal(recent.points.length, 16);
assert.equal(recent.excluded.length, 4);
assert.ok(recent.points.every((p) => p.observation.metric === "mrr" && p.observation.selection === "platform"));
assert.deepEqual(new Set(recent.excluded.map((p) => p.observation.company)), new Set(["AI016", "MR013", "MR014"]));
assert.deepEqual(new Set(recent.excluded.map((p) => p.observation.company)), new Set(["AI016", "MR013", "MR014", "MR017"]));
assert.ok(recent.points.some((p) => p.company.name === "TrackAI" && p.observation.connectionExpired), "dated recent value is not a claim of active connection");
const older = platformIncomeSample(snapshot, { windowDays: 90 });
assert.equal(older.points.length, 15);
assert.equal(older.points.length, 18);
assert.ok(older.points.some((p) => p.company.id === "AI016"));
assert.ok(!recent.points.some((p) => p.company.id === "MR017"));
assert.ok(older.points.some((p) => p.company.id === "MR017" && p.observation.connectionExpired),
"an expired connection retains its historical observation date in the wider window");
assert.equal(recent.points.find((p) => p.company.id === "MR018")?.value, 67864,
"an older Markdown cache must not replace or duplicate the later web snapshot");
assert.equal(recent.points.find((p) => p.company.id === "MR019")?.value, 65431,
"period-table revenue and revenue snapshots must not replace MRR");
const scaled = platformIncomeSample(snapshot, { windowDays: 7, annualize: true });
const monthlyGroups = groupIncome(recent.points);
const annualGroups = groupIncome(scaled.points);
@@ -279,7 +381,7 @@ test("annual launch archives exclude older companies only with evidence and reus
assert.equal(batch.complete, false);
assert.equal(batch.records.length, 19);
const older = batch.records.filter((r) => r.status === "outside-cohort");
assert.equal(older.length, 7);
assert.equal(older.length, 16);
assert.ok(older.every((r) => !r.companyId && r.evidenceIds?.length));
assert.equal(batch.records.find((r) => r.name === "Trace")?.companyId, "YC30687");
const trace = snapshot.companies.find((c) => c.id === "YC30687")!;
@@ -300,7 +402,7 @@ test("unreconciled directory financials remain source reviews instead of income
assert.ok(review.evidenceIds.includes("PH-TANA-DR"));
assert.equal(reviewCompanyIds(snapshot, review).length, 0);
assert.ok(!snapshot.observations.some((o) => o.evidence === "PH-TANA-DR"));
assert.equal(snapshot.candidateBatches!.flatMap((b) => b.records).filter((r) => r.status === "pending-year").length, 21);
assert.equal(snapshot.candidateBatches!.flatMap((b) => b.records).filter((r) => r.status === "pending-year").length, 18);
});
test("product pivots and founder histories do not become new company years or current revenue", () => {
@@ -313,6 +415,15 @@ test("product pivots and founder histories do not become new company years or cu
assert.equal(candidate.companyId, pivot.id);
assert.ok(!platformIncomeSample(snapshot, { windowDays: null }).points.some((point) => point.company.id === pivot.id));
assert.ok(!snapshot.observations.some((o) => ["PH-FUNDRAISLY-ABOUT", "LF-PIVOT", "LF-HISTORY"].includes(o.evidence)), "other ventures' ARR, users and funding must not transfer");
const candidates = snapshot.candidateBatches!.flatMap((b) => b.records);
assert.equal(candidates.find((r) => r.id === "PH26-12")?.status, "outside-cohort",
"a brand iteration with an earlier business chain is not a new company");
const happycapy = candidates.find((r) => r.id === "PH26-04")!;
assert.ok(happycapy.evidenceIds?.includes("FOLLOWUP-CAPY-ACRA"));
assert.equal(happycapy.status, "pending-year", "a seller's registration does not establish a new global business");
assert.equal(happycapy.companyId, undefined);
assert.ok(!snapshot.observations.some((o) => o.evidence.startsWith("FOLLOWUP-")),
"registration, funding, app-store prices and client amounts are not company income");
});
test("follow-up evidence can admit an uncertain directory candidate without rewriting its original status", () => {
@@ -335,7 +446,7 @@ test("Chinese discovery entries preserve unread articles and use separate primar
assert.equal(batch.records.length, 4);
assert.ok(batch.records.every((row) => row.kind === "company" && row.profileStatus === "unavailable"));
assert.equal(batch.records.filter((row) => row.status === "linked").length, 2);
assert.equal(batch.records.find((row) => row.name === "Utopai")?.status, "pending-year");
assert.equal(batch.records.find((row) => row.name === "Utopai")?.status, "outside-cohort");
assert.ok(!snapshot.observations.some((o) => ["PS-CODE", "SG-START", "SG-APR", "SSI-HISTORY"].includes(o.evidence)), "task costs, investment, capacity and financing are not revenue");
const prism = snapshot.companies.find((c) => c.id === "AI075")!;
assert.equal(prism.team, null, "a team range or linked employees must not become exact headcount");
File diff suppressed because it is too large Load Diff
+90 -34
View File
@@ -1,5 +1,5 @@
{
"version": "2026-10-30-9",
"version": "2026-10-31-industry-11",
"scopeConfirmedAt": "2026-10-30",
"acceptance": "以确认的公开来源清单为边界,逐条核对并标明缺项。",
"provenance": "S01—S12恢复自已提交用户的《AI公司研究_来源与分类模板》来源清单;S13—S16单列后续已使用的补充证据渠道,不将使用次数当作全量覆盖。",
@@ -15,7 +15,7 @@
],
"initialAccess": "官网目录可访问",
"boundary": "已读取八个2025—2026批次镜像候选池1428条,再核YC官方页面;不声称覆盖全部YC历年或未公开公司。",
"nextCheck": "继续复核规则初分和缺项;候选明细见逐条台账。",
"nextCheck": "原新增目录37条深化并保留发现来源;本快照基础项639条主营人工复核。原C07主营规则池184项已读:135保留、49迁移;原C07标签池68项全部已读:39保留、29迁移。合计252项主营复核,不等所有公司财务/技术已核。Locke历史超过50万年化收入运行率不是ARR;Donkey/TrueMile贸易与运输、保险经纪和TPA服务不当同质SaaS。Veltha官网/TPA定位、Mount与AGI Deploy、TrueMile家族业务沿革、AGI官网缺项保留。付费采用不推收入,客户资产、旧公司及演示金额排除。C05原主营规则89项成果待另一分支整合;其他基础资料、全部公开来源及财务/技术缺项仍需继续核对。",
"limitation": "批次不等于成立年;不代表全部公司"
},
{
@@ -28,8 +28,8 @@
"techcrunch.com"
],
"initialAccess": "AI栏目可读取",
"boundary": "栏目首页选择的3条公司线索已追溯原始资料:Wabi据公开名录2025年份关联公司;DreamHQ与Thrive Holdings成立年待核,后者2025发布公告另存。不是首页全部条目或两年档案;公开分类接口与RSS普通HTTP请求返回403。",
"nextCheck": "补可公开取得的2025、2026档案及剩余分页;核DreamHQ、Thrive Holdings成立依据及Wabi精确注册、团队和付费口径。",
"boundary": "2026首页定向3项已登记,另从AI第3—4页登记5项:3项关联公司、1项范围外、1项待核;不等全部卡片核验。另已读2025年度档案入口与AI第二页,并完整登记一篇2025企业竞赛名单32项:复用1项、范围外29项、待核2项;每项均尝试原始链接,未知与不可读保留。单篇32项不等全部200参赛者、全部分页或两年全量;RSS/API普通请求受限。 另从AI第5—6页定向4项:2项原始资料归范围外,2项原始成立/年份待核;不是全部卡片、参赛者或档案。 首次登记2025政府/法律名单正文11项(标题称10)时:7项范围外或既有业务、4项成立/主体待核;当时未新增主公司或ARR,非全部200或两年档案。 另将2025金融/地产名单14项全部登记:11项范围外/既有业务,3项年份/主体/AI待核;不新增公司或收入,不等其余竞赛类别。 另将2025生物技术名单9项全部登记:8项范围外或既有业务,1项产品与主体对应待核;不新增公司或收入,非全部竞赛类别。 后续补证:政府/法律名单Aparti关联2025主公司,Pytho按既有业务排除;该11项现1关联、8范围外、2待核,原始访问记录保留。",
"nextCheck": "继续核本篇2项的成立、主体及AI主营,优先解决Billow批次与发现年份时序、Sponstar原始年份冲突;既有业务的法律沿革及不可读项仍留缺;沿普通2025档案和AI分页扩展其余来源台账,区分参赛年、批次与公司成立年。 继续核Laplace原始年份及Eka精确注册主体;Recursive的2025/2026年份冲突保留。 继续核Honeyjar主字段与转载的年份差异、Blueprints原始可读资料与旧建筑业务主体;不把包价、演示金额或建设项目价值当ARR。 本篇待核Aparti精确注册、JustiGuide的2024/2026语义、orchestraos主体与年份及Pytho原始登记;补其余竞赛类别和年度档案,不用同名链接替代核查。 补ticker原始可读页面、Investwise主营及主体、Soundspace成立与AI实际交付;保留Genia结果收费与首页套餐的合同缺项,不把交易、返佣或客户资产年化为ARR。 补Avara可读条款和Lumos登记号对应、其他生物技术候选的AI实际作用及地域主体链;研发、临床、审批计划和药企赞助与当前商业收入分开。 Aparti已补公司自述成立,精确注册/付费及Amie收入仍待核;Pytho已由官网链接原始采访补范围外,不再列年份待核,精确注册留缺。",
"limitation": "需追溯原始披露,部分文章受访问限制",
"accessRecheckedAt": "2026-10-30"
},
@@ -44,7 +44,7 @@
"app.dealroom.co"
],
"initialAccess": "官网可读取",
"boundary": "已读Tana与Wabi公开公司档案:前者财务字段不一致留缺,后者2025成立字段与官方业务关联入表;尚未取得两年成立筛选结果集,两个个案不等于目录全量。",
"boundary": "已读Tana、Wabi及Laplace公开公司档案:Tana财务字段不一致留缺,Wabi2025字段与官方业务关联入表,Laplace未取得成立字段而保留候选;尚未取得两年成立筛选结果集,三个个案不等目录全量。 另读取Aparti公开档案并以公司自述成立关联;四个个案仍不等两年筛选全集,AI生成简介、上线与融资估值分开。",
"nextCheck": "确认公开可取得的两年成立筛选清单;逐项追溯财务原始凭证及字段口径,受权限限制保留缺项。",
"limitation": "完整档案需权限;概览及财务表不能未经勾稽直接导入ARR",
"accessRecheckedAt": "2026-10-30"
@@ -60,8 +60,8 @@
"producthunt.com"
],
"initialAccess": "官网可读取",
"boundary": "2026与2025年度档案各登记首个公开响应19条,共38条候选/推广;2025另有一个仅图片位置未解析。已逐条保留关联、范围外与年份待核状态,未穷尽两年档案。",
"nextCheck": "继续补两年其余分页/日期、仅图片位置及18条年份待核候选;保留Pazi日期冲突,核清品牌与主体,避免转移历史业绩。",
"boundary": "两年公开响应各19条,共38条。当前5条关联公司、2条范围外对照、26条范围外/既有业务、3条年份待核、2条推广。旧版本、品牌与法律主体分开,Tapflow据2024原版记录排除,Happycapy仍待核;非两年全量。 Pazi补既有业务链后范围外,保留字段冲突;Happycapy iOS销售方与ACRA注册日已核,全球产品/旧业务沿革仍待核。",
"nextCheck": "继续补两年其余分页/日期、仅图片位置及3条年份待核候选;保留Pazi日期冲突与旧业务排除依据,核清其余品牌与主体,避免转移历史业绩。 Happycapy的新加坡销售法人不自动等新创业;继续补网页签约方与集团业务沿革。",
"limitation": "上线时间不等于成立时间",
"accessRecheckedAt": "2026-10-30"
},
@@ -90,9 +90,22 @@
"bvp.com"
],
"initialAccess": "报告页面可读取",
"boundary": "用于行业分类和增长研究背景;不是新成立公司普查。",
"nextCheck": "核报告样本、方法与年份,不能将组合公司清单当总体。",
"limitation": "投资机构选样偏差"
"boundary": "主动选取20家高增长AI企业,含组合内外公司;非随机普查;已核方法与增长/毛利基准段;未取得20家逐公司底表。仅背景基准,不构成两年新公司名录。",
"nextCheck": "高增长选样和投资机构利益关系;不能把头部样本均值当各领域平均或新成立公司总体;需要公开可核底表时另补,不把缺项当已完成。",
"limitation": "投资机构选样偏差",
"accessRecheckedAt": "2026-10-30",
"methodology": {
"sample": "主动选取20家高增长AI企业,含组合内外公司;非随机普查",
"unit": "企业增长与毛利基准,按商业化后的第几年比较",
"period": "2025-08-13报告;商业化年数不等法律成立年",
"access": "已核方法与增长/毛利基准段;未取得20家逐公司底表",
"use": "用于提出留存、毛利、资本效率假设及分类线索",
"limit": "高增长选样和投资机构利益关系;不能把头部样本均值当各领域平均或新成立公司总体",
"evidenceIds": [
"CTX-BVP"
],
"checkedAt": "2026-10-30"
}
},
{
"id": "S07",
@@ -105,9 +118,22 @@
"www.menlovc.com"
],
"initialAccess": "报告页面可读取",
"boundary": "用于企业采购背景;不把调研比例转换为公司ARR。",
"nextCheck": "保留调研方法、采购样本及与新公司样本的差异。",
"limitation": "口径与样本需查看方法说明"
"boundary": "495名美国企业AI采购/开发决策者,企业已主动使用AI;已核公开方法、范围及限制;受访个体与公司估算底表未取得。仅背景基准,不构成两年新公司名录。",
"nextCheck": "仅美国企业;排除芯片、推理/模型服务及既有软件内嵌AI功能,不能与包含这些项目的总量直接比较或分摊为单家公司ARR;需要公开可核底表时另补,不把缺项当已完成。",
"limitation": "口径与样本需查看方法说明",
"accessRecheckedAt": "2026-10-30",
"methodology": {
"sample": "495名美国企业AI采购/开发决策者,企业已主动使用AI",
"unit": "采购调查与自下而上的市场支出估算,非公司逐项ARR",
"period": "2025-11-07至11-25调查,市场判断截至2025-12",
"access": "已核公开方法、范围及限制;受访个体与公司估算底表未取得",
"use": "用于买家、采购方式和任务方向假设",
"limit": "仅美国企业;排除芯片、推理/模型服务及既有软件内嵌AI功能,不能与包含这些项目的总量直接比较或分摊为单家公司ARR",
"evidenceIds": [
"CTX-MENLO"
],
"checkedAt": "2026-10-30"
}
},
{
"id": "S08",
@@ -117,12 +143,26 @@
"status": "context",
"hosts": [
"stripe.com",
"www.stripe.com"
"www.stripe.com",
"assets.stripeassets.com"
],
"initialAccess": "年度信可读取",
"boundary": "年度平台观察,不提供可公开穷尽的新AI商户名录。",
"nextCheck": "与TrustMRR支付重建数据分开;不据平台总量推算单家公司。",
"limitation": "平台样本不能代表全部AI公司"
"boundary": "2025年开始使用Stripe的商户分组,另有Atlas注册企业分组;已读并视觉核对PDF第1、3页;未取得商户明细与完整分组计算底表。仅背景基准,不构成两年新公司名录。",
"nextCheck": "开始使用平台不等成立年,也非AI公司专属样本;交易量不等Stripe收入,门槛数量翻倍不等ARR均值翻倍;不与TrustMRR公开者样本混算;需要公开可核底表时另补,不把缺项当已完成。",
"limitation": "平台样本不能代表全部AI公司",
"accessRecheckedAt": "2026-10-30",
"methodology": {
"sample": "2025年开始使用Stripe的商户分组,另有Atlas注册企业分组",
"unit": "平台交易量、商户增长及达到收入门槛的数量;分母不同",
"period": "2025年度;2026-02-24年度信",
"access": "已读并视觉核对PDF第1、3页;未取得商户明细与完整分组计算底表",
"use": "用于支付、全球市场及首次收费速度假设",
"limit": "开始使用平台不等成立年,也非AI公司专属样本;交易量不等Stripe收入,门槛数量翻倍不等ARR均值翻倍;不与TrustMRR公开者样本混算",
"evidenceIds": [
"CTX-STRIPE"
],
"checkedAt": "2026-10-30"
}
},
{
"id": "S09",
@@ -135,9 +175,22 @@
"saas-capital.com"
],
"initialAccess": "研究页可读取",
"boundary": "B2B订阅软件基准,不能直接代表新AI公司、硬件或服务企业。",
"nextCheck": "按报告样本及营收规模使用,不强套为行业ARR分布。",
"limitation": "不直接套用硬件与项目制业务"
"boundary": "第15次年度调查,超过1000家私营B2B SaaS公司;仅读公开摘要;完整下载需填写表单,未提交表单或取得完整底表。仅背景基准,不构成两年新公司名录。",
"nextCheck": "样本不是两年新AI公司;增长率中位数不能当ARR中位数;完整抽样与指标细节仍缺,不外推至硬件、项目服务或所有AI模式;需要公开可核底表时另补,不把缺项当已完成。",
"limitation": "不直接套用硬件与项目制业务",
"accessRecheckedAt": "2026-10-30",
"methodology": {
"sample": "第15次年度调查,超过1000家私营B2B SaaS公司",
"unit": "2025年度增长率与留存等公开摘要;非ARR金额分布",
"period": "2026报告回顾2025增长;不把报告年当公司成立年",
"access": "仅读公开摘要;完整下载需填写表单,未提交表单或取得完整底表",
"use": "用于B2B订阅软件的规模、融资和留存分层假设",
"limit": "样本不是两年新AI公司;增长率中位数不能当ARR中位数;完整抽样与指标细节仍缺,不外推至硬件、项目服务或所有AI模式",
"evidenceIds": [
"CTX-SAASCAP"
],
"checkedAt": "2026-10-30"
}
},
{
"id": "S10",
@@ -151,8 +204,8 @@
"pro.jiqizhixin.com"
],
"initialAccess": "首页可读取",
"boundary": "已登记4篇定向检索线索;三篇直达仅登录外壳,一篇读取失败。2项经公开原始资料补证关联公司,1项范围外,1项仍待核;未阅读全文或穷尽两年档案。",
"nextCheck": "补公开可读文章、2025档案及Utopai原始成立材料;受限正文留缺,不以搜索摘录代全文。",
"boundary": "已登记4篇定向检索线索;三篇直达仅登录外壳,一篇读取失败。2项经公开原始资料关联公司,SSI与Utopai经原始历史材料判为范围外,0项成立年待核;未阅读全文或穷尽两年档案。",
"nextCheck": "补公开可读文章及2025—2026档案;受限正文留缺,不以搜索摘录代全文,更名与产品发布不新增公司。",
"limitation": "全文访问受限;仅作发现时明确标识,技术和财务另追原始资料",
"accessRecheckedAt": "2026-10-30"
},
@@ -161,15 +214,16 @@
"name": "36氪",
"url": "https://www.36kr.com/",
"role": "reporting",
"status": "access-limited",
"status": "targeted",
"hosts": [
"www.36kr.com",
"36kr.com"
"36kr.com",
"eu.36kr.com"
],
"initialAccess": "安全检测,内容未核验",
"boundary": "2026-10-30仍返回安全检测页面,未把首页作为已读文章。",
"nextCheck": "使用可公开读取的具体文章或其他原始来源;不绕过访问保护。",
"limitation": "暂不作为已读取内容证据",
"boundary": "此前首页安全检测仍保留;本轮4项定向检索及相关段落补查,3项据政府/公司/交易当事方资料关联、1项按原始登记归范围外。清析检索与直达正文不一致未确认关联,其他文章为平台发布/授权转载,不代表两年全量。 另登记合木、昆腾动力、寰宇心生3项报道相关段落,原始成立/主体留缺,未新增主公司表;不是全量。",
"nextCheck": "墨甲已由交易当事方原始公告补足2025成立及历史期间收入,母公司项目沿革和当前财务仍待核;继续补清析发现文章关联及未读正文,建两年公开文章台账,不绕过访问保护。 补本组三项公司/孵化/投资原始资料,避免融资与研究院、母体业绩归属混淆。",
"limitation": "仅核具体相关段落,检索和直达差异、转载与原始披露分别记录。",
"accessRecheckedAt": "2026-10-30"
},
{
@@ -180,7 +234,7 @@
"status": "per-company",
"hosts": [],
"initialAccess": "按公司逐一核验",
"boundary": "公司官网和官方材料逐条绑定证据;不是可预先列尽的全站候选池。",
"boundary": "公司官网和官方材料逐条绑定证据;不是可预先列尽的全站候选池。 定向新增企智芯与九昇智能的2025成立自述及交付资料,精确注册、实际客户与收入未核;两个官网不代表中文公司全集。",
"nextCheck": "核定价、主体、交付与收入定义,自述与独立验证分开。",
"limitation": "自述不等于独立审计"
},
@@ -194,8 +248,8 @@
"trustmrr.com"
],
"initialAccess": "见具体证据访问记录",
"boundary": "已采集产品与支付快照;公开者自选,不是全部商户或独立财务审计。",
"nextCheck": "核同步日、连接失效、支付平台公式及产品/法人归属。",
"boundary": "已采集产品与支付快照;公开者自选,不是全部商户或独立财务审计。 另定向登记LLM Gateway、Gomotion AI(iker)及Promptwise,补官网、条款、公开Markdown与缓存差异;API文档明确USD分单位,未调用认证接口,不等全平台。",
"nextCheck": "核同步日、连接失效、支付平台公式及产品/法人归属。 补网关充值/供应商成本/订阅收入勾稽、Gomotion独立主体与官网、Promptwise期间表和快照窗口;保留过期连接及不同缓存时点,不用当前访问日刷新财务日期。",
"limitation": "来源可读不等于声明已独立验证。"
},
{
@@ -208,8 +262,8 @@
"speedrun.a16z.com"
],
"initialAccess": "见具体证据访问记录",
"boundary": "投资机构公司介绍定向样本,收入数字仍多来自公司。",
"nextCheck": "核币种、相对时间及管线/履历;未读取的页面保持缺项。",
"boundary": "投资机构公司介绍定向样本,收入数字仍多来自公司。 新增8项2026名录定向核查并登记候选台账(7项SR007、Bota SR006),不等全部批次或两年目录。 另登记8项2025年定向公司(SR005一项、SR006五项、SR007两项);动态目录静态仅外壳、浏览器超时,不认定全量。",
"nextCheck": "核币种、相对时间及管线/履历;未读取的页面保持缺项。 区分live ARR、signed/contracted ARR、固定法律承诺费用、管线、目标交易量及客户收益;Nexor原$不自动当USD,财务日历截至日和合同底表留缺。 补Hotbox两个ARR时点、OneGC试点包含关系及律所/平台收入拆分、Omi等待名单付费转化、DRS应用与合并口径、Cedar签约履行和Loops可读官网。",
"limitation": "来源可读不等于声明已独立验证。"
},
{
@@ -232,9 +286,11 @@
"url": null,
"role": "supplement",
"status": "targeted",
"hosts": [],
"hosts": [
"getlatka.com"
],
"initialAccess": "见具体证据访问记录",
"boundary": "未归入上述来源的公开材料按原始证据链接逐条保留;不视为新增已批准的完整候选池。",
"boundary": "未归入上述来源的公开材料按原始证据链接逐条保留;不视为新增已批准的完整候选池。 本轮读取GetLatka单页历史年收入估算,方法底表未取得,不视为新增完整候选池。",
"nextCheck": "逐项核来源归属、可靠性及可读正文,不能将搜索摘要当全文。",
"limitation": "来源可读不等于声明已独立验证。"
}
+52 -28
View File
@@ -1,6 +1,6 @@
{
"sourceVersion": "2026-10-30-tc-2",
"asOf": "2026-10-30",
"sourceVersion": "2026-10-31-industry-11",
"asOf": "2026-10-31",
"metric": "platform-mrr-usd",
"sampleUnit": "product-or-business",
"note": "独立Python复算参考值,仅验证样本计算;不是行业估计或财务审计。",
@@ -21,7 +21,9 @@
"MR011",
"MR012",
"MR015",
"MR016"
"MR016",
"MR018",
"MR019"
],
"groups": [
{
@@ -35,7 +37,7 @@
"MR016"
],
"n": 4,
"mean": 34815.0,
"mean": 34815,
"median": 36723.0,
"sampleDeviation": 16334.54845820151,
"cv": 0.46918134304758036,
@@ -50,7 +52,7 @@
"MR001"
],
"n": 1,
"mean": 53101.0,
"mean": 53101,
"median": 53101,
"sampleDeviation": null,
"cv": null,
@@ -65,7 +67,7 @@
"MR002"
],
"n": 1,
"mean": 62423.0,
"mean": 62423,
"median": 62423,
"sampleDeviation": null,
"cv": null,
@@ -79,15 +81,17 @@
"products": [
"MR003",
"MR004",
"MR005"
"MR005",
"MR018",
"MR019"
],
"n": 3,
"mean": 41693.0,
"median": 49069,
"sampleDeviation": 28111.375064197768,
"cv": 0.6742468775141575,
"meanWithoutLargest": 29849.0,
"largestRemovalDrop": 0.2840764636749574
"n": 5,
"mean": 51674.8,
"median": 65381,
"sampleDeviation": 24138.82269705795,
"cv": 0.4671294847209461,
"meanWithoutLargest": 47627.5,
"largestRemovalDrop": 0.07832250923080501
},
{
"category": "C09",
@@ -105,7 +109,7 @@
"median": 20483,
"sampleDeviation": 11942.947278624319,
"cv": 0.4711370488466823,
"meanWithoutLargest": 20046.0,
"meanWithoutLargest": 20046,
"largestRemovalDrop": 0.20920581320120557
}
]
@@ -127,9 +131,27 @@
"MR012",
"MR015",
"MR016",
"AI016"
"AI016",
"MR017",
"MR018",
"MR019"
],
"groups": [
{
"category": "C02",
"provider": "Stripe",
"model": "模型网关订阅与用量",
"products": [
"MR017"
],
"n": 1,
"mean": 78764,
"median": 78764,
"sampleDeviation": null,
"cv": null,
"meanWithoutLargest": null,
"largestRemovalDrop": null
},
{
"category": "C06",
"provider": "Stripe",
@@ -146,7 +168,7 @@
"median": 40112,
"sampleDeviation": 174786.78744916618,
"cv": 1.5505509613536426,
"meanWithoutLargest": 34815.0,
"meanWithoutLargest": 34815,
"largestRemovalDrop": 0.6911526751687284
},
{
@@ -157,7 +179,7 @@
"MR001"
],
"n": 1,
"mean": 53101.0,
"mean": 53101,
"median": 53101,
"sampleDeviation": null,
"cv": null,
@@ -172,7 +194,7 @@
"MR002"
],
"n": 1,
"mean": 62423.0,
"mean": 62423,
"median": 62423,
"sampleDeviation": null,
"cv": null,
@@ -186,15 +208,17 @@
"products": [
"MR003",
"MR004",
"MR005"
"MR005",
"MR018",
"MR019"
],
"n": 3,
"mean": 41693.0,
"median": 49069,
"sampleDeviation": 28111.375064197768,
"cv": 0.6742468775141575,
"meanWithoutLargest": 29849.0,
"largestRemovalDrop": 0.2840764636749574
"n": 5,
"mean": 51674.8,
"median": 65381,
"sampleDeviation": 24138.82269705795,
"cv": 0.4671294847209461,
"meanWithoutLargest": 47627.5,
"largestRemovalDrop": 0.07832250923080501
},
{
"category": "C09",
@@ -212,7 +236,7 @@
"median": 20483,
"sampleDeviation": 11942.947278624319,
"cv": 0.4711370488466823,
"meanWithoutLargest": 20046.0,
"meanWithoutLargest": 20046,
"largestRemovalDrop": 0.20920581320120557
}
]
+24 -3
View File
@@ -55,6 +55,26 @@ def source_register():
def context_methods():
evidence = {e['id']: e for e in DATA['evidence']}
rows = [s for s in SOURCES['sources'] if s.get('methodology')]
assert {s['id'] for s in rows} == {'S06', 'S07', 'S08', 'S09'}
lines = ['', '## 背景基准适用性与缺项', '',
'以下核对背景资料的方法及适用范围,不是公司名录的完整覆盖。原始方法记录与全局核查随研究版本进入同一数据库;没有把行业汇总值新增为任何公司的收入观测。', '',
'| 来源与证据 | 样本与统计单位 | 时间 | 本次读取范围 | 可用于什么 | 主要限制 |',
'| --- | --- | --- | --- | --- | --- |']
for source in rows:
m = source['methodology']
assert m['checkedAt'] <= DATA['asOf']
assert all(m.get(key) for key in ('sample', 'unit', 'period', 'access', 'use', 'limit', 'evidenceIds'))
assert all(eid in evidence and evidence[eid]['companyId'] is None for eid in m['evidenceIds'])
assert not any(o['evidence'] in m['evidenceIds'] for o in DATA['observations'])
proof = '、'.join(f"[{eid}]({evidence[eid]['url']})" for eid in m['evidenceIds'])
lines.append(f"| {source['id']} {source['name']}:{proof} | {m['sample']};{m['unit']} | {m['period']} | {m['access']} | {m['use']} | {m['limit']} |")
lines += ['', '本表的应用解释属于研究判断。选样、分母、时间或指标不同会造成表观差异;这些差异不单独证明财务造假,也不能作为给未知公司填补收入的依据。', '']
return lines
def candidate_register():
batches = DATA.get('candidateBatches', [])
companies = {c['id']: c for c in DATA['companies']}
@@ -67,7 +87,7 @@ def candidate_register():
lines = []
statuses = {'pending-year': '成立年待核', 'linked': '已关联公司',
'existing-outside-cohort': '已有关联,年份范围外',
'outside-cohort': '成立年范围外,不新增公司', 'promotion': '推广活动,非公司新增'}
'outside-cohort': '范围外或既有业务,不新增公司', 'promotion': '推广活动,非公司新增'}
for batch in batches:
assert batch['sourceId'] in known_sources
assert batch['accessedAt'] <= DATA['asOf']
@@ -171,8 +191,8 @@ def render():
f"| 其他目录基础项 | {depth-reviewed} | 规则初分或资料不足,继续待复核 |",
f"| 有团队数字 | {sum(c['team'] is not None for c in companies)} | 多为自报Team Size或Employees,时点与口径不统一 |",
f"| 专项技术证据记录 | {len(companies)-technical['unreviewed']} | 其中{technical['description']}条自述、{technical['public-artifact']}条公开材料、{technical['independent-test']}条第三方评测 |",
f"| 公司披露收入线索 | {income['self']} | 含历史、下界、币种不明及产品业务声明,不等于可比ARR |",
f"| 仅平台收入线索 | {income['platform']} | 同时有公司披露者计入上一行,不重复计数 |",
f"| 企业披露收入线索 | {income['self']} | 含公司或交易当事方披露、历史、下界及币种不明,不等于可比ARR |",
f"| 仅平台收入线索 | {income['platform']} | 同时有企业披露者计入上一行,不重复计数 |",
f"| 仅第三方估算 | {income['estimate']} | 不与直接披露混合 |",
f"| 暂无可用收入数字 | {income['unknown']} | 包含尚未逐家搜索的目录项,不代表没有收入 |", '',
'## 业务分类与数据入口', '',
@@ -199,6 +219,7 @@ def render():
lines.append(f"| 其他官网、评测及公开资料 | {len(others)} | {len({e['url'] for e in others})} | 按每条来源限制理解,不能统一当作独立证据 |")
lines += directory_rechecks()
lines += source_register()
lines += context_methods()
lines += candidate_register()
lines += ['', '## 尚未满足的研究条件', '',
'行业ARR比较仍缺统一业务截止日、币种、经常性定义和业务边界。现有平台MRR可描述已采集产品,不足以代表全部新公司。技术评测只覆盖所列版本与任务,不宜跨领域汇总为统一强弱分数。', '',
+2 -2
View File
@@ -3,7 +3,7 @@
set -eu
cd /opt/endorphin-tabularium-docs/current
case "${1:-}" in
list|get|save|history|export|backup|import)
list|get|save|history|export|backup|import|key-create|key-list|key-revoke)
docker compose -p endorphin-tabularium-docs --env-file /etc/endorphin-tabularium-docs/production.env --env-file release.env -f compose.yml exec -T docs node apps/docs/docs-cli.cjs "$@" ;;
*) echo 'Usage: docs list | get <slug> | save < JSON | history <sourceSlug> | export <private-file> | backup <private-file> | import < JSON' >&2; exit 1 ;;
*) echo 'Usage: docs key-create <name> <private-file> [read|read-write] [days] | key-list | key-revoke <id> | list | get <slug> | save < JSON | history <sourceSlug> | export <private-file> | backup <private-file> | import < JSON' >&2; exit 1 ;;
esac
+196
View File
@@ -0,0 +1,196 @@
"""Upload documents through the database API; credentials stay in a private file."""
import argparse
import json
import os
from pathlib import Path
import stat
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
import uuid
class DocumentClientError(ValueError):
pass
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, req, fp, code, msg, headers, newurl):
# Never forward a bearer credential to a redirected host or protocol.
return None
def server_url(value):
parsed = urllib.parse.urlsplit(value)
if (parsed.username or parsed.password or parsed.query or parsed.fragment
or parsed.path not in ('', '/') or not parsed.hostname):
raise ValueError('Provide a server origin without credentials, path, query or fragment')
if parsed.scheme != 'https' and not (
parsed.scheme == 'http' and parsed.hostname in ('localhost', '127.0.0.1', '::1')):
raise ValueError('Use HTTPS or an HTTP loopback connection through a trusted SSH tunnel')
return value.rstrip('/')
def read_token(filename):
path = Path(filename)
if stat.S_IMODE(path.stat().st_mode) & 0o077:
raise ValueError('API key file must be private (chmod 600)')
value = path.read_text().strip()
if value.startswith('{'):
value = json.loads(value)['token']
if not isinstance(value, str) or not value.startswith('tdoc_') or len(value) != 48:
raise ValueError('Invalid document API key file')
return value
def request(base, token, method, path, payload=None, request_id=None):
data = None if payload is None else json.dumps(payload, ensure_ascii=False).encode()
if data is not None and len(data) > 8 * 1024 * 1024:
raise ValueError('Request exceeds 8 MiB; split into smaller atomic batches')
headers = {'Authorization': 'Bearer ' + token, 'Accept': 'application/json'}
if data is not None:
headers['Content-Type'] = 'application/json'
headers['Idempotency-Key'] = request_id
opener = urllib.request.build_opener(NoRedirect())
for attempt in range(2):
req = urllib.request.Request(base + path, data=data, method=method, headers=headers)
try:
with opener.open(req, timeout=30) as response:
return json.load(response)
except urllib.error.HTTPError as error:
if error.code in (502, 503, 504) and attempt == 0:
error.close()
time.sleep(1)
continue
status = error.code
error.close()
explanations = {400: 'Invalid document or pagination', 401: 'Key missing, expired or revoked',
403: 'Read-only key', 404: 'Document or revision not found',
409: 'Revision or request-id conflict; read current state before retrying',
413: 'Request too large', 503: 'Database unavailable'}
raise DocumentClientError(f'HTTP {status}: ' + explanations.get(status, 'Request failed')) from None
except (urllib.error.URLError, TimeoutError):
if attempt == 0:
time.sleep(1)
continue
raise DocumentClientError('Connection failed; retry using the same request id and unchanged input') from None
def document_batches(documents, max_bytes=8 * 1024 * 1024):
if not isinstance(documents, list) or not documents:
raise DocumentClientError('Provide a nonempty documents array')
batches, chunk, size, seen = [], [], 20, set()
for document in documents:
if not isinstance(document, dict) or not isinstance(document.get('sourceSlug'), str):
raise DocumentClientError('Every document needs a sourceSlug')
if document['sourceSlug'] in seen:
raise DocumentClientError('Duplicate sourceSlug in upload')
seen.add(document['sourceSlug'])
encoded = len(json.dumps(document, ensure_ascii=False).encode()) + 2
if encoded + 20 > max_bytes:
raise DocumentClientError('A document exceeds the request limit')
if chunk and (len(chunk) == 100 or size + encoded > max_bytes):
batches.append(chunk)
chunk, size = [], 20
chunk.append(document)
size += encoded
batches.append(chunk)
return batches
def main():
os.umask(0o077)
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--url', default=os.environ.get('TABULARIUM_DOCS_URL'))
parser.add_argument('--key-file', default=os.environ.get('TABULARIUM_DOCS_KEY_FILE'))
sub = parser.add_subparsers(dest='command', required=True)
listing = sub.add_parser('list')
listing.add_argument('--limit', type=int, default=50)
listing.add_argument('--offset', type=int, default=0)
reading = sub.add_parser('get')
reading.add_argument('slug')
mode = reading.add_mutually_exclusive_group()
mode.add_argument('--history', action='store_true')
mode.add_argument('--revision', type=int)
reading.add_argument('--limit', type=int, default=50)
reading.add_argument('--offset', type=int, default=0)
sections = sub.add_parser('outline', help='Read chapter titles and internal summaries without bodies')
sections.add_argument('slug')
sections.add_argument('--revision', type=int)
section = sub.add_parser('sections', help='Read selected chapter bodies')
section.add_argument('slug')
section.add_argument('ids', nargs='+')
section.add_argument('--revision', type=int)
editing = sub.add_parser('edit-sections', help='Atomically apply chapter operations from private JSON')
editing.add_argument('slug')
editing.add_argument('file')
editing.add_argument('--request-id')
saving = sub.add_parser('save')
saving.add_argument('file', help='Private JSON file or - for stdin; one document or {documents: [...]}')
saving.add_argument('--request-id', help='Reuse this id only with the same input when retrying')
args = parser.parse_args()
if not args.url or not args.key_file:
parser.error('Set --url and --key-file or TABULARIUM_DOCS_URL and TABULARIUM_DOCS_KEY_FILE')
try:
base = server_url(args.url)
token = read_token(args.key_file)
if args.command == 'list':
result = request(base, token, 'GET', '/api/documents?' + urllib.parse.urlencode(
{'limit': args.limit, 'offset': args.offset}))
elif args.command == 'get':
path = '/api/documents/item/' + urllib.parse.quote(args.slug, safe='/')
if args.history:
path += '?' + urllib.parse.urlencode({'history': 1, 'limit': args.limit, 'offset': args.offset})
elif args.revision is not None:
path += '?' + urllib.parse.urlencode({'revision': args.revision})
result = request(base, token, 'GET', path)
elif args.command in ('outline', 'sections', 'edit-sections'):
path = '/api/documents/sections/' + urllib.parse.quote(args.slug, safe='/')
if args.command == 'edit-sections':
payload = json.load(sys.stdin) if args.file == '-' else json.loads(Path(args.file).read_text())
request_id = args.request_id or str(uuid.uuid4())
if not request_id or len(request_id) > 100 or any(c not in 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._:-' for c in request_id):
raise ValueError('Invalid request id')
print('Request id: ' + request_id, file=sys.stderr)
result = request(base, token, 'POST', path, payload, request_id)
else:
query = []
if args.revision is not None:
query.append(('revision', args.revision))
if args.command == 'sections':
query.extend(('section', section_id) for section_id in args.ids)
if query:
path += '?' + urllib.parse.urlencode(query)
result = request(base, token, 'GET', path)
else:
payload = json.load(sys.stdin) if args.file == '-' else json.loads(Path(args.file).read_text())
request_id = args.request_id or str(uuid.uuid4())
if not request_id or len(request_id) > 100 or any(c not in 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._:-' for c in request_id):
raise ValueError('Invalid request id')
print('Request id: ' + request_id, file=sys.stderr)
if isinstance(payload, dict) and 'documents' in payload:
if set(payload) != {'documents'}:
raise DocumentClientError('Batch payload must contain only documents')
chunks = document_batches(payload['documents'])
receipts, replayed = [], 0
for index, chunk in enumerate(chunks, 1):
batch_id = f'{request_id}:{index}'
saved = request(base, token, 'POST', '/api/documents/batch', {'documents': chunk}, batch_id)
receipts.extend(saved['documents'])
replayed += int(saved['replayed'])
print(f'Batch {index}/{len(chunks)} committed; {len(receipts)} documents total; request id: {batch_id}', file=sys.stderr)
result = {'documents': receipts, 'batches': len(chunks), 'replayedBatches': replayed, 'requestId': request_id}
else:
result = request(base, token, 'POST', '/api/documents', payload, request_id)
print(json.dumps(result, ensure_ascii=False, indent=2))
except DocumentClientError as error:
raise SystemExit(str(error)) from None
except (ValueError, OSError, KeyError, TypeError):
# Never print exception objects that could contain credentials or document data.
raise SystemExit('Document request failed. Check the URL, private key file, JSON schema and server response; credentials omitted.') from None
if __name__ == '__main__':
main()
+22 -3
View File
@@ -89,23 +89,42 @@ def link(cid):
def render():
points, excluded = sample(7)
grouped = groups(points)
described = [(key, stats(values)) for key, values in sorted(grouped.items())]
sizes = [value['n'] for _, value in described]
eligible = [(key, value) for key, value in described if value['n'] >= 3]
interpretation = '各组均不足3个样本,暂不比较分组离散度或收入高低。'
if eligible:
highest_key, highest = max(eligible, key=lambda item: item[1]['mean'])
sensitive_key, sensitive = max(eligible, key=lambda item: item[1]['largestRemovalDrop'] or 0)
cvs = [value['cv'] for _, value in eligible if value['cv'] is not None]
cv_text = f"CV范围为{min(cvs):.3f}—{max(cvs):.3f}。" if cvs else '各组CV均不可计算。'
interpretation = (
f"在至少3个样本的组内,当前均值最高的是{CATEGORIES[highest_key[0]]} · {highest_key[2]}"
f"({highest_key[1]},{money(highest['mean'])}美元每月)。{cv_text}"
f"剔除最大值后均值下降最多的是{CATEGORIES[sensitive_key[0]]} · {sensitive_key[2]}"
f"({sensitive_key[1]},下降{sensitive['largestRemovalDrop'] or 0:.1%})。"
'这些只是当前采集点的描述,不据微小CV差异宣称某领域更稳定。'
)
size_text = f"各组目前只有{min(sizes)}至{max(sizes)}个产品" if sizes else '当前没有可比较的产品'
lines = [
'# AI 公司收入样本分布快照', '',
f"截至 {DATA['asOf']},7日窗口内有 {len(points)} 个产品的美元平台MRR可纳入分组描述。创意软件订阅组的样本均值较高、CV也较大;营销软件组与消费者个人应用组的CV接近。各组只有1至5个产品,尚不足以估计行业总体或造假概率。", '',
f"截至 {DATA['asOf']},7日窗口内有 {len(points)} 个产品的美元平台MRR可纳入分组描述。按业务、支付平台和收费模式分别计算均值、中位数与离散度。{size_text},尚不足以估计行业总体或造假概率。", '',
f"这是已采集资料的固定快照。[动态工作台]({BASE})提供筛选与原始证据。窗口按最后同步日计算,不保证所有点同日,也不是{len(DATA['companies']):,}条公司/产品记录的收入普查。", '',
'## 7日窗口的分组结果', '',
'MRR单位为美元每月。CV为样本标准差除以均值;最大值剔除降幅表示移除最高收入产品后均值下降的比例。单个样本不计算离散度。', '',
'| 业务及收费模式 | 平台 | 样本数 | 均值 | 中位数 | CV | 最大值剔除降幅 |',
'| --- | --- | --- | --- | --- | --- | --- |',
]
for key, values in sorted(groups(points).items()):
for key, values in sorted(grouped.items()):
cat, provider, model = key
v = stats(values)
cv = f"{v['cv']:.3f}" if v['cv'] is not None else '不计算'
drop = f"{v['largestRemovalDrop']:.1%}" if v['largestRemovalDrop'] is not None else '不计算'
lines.append(f"| {CATEGORIES[cat]} · {model} | {provider} | {v['n']} | {money(v['mean'])} | {money(v['median'])} | {cv} | {drop} |")
lines += ['',
'在至少3个样本的组内,内容创意软件订阅的均值较高,但移除最大样本后的下降也最大。营销软件与个人应用的CV仅有微小差别,不据此宣称哪个领域更稳定。两个内容类单样本组虽有更高数值,不能作为典型收入或领域排名。', '',
interpretation, '',
'单个样本不计算离散度,也不能作为典型收入或领域排名;不同收费模式和支付平台的分组保留各自含义。', '',
'均值和中位数接近只描述当前采集点,并不证明样本具有代表性。平台公开者、成立年、规模、获客渠道与收费方式均可能影响结果。', '',
'## 样本与证据入口', '',
'点击产品名称可打开公司明细,再查看平台记录及其限制。连接已过期的近期快照仍按所标日期保留,并不表示连接当前有效。', '',
+110
View File
@@ -0,0 +1,110 @@
"""Lightweight client tests: no HTTP server, compilation or real credentials."""
import importlib.util
import io
import json
from pathlib import Path
import tempfile
import sys
import unittest
from unittest.mock import patch
import urllib.error
import urllib.request
spec = importlib.util.spec_from_file_location('documents_client', Path(__file__).with_name('documents-client.py'))
client = importlib.util.module_from_spec(spec)
spec.loader.exec_module(client)
class ClientTests(unittest.TestCase):
def test_section_reads_select_bodies_and_revision_without_full_document_fetch(self):
argv = ['client', '--url', 'https://docs.example', '--key-file', '/private/key',
'sections', 'endor/example', 'preamble', 's_one', '--revision', '3']
with patch.object(sys, 'argv', argv), patch.object(client, 'read_token', return_value='test-token'), \
patch.object(client, 'request', return_value={'revision': 3, 'sections': []}) as request, \
patch.object(sys, 'stdout', io.StringIO()):
client.main()
request.assert_called_once_with('https://docs.example', 'test-token', 'GET',
'/api/documents/sections/endor/example?revision=3&section=preamble&section=s_one')
def test_section_edit_keeps_payload_and_retry_id(self):
payload = {'revision': 3, 'operations': [{'op': 'update', 'id': 's_one', 'summary': 'Internal summary'}]}
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'edit.json'
path.write_text(json.dumps(payload))
argv = ['client', '--url', 'https://docs.example', '--key-file', '/private/key',
'edit-sections', 'endor/example', str(path), '--request-id', 'edit-1']
with patch.object(sys, 'argv', argv), patch.object(client, 'read_token', return_value='test-token'), \
patch.object(client, 'request', return_value={'revision': 4}) as request, \
patch.object(sys, 'stdout', io.StringIO()), patch.object(sys, 'stderr', io.StringIO()):
client.main()
request.assert_called_once_with('https://docs.example', 'test-token', 'POST',
'/api/documents/sections/endor/example', payload, 'edit-1')
def test_only_tls_or_loopback_origins_are_accepted(self):
self.assertEqual(client.server_url('https://docs.example/'), 'https://docs.example')
self.assertEqual(client.server_url('http://127.0.0.1:18135'), 'http://127.0.0.1:18135')
for url in ['http://docs.example', 'https://user:secret@docs.example', 'file:///tmp/data',
'https://docs.example/api', 'https://docs.example/?token=secret']:
with self.assertRaises(ValueError):
client.server_url(url)
def test_key_file_permissions_and_formats(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'key.json'
token = 'tdoc_' + 'x' * 43
path.write_text(json.dumps({'token': token}))
path.chmod(0o600)
self.assertEqual(client.read_token(path), token)
path.chmod(0o644)
with self.assertRaises(ValueError):
client.read_token(path)
def test_redirects_never_forward_bearer_credentials(self):
request = urllib.request.Request('https://docs.example', headers={'Authorization': 'Bearer test-only'})
self.assertIsNone(client.NoRedirect().redirect_request(request, None, 302, '', {}, 'https://other.example'))
def test_network_retry_preserves_idempotency_and_body(self):
calls = []
class Opener:
def open(self, request, timeout):
calls.append(request)
if len(calls) == 1:
raise urllib.error.URLError('temporary network failure')
return io.BytesIO(b'{"replayed":true}')
with patch.object(client.urllib.request, 'build_opener', return_value=Opener()), patch.object(client.time, 'sleep'):
result = client.request('https://docs.example', 'test-token', 'POST', '/api/documents', {'revision': 0}, 'request-1')
self.assertTrue(result['replayed'])
self.assertEqual(calls[0].data, calls[1].data)
self.assertEqual(calls[0].get_header('Idempotency-key'), 'request-1')
self.assertEqual(calls[1].get_header('Idempotency-key'), 'request-1')
def test_large_uploads_are_split_by_count_and_bytes(self):
documents = [{'sourceSlug': str(i), 'content': 'a' * 20} for i in range(201)]
batches = client.document_batches(documents)
self.assertEqual([len(batch) for batch in batches], [100, 100, 1])
self.assertEqual([doc for batch in batches for doc in batch], documents)
bounded = client.document_batches(documents[:5], max_bytes=150)
self.assertTrue(all(len(json.dumps({'documents': batch}, ensure_ascii=False).encode()) <= 150 for batch in bounded))
def test_duplicate_identifiers_and_oversized_items_fail_before_upload(self):
with self.assertRaises(ValueError):
client.document_batches([{'sourceSlug': 'same'}, {'sourceSlug': 'same'}])
with self.assertRaises(ValueError):
client.document_batches([{'sourceSlug': 'one', 'content': 'a' * 500}], max_bytes=100)
def test_conflict_is_reported_without_printing_server_content_or_secret(self):
class Opener:
def open(self, request, timeout):
raise urllib.error.HTTPError(request.full_url, 409, 'Conflict', {}, io.BytesIO(b'{"error":"private-content"}'))
with patch.object(client.urllib.request, 'build_opener', return_value=Opener()):
with self.assertRaisesRegex(client.DocumentClientError, 'HTTP 409') as result:
client.request('https://docs.example', 'private-token', 'POST', '/api/documents', {}, 'request-1')
self.assertNotIn('private-token', str(result.exception))
self.assertNotIn('private-content', str(result.exception))
if __name__ == '__main__':
unittest.main()