79 lines
4.3 KiB
Markdown
79 lines
4.3 KiB
Markdown
# Shared research API (in progress)
|
||
|
||
This work belongs to Thesisly 研迹 and uses Foundry Core rather than a separate
|
||
application backend. The corpus is public; account sessions and future private
|
||
collections remain project scoped.
|
||
|
||
## Current implementation boundary
|
||
|
||
`internal/research` implements the HTTP contract and a PostgreSQL repository
|
||
based on the fixed Paper Atlas `a3b677b` schema (TEXT/BIGINT columns, native
|
||
English title/abstract search index). `cmd/foundry` enables it only when
|
||
`FOUNDRY_RESEARCH_DATABASE_URL` is configured. No live research connection has
|
||
been configured during the ongoing migration.
|
||
The existing SQLite implementation was withdrawn. Wait for the separate Paper
|
||
Atlas PostgreSQL migration to finish, inspect the published schema and indexes,
|
||
then verify the queries, indexes and full-corpus performance with a
|
||
least-privilege read-only connection before enabling it in the existing server.
|
||
Do not add `FOUNDRY_RESEARCH_DATABASE`.
|
||
|
||
The server-side deployment may load a protected
|
||
`/opt/endoleap-infra/foundry/research.env` containing
|
||
`FOUNDRY_RESEARCH_DATABASE_URL` and `FOUNDRY_RESEARCH_SCHEMA` (default `research`).
|
||
Keep the connection string out of Git, Actions variables, browser configuration
|
||
and command arguments. Use a dedicated role with schema USAGE and table SELECT
|
||
only. Core uses a separate two-connection pool, read-only connection defaults and
|
||
read-only transactions. Startup checks the published schema/SELECT permissions;
|
||
an explicitly enabled but invalid connection fails candidate readiness.
|
||
|
||
Deployment migration defaults now follow the shared `foundry-postgres`
|
||
container name. `FOUNDRY_POSTGRES_CONTAINER` or the existing `--container` option
|
||
can select a different container without changing database/table names.
|
||
|
||
## Routes
|
||
|
||
- `GET /projects/{project}/research/papers`
|
||
- `GET /projects/{project}/research/papers/{id}`
|
||
|
||
Only configured Foundry projects are accepted. Reading the public corpus does
|
||
not require a session; these routes do not expose private data or accept writes.
|
||
|
||
List parameters are `q` (200 Unicode characters, 20 words), `searchIn`
|
||
(`text` or `author`), `topic` (an exact arXiv category), `sort` (`recent` or
|
||
`title`), `page` (1–1000), and `pageSize` (1–50, default 8). Unknown, malformed
|
||
or repeated parameters return 400. `recent` means descending stable paper ID,
|
||
not publication date. Search terms must become bound query parameters, never
|
||
raw SQL or user-supplied full-text syntax.
|
||
|
||
Title/abstract search uses the migration's English vector with
|
||
`plainto_tsquery`; author search uses simple-language terms over the stored
|
||
author JSON. Both treat input as terms, never full-text operators. Author search
|
||
and topic filtering still need full-corpus execution-plan/performance checks
|
||
after publication; the migration's title/abstract index does not index authors.
|
||
|
||
List responses contain `items`, `total`, `topics`, `page`, and `pageSize`.
|
||
Each paper preserves original title/abstract, authors, source dates, topics and
|
||
source links. Publication decisions need source evidence; absent decisions stay
|
||
unknown. No invented repositories, metrics or generated research conclusions.
|
||
|
||
The repository must count and read each list in one read-only snapshot. The
|
||
HTTP layer permits two in-flight queries with an eight-second context budget.
|
||
Busy requests return 429 with `Retry-After: 1`; deadlines return 504; storage
|
||
errors return a generic 503 without database details. Missing papers return 404.
|
||
|
||
## Validation
|
||
|
||
`go test -race -count=1 ./internal/research` covers routing, query limits,
|
||
Unicode, anonymous reads, rejected mutations, project lookup, errors, saturation
|
||
and timeout slot release. Run compilation/tests only on the user-confirmed ECS
|
||
runner. These transport tests do not prove a database connection, full-corpus
|
||
query performance or production availability.
|
||
|
||
The PostgreSQL repository batches source/tag/identifier/submission queries in
|
||
the same read-only repeatable-read transaction. Author tuples are displayed as
|
||
given name, family name and suffix; null abstracts/dates remain null. Publication
|
||
status comes from the latest observed decision, using decision date and ID as
|
||
tie-breakers, and stays unknown when no decision exists. Unit checks cover
|
||
author shapes, bound user values and schema qualification; real PostgreSQL
|
||
integration checks remain pending the migration.
|