feat: establish Hestia server with preserved deployment resources
Verify Hestia server / verify (push) Failing after 0s

This commit is contained in:
2026-10-20 01:10:02 +08:00
commit 85ca0cd124
54 changed files with 3713 additions and 0 deletions
+15
View File
@@ -0,0 +1,15 @@
.git
target
data
work
outputs
ci-output
.env
*.db
*.db-shm
*.db-wal
**/.git
**/target
**/AGENTS.md
**/work
**/ci-output
+8
View File
@@ -0,0 +1,8 @@
# Docker Compose reads .env; programs read exported host settings.
ASTER_PORT=8080
ASTER_LISTEN=127.0.0.1:8080
# JSON host information, separate from databases:
ASTER_STATE_DIR=data/host
# PostgreSQL provisioning only: credentials are not consumed by the VM host.
# Database-backed objects will receive an explicit DatabaseConfig.
POSTGRES_PASSWORD=replace-with-a-random-hex-password
+24
View File
@@ -0,0 +1,24 @@
# Hestia Server automation
[workflows](workflows/README.md) contains repository workflows.
## CI checkout prerequisites
All workflows use `runs-on: ces-control`; no shared Runner configuration is changed.
Configure repository or organization secrets `CODEX_BOT_SSH_KEY` (the dedicated
codex-bot private SSH key) and `GITEA_SSH_KNOWN_HOSTS` (the independently verified
known_hosts entry for `[gitea.entroia.com]:22222`). The bot needs read access to the
repository and its Aster submodule. Workflows use an isolated SSH command with
`-F /dev/null`, `IdentitiesOnly=yes`, an explicit key and port, strict host-key
checking, and temporary credential files removed after checkout. Keys are never
written into tracked files. Missing secrets fail checkout explicitly.
The SSH submodule URL's repository owner is separate from the authenticated bot.
Local Git operations must use the same dedicated bot identity; configure the
checkout's isolated command in `GIT_SSH_COMMAND` for submodule initialization.
Workflow definitions do not prove CI passed. CI/deployment state is not queried
automatically. Local runtime checks are not performed for Aster, Lyra or Hestia;
Tessera permits local native builds/window runs, with automated tests in CI.
Hestia deployment uses a separate manual-only [deploy.yml](workflows/deploy.yml); ordinary pushes and pull requests only verify.
+5
View File
@@ -0,0 +1,5 @@
# Hestia Server workflows
[ci.yml](ci.yml) verifies this project on main pushes, pull requests and manual dispatch. It runs on ces-control, with a 60-minute timeout. Checkout and secret requirements are in [the parent README](../README.md).
[deploy.yml](deploy.yml) is main-only and manual-only. It repeats verification, builds the runtime image, then runs the existing deployment script. Production resource names, paths, lock and database mounts remain unchanged.
+48
View File
@@ -0,0 +1,48 @@
name: Verify Hestia server
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
jobs:
verify:
runs-on: ces-control
timeout-minutes: 60
env:
DOCKER_BUILDKIT: "0"
steps:
- name: Checkout with codex-bot
env:
CODEX_BOT_SSH_KEY: ${{ secrets.CODEX_BOT_SSH_KEY }}
GITEA_SSH_KNOWN_HOSTS: ${{ secrets.GITEA_SSH_KNOWN_HOSTS }}
SOURCE_SHA: ${{ gitea.sha }}
SOURCE_REPOSITORY: ${{ gitea.repository }}
run: |
set -eu
test -n "$CODEX_BOT_SSH_KEY"
test -n "$GITEA_SSH_KNOWN_HOSTS"
case "$SOURCE_SHA" in *[!0-9a-f]*|'') exit 1;; esac
test "${#SOURCE_SHA}" -eq 40
umask 077
checkout_keys=$(mktemp -d)
trap 'rm -rf "$checkout_keys"' EXIT
printf '%s\n' "$CODEX_BOT_SSH_KEY" > "$checkout_keys/key"
printf '%s\n' "$GITEA_SSH_KNOWN_HOSTS" > "$checkout_keys/known_hosts"
unset CODEX_BOT_SSH_KEY GITEA_SSH_KNOWN_HOSTS
export GIT_SSH_COMMAND="ssh -F /dev/null -o IdentitiesOnly=yes -i $checkout_keys/key -p 22222 -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$checkout_keys/known_hosts"
git init .
source_url="ssh://git@gitea.entroia.com:22222/$SOURCE_REPOSITORY.git"
if git remote get-url origin >/dev/null 2>&1; then
git remote set-url origin "$source_url"
else
git remote add origin "$source_url"
fi
git fetch --no-tags --depth=1 origin "$SOURCE_SHA"
git checkout --detach --force FETCH_HEAD
git submodule update --init --recursive
- name: Verify SQLite, PostgreSQL and HTTP host
env:
ASTER_RELEASE_SHA: ${{ gitea.sha }}
run: sh scripts/ci-ces-control.sh
+55
View File
@@ -0,0 +1,55 @@
name: Deploy Hestia manually
on:
workflow_dispatch:
concurrency:
group: endorphin-aster-vm-production
cancel-in-progress: false
jobs:
verify-and-deploy:
if: gitea.ref == 'refs/heads/main'
runs-on: ces-control
timeout-minutes: 60
env:
DOCKER_BUILDKIT: "0"
ASTER_RELEASE_SHA: ${{ gitea.sha }}
steps:
- name: Checkout with codex-bot
env:
CODEX_BOT_SSH_KEY: ${{ secrets.CODEX_BOT_SSH_KEY }}
GITEA_SSH_KNOWN_HOSTS: ${{ secrets.GITEA_SSH_KNOWN_HOSTS }}
SOURCE_SHA: ${{ gitea.sha }}
SOURCE_REPOSITORY: ${{ gitea.repository }}
run: |
set -eu
test -n "$CODEX_BOT_SSH_KEY"
test -n "$GITEA_SSH_KNOWN_HOSTS"
case "$SOURCE_SHA" in *[!0-9a-f]*|'') exit 1;; esac
test "${#SOURCE_SHA}" -eq 40
umask 077
checkout_keys=$(mktemp -d)
trap 'rm -rf "$checkout_keys"' EXIT
printf '%s\n' "$CODEX_BOT_SSH_KEY" > "$checkout_keys/key"
printf '%s\n' "$GITEA_SSH_KNOWN_HOSTS" > "$checkout_keys/known_hosts"
unset CODEX_BOT_SSH_KEY GITEA_SSH_KNOWN_HOSTS
export GIT_SSH_COMMAND="ssh -F /dev/null -o IdentitiesOnly=yes -i $checkout_keys/key -p 22222 -o StrictHostKeyChecking=yes -o UserKnownHostsFile=$checkout_keys/known_hosts"
git init .
source_url="ssh://git@gitea.entroia.com:22222/$SOURCE_REPOSITORY.git"
if git remote get-url origin >/dev/null 2>&1; then
git remote set-url origin "$source_url"
else
git remote add origin "$source_url"
fi
git fetch --no-tags --depth=1 origin "$SOURCE_SHA"
git checkout --detach --force FETCH_HEAD
git submodule update --init --recursive
- name: Verify SQLite, PostgreSQL and HTTP host
env:
ASTER_RELEASE_SHA: ${{ gitea.sha }}
run: sh scripts/ci-ces-control.sh
- name: Build verified production image
run: docker build --target runtime -t "endorphin-aster-vm:$ASTER_RELEASE_SHA" .
- name: Deploy with existing resource identifiers
run: sh scripts/deploy-ces-control.sh
+11
View File
@@ -0,0 +1,11 @@
/target/
/data/
/work/
/outputs/
/ci-output/
.env
*.db
*.db-shm
*.db-wal
__pycache__/
AGENTS.md
+3
View File
@@ -0,0 +1,3 @@
[submodule "vendor/aster"]
path = vendor/aster
url = ssh://git@gitea.entroia.com:22222/Zayden-Jung/endorphin-aster-vm.git
Generated
+1888
View File
File diff suppressed because it is too large Load Diff
+22
View File
@@ -0,0 +1,22 @@
[workspace]
members = ["crates/server", "crates/admin", "crates/host", "crates/storage"]
resolver = "2"
[workspace.package]
version = "0.1.0"
edition = "2024"
rust-version = "1.91"
license = "MIT"
[workspace.dependencies]
core = { path = "vendor/aster/crates/core" }
storage = { path = "crates/storage" }
host = { path = "crates/host" }
axum = { version = "0.8", default-features = false, features = ["http1", "json", "tokio"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sqlx = { version = "0.8.6", default-features = false, features = ["runtime-tokio", "tls-rustls-ring", "sqlite", "postgres"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread", "net", "signal", "fs"] }
[workspace.lints.rust]
unsafe_code = "forbid"
+32
View File
@@ -0,0 +1,32 @@
FROM rust:1.91.1-bookworm AS source
WORKDIR /build
ENV CARGO_BUILD_JOBS=1
COPY Cargo.toml Cargo.lock ./
COPY crates ./crates
COPY vendor ./vendor
COPY scripts ./scripts
FROM source AS ci
RUN apt-get update && apt-get install -y --no-install-recommends python3 \
&& rm -rf /var/lib/apt/lists/* \
&& rustup component add rustfmt clippy
ENV CARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0
CMD ["sh", "scripts/check.sh"]
FROM source AS build
RUN cargo build --locked --release -p server -p admin
FROM debian:bookworm-slim AS runtime
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl \
&& rm -rf /var/lib/apt/lists/* \
&& useradd --system --uid 10001 --create-home aster \
&& mkdir -p /data/host && chown -R aster:aster /data
COPY --from=build /build/target/release/server /usr/local/bin/server
COPY --from=build /build/target/release/cli /usr/local/bin/cli
USER 10001
WORKDIR /data
ENV ASTER_LISTEN=0.0.0.0:8080 ASTER_STATE_DIR=/data/host
EXPOSE 8080
HEALTHCHECK --interval=10s --timeout=6s --start-period=10s --retries=3 \
CMD curl --fail --silent http://127.0.0.1:8080/health || exit 1
CMD ["server"]
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Aster VM contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+22
View File
@@ -0,0 +1,22 @@
# Hestia Server
Hestia 是独立 Endor HTTP 宿主。使用固定版本的 Aster 对象与 VM 库;当前仅提供 /health 和只读 /bootstrap,没有源码执行或认证 API。
源码迁自 Aster 提交 51cd64e6884c1036564c22e5de582c083d6db142。
| 入口 | 职责 |
| --- | --- |
| [crates](crates/README.md) | HTTP server、兼容管理命令 admin、文件信息 host、显式数据库适配 storage |
| [vendor](vendor/README.md) | 固定 Aster 子模块 |
| [scripts](scripts/README.md) | 验证、HTTP/文件信息冒烟和生产部署 |
| [docs](docs/README.md) | 配置、数据保护和部署契约 |
| [.gitea](.gitea/README.md) | ces-control CI,以及独立手动 CD |
| Dockerfile | 远端 source/ci/build/runtime 阶段 |
| docker-compose*.yml | 兼容既有服务和数据库资源的部署定义 |
| Cargo.toml / Cargo.lock | Rust 1.91+ 宿主工作区和锁定依赖 |
| .env.example / .gitignore / .dockerignore / LICENSE | 示例、排除规则与许可 |
Server 默认监听 127.0.0.1:8080,可由 ASTER_LISTEN 调整;宿主 JSON 存放于 ASTER_STATE_DIR。二进制 server 保留原名。admin 包产出兼容命令 cli,只提供 bootstrap、metadata put/get/import 和 help,供已有部署脚本使用;Endor run/eval/parse 属于独立 Lyra。
Host 和 Server 不连接数据库。Storage 仅在显式 DatabaseConfig 下连接,不创建应用表;真实 PostgreSQL 与 SQLite 测试均由 Hestia CI 承担。生产既有数据库、卷和目录完整保留。
main 推送和 PR 只验证;生产部署须显式手动触发 deploy.yml,且再次验证成功后才构建并发布。保留旧 ASTER_* 变量、Compose 项目/服务、镜像名和生产路径,不把拆仓当成数据迁移。见 [部署](docs/operations/deployment.md)。本地不构建/测试或运行 Docker,不主动查询 CI/部署。CI 所需 bot SSH secrets 见 [.gitea](.gitea/README.md)。
+10
View File
@@ -0,0 +1,10 @@
# Hestia 宿主包
| 目录 | 职责 |
| --- | --- |
| [server](server/README.md) | HTTP server,依赖固定 Aster core 和本地 host |
| [admin](admin/README.md) | 兼容 cli 二进制,只提供 bootstrap 与文件信息管理 |
| [host](host/README.md) | UTF-8 JSON 基础信息,不连接数据库 |
| [storage](storage/README.md) | 特殊类未来使用的显式 SQLite/PostgreSQL 适配 |
Aster 代码位于 vendor/aster,不复制引擎实现。返回 [项目入口](../README.md)。
+17
View File
@@ -0,0 +1,17 @@
[package]
name = "admin"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[[bin]]
name = "cli"
path = "src/main.rs"
[dependencies]
core.workspace = true
host.workspace = true
[lints]
workspace = true
+3
View File
@@ -0,0 +1,3 @@
# 兼容管理程序
Cargo 包名 admin,二进制 cli。仅提供 bootstrap、metadata put/get/import 和 help,保持既有部署脚本兼容。源码位于 [src](src/README.md),Cargo.toml 声明固定 Aster core 和本地 host 依赖。语言执行命令属于独立 Lyra。返回 [crates](../README.md)。
+3
View File
@@ -0,0 +1,3 @@
# 管理命令实现
main.rs 负责对象引导与释放、宿主 JSON 读写和缺失键导入;不提供 run/eval/parse,也不连接数据库。运行验证由 scripts/smoke.py 覆盖。返回 [admin](../README.md)。
+68
View File
@@ -0,0 +1,68 @@
use std::error::Error;
use std::process::ExitCode;
use endor_core::{Value, Vm};
use host::MetadataStore;
const USAGE: &str = "Usage: cli [bootstrap | metadata put <key> <value> | metadata get <key> | metadata import | help]";
fn main() -> ExitCode {
match run() {
Ok(()) => ExitCode::SUCCESS,
Err(error) => {
eprintln!("cli: {error}");
ExitCode::FAILURE
}
}
}
fn run() -> Result<(), Box<dyn Error>> {
let args: Vec<String> = std::env::args().skip(1).collect();
match args
.iter()
.map(String::as_str)
.collect::<Vec<_>>()
.as_slice()
{
[] | ["bootstrap"] => {
let mut vm = Vm::new()?;
let first = vm.new_object()?;
let second = vm.new_object()?;
vm.set_attr(
first,
"message",
Value::Text("Hello from EndorObject".into()),
)?;
println!("runtime=EndorObject type_system=disabled");
println!(
"objects_distinct={} heap_objects={}",
first != second,
vm.object_count()
);
vm.release(first)?;
vm.release(second)?;
println!("released heap_objects={}", vm.object_count());
println!("bootstrap uses no database");
}
["help"] | ["--help"] | ["-h"] => println!("{USAGE}"),
["metadata", "put", key, value] => {
MetadataStore::from_env()?.put(key, value)?;
println!("stored (file)");
}
["metadata", "get", key] => {
println!(
"{}",
MetadataStore::from_env()?
.get(key)?
.as_deref()
.unwrap_or("(missing)")
);
}
["metadata", "import"] => {
let count = MetadataStore::from_env()?.import_missing(std::io::stdin().lock())?;
println!("imported {count} missing entries (file)");
}
_ => return Err(USAGE.into()),
}
Ok(())
}
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "host"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
serde.workspace = true
serde_json.workspace = true
[dev-dependencies]
tempfile = "3"
[lints]
workspace = true
+11
View File
@@ -0,0 +1,11 @@
# host
宿主基础信息的文件层,与 Core 和数据库 Storage 分开。CLI/Server 使用它保存 host.version、deployment.release 和其他命名信息;不打开数据库,不保存 VM 堆。
| 内容 | 职责 |
| --- | --- |
| Cargo.toml | host 包、Serde/JSON 依赖及远端检查依赖 |
| [src](src/README.md) | MetadataStore、UTF-8 JSON 文件与迁移导入 |
| [tests](tests/README.md) | 持久读取、替换、并发、坏记录、路径边界和幂等导入 |
ASTER_STATE_DIR 指定信息目录;默认 data/host,容器使用 /data/host。协议、生命周期及旧数据库一次性迁移见 [宿主信息](../../docs/operations/host-information.md)。返回 [工作区](../README.md)。
+8
View File
@@ -0,0 +1,8 @@
# Host 源码导航
| 文件 | 职责 |
| --- | --- |
| [lib.rs](lib.rs) | 导出 MetadataStore,明确文件层不依赖 VM 或 SQL |
| [metadata.rs](metadata.rs) | 目录配置、键校验、逐键 JSON 读取/同步写入、原子发布、仅导入缺失项 |
文件数据以 UTF-8 JSON 保存,操作系统路径用 PathBuf;坏记录不自动覆盖。每个键独立发布,不提供多键事务。旧库导入不覆盖新文件,规则见 [宿主信息](../../../docs/operations/host-information.md)。返回 [host](../README.md)。
+4
View File
@@ -0,0 +1,4 @@
//! Host information stored in UTF-8 JSON files, independently of the VM and SQL.
mod metadata;
pub use metadata::MetadataStore;
+172
View File
@@ -0,0 +1,172 @@
use std::fs::{self, OpenOptions};
use std::io::{self, Read, Write};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use serde::{Deserialize, Serialize};
static NEXT_TEMP: AtomicU64 = AtomicU64::new(0);
const FORMAT_VERSION: u32 = 1;
#[derive(Deserialize)]
struct LegacyRecord {
key: String,
value: String,
}
#[derive(Serialize, Deserialize)]
struct Record {
format_version: u32,
key: String,
value: String,
}
/// One independently replaceable JSON file per ASCII metadata key. Clones
/// share a directory, not a cached map; reads always observe the file system.
#[derive(Debug, Clone)]
pub struct MetadataStore {
directory: PathBuf,
}
impl MetadataStore {
/// Does not create the directory until the first write.
pub fn new(directory: impl Into<PathBuf>) -> io::Result<Self> {
let directory = directory.into();
if directory.as_os_str().is_empty() {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"metadata directory is empty",
));
}
Ok(Self { directory })
}
/// Preserve OS paths without requiring the environment value to be UTF-8.
pub fn from_env() -> io::Result<Self> {
Self::new(
std::env::var_os("ASTER_STATE_DIR")
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from("data/host")),
)
}
pub fn directory(&self) -> &Path {
&self.directory
}
pub fn get(&self, key: &str) -> io::Result<Option<String>> {
let path = self.path(key)?;
let bytes = match fs::read(path) {
Ok(bytes) => bytes,
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None),
Err(error) => return Err(error),
};
let record: Record = serde_json::from_slice(&bytes)
.map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;
if record.format_version != FORMAT_VERSION || record.key != key {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
"metadata key or format version mismatch",
));
}
Ok(Some(record.value))
}
/// Writes a new sibling file, syncs it, then publishes it with rename.
/// Different keys do not share a read/modify/write map. For the same key,
/// the last successful rename wins. This is not a multi-key transaction.
pub fn put(&self, key: &str, value: &str) -> io::Result<()> {
self.store(key, value, true).map(|_| ())
}
/// One-time migration input: a JSON array of key/value string records.
/// Validate all keys first, and never overwrite an existing file record.
pub fn import_missing(&self, reader: impl Read) -> io::Result<usize> {
let records: Vec<LegacyRecord> = serde_json::from_reader(reader)
.map_err(|error| io::Error::new(io::ErrorKind::InvalidData, error))?;
for record in &records {
self.path(&record.key)?;
}
let mut imported = 0;
for record in records {
if self.store(&record.key, &record.value, false)? {
imported += 1;
} else if self.get(&record.key)?.is_none() {
return Err(io::Error::new(
io::ErrorKind::NotFound,
"existing metadata disappeared during import",
));
}
}
Ok(imported)
}
fn store(&self, key: &str, value: &str, replace: bool) -> io::Result<bool> {
let destination = self.path(key)?;
let mut contents = serde_json::to_vec_pretty(&Record {
format_version: FORMAT_VERSION,
key: key.into(),
value: value.into(),
})
.map_err(io::Error::other)?;
contents.push(b'\n');
fs::create_dir_all(&self.directory)?;
for _ in 0..32 {
let sequence = NEXT_TEMP.fetch_add(1, Ordering::Relaxed);
let temporary = self
.directory
.join(format!(".{key}.{}.{sequence}.tmp", std::process::id()));
let mut options = OpenOptions::new();
options.write(true).create_new(true);
#[cfg(unix)]
{
use std::os::unix::fs::OpenOptionsExt;
options.mode(0o600);
}
let mut file = match options.open(&temporary) {
Ok(file) => file,
Err(error) if error.kind() == io::ErrorKind::AlreadyExists => continue,
Err(error) => return Err(error),
};
let result = (|| {
file.write_all(&contents)?;
file.sync_all()?;
drop(file);
if replace {
fs::rename(&temporary, &destination)?;
Ok(true)
} else {
// Atomic create-if-absent: a concurrent writer's record
// cannot be replaced by the legacy import.
match fs::hard_link(&temporary, &destination) {
Ok(()) => Ok(true),
Err(error) if error.kind() == io::ErrorKind::AlreadyExists => Ok(false),
Err(error) => Err(error),
}
}
})();
let _ = fs::remove_file(&temporary);
return result;
}
Err(io::Error::new(
io::ErrorKind::AlreadyExists,
"cannot reserve a metadata temporary file",
))
}
fn path(&self, key: &str) -> io::Result<PathBuf> {
if key.is_empty()
|| key.len() > 128
|| !key.as_bytes()[0].is_ascii_alphanumeric()
|| !key
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'.' | b'_' | b'-'))
{
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"metadata keys must start with an ASCII letter/digit and contain at most 128 ASCII letters, digits, dots, underscores or hyphens",
));
}
Ok(self.directory.join(format!("{key}.json")))
}
}
+5
View File
@@ -0,0 +1,5 @@
# Host 文件信息检查
[metadata.rs](metadata.rs) 检查缺失读取不建目录、日文/中文/emoji 值和日文路径、重开与同键替换、非法键不越界、坏 JSON/版本/键不匹配不改原文件、并发替换可见完整记录,以及旧记录幂等导入和保留更新值。
检查使用临时目录,仅在指定远端环境运行。本地不执行文件行为测试;当前是否通过应以对应提交的 CI 为准。命令见 [验证](../../../vendor/aster/docs/operations/validation.md),返回 [host](../README.md)。
+131
View File
@@ -0,0 +1,131 @@
use host::MetadataStore;
#[test]
fn missing_reads_do_not_create_state() {
let temporary = tempfile::tempdir().unwrap();
let directory = temporary.path().join("state");
let store = MetadataStore::new(&directory).unwrap();
assert_eq!(store.get("missing").unwrap(), None);
assert!(!directory.exists());
}
#[test]
fn utf8_values_survive_reopening_and_same_key_replacement() {
let temporary = tempfile::tempdir().unwrap();
let directory = temporary.path().join("日本語");
let store = MetadataStore::new(&directory).unwrap();
store.put("host.version", "0.1.0").unwrap();
store.put("project.note", "日本語・中文・😀").unwrap();
let reopened = MetadataStore::new(&directory).unwrap();
assert_eq!(
reopened.get("project.note").unwrap().as_deref(),
Some("日本語・中文・😀")
);
reopened.put("host.version", "0.2.0").unwrap();
assert_eq!(store.get("host.version").unwrap().as_deref(), Some("0.2.0"));
assert_eq!(
store.get("project.note").unwrap().as_deref(),
Some("日本語・中文・😀")
);
let record: serde_json::Value =
serde_json::from_slice(&std::fs::read(directory.join("project.note.json")).unwrap())
.unwrap();
assert_eq!(record["format_version"], 1);
assert_eq!(record["key"], "project.note");
}
#[test]
fn keys_cannot_escape_the_state_directory() {
let temporary = tempfile::tempdir().unwrap();
let directory = temporary.path().join("state");
let store = MetadataStore::new(&directory).unwrap();
for key in [
"",
"../outside",
"/outside",
"nested/key",
"nested\\key",
".hidden",
"日本語",
] {
assert!(store.put(key, "invalid").is_err(), "{key}");
assert!(store.get(key).is_err(), "{key}");
}
assert!(store.put(&"x".repeat(129), "invalid").is_err());
assert!(!directory.exists());
}
#[test]
fn corrupt_or_mismatched_records_fail_without_being_rewritten() {
let temporary = tempfile::tempdir().unwrap();
let store = MetadataStore::new(temporary.path()).unwrap();
let path = temporary.path().join("host.version.json");
for bytes in [
b"not-json".as_slice(),
b"\xff".as_slice(),
br#"{"format_version":2,"key":"host.version","value":"old"}"#.as_slice(),
br#"{"format_version":1,"key":"wrong","value":"old"}"#.as_slice(),
] {
std::fs::write(&path, bytes).unwrap();
assert!(store.get("host.version").is_err());
assert_eq!(std::fs::read(&path).unwrap(), bytes);
}
}
#[test]
fn concurrent_replacements_expose_complete_records_and_preserve_other_keys() {
let temporary = tempfile::tempdir().unwrap();
let store = MetadataStore::new(temporary.path()).unwrap();
store.put("independent", "keep").unwrap();
let a = "a".repeat(8192);
let b = "b".repeat(8192);
store.put("shared", &a).unwrap();
std::thread::scope(|scope| {
for value in [&a, &b] {
let store = &store;
scope.spawn(move || {
for _ in 0..8 {
store.put("shared", value).unwrap();
}
});
}
for _ in 0..64 {
let value = store.get("shared").unwrap().unwrap();
assert!(value == a || value == b);
}
});
assert_eq!(store.get("independent").unwrap().as_deref(), Some("keep"));
assert!(std::fs::read_dir(temporary.path()).unwrap().all(|entry| {
!entry
.unwrap()
.file_name()
.to_string_lossy()
.ends_with(".tmp")
}));
}
#[test]
fn legacy_import_is_repeatable_and_preserves_newer_file_values() {
let temporary = tempfile::tempdir().unwrap();
let store = MetadataStore::new(temporary.path()).unwrap();
store.put("host.version", "new").unwrap();
let input =
br#"[{"key":"host.version","value":"old"},{"key":"project.note","value":"legacy"}]"#;
assert_eq!(store.import_missing(input.as_slice()).unwrap(), 1);
assert_eq!(store.import_missing(input.as_slice()).unwrap(), 0);
assert_eq!(store.get("host.version").unwrap().as_deref(), Some("new"));
assert_eq!(
store.get("project.note").unwrap().as_deref(),
Some("legacy")
);
}
#[test]
fn invalid_import_keys_are_rejected_before_any_record_is_written() {
let temporary = tempfile::tempdir().unwrap();
let directory = temporary.path().join("state");
let store = MetadataStore::new(&directory).unwrap();
let input = br#"[{"key":"valid","value":"keep"},{"key":"../outside","value":"invalid"}]"#;
assert!(store.import_missing(input.as_slice()).is_err());
assert!(!directory.exists());
}
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "server"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
core.workspace = true
host.workspace = true
axum.workspace = true
serde.workspace = true
tokio.workspace = true
[lints]
workspace = true
+10
View File
@@ -0,0 +1,10 @@
# server
最小常驻 HTTP 宿主,提供存活检查和只读 VM 信息。通过 Arc 共享 Vm,启动不读取数据库配置、不连接或写入数据库;通过 host 包把 host.version 写入文件。特殊类的数据库能力尚未接入 HTTP 宿主。
| 内容 | 职责 |
| --- | --- |
| Cargo.toml | core/host、Axum、Tokio、Serde 依赖;无 storage 依赖 |
| [src](src/README.md) | AppState、启动、路由、信号与退出 |
HTTP 返回内容见 [运行指南](../../vendor/aster/docs/operations/README.md),监听变量见 [配置](../../docs/operations/configuration.md),预置 PostgreSQL 资源见 [部署](../../docs/operations/deployment.md)。没有语言执行接口或可变 VM 请求上下文。返回 [工作区](../README.md)。
+13
View File
@@ -0,0 +1,13 @@
# Server 入口
[main.rs](main.rs) 包含所有宿主启动代码:
| 部分 | 职责 |
| --- | --- |
| AppState | 共享只读 Arc<Vm> |
| main/run | 创建空 VM、绑定地址、设置退出信号、写文件 host.version、组装路由;不创建数据库连接 |
| health | 返回 200 和 status=ok,仅检查 HTTP 宿主可响应 |
| bootstrap | 返回对象模型、type_system=false 和堆对象数量,不分配对象 |
| shutdown_signal | Ctrl-C;Unix 上同时接受 SIGTERM,触发 Axum 优雅退出 |
/health 不表示 PostgreSQL、特殊类查询或执行脚本正常;数据库检查应属于实际消费连接的对象。宿主独立于数据库的行为由远端 smoke.py 验证,见 [验证](../../../vendor/aster/docs/operations/validation.md)。改变端点同步 [运行指南](../../../vendor/aster/docs/operations/README.md) 和 [scripts](../../../scripts/README.md)。返回 [server](../README.md)。
+77
View File
@@ -0,0 +1,77 @@
use std::error::Error;
use std::future::Future;
use std::process::ExitCode;
use std::sync::Arc;
use axum::{Json, Router, extract::State, routing::get};
use endor_core::Vm;
use host::MetadataStore;
use serde::Serialize;
#[derive(Clone)]
struct AppState {
vm: Arc<Vm>,
}
#[derive(Serialize)]
struct Health {
status: &'static str,
}
#[derive(Serialize)]
struct BootstrapInfo {
object_model: &'static str,
type_system: bool,
heap_objects: usize,
}
#[tokio::main]
async fn main() -> ExitCode {
match run().await {
Ok(()) => ExitCode::SUCCESS,
Err(error) => {
eprintln!("server: {error}");
ExitCode::FAILURE
}
}
}
async fn run() -> Result<(), Box<dyn Error>> {
let vm = Arc::new(Vm::new()?);
let address = std::env::var("ASTER_LISTEN").unwrap_or_else(|_| "127.0.0.1:8080".into());
let listener = tokio::net::TcpListener::bind(&address).await?;
let shutdown = shutdown_signal()?;
MetadataStore::from_env()?.put("host.version", env!("CARGO_PKG_VERSION"))?;
let app = Router::new()
.route("/health", get(health))
.route("/bootstrap", get(bootstrap))
.with_state(AppState { vm });
println!("Endor host listening on {}", listener.local_addr()?);
axum::serve(listener, app)
.with_graceful_shutdown(shutdown)
.await?;
Ok(())
}
async fn health() -> Json<Health> {
Json(Health { status: "ok" })
}
async fn bootstrap(State(state): State<AppState>) -> Json<BootstrapInfo> {
Json(BootstrapInfo {
object_model: "EndorObject",
type_system: false,
heap_objects: state.vm.object_count(),
})
}
fn shutdown_signal() -> std::io::Result<impl Future<Output = ()>> {
#[cfg(unix)]
let mut terminate = tokio::signal::unix::signal(tokio::signal::unix::SignalKind::terminate())?;
Ok(async move {
#[cfg(unix)]
tokio::select! { _ = tokio::signal::ctrl_c() => {}, _ = terminate.recv() => {} }
#[cfg(not(unix))]
let _ = tokio::signal::ctrl_c().await;
})
}
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "storage"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license.workspace = true
[dependencies]
sqlx.workspace = true
tokio.workspace = true
[dev-dependencies]
tempfile = "3"
[lints]
workspace = true
+11
View File
@@ -0,0 +1,11 @@
# storage
按需数据库连接适配层,供将来需要数据库的特殊类或宿主适配器使用。依赖 SQLx/Tokio,不依赖 VM 内核;普通对象、CLI 和 Server 不连接这层。
| 内容 | 入口 |
| --- | --- |
| Cargo.toml | SQLx、Tokio 与远端检查依赖 |
| [src](src/README.md) | 显式 DatabaseConfig、连接池与关闭 |
| [tests](tests/README.md) | 连接不建表、调用方 SQL、隔离/持久化、真实 PostgreSQL |
没有默认数据库、自动环境变量读取、基础信息表或通用元数据 put/get。特殊类的名称、布局与查询接口尚未定义;当前只保留连接能力。设计边界见 [数据库对象](../../docs/runtime/database-objects.md),参数见 [配置](../../docs/operations/configuration.md)。返回 [工作区](../README.md)。
+7
View File
@@ -0,0 +1,7 @@
# 数据库连接源码导航
[lib.rs](lib.rs) 是唯一实现文件:DatabaseConfig 显式选择 SQLite 文件/内存或 PostgreSQL;DatabaseConnection 提供 connect、sqlite_pool/postgres_pool、backend_name、ping 和 close;DatabaseError 对公开错误脱敏。
connect 只建立连接池,不执行应用 DDL,不保存基础信息。没有 schema.sql、初始化表、put/get 或 from_env。业务表和查询由未来的数据库特殊类决定;本层不替它定义数据模型。
克隆共享连接池,close 会关闭所有克隆;具体生命周期及限制见 [数据库对象](../../../docs/runtime/database-objects.md)。配置变动同步 [配置](../../../docs/operations/configuration.md) 和 [tests](../tests/README.md)。返回 [storage](../README.md)。
+181
View File
@@ -0,0 +1,181 @@
//! Explicit database connections for database-backed objects.
//! Connecting never creates application tables or stores VM/host metadata.
use std::fmt;
use std::path::PathBuf;
use std::str::FromStr;
use std::time::Duration;
use sqlx::postgres::{PgConnectOptions, PgPoolOptions};
use sqlx::sqlite::{SqliteConnectOptions, SqliteJournalMode, SqlitePoolOptions};
use sqlx::{PgPool, SqlitePool};
/// An explicit PostgreSQL configuration never falls back to SQLite.
#[derive(Clone)]
pub enum DatabaseConfig {
SqliteFile(PathBuf),
SqliteMemory,
Postgres(String),
}
impl fmt::Debug for DatabaseConfig {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::SqliteFile(path) => f.debug_tuple("SqliteFile").field(path).finish(),
Self::SqliteMemory => f.write_str("SqliteMemory"),
Self::Postgres(_) => f.write_str("Postgres([REDACTED])"),
}
}
}
#[derive(Debug)]
pub enum DatabaseError {
Config(&'static str),
Directory(std::io::Error),
Database(sqlx::Error),
}
impl fmt::Display for DatabaseError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Config(message) => f.write_str(message),
Self::Directory(_) => f.write_str("cannot create SQLite parent directory"),
// CLI/HTTP logs must never print a connection URL or credentials.
Self::Database(_) => {
f.write_str("database operation failed; no fallback was attempted")
}
}
}
}
impl std::error::Error for DatabaseError {
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
match self {
Self::Config(_) => None,
Self::Directory(error) => Some(error),
Self::Database(error) => Some(error),
}
}
}
impl From<sqlx::Error> for DatabaseError {
fn from(error: sqlx::Error) -> Self {
Self::Database(error)
}
}
#[derive(Clone)]
enum Backend {
Sqlite(SqlitePool),
Postgres(PgPool),
}
/// Created only on explicit request by an object or host adapter. Clones share
/// a pool; close() closes it for all users. There is no automatic schema or data.
/// In-memory SQLite uses one connection to preserve its database lifetime.
#[derive(Clone)]
pub struct DatabaseConnection {
backend: Backend,
}
impl DatabaseConnection {
/// Connects eagerly to the explicit backend without executing application DDL.
pub async fn connect(config: &DatabaseConfig) -> Result<Self, DatabaseError> {
let backend = match config {
DatabaseConfig::Postgres(url) => {
if !(url.starts_with("postgres://") || url.starts_with("postgresql://")) {
return Err(DatabaseError::Config(
"expected a PostgreSQL connection URL",
));
}
let options = PgConnectOptions::from_str(url)
.map_err(|_| DatabaseError::Config("invalid PostgreSQL connection URL"))?;
Backend::Postgres(
PgPoolOptions::new()
.max_connections(5)
.acquire_timeout(Duration::from_secs(5))
.connect_with(options)
.await?,
)
}
DatabaseConfig::SqliteFile(path) => {
if path.as_os_str().is_empty() || path == std::path::Path::new(":memory:") {
return Err(DatabaseError::Config(
"expected a nonempty file path; use SqliteMemory for :memory:",
));
}
if let Some(parent) = path.parent().filter(|p| !p.as_os_str().is_empty()) {
tokio::fs::create_dir_all(parent)
.await
.map_err(DatabaseError::Directory)?;
}
let options = SqliteConnectOptions::new()
.filename(path)
.create_if_missing(true)
.journal_mode(SqliteJournalMode::Wal)
.busy_timeout(Duration::from_secs(5));
Backend::Sqlite(
SqlitePoolOptions::new()
.max_connections(5)
.acquire_timeout(Duration::from_secs(5))
.connect_with(options)
.await?,
)
}
DatabaseConfig::SqliteMemory => {
let options = SqliteConnectOptions::new().in_memory(true);
Backend::Sqlite(
SqlitePoolOptions::new()
.max_connections(1)
.min_connections(1)
.idle_timeout(None)
.max_lifetime(None)
.connect_with(options)
.await?,
)
}
};
Ok(Self { backend })
}
/// Backend-specific access for the database-backed object implementation.
pub fn sqlite_pool(&self) -> Option<&SqlitePool> {
match &self.backend {
Backend::Sqlite(pool) => Some(pool),
Backend::Postgres(_) => None,
}
}
pub fn postgres_pool(&self) -> Option<&PgPool> {
match &self.backend {
Backend::Postgres(pool) => Some(pool),
Backend::Sqlite(_) => None,
}
}
pub fn backend_name(&self) -> &'static str {
match &self.backend {
Backend::Sqlite(_) => "sqlite",
Backend::Postgres(_) => "postgresql",
}
}
pub async fn ping(&self) -> Result<(), DatabaseError> {
match &self.backend {
Backend::Sqlite(pool) => {
sqlx::query("SELECT 1").execute(pool).await?;
}
Backend::Postgres(pool) => {
sqlx::query("SELECT 1").execute(pool).await?;
}
}
Ok(())
}
/// Explicitly closes the shared pool, including all DatabaseConnection clones.
pub async fn close(&self) {
match &self.backend {
Backend::Sqlite(pool) => pool.close().await,
Backend::Postgres(pool) => pool.close().await,
}
}
}
+8
View File
@@ -0,0 +1,8 @@
# 数据库连接检查
| 文件 | 覆盖与前提 |
| --- | --- |
| [storage.rs](storage.rs) | 显式错误配置、凭据脱敏、SQLite 连接后无应用表、克隆共享与独立内存隔离、调用方自建数据跨文件重连、目录错误、PostgreSQL 失败不降级 |
| [postgres.rs](postgres.rs) | 真实连接后 public 无表;测试在事务中创建临时 class_rows、参数绑定和读取,提交自动删除临时表,再连接;默认 ignore,需全新专用 ASTER_TEST_POSTGRES_URL |
所有业务表都由测试自己创建,连接层不得自动初始化基础信息表。临时 SQLite 文件和 PostgreSQL 测试库由远端 CI 隔离,不使用生产数据。检查命令见 [验证](../../../vendor/aster/docs/operations/validation.md),原生连接能力边界见 [数据库对象](../../../docs/runtime/database-objects.md)。本地不运行测试或下载依赖。返回 [storage](../README.md)。
+53
View File
@@ -0,0 +1,53 @@
use storage::{DatabaseConfig, DatabaseConnection};
/// Run against a fresh disposable database, never a production or shared schema.
#[tokio::test]
#[ignore = "requires ASTER_TEST_POSTGRES_URL pointing to a fresh disposable PostgreSQL database"]
async fn postgres_connects_without_schema_and_executes_caller_owned_sql() {
let url = std::env::var("ASTER_TEST_POSTGRES_URL")
.expect("set ASTER_TEST_POSTGRES_URL to a fresh disposable PostgreSQL database");
let config = DatabaseConfig::Postgres(url);
let database = DatabaseConnection::connect(&config).await.unwrap();
assert_eq!(database.backend_name(), "postgresql");
assert!(database.sqlite_pool().is_none());
database.ping().await.unwrap();
let pool = database.postgres_pool().unwrap();
let tables: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public'",
)
.fetch_one(pool)
.await
.unwrap();
assert_eq!(tables, 0, "connecting must not create application tables");
let mut transaction = pool.begin().await.unwrap();
sqlx::query(
"CREATE TEMP TABLE class_rows (id INTEGER PRIMARY KEY, value TEXT NOT NULL) ON COMMIT DROP",
)
.execute(&mut *transaction)
.await
.unwrap();
sqlx::query("INSERT INTO class_rows (id, value) VALUES ($1, $2)")
.bind(1_i64)
.bind("你好 ' ; $1")
.execute(&mut *transaction)
.await
.unwrap();
let value: String = sqlx::query_scalar("SELECT value FROM class_rows WHERE id = $1")
.bind(1_i64)
.fetch_one(&mut *transaction)
.await
.unwrap();
assert_eq!(value, "你好 ' ; $1");
transaction.commit().await.unwrap();
database.close().await;
let reopened = DatabaseConnection::connect(&config).await.unwrap();
reopened.ping().await.unwrap();
let tables: i64 = sqlx::query_scalar(
"SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public'",
)
.fetch_one(reopened.postgres_pool().unwrap())
.await
.unwrap();
assert_eq!(tables, 0);
reopened.close().await;
}
+130
View File
@@ -0,0 +1,130 @@
use storage::{DatabaseConfig, DatabaseConnection};
#[test]
fn postgres_configuration_debug_redacts_credentials() {
let config = DatabaseConfig::Postgres("postgres://user:secret@localhost/db".into());
assert!(!format!("{config:?}").contains("secret"));
}
#[tokio::test]
async fn explicit_invalid_backends_fail_without_fallback() {
for config in [
DatabaseConfig::Postgres(String::new()),
DatabaseConfig::Postgres("mysql://localhost/db".into()),
DatabaseConfig::SqliteFile("".into()),
DatabaseConfig::SqliteFile(":memory:".into()),
] {
assert!(DatabaseConnection::connect(&config).await.is_err());
}
}
#[tokio::test]
async fn memory_connections_start_without_tables_and_clones_share_the_pool() {
let database = DatabaseConnection::connect(&DatabaseConfig::SqliteMemory)
.await
.unwrap();
let other = DatabaseConnection::connect(&DatabaseConfig::SqliteMemory)
.await
.unwrap();
assert!(database.postgres_pool().is_none());
let pool = database.sqlite_pool().unwrap();
let tables: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM sqlite_master WHERE type = 'table'")
.fetch_one(pool)
.await
.unwrap();
assert_eq!(tables, 0, "connecting must not create application tables");
// This schema belongs to the test's simulated database-backed object.
sqlx::query("CREATE TABLE class_rows (id INTEGER PRIMARY KEY, value TEXT NOT NULL)")
.execute(pool)
.await
.unwrap();
sqlx::query("INSERT INTO class_rows (id, value) VALUES (?, ?)")
.bind(1_i64)
.bind("你好 ' ? $1 ;")
.execute(pool)
.await
.unwrap();
let cloned = database.clone();
let value: String = sqlx::query_scalar("SELECT value FROM class_rows WHERE id = ?")
.bind(1_i64)
.fetch_one(cloned.sqlite_pool().unwrap())
.await
.unwrap();
assert_eq!(value, "你好 ' ? $1 ;");
let other_tables: i64 =
sqlx::query_scalar("SELECT COUNT(*) FROM sqlite_master WHERE type = 'table'")
.fetch_one(other.sqlite_pool().unwrap())
.await
.unwrap();
assert_eq!(other_tables, 0);
database.close().await;
assert!(cloned.ping().await.is_err());
other.close().await;
}
#[tokio::test]
async fn explicit_sqlite_file_preserves_only_caller_created_data_across_reconnects() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("nested/data/class.db");
let config = DatabaseConfig::SqliteFile(path.clone());
let database = DatabaseConnection::connect(&config).await.unwrap();
let pool = database.sqlite_pool().unwrap();
let tables: i64 = sqlx::query_scalar("SELECT COUNT(*) FROM sqlite_master WHERE type = 'table'")
.fetch_one(pool)
.await
.unwrap();
assert_eq!(tables, 0);
sqlx::query("CREATE TABLE class_rows (id INTEGER PRIMARY KEY, value TEXT NOT NULL)")
.execute(pool)
.await
.unwrap();
sqlx::query("INSERT INTO class_rows (id, value) VALUES (?, ?)")
.bind(1_i64)
.bind("caller-owned data")
.execute(pool)
.await
.unwrap();
database.close().await;
assert!(path.is_file());
let reopened = DatabaseConnection::connect(&config).await.unwrap();
let value: String = sqlx::query_scalar("SELECT value FROM class_rows WHERE id = ?")
.bind(1_i64)
.fetch_one(reopened.sqlite_pool().unwrap())
.await
.unwrap();
assert_eq!(value, "caller-owned data");
let tables: Vec<String> =
sqlx::query_scalar("SELECT name FROM sqlite_master WHERE type = 'table'")
.fetch_all(reopened.sqlite_pool().unwrap())
.await
.unwrap();
assert_eq!(tables, vec!["class_rows"]);
reopened.close().await;
}
#[tokio::test]
async fn invalid_sqlite_directory_reports_error() {
let dir = tempfile::tempdir().unwrap();
let file = dir.path().join("file");
std::fs::write(&file, "occupied").unwrap();
assert!(
DatabaseConnection::connect(&DatabaseConfig::SqliteFile(file.join("class.db")))
.await
.is_err()
);
}
#[tokio::test]
async fn postgres_connection_failure_is_an_error() {
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!(
"postgres://aster:secret@{}/aster?sslmode=disable",
listener.local_addr().unwrap()
);
let error = match DatabaseConnection::connect(&DatabaseConfig::Postgres(url)).await {
Ok(_) => panic!("unreachable PostgreSQL must fail"),
Err(error) => error,
};
assert!(!error.to_string().contains("secret"));
assert!(error.to_string().contains("no fallback"));
}
+41
View File
@@ -0,0 +1,41 @@
name: endorphin-aster-vm
services:
aster:
image: ${ASTER_IMAGE:?set ASTER_IMAGE to the verified release image}
init: true
ports:
- "127.0.0.1:${ASTER_PORT:-18081}:8080"
environment:
ASTER_LISTEN: 0.0.0.0:8080
ASTER_STATE_DIR: /data/host
volumes:
- /var/lib/endorphin-aster-vm/host:/data/host
restart: unless-stopped
stop_grace_period: 20s
logging:
driver: json-file
options:
max-size: 10m
max-file: "3"
postgres:
image: postgres:17-bookworm
environment:
POSTGRES_DB: aster
POSTGRES_USER: aster
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}
volumes:
- /var/lib/endorphin-aster-vm/postgres:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -h 127.0.0.1 -U aster -d aster"]
interval: 5s
timeout: 3s
retries: 20
restart: unless-stopped
stop_grace_period: 30s
logging:
driver: json-file
options:
max-size: 10m
max-file: "3"
+31
View File
@@ -0,0 +1,31 @@
# Standalone alternative: docker compose -f docker-compose.postgres.yml up --build -d
services:
aster:
build: .
ports:
- "127.0.0.1:${ASTER_PORT:-8080}:8080"
environment:
ASTER_LISTEN: 0.0.0.0:8080
ASTER_STATE_DIR: /data/host
volumes:
- aster-host:/data/host
restart: unless-stopped
postgres:
image: postgres:17-bookworm
environment:
POSTGRES_DB: aster
POSTGRES_USER: aster
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
volumes:
- aster-postgres:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U aster -d aster"]
interval: 5s
timeout: 3s
retries: 10
restart: unless-stopped
volumes:
aster-host:
aster-postgres:
+14
View File
@@ -0,0 +1,14 @@
services:
aster:
build: .
ports:
- "127.0.0.1:${ASTER_PORT:-8080}:8080"
environment:
ASTER_LISTEN: 0.0.0.0:8080
ASTER_STATE_DIR: /data/host
volumes:
- aster-sqlite:/data
restart: unless-stopped
volumes:
aster-sqlite:
+8
View File
@@ -0,0 +1,8 @@
# Hestia 文档
| 入口 | 内容 |
| --- | --- |
| [operations](operations/README.md) | 运行、配置、部署、验证和宿主信息 |
| [runtime](runtime/README.md) | 显式数据库适配边界 |
语言对象、字节码和编译说明以固定 Aster 子模块文档为准;命令行源码执行属于 Lyra。返回 [项目入口](../README.md)。
+11
View File
@@ -0,0 +1,11 @@
# Hestia 运维
| 文件 | 内容 |
| --- | --- |
| README.md | 本目录入口 |
| [configuration.md](configuration.md) | 兼容环境变量和显式配置 |
| [host-information.md](host-information.md) | JSON 基础信息、读写与只读旧库导入 |
| [deployment.md](deployment.md) | 手动部署及生产数据保护 |
| [validation.md](validation.md) | 远端 CI 范围 |
HTTP binary server 提供 /health 与只读 /bootstrap;admin 包的 cli 提供对象引导和文件信息管理。两者不使用数据库;没有语言执行 HTTP API。返回 [文档索引](../README.md)。
+73
View File
@@ -0,0 +1,73 @@
# 配置、UTF-8 与进程边界
## 当前选择
当前先明确少量稳定边界,不引入 PyPreConfig/PyConfig 式两阶段初始化,也不新增 VM 配置文件、环境变量前缀或配置框架。Vm::new 从空堆启动;监听与网络由宿主处理,数据库连接仅由真正消费它的对象显式请求。
| 配置层 | 当前入口 | 责任 |
| --- | --- | --- |
| 内核 | Vm::new、显式创建对象/函数;call_with_limits/execute_with_limits | 可预测的运行时语义,不读取环境变量、修改 locale、注册信号或加载 .env |
| 宿主 | Server 启动代码与 host::MetadataStore | 读取监听/文件信息目录;基础信息存 JSON 文件,不读取数据库配置 |
| 数据库对象适配 | DatabaseConnection::connect(&DatabaseConfig) | 显式选择后端后连接,不预设应用表;特殊类尚未接入 |
| 部署 | Compose 与 ces-control 脚本 | 端口映射、数据库密码、持久卷和镜像版本 |
| 构建/CI | Cargo.toml、Cargo.lock、Dockerfile、工作流 | 工具链、依赖锁定与验证环境;不作为语言运行语义配置 |
内核不直接依赖 HTTP、SQLx 或操作系统 locale 接口。这个隔离来自现有代码与分层,不是已经实现的独立安全沙箱。
## 为什么不照搬 CPython 的 locale 初始化
CPython 的 PyPreConfig 支持选择分配器、UTF-8 模式以及是否配置 LC_CTYPE,以处理它自己的启动和兼容性需求。Python 官方也提供忽略环境、保留 LC_CTYPE、不注册信号处理的隔离初始化。当前 Endor 通过 Rust 宿主接收已构造的值,没有需要复刻这套预初始化的依赖环。[CPython 初始化文档](https://docs.python.org/3.14/c-api/init_config.html)。
LC_CTYPE 处理字符分类、大小写和多字节字符相关约定;小数点和数字分组属于 LC_NUMERIC。只调整 LC_CTYPE 不能直接等同于修改数字小数点,这两类配置应区分。[locale 类别说明](https://docs.python.org/3.14/library/locale.html#locale.LC_CTYPE)。
云原生环境仍会遇到进程变量、终端、文件路径与外部工具的问题;较少跨语言互操作不代表可以省略输入边界。当前解决方式是让核心语义独立于进程区域设置,把必要的解码留在宿主入口。
## 当前文本和数字行为
- Value::Text 与 EndorStringObject 使用 Rust String,内容始终是有效 UTF-8。字符访问按 Unicode 标量值,不按字节或字形簇;不支持孤立代理码点。[Rust String](https://doc.rust-lang.org/std/string/struct.String.html#utf-8)。
- Number 的整数与十进制构造器明确检查 ASCII 数字和语法;十进制点为句点,不根据 LANG/LC_NUMERIC 切换为逗号。当前数值比较、哈希和算术尚未实现。
- 内核没有 setlocale 调用;不会为了启动 VM 修改 LANG、LC_ALL 或 LC_CTYPE。外部库、终端、字体或下游工具自身的行为不在这项保证之内。
- 源码 parse/run/eval 属于独立 Lyra CLI,使用 Aster EndorReader;Hestia 管理 cli 不执行源码。来源与解码失败契约见 [阅读器](../../vendor/aster/docs/runtime/reader.md)。Frontend 提供基础标识符与语法解析,见 [源码前端](../../vendor/aster/docs/runtime/frontend.md)。尚无通用字节串对象、语言文件系统 API 或非 UTF-8 源码自动转码入口。
日文与外部 Shift_JIS/CP932 的处理见 [文本编码](../../vendor/aster/docs/runtime/text-encoding.md);内部 UTF-8 不因地区切换。
## 已有进程与部署变量
| 名称 | 使用位置与默认值 | 错误和优先级 |
| --- | --- | --- |
| ASTER_STATE_DIR | Host 文件信息目录;默认 data/host,容器 /data/host | 使用 var_os 保留操作系统路径;显式空值报错,不影响数据库选择 |
| ASTER_LISTEN | Server;默认 127.0.0.1:8080 | 当前 var(...).unwrap_or_else:缺失或非 Unicode 都用默认值;有效 Unicode 的错误地址在绑定时失败 |
| ASTER_PORT | Compose 宿主端口映射 | 独立方案默认 8080,ces-control 默认 18081;不是 Vm 配置 |
| POSTGRES_PASSWORD | PostgreSQL Compose/生产环境文件 | Compose 必填;示例使用随机十六进制以避免 URL 转义歧义;不写入仓库 |
| ASTER_IMAGE | ces-control Compose | 必填已构建的版本镜像,通常由 release.env 提供 |
| ASTER_RELEASE_SHA | CI 与部署脚本 | 必须为 40 位小写十六进制提交 SHA;用于镜像和版本目录 |
| ASTER_TEST_POSTGRES_URL | 指定环境显式运行 PostgreSQL 集成测试 | 专用测试库 URL;不是生产数据库配置 |
文件信息命令及 Server 通过 host 包读取 ASTER_STATE_DIR,Server 还读取 ASTER_LISTEN;不会把 DATABASE_URL/SQLITE_PATH 作为基础信息存储配置。程序不自动加载 .env,Cargo 运行时须显式导出监听变量。Compose 会按其规则读取 .env;显式 --env-file 的生产文件由部署脚本管理。模板见 [.env.example](../../.env.example),执行链见 [部署](deployment.md)。
## 显式数据库配置
DatabaseConfig 没有默认值和 from_env,仅允许显式构造:Postgres(url)、SqliteFile(path)、SqliteMemory。没有默认 data/aster.db,也没有应用启动时的全局连接。connect 不执行应用 DDL,不保存基础信息;基础信息另由 [host 文件层](host-information.md) 保存;未来的特殊类负责消费连接和定义数据,见 [数据库对象](../runtime/database-objects.md)。
SQLite 文件显式连接时创建父目录与文件、启用 WAL,最多 5 个连接、busy timeout 5 秒;内存库单连接、不因池轮换销毁。PostgreSQL 最多 5 个连接、获取超时 5 秒,错误不降级。数据库 URL 的 Debug 和公开错误脱敏;底层错误 source 保留,不应不加处理地公开输出。
## 已有执行限额
ExecutionLimits 是每次调用显式传入的普通 Rust 值,不是进程全局开关;字段定义见 [frame.rs](../../vendor/aster/crates/core/src/frame.rs)。
| 字段 | 默认值 | 作用 |
| --- | --- | --- |
| max_frames | 1000 | 本次执行允许的活动语言帧数量 |
| max_instructions | usize::MAX | 本次执行共享的逻辑指令预算;默认不设实用的有限预算 |
call_with_limits/execute_with_limits 使用自定义值;嵌套语言调用共用计数。预算不能抢占正在执行的 Native 回调,也没有超时、内存上限或异步取消能力;不能据此视为安全执行不可信代码的完整方案。
## 保留的入口缺口与以后再做的内容
CLI 当前使用 std::env::args,遇到非 Unicode 命令行参数可能 panic,并没有统一转换成友好的配置错误。[Rust args 文档](https://doc.rust-lang.org/std/env/fn.args.html#panics)。Server 的非 Unicode ASTER_LISTEN 也尚未严格报错;宿主输入错误接口尚未统一。
以后增加宿主输入时,应明确使用 OsString/PathBuf 保留系统路径,或显式拒绝不符合文本协议的输入;不要通过修改整个进程 locale 让隐式解码暂时成功。未来源码文本默认 UTF-8、协议数字固定格式是设计方向,只有加载器实现后才能写成已实现行为。
现在不新增通用 VmConfig。出现构造期策略时再集中为实例级不可变配置;进程参数由宿主解析后传入,不让 Vm::new 自动读取一组隐含开关。时区、模块路径、凭据注入、内存配额和网络能力也按实际实现分别设计,不预设已存在。
返回 [运行指南](../../vendor/aster/docs/operations/README.md) 或 [文档索引](../../vendor/aster/docs/README.md)。
+22
View File
@@ -0,0 +1,22 @@
# Hestia Docker 部署
本仓库承接原 Aster HTTP 宿主部署定义,不代表远端已迁移或正在运行本版本。普通 push/PR 只验证;.gitea/workflows/deploy.yml 仅 main 手动触发,先完整验证 SQLite、真实 PostgreSQL 和宿主流程,再构建 runtime 镜像并调用 scripts/deploy-ces-control.sh。
Dockerfile 的 source 阶段复制宿主工作区、固定 Aster 子模块和脚本;ci 阶段提供检查环境;build 构建 server 和 admin,runtime 继续安装 server 和兼容 cli,UID 10001,/data,健康检查 /health。管理 cli 只负责 bootstrap 和 metadata,不包含 Lyra 的语言执行命令。
| 兼容标识 | 保留原因 |
| --- | --- |
| Compose 项目 endorphin-aster-vm、服务 aster/postgres | 复用既有服务,不创建替代生产实例 |
| 镜像 endorphin-aster-vm:<Hestia SHA> | 沿用部署脚本接口,SHA 现指宿主提交 |
| ASTER_RELEASE_SHA、ASTER_IMAGE、ASTER_PORT、ASTER_LISTEN、ASTER_STATE_DIR | 保持配置兼容 |
| /opt/endorphin-aster-vm | release.env、releases、current 及回退记录 |
| /etc/endorphin-aster-vm/production.env | 原有生产配置与数据库凭据 |
| /var/lib/endorphin-aster-vm/postgres | 原有 PostgreSQL 数据,禁止删除 |
| /var/lib/endorphin-aster-vm/host | 原有 JSON 文件信息与导入标记 |
| /var/lock/endorphin-aster-vm-deploy.lock | 串行切换生产版本 |
生产 Compose 继续读取 docker-compose.ces-control.yml。默认 docker-compose.yml 和独立 docker-compose.postgres.yml 保留原命名卷、环境与端口接口;不因拆仓改变卷或 PostgreSQL major 版本。
部署只接受已构建镜像。健康、对象引导和释放检查成功后,按需只读导出旧 public.aster_metadata,并由 cli metadata import 仅创建缺失 JSON;不覆盖新文件,不写旧表,不删除原库。最后写 deployment.release,原子切换 current。失败时保留数据并尝试恢复上个健康应用版本;应用回退不等于数据库回退。
迁移仓库和移除 Aster CD 均不会执行以上部署命令,不停止现有容器,不改共享 Runner。具体文件信息协议见 [host-information.md](host-information.md),验证边界见 [validation.md](validation.md)。返回 [运维](README.md)。
+47
View File
@@ -0,0 +1,47 @@
# 宿主基础信息:文件保存
基础信息不使用数据库。host 包保存逐键 UTF-8 JSON,CLI 提供读写,Server 启动写 host.version,部署检查通过后写 deployment.release。Storage 的数据库连接仅给未来特殊类使用。
## 文件与配置
ASTER_STATE_DIR 使用操作系统路径,默认 data/host;显式空值报错。读取一个不存在的键返回 None,不创建目录;写入时创建目录。容器为 /data/host;生产挂载到 /var/lib/endorphin-aster-vm/host,独立 PostgreSQL Compose 使用 aster-host 卷。
例如 host.version 对应 host.version.json:
```json
{
"format_version": 1,
"key": "host.version",
"value": "0.1.0"
}
```
控制键以 ASCII 字母或数字开头,最长 128 字节,其余只能是字母、数字、点、下划线或连字符。它是信息命名协议,不是文本编码限制;值支持日文、中文、emoji 等 Unicode 文本。用户文本不应被当作文件路径或控制键。
文件必须是有效 UTF-8 JSON,版本和 key 必须匹配请求。损坏记录返回错误,不清空、不用默认值偷偷替换。文件不属于 VM 堆快照,也不存数据库凭据。
## 读写与生命周期
CLI 的 metadata put/get 通过 host::MetadataStore 访问文件,不连接 SQL:
```sh
cli metadata put project.note '日本語・中文・😀'
cli metadata get project.note
cli metadata get host.version
```
MetadataStore::clone 只共享目录路径,没有缓存;每次读取重新打开文件。写入先建立同目录唯一临时文件,写完并 sync_all,再通过 rename 发布;Unix 新文件权限 0600。不同键分别写文件,不会因整张 map 的读改写丢失其他键;同键并发是最后一次成功发布生效。
没有跨键事务、分布式锁或共享网络文件系统一致性保证;文件同步与原子发布也不等于数据库的完整断电持久性协议。异常进程终止可能留下隐藏临时文件,正常失败会清理自己的临时文件。
## 旧基础信息的一次性迁移
ces-control 发布时,在 host/.legacy-imported 标记缺失的情况下,只读检查 PostgreSQL 原 public.aster_metadata 表。存在时,使用 UTF-8 客户端导出 key/value JSON,再经 cli metadata import 的标准输入导入。该命令只创建缺失文件,不覆盖 Server 刚写入的版本或其他已有文件;临时导出文件以私有权限创建并清理,不打印数据内容。
只导入缺失项通过同目录临时文件的原子 hard_link 实现,因此运行目录应使用支持该操作的本地文件系统。导入可重复执行;只有导入成功或源表不存在才写完成标记。损坏记录、不支持的控制键或 I/O 错误会停止发布,保留原库数据,不能当作迁移成功。
本迁移针对已配置的 PostgreSQL 生产路径,不自动扫描或删除其他 SQLite 文件。原数据库表与数据不会被修改/删除。此处说明发布脚本行为,不代表远端已执行成功;按约定不主动跟踪 CI 或服务器。
部署记录仍以版本镜像、release.env 与 current 链接为发布来源;deployment.release.json 是文件信息记录,不是数据库迁移表。脚本见 [scripts](../../scripts/README.md)。
返回 [运行指南](../../vendor/aster/docs/operations/README.md) 或 [文档索引](../../vendor/aster/docs/README.md)。
+9
View File
@@ -0,0 +1,9 @@
# Hestia 验证
仅在 ces-control 执行 Docker、构建、测试、Clippy 和冒烟;本地不执行,不主动查询结果。
ci.yml 在 main push、PR 和手动触发时运行 scripts/ci-ces-control.sh。检查容器执行格式、cargo test --workspace --locked、Clippy,再显式运行 storage 的真实 PostgreSQL ignored 测试和 smoke.py。测试 PostgreSQL 使用独立临时容器、私有网络和 tmpfs,不挂载生产数据;SQLite 和 PostgreSQL 都验证不自动建应用表。
smoke.py 验证管理 cli 的对象释放、JSON 跨进程读写/缺失导入、Server 的健康/只读对象信息和 SIGTERM 退出,并检查失效数据库变量不导致隐式连接或创建 SQLite。Endor 五段样例的真实 run/eval 报告属于 Lyra CI,不由 Hestia 执行。
生产部署位于单独手动工作流,必须重复通过以上验证。SSH 凭据要求见 ../../.gitea/README.md。工作流定义不代表实际通过,未查询结果如实注明。返回 [运维](README.md)。
+3
View File
@@ -0,0 +1,3 @@
# 宿主运行时适配
[database-objects.md](database-objects.md) 描述为未来特殊类提供的显式数据库适配;没有默认 VM 数据库或自动应用表。引擎本身不依赖此适配。返回 [文档索引](../README.md)。
+36
View File
@@ -0,0 +1,36 @@
# 数据库特殊类与连接边界
数据库连接用于某类明确需要数据库的特殊对象,不用于 VM 版本、部署版本、普通对象信息或统一 key/value 元数据。当前尚未定义该特殊类的名字、载荷布局或语言接口,本页说明已保留的连接能力及接入边界。
## 当前分层
- Objects 负责核心对象与私有堆,Core 负责 VM 和执行;两者不依赖 SQLx 或数据库。
- Storage 仅暴露 DatabaseConfig、DatabaseConnection 和 DatabaseError。
- CLI/Server 不依赖 Storage,启动、bootstrap 和 health 不访问数据库;基础信息由独立 host 文件层处理。
- 特殊类以后由宿主实现按需取得数据库连接;尚未自动挂入 Vm 或对象工厂。
## 显式建立连接
调用方先构造 DatabaseConfig::Postgres(url)、SqliteFile(path) 或 SqliteMemory,再 await DatabaseConnection::connect。没有 from_env、默认 SQLite 路径或应用启动时的全局数据库。
connect 会建立真实连接,并可在显式请求 SQLite 文件时创建父目录和文件,但不创建任何应用表。SQLite WAL、连接协商等驱动自身操作不等于应用 schema 初始化。不会创建基础信息表,也不会写入运行或发布信息。
sqlite_pool/postgres_pool 返回匹配后端的 SQLx 池借用,另一后端返回 None。特殊类以后可在自己的适配实现里执行参数化查询、事务及由该类定义的数据结构;本层不预设通用 KV、ORM、迁移或序列化协议。现有 SQLx API 见 [Pool 文档](https://docs.rs/sqlx/0.8.6/sqlx/struct.Pool.html)。
## 生命周期
DatabaseConnection::clone 共享连接池;close().await 显式关闭所有克隆使用的池,并等待已借出连接归还。取出的 SQLx 池还能被调用方克隆,因此单个包装值销毁不保证所有使用者立即结束。
未来将连接放进原生载荷时,必须另行定义异步关闭、共享所有权和错误传播;当前 VM 的同步 Drop/Native 回调不等于已经完成异步数据库类。连接跨进程不能复用,也不代表 VM 快照。
## 既有数据和部署
原先的基础信息存储已从代码中移除。已有数据库表、卷、密码和服务器数据未执行删除或迁移;它们是否存在、有哪些记录不由本次代码修改推断。
生产继续保留预置 PostgreSQL 服务与既有数据目录,等待实际类的消费接口。应用容器不注入数据库连接变量,当前不消费该服务。部署版本通过镜像标签、release.env 和 current 链接识别,基础信息写入文件;发布时仅为旧数据做一次只读导入,不继续写数据库。详见 [宿主信息](../operations/host-information.md)。
## 验证
SQLite 检查先确认新连接没有应用表,再由测试自己创建 class_rows 验证查询和重连。真实 PostgreSQL 使用全新临时测试库,检查连接不建表,并在事务中创建提交后自动删除的临时表。这些是适配层测试,不代表特殊类已实现。
文件定位见 [storage](../../crates/storage/README.md),运行与生产资源见 [部署](../operations/deployment.md)。返回 [运行时索引](../../vendor/aster/docs/runtime/README.md)。
+11
View File
@@ -0,0 +1,11 @@
# Hestia 脚本
| 文件 | 职责 |
| --- | --- |
| check.sh | 远端格式、工作区测试和 Clippy |
| ci-ces-control.sh | 专用容器/临时 PostgreSQL,验证 SQLite、真实 PostgreSQL 和宿主行为 |
| smoke.py | 实际 server 与管理 cli 的 HTTP、SIGTERM、JSON 持久化和无隐式数据库访问检查 |
| deploy-ces-control.sh | 已验证镜像部署、只读旧信息导入、健康检查和失败回退 |
| README.md | 本目录入口 |
CI 使用 Hestia 专用容器/缓存名称。生产脚本保留旧 endorphin-aster-vm 资源标识,不能删除数据库或卷。只在指定远端环境执行。返回 [项目入口](../README.md)。
+6
View File
@@ -0,0 +1,6 @@
#!/bin/sh
# Run only in the designated remote validation environment.
set -eu
cargo fmt --all --check
cargo test --workspace --locked
cargo clippy --workspace --all-targets --locked -- -D warnings
+58
View File
@@ -0,0 +1,58 @@
#!/bin/sh
# Run only on the designated remote CI host, never on the developer's machine.
set -eu
cd "$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)"
release_sha=${ASTER_RELEASE_SHA:?ASTER_RELEASE_SHA is required}
case "$release_sha" in *[!0-9a-f]*|'') echo "Invalid release SHA" >&2; exit 1;; esac
test "${#release_sha}" -eq 40
ci_id="${GITHUB_RUN_ID:-manual}-$$"
network="endorphin-hestia-server-ci-$ci_id"
postgres="endorphin-hestia-server-ci-pg-$ci_id"
checks="endorphin-hestia-server-ci-checks-$ci_id"
image="endorphin-hestia-server-ci:$release_sha"
cleanup() {
docker rm -f "$checks" "$postgres" >/dev/null 2>&1 || true
docker network rm "$network" >/dev/null 2>&1 || true
}
trap cleanup EXIT
trap 'exit 130' INT
trap 'exit 143' TERM
docker build --target ci -t "$image" .
docker network create "$network" >/dev/null
# Disposable test database: no production mounts and no published ports.
docker run -d --name "$postgres" --network "$network" --network-alias postgres \
--env POSTGRES_USER=aster --env POSTGRES_DB=aster \
--env POSTGRES_PASSWORD=aster-ci-only \
--tmpfs /var/lib/postgresql/data postgres:17-bookworm >/dev/null
ready=false
for attempt in $(seq 1 60); do
if docker exec "$postgres" pg_isready -h 127.0.0.1 -U aster -d aster >/dev/null 2>&1; then
ready=true
break
fi
sleep 1
done
if [ "$ready" != true ]; then
echo "CI PostgreSQL did not become ready" >&2
exit 1
fi
check_status=0
docker run --name "$checks" --init --network "$network" \
--mount type=volume,src=endorphin-hestia-server-ci-cargo,dst=/usr/local/cargo/registry \
--mount type=volume,src=endorphin-hestia-server-ci-target,dst=/build/target \
--env ASTER_TEST_POSTGRES_URL='postgres://aster:aster-ci-only@postgres:5432/aster?sslmode=disable' \
--env ASTER_RELEASE_SHA="$release_sha" \
"$image" sh -eu -c '
checks=0
sh scripts/check.sh || checks=$?
test "$checks" -eq 0
cargo test -p storage --test postgres --locked -- --ignored
python3 scripts/smoke.py
' || check_status=$?
exit "$check_status"
+119
View File
@@ -0,0 +1,119 @@
#!/bin/sh
# Deploy only the image produced after the remote database checks succeed.
set -eu
cd "$(CDPATH= cd -- "$(dirname -- "$0")/.." && pwd)"
test "$(id -u)" -eq 0 || { echo "Deployment requires root on ces-control" >&2; exit 1; }
release_sha=${ASTER_RELEASE_SHA:?ASTER_RELEASE_SHA is required}
case "$release_sha" in *[!0-9a-f]*|'') echo "Invalid release SHA" >&2; exit 1;; esac
test "${#release_sha}" -eq 40
image="endorphin-aster-vm:$release_sha"
docker image inspect "$image" >/dev/null
umask 077
root=/opt/endorphin-aster-vm
config=/etc/endorphin-aster-vm/production.env
release="$root/releases/$release_sha"
mkdir -p "$release" /etc/endorphin-aster-vm /var/lib/endorphin-aster-vm/postgres /var/lib/endorphin-aster-vm/host
chown 10001:10001 /var/lib/endorphin-aster-vm/host
chmod 700 /var/lib/endorphin-aster-vm/host
exec 9>/var/lock/endorphin-aster-vm-deploy.lock
flock 9
if [ ! -f "$config" ]; then
password=$(openssl rand -hex 24)
printf 'POSTGRES_PASSWORD=%s\nASTER_PORT=18081\n' "$password" > "$config"
unset password
fi
chmod 600 "$config"
cp docker-compose.ces-control.yml "$release/compose.yml"
printf 'ASTER_IMAGE=%s\n' "$image" > "$release/release.env"
previous=$(readlink -f "$root/current" 2>/dev/null || true)
legacy_export=
cleanup_export() {
if [ -n "$legacy_export" ]; then rm -f "$legacy_export"; fi
}
trap cleanup_export EXIT
compose_at() {
location=$1
shift
docker compose -p endorphin-aster-vm --env-file "$config" \
--env-file "$location/release.env" -f "$location/compose.yml" "$@"
}
rollback() {
echo "Deployment failed; application data has been retained" >&2
if [ -n "$previous" ] && [ "$previous" != "$release" ] && [ -f "$previous/compose.yml" ]; then
echo "Restoring the previously healthy release" >&2
compose_at "$previous" up -d --no-build --wait --wait-timeout 180 || true
fi
}
if ! compose_at "$release" up -d --no-build --wait --wait-timeout 180; then
rollback
exit 1
fi
if ! health=$(compose_at "$release" exec -T aster curl --fail --silent http://127.0.0.1:8080/health); then
rollback
exit 1
fi
printf '%s' "$health" | python3 -c 'import json,sys; d=json.load(sys.stdin); assert d == {"status":"ok"}' || {
rollback
exit 1
}
if ! bootstrap=$(compose_at "$release" exec -T aster cli bootstrap); then
rollback
exit 1
fi
printf '%s' "$bootstrap" | python3 -c 'import sys; s=sys.stdin.read(); assert "runtime=EndorObject type_system=disabled" in s; assert "released heap_objects=0" in s' || {
rollback
exit 1
}
# Import legacy basic information once, without changing database records.
# Files are authoritative; cli import only creates records that are missing.
if [ ! -f /var/lib/endorphin-aster-vm/host/.legacy-imported ]; then
if ! has_legacy=$(compose_at "$release" exec -T postgres env PGCLIENTENCODING=UTF8 psql -X -U aster -d aster -A -t -v ON_ERROR_STOP=1 \
-c "SELECT to_regclass('public.aster_metadata') IS NOT NULL"); then
rollback
exit 1
fi
if [ "$has_legacy" = t ]; then
legacy_export=$(mktemp)
if ! compose_at "$release" exec -T postgres env PGCLIENTENCODING=UTF8 psql -X -U aster -d aster -A -t -v ON_ERROR_STOP=1 \
-c "SELECT COALESCE(json_agg(json_build_object('key', key, 'value', value)), '[]'::json) FROM public.aster_metadata" > "$legacy_export"; then
rollback
exit 1
fi
if ! compose_at "$release" exec -T aster cli metadata import < "$legacy_export"; then
rollback
exit 1
fi
rm -f "$legacy_export"
legacy_export=
elif [ "$has_legacy" != f ]; then
echo "Cannot determine legacy information schema" >&2
rollback
exit 1
fi
if ! touch /var/lib/endorphin-aster-vm/host/.legacy-imported; then
rollback
exit 1
fi
fi
# File-backed host information; no SQL table or database write.
if ! compose_at "$release" exec -T aster cli metadata put deployment.release "$release_sha"; then
rollback
exit 1
fi
if ! actual=$(compose_at "$release" exec -T aster cli metadata get deployment.release); then
rollback
exit 1
fi
if [ "$actual" != "$release_sha" ]; then
rollback
exit 1
fi
ln -sfn "$release" "$root/current.next"
mv -Tf "$root/current.next" "$root/current"
compose_at "$release" ps
echo "Deployed release=$release_sha postgres=provisioned endpoint=127.0.0.1:18081"
echo "Persistent PostgreSQL data: /var/lib/endorphin-aster-vm/postgres"
+88
View File
@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""Remote CI end-to-end smoke test, no external database required."""
import json
import os
from pathlib import Path
import signal
import socket
import subprocess
import tempfile
import time
import urllib.request
ROOT = Path(__file__).resolve().parents[1]
def run(*args, env):
return subprocess.run(args, cwd=ROOT, env=env, check=True, text=True, capture_output=True).stdout.strip()
def main():
env = os.environ.copy()
run("cargo", "build", "--workspace", "--locked", env=env)
# Cargo may use a configured CARGO_TARGET_DIR; discover the actual binaries.
metadata = json.loads(run("cargo", "metadata", "--no-deps", "--format-version", "1", env=env))
binary_dir = Path(metadata["target_directory"]) / "debug"
cli = str(binary_dir / "cli")
server = str(binary_dir / "server")
with tempfile.TemporaryDirectory(prefix="aster-smoke-") as directory:
env["ASTER_STATE_DIR"] = str(Path(directory) / "host-info")
env["SQLITE_PATH"] = str(Path(directory) / "nested" / "aster.db")
# Neither executable may use database settings for basic VM information.
env["DATABASE_URL"] = "invalid-database-url"
bootstrap = run(cli, "bootstrap", env=env)
assert "runtime=EndorObject type_system=disabled" in bootstrap, bootstrap
assert "objects_distinct=true heap_objects=2" in bootstrap, bootstrap
assert "released heap_objects=0" in bootstrap, bootstrap
assert "stored (file)" in run(cli, "metadata", "put", "project.note", "日本語・中文・😀", env=env)
assert run(cli, "metadata", "get", "project.note", env=env) == "日本語・中文・😀"
legacy = json.dumps([{"key": "project.note", "value": "old"},
{"key": "legacy.note", "value": "preserved"}])
subprocess.run([cli, "metadata", "import"], input=legacy, cwd=ROOT, env=env,
check=True, text=True, capture_output=True)
assert run(cli, "metadata", "get", "project.note", env=env) == "日本語・中文・😀"
assert run(cli, "metadata", "get", "legacy.note", env=env) == "preserved"
assert not Path(env["SQLITE_PATH"]).exists(), "CLI must not open a database"
with socket.socket() as sock:
sock.bind(("127.0.0.1", 0))
port = sock.getsockname()[1]
env["ASTER_LISTEN"] = f"127.0.0.1:{port}"
with tempfile.TemporaryFile(mode="w+") as log:
process = subprocess.Popen([server], cwd=ROOT, env=env, stdout=log, stderr=log)
try:
deadline = time.monotonic() + 15
while True:
try:
with urllib.request.urlopen(f"http://127.0.0.1:{port}/health", timeout=1) as response:
health = json.load(response)
break
except (OSError, ValueError):
if process.poll() is not None or time.monotonic() >= deadline:
log.seek(0)
raise AssertionError(f"server did not become healthy: {log.read()}")
time.sleep(0.1)
assert health == {"status": "ok"}, health
for _ in range(2):
with urllib.request.urlopen(f"http://127.0.0.1:{port}/bootstrap", timeout=2) as response:
info = json.load(response)
assert info == {"object_model": "EndorObject", "type_system": False, "heap_objects": 0}, info
finally:
process.send_signal(signal.SIGTERM)
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
raise AssertionError("server did not shut down gracefully")
assert process.returncode == 0, process.returncode
assert not Path(env["SQLITE_PATH"]).exists(), "Server must not open a database"
assert run(cli, "metadata", "get", "host.version", env=env) == "0.1.0"
record = json.loads((Path(env["ASTER_STATE_DIR"]) / "host.version.json").read_text(encoding="utf-8"))
assert record == {"format_version": 1, "key": "host.version", "value": "0.1.0"}
assert run(cli, "metadata", "get", "project.note", env=env) == "日本語・中文・😀"
print("PASS: no implicit database access; file metadata persistence; HTTP health/bootstrap; SIGTERM shutdown")
if __name__ == "__main__":
main()
+11
View File
@@ -0,0 +1,11 @@
# Aster 引擎依赖
[aster](aster/README.md) 是独立 Git 子模块;父仓库 gitlink 固定具体提交,.gitmodules 保存 SSH URL。使用专用 codex-bot 的隔离 SSH 命令初始化:
```sh
git submodule update --init --recursive
```
根 Cargo.toml 通过子模块路径链接库;子模块保留自己的 Cargo workspace。父仓库 Cargo.lock 负责宿主依赖图。构建宿主会自动编译所需引擎库,无需启动 Aster 服务。更新时先取得明确的已发布 Aster commit,再 checkout --detach 并提交父仓库 gitlink;不要提交指向未发布引擎提交的宿主版本。
CI 获取私有子模块的 bot SSH secrets 要求见 [.gitea](../.gitea/README.md)。Aster 源码和本目录 README 是本目录直接子项。返回 [项目入口](../README.md)。
Vendored Submodule
+1
Submodule vendor/aster added at 55475a9d4b