Compare commits
10 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 08aa0d69b6 | |||
| 81c2d2e6df | |||
| d80854deae | |||
| 9e5d805f5a | |||
| 4d5e6e2512 | |||
| cc8ddba7aa | |||
| c7ec5f7eb2 | |||
| 33719a549b | |||
| 17cec9ff06 | |||
| 4e5832c14b |
@@ -0,0 +1,39 @@
|
||||
name: Deploy Foundry Core
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'cmd/**'
|
||||
- 'internal/**'
|
||||
- 'go.mod'
|
||||
- 'go.sum'
|
||||
- 'deploy/**'
|
||||
- '.gitea/workflows/deploy.yml'
|
||||
|
||||
concurrency:
|
||||
group: foundry-core-main-deploy
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
if: gitea.ref == 'refs/heads/main' && gitea.event_name == 'push'
|
||||
runs-on: ces-control
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-go@v5
|
||||
with:
|
||||
go-version-file: go.mod
|
||||
cache: false
|
||||
- name: Verify and build
|
||||
run: |
|
||||
set -eu
|
||||
test -z "$(gofmt -l cmd internal)"
|
||||
go test -mod=readonly ./...
|
||||
CGO_ENABLED=0 go build -mod=readonly -o bin/foundry ./cmd/foundry
|
||||
- name: Deploy and verify backend readiness
|
||||
env:
|
||||
GOOGLE_CLIENT_ID: ${{ secrets.GOOGLE_CLIENT_ID }}
|
||||
GOOGLE_CLIENT_SECRET: ${{ secrets.GOOGLE_CLIENT_SECRET }}
|
||||
FOUNDRY_PROJECTS: ${{ vars.FOUNDRY_PROJECTS }}
|
||||
run: bash deploy/release.sh bin/foundry
|
||||
@@ -7,9 +7,15 @@
|
||||
1. 在 Google Cloud 创建一个 Web application OAuth 客户端。授权重定向 URI 填写 `https://<API 域名>/auth/google/callback`。所有项目信任同一个客户端,因此共享 Google 同意页面的应用名称。
|
||||
2. 在服务器私网准备 PostgreSQL 13 或以上版本。在认证库执行一次 `db/001-auth.sql`(使用 `psql -v ON_ERROR_STOP=1`),给运行账号授予这四张表所需的查询及增删改权限。
|
||||
3. 按 `.env.example` 注入运行环境变量。每个项目配置一个固定的前端回调地址,本地开发可用 localhost。将 Google 客户端密钥和数据库连接串存入服务器的密钥管理设施。
|
||||
4. 由 Gitea Actions 在 `ces-control` 上使用 Go 1.25 构建并检查。部署时运行 `foundry` 二进制,通过 HTTPS 反向代理转发至服务监听地址。通过 `GET /healthz` 检查进程,通过 `GET /readyz` 检查认证库。
|
||||
4. 由 Gitea Actions 在 `ces-control` 上使用 Go 1.25 构建并检查。部署工作流仅处理 `main` 上影响后端或部署的改动,将二进制启动在服务器 Docker 私网,由 Traefik 提供 HTTPS。通过 `GET /healthz` 检查进程,通过 `GET /readyz` 检查认证库。
|
||||
5. 代理避免记录 Google 回调的查询参数及认证请求头。设置入口限流,并定时删除三张临时表(`oauth_attempts`、`login_grants`、`project_sessions`)中 `expires_at <= now()` 的记录。
|
||||
|
||||
## Actions 部署配置
|
||||
|
||||
在仓库的 Actions Secrets 配置 `GOOGLE_CLIENT_ID` 和 `GOOGLE_CLIENT_SECRET`,在 Actions Variables 配置 `FOUNDRY_PROJECTS`。变量值为 JSON 对象,例如 `{"prompt":"http://localhost:17321/auth/callback"}`。部署读取服务器上预先配置的 `/opt/endoleap-infra/foundry/database.env`,OAuth 值由 Actions 在启动容器时注入。
|
||||
|
||||
Google OAuth 的授权回调地址为 `https://api.foundry.entroia.com/auth/google/callback`。工作流会验证数据库就绪、匿名请求返回 401 和 HTTPS 就绪。前端接入后,可在浏览器用 Google 测试账号验证完整登录、回调、兑换和会话。
|
||||
|
||||
## 前端接入
|
||||
|
||||
1. 在浏览器用 Web Crypto 生成 32 字节随机值,编码为无填充 Base64URL,作为 verifier 暂存在当前 tab 的 sessionStorage。对 verifier 字符串做 SHA-256 并用相同编码得到 challenge。
|
||||
@@ -22,3 +28,36 @@
|
||||
## 验收
|
||||
|
||||
用同一个 Google 测试账号分别登录两个项目,确认用户 ID 不同。用 A 的会话请求 B 的 `me` 应返回 401。验证兑换码重放被拒绝,并确认退出 A 后 B 的会话仍有效。
|
||||
|
||||
## Codex execution
|
||||
|
||||
When enabled, `POST /projects/{project}/codex/exec` accepts a JSON object
|
||||
containing only a nonempty `prompt` string and returns `{"result":"…"}`.
|
||||
Use `Authorization: Bearer <token>` with a dedicated 32-byte random token
|
||||
encoded as 64 hex characters. Ordinary OAuth sessions cannot execute Codex.
|
||||
Missing, unknown and revoked tokens return 401. A valid token for a different
|
||||
project returns 403. Requests are limited to 32 KiB, CLI output to 512 KiB,
|
||||
and each execution to 90 seconds. The worker accepts one execution at a time
|
||||
and returns 429 when busy.
|
||||
|
||||
On the deployment host, provision `/opt/endoleap-infra/foundry/codex` with:
|
||||
|
||||
- `grants/projects.json` containing a JSON object mapping lowercase hex SHA-256
|
||||
digests of the **decoded** token bytes to configured project names. Use mode
|
||||
0700 for the directory and 0600 for the file, readable by UID 1000. Rotate or
|
||||
revoke tokens by atomically replacing the file within the mounted directory.
|
||||
A revocation applies to subsequent requests.
|
||||
- `home` owned by UID 1000 with an isolated Codex login in `auth.json`.
|
||||
Keep this home dedicated to the worker. Supply the login only and keep user
|
||||
configuration, MCP servers, skills, and other workload files out of it.
|
||||
|
||||
With that directory configured, the next normal Foundry deployment starts the
|
||||
worker using the Linux binary built by ces-control and mounts the worker socket
|
||||
and grants directory read-only. To replace an existing worker, stop and remove
|
||||
its container, then run `bash deploy/codex-worker.sh /absolute/path/to/foundry`
|
||||
with the desired ces-control build. The worker
|
||||
runs the installed Codex CLI with fixed read-only, no-approval arguments and
|
||||
disables command execution tools. It receives prompts via stdin and returns
|
||||
the final agent message. The worker has no published port and Foundry receives
|
||||
neither the Codex home nor the Docker socket. Keep both containers on the same
|
||||
host for Unix socket access.
|
||||
|
||||
+11
-2
@@ -10,6 +10,7 @@ import (
|
||||
"time"
|
||||
|
||||
"gitea.entroia.com/Zayden-Jung/foundry-core/internal/auth"
|
||||
"gitea.entroia.com/Zayden-Jung/foundry-core/internal/codex"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/stdlib"
|
||||
)
|
||||
@@ -22,6 +23,9 @@ func env(key, fallback string) string {
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) == 2 && os.Args[1] == "codex-worker" {
|
||||
log.Fatal(codex.ServeWorker("/run/foundry-codex/worker.sock"))
|
||||
}
|
||||
origin, err := url.Parse(os.Getenv("PUBLIC_ORIGIN"))
|
||||
if err != nil || origin.Scheme != "https" || origin.Hostname() == "" || origin.User != nil ||
|
||||
(origin.Path != "" && origin.Path != "/") || origin.RawQuery != "" || origin.Fragment != "" {
|
||||
@@ -45,10 +49,15 @@ func main() {
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
mux := http.NewServeMux()
|
||||
if err := codex.Register(mux, callbacks, os.Getenv("FOUNDRY_CODEX_SOCKET"), os.Getenv("FOUNDRY_CODEX_GRANTS")); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
mux.Handle("/", handler)
|
||||
server := &http.Server{
|
||||
Addr: net.JoinHostPort(env("HOST", "127.0.0.1"), env("PORT", "8080")),
|
||||
Handler: handler, ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second,
|
||||
WriteTimeout: 30 * time.Second, IdleTimeout: 60 * time.Second,
|
||||
Handler: mux, ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second,
|
||||
WriteTimeout: 110 * time.Second, IdleTimeout: 60 * time.Second,
|
||||
}
|
||||
log.Fatal(server.ListenAndServe())
|
||||
}
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
base=/opt/endoleap-infra
|
||||
dir=$base/foundry/codex
|
||||
binary=$(realpath "$1")
|
||||
test -x "$binary"
|
||||
test -s "$dir/home/auth.json"
|
||||
test -s "$dir/grants/projects.json"
|
||||
install -d -m 700 "$dir/run"
|
||||
chown 1000:1000 "$dir/run"
|
||||
|
||||
# The host operator supplies an isolated Codex home and starts a single worker.
|
||||
# The Docker default bridge supplies egress independently of Foundry's network.
|
||||
docker run -d --name foundry-codex-worker --restart unless-stopped --init \
|
||||
--network bridge --user 1000:1000 --cap-drop ALL --security-opt no-new-privileges \
|
||||
--read-only --pids-limit 128 --memory 384m --tmpfs /tmp:rw,nosuid,nodev \
|
||||
--tmpfs /work:ro,nosuid,nodev --workdir /work \
|
||||
--mount "type=bind,source=$binary,target=/opt/foundry,readonly" \
|
||||
--mount "type=bind,source=$base/codex-cli,target=/opt/codex,readonly" \
|
||||
--mount "type=bind,source=$dir/home,target=/home/node/.codex" \
|
||||
--mount "type=bind,source=$dir/run,target=/run/foundry-codex" \
|
||||
--mount "type=bind,source=/etc/ssl/certs/ca-certificates.crt,target=/etc/ssl/certs/ca-certificates.crt,readonly" \
|
||||
node:24-bookworm-slim /opt/foundry codex-worker
|
||||
@@ -0,0 +1,113 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
umask 077
|
||||
|
||||
base=/opt/endoleap-infra
|
||||
foundry=$base/foundry
|
||||
origin=https://api.foundry.entroia.com
|
||||
candidate=foundry-core-candidate
|
||||
network=endoleap-dev_default
|
||||
|
||||
for name in GOOGLE_CLIENT_ID GOOGLE_CLIENT_SECRET; do
|
||||
value=${!name-}
|
||||
if [[ -z "$value" || "$value" == *$'\n'* || "$value" == *$'\r'* ]]; then
|
||||
printf 'Missing or invalid %s\n' "$name" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
# Accept formatted JSON from Actions Variables and never print its contents.
|
||||
FOUNDRY_PROJECTS=$(python3 - <<'PYCONFIG'
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
raw = os.environ.get("FOUNDRY_PROJECTS", "")
|
||||
if not raw.strip():
|
||||
sys.exit("FOUNDRY_PROJECTS was not injected from Actions Variables")
|
||||
try:
|
||||
projects = json.loads(raw)
|
||||
except ValueError:
|
||||
sys.exit("FOUNDRY_PROJECTS must be valid JSON")
|
||||
if not isinstance(projects, dict) or not projects or any(
|
||||
not isinstance(callback, str) or not callback.strip()
|
||||
for callback in projects.values()
|
||||
):
|
||||
sys.exit("FOUNDRY_PROJECTS must map project names to callback URLs")
|
||||
print(json.dumps(projects, separators=(",", ":")))
|
||||
PYCONFIG
|
||||
)
|
||||
export FOUNDRY_PROJECTS
|
||||
|
||||
test -s "$foundry/database.env"
|
||||
docker network inspect "$network" >/dev/null
|
||||
|
||||
release=$(mktemp -d "$foundry/release.XXXXXXXX")
|
||||
install -m 755 "$1" "$release/foundry"
|
||||
export HOST=0.0.0.0 PORT=8080 PUBLIC_ORIGIN="$origin"
|
||||
codex_mounts=()
|
||||
codex_dir=$foundry/codex
|
||||
if [[ -d "$codex_dir" ]]; then
|
||||
if ! docker inspect -f '{{.State.Running}}' foundry-codex-worker 2>/dev/null | grep -qx true; then
|
||||
bash "$(dirname "$0")/codex-worker.sh" "$release/foundry" >/dev/null
|
||||
fi
|
||||
for attempt in {1..20}; do
|
||||
[[ ! -S "$codex_dir/run/worker.sock" ]] || break
|
||||
sleep 0.25
|
||||
done
|
||||
test -S "$codex_dir/run/worker.sock"
|
||||
test -s "$codex_dir/grants/projects.json"
|
||||
codex_mounts=(
|
||||
--mount "type=bind,source=$codex_dir/run,target=/run/foundry-codex,readonly"
|
||||
--mount "type=bind,source=$codex_dir/grants,target=/etc/foundry-codex,readonly"
|
||||
-e FOUNDRY_CODEX_SOCKET=/run/foundry-codex/worker.sock
|
||||
-e FOUNDRY_CODEX_GRANTS=/etc/foundry-codex/projects.json
|
||||
)
|
||||
fi
|
||||
|
||||
docker run -d --name "$candidate" "${codex_mounts[@]}" --network "$network" \
|
||||
--env-file "$foundry/database.env" -e HOST -e PORT -e PUBLIC_ORIGIN \
|
||||
-e GOOGLE_CLIENT_ID -e GOOGLE_CLIENT_SECRET -e FOUNDRY_PROJECTS \
|
||||
--user 1000:1000 --cap-drop ALL --security-opt no-new-privileges \
|
||||
--read-only --memory 128m \
|
||||
--mount "type=bind,source=$release/foundry,target=/opt/foundry,readonly" \
|
||||
node:24-bookworm-slim /opt/foundry >/dev/null
|
||||
|
||||
cleanup_candidate() {
|
||||
docker rm -f "${live:-$candidate}" >/dev/null 2>&1 || true
|
||||
}
|
||||
trap cleanup_candidate EXIT
|
||||
|
||||
docker exec "$candidate" node -e '
|
||||
const assert = require("node:assert/strict");
|
||||
async function verify() {
|
||||
const base = "http://127.0.0.1:8080";
|
||||
for (let attempt = 0; ; attempt++) {
|
||||
try {
|
||||
const ready = await fetch(base + "/readyz", {signal: AbortSignal.timeout(3000)});
|
||||
if (ready.ok) break;
|
||||
} catch (error) {
|
||||
if (attempt === 9) throw error;
|
||||
}
|
||||
assert.ok(attempt < 9);
|
||||
await new Promise(resolve => setTimeout(resolve, 1000));
|
||||
}
|
||||
const project = Object.keys(JSON.parse(process.env.FOUNDRY_PROJECTS))[0];
|
||||
const route = base + "/projects/" + encodeURIComponent(project) + "/auth";
|
||||
assert.equal((await fetch(route + "/me")).status, 401);
|
||||
if (process.env.FOUNDRY_CODEX_SOCKET) {
|
||||
const exec = base + "/projects/" + encodeURIComponent(project) + "/codex/exec";
|
||||
assert.equal((await fetch(exec, {method: "POST"})).status, 401);
|
||||
}
|
||||
}
|
||||
verify().catch(() => { console.error("Foundry readiness or anonymous access check failed"); process.exit(1); });
|
||||
'
|
||||
|
||||
# Keep the previous running container until the new candidate has passed checks.
|
||||
# Traefik reads the route atomically from its watched config directory.
|
||||
id=$(docker inspect -f '{{.Id}}' "$candidate")
|
||||
live=foundry-core-${id:0:12}
|
||||
docker rename "$candidate" "$live"
|
||||
docker update --restart unless-stopped "$live" >/dev/null
|
||||
bash "$(dirname "$0")/route.sh" "$live"
|
||||
trap - EXIT
|
||||
printf 'Foundry readiness and unauthenticated rejection verified.\n'
|
||||
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
umask 077
|
||||
cd /opt/endoleap-infra
|
||||
live=$1
|
||||
[[ "$live" =~ ^foundry-core-[a-f0-9]{12}$ ]]
|
||||
|
||||
install -d -m 700 dev/foundry-edge
|
||||
edge=dev/foundry-edge
|
||||
route=$edge/foundry.yml
|
||||
previous=
|
||||
if [[ -f "$route" ]]; then
|
||||
previous=$(sed -n 's@.*url: http://\(foundry-core-[a-f0-9]\{12\}\):8080.*@\1@p' "$route")
|
||||
fi
|
||||
backup=$(mktemp)
|
||||
if [[ -f "$route" ]]; then cp "$route" "$backup"; fi
|
||||
rollback() {
|
||||
if [[ -s "$backup" ]]; then
|
||||
cp "$backup" "$edge/.restore"
|
||||
mv "$edge/.restore" "$route"
|
||||
else
|
||||
rm -f "$route"
|
||||
fi
|
||||
rm -f "$backup"
|
||||
}
|
||||
trap rollback EXIT
|
||||
|
||||
touch "$edge/acme.json"
|
||||
chmod 600 "$edge/acme.json"
|
||||
cat > "$edge/.foundry.yml" <<YAML
|
||||
http:
|
||||
routers:
|
||||
foundry:
|
||||
entryPoints: [websecure]
|
||||
rule: Host(\`api.foundry.entroia.com\`)
|
||||
middlewares: [foundry-limit]
|
||||
service: foundry
|
||||
tls:
|
||||
certResolver: public
|
||||
middlewares:
|
||||
foundry-limit:
|
||||
rateLimit:
|
||||
average: 30
|
||||
burst: 60
|
||||
services:
|
||||
foundry:
|
||||
loadBalancer:
|
||||
servers:
|
||||
- url: http://$live:8080
|
||||
YAML
|
||||
mv "$edge/.foundry.yml" "$route"
|
||||
cat > foundry-edge.yml <<'YAML'
|
||||
services:
|
||||
traefik:
|
||||
command:
|
||||
- --entrypoints.web.address=:8080
|
||||
- --entrypoints.websecure.address=:8443
|
||||
- --ping=true
|
||||
- --ping.entrypoint=web
|
||||
- --api.dashboard=false
|
||||
- --providers.file.directory=/edge
|
||||
- --certificatesresolvers.public.acme.storage=/edge/acme.json
|
||||
- --certificatesresolvers.public.acme.tlschallenge=true
|
||||
ports:
|
||||
- "443:8443"
|
||||
volumes:
|
||||
- ./dev/foundry-edge:/edge
|
||||
YAML
|
||||
docker compose -p endoleap-dev --env-file dev/env -f compose.yml -f dev.yml -f foundry-edge.yml up -d --no-deps traefik
|
||||
|
||||
curl --fail --silent --show-error --output /dev/null --max-time 6 \
|
||||
--retry 15 --retry-delay 4 --retry-all-errors \
|
||||
--resolve api.foundry.entroia.com:443:127.0.0.1 \
|
||||
https://api.foundry.entroia.com/readyz
|
||||
|
||||
trap - EXIT
|
||||
rm -f "$backup"
|
||||
if [[ "$previous" =~ ^foundry-core-[a-f0-9]{12}$ && "$previous" != "$live" ]]; then
|
||||
docker stop "$previous" >/dev/null || printf 'Previous container needs manual cleanup.\n' >&2
|
||||
fi
|
||||
@@ -0,0 +1,143 @@
|
||||
package codex
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"crypto/subtle"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
maxRequest = 32 << 10
|
||||
maxOutput = 512 << 10
|
||||
timeout = 90 * time.Second
|
||||
)
|
||||
|
||||
type task struct {
|
||||
Prompt string `json:"prompt"`
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, value any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(value)
|
||||
}
|
||||
|
||||
func fail(w http.ResponseWriter, status int, code string) {
|
||||
writeJSON(w, status, map[string]string{"error": code})
|
||||
}
|
||||
|
||||
func readTask(w http.ResponseWriter, r *http.Request, limit int64) (task, bool) {
|
||||
var input task
|
||||
decoder := json.NewDecoder(http.MaxBytesReader(w, r.Body, limit))
|
||||
decoder.DisallowUnknownFields()
|
||||
err := decoder.Decode(&input)
|
||||
if err == nil {
|
||||
err = decoder.Decode(new(any))
|
||||
if err == io.EOF && strings.TrimSpace(input.Prompt) != "" {
|
||||
return input, true
|
||||
}
|
||||
}
|
||||
var tooLarge *http.MaxBytesError
|
||||
if errors.As(err, &tooLarge) {
|
||||
fail(w, 413, "request_too_large")
|
||||
} else {
|
||||
fail(w, 400, "invalid_task")
|
||||
}
|
||||
return task{}, false
|
||||
}
|
||||
|
||||
// Register enables the public endpoint only when both configuration files are supplied.
|
||||
// Each credential digest belongs to exactly one project.
|
||||
func Register(mux *http.ServeMux, projects map[string]string, socket, grantsFile string) error {
|
||||
if socket == "" && grantsFile == "" {
|
||||
return nil
|
||||
}
|
||||
if socket == "" || grantsFile == "" {
|
||||
return errors.New("Codex requires both a socket and a grants file")
|
||||
}
|
||||
transport := &http.Transport{
|
||||
DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
|
||||
return (&net.Dialer{}).DialContext(ctx, "unix", socket)
|
||||
},
|
||||
DisableKeepAlives: true,
|
||||
}
|
||||
client := &http.Client{
|
||||
Transport: transport,
|
||||
Timeout: timeout + 2*time.Second,
|
||||
CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse },
|
||||
}
|
||||
mux.HandleFunc("POST /projects/{project}/codex/exec", func(w http.ResponseWriter, r *http.Request) {
|
||||
scheme, token, _ := strings.Cut(r.Header.Get("Authorization"), " ")
|
||||
key, err := hex.DecodeString(token)
|
||||
if !strings.EqualFold(scheme, "Bearer") || err != nil || len(key) != 32 {
|
||||
w.Header().Set("WWW-Authenticate", "Bearer")
|
||||
fail(w, 401, "unauthorized")
|
||||
return
|
||||
}
|
||||
data, err := os.ReadFile(grantsFile)
|
||||
var grants map[string]string
|
||||
if err != nil || json.Unmarshal(data, &grants) != nil {
|
||||
fail(w, 503, "codex_unavailable")
|
||||
return
|
||||
}
|
||||
hash := sha256.Sum256(key)
|
||||
var scope string
|
||||
for digest, project := range grants {
|
||||
value, err := hex.DecodeString(digest)
|
||||
if err == nil && subtle.ConstantTimeCompare(hash[:], value) == 1 {
|
||||
scope = project
|
||||
}
|
||||
}
|
||||
if scope == "" {
|
||||
w.Header().Set("WWW-Authenticate", "Bearer")
|
||||
fail(w, 401, "unauthorized")
|
||||
return
|
||||
}
|
||||
project := r.PathValue("project")
|
||||
if _, exists := projects[project]; !exists || scope != project {
|
||||
fail(w, 403, "forbidden")
|
||||
return
|
||||
}
|
||||
input, ok := readTask(w, r, maxRequest)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
body, _ := json.Marshal(input)
|
||||
req, err := http.NewRequestWithContext(r.Context(), http.MethodPost, "http://codex/exec", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
fail(w, 502, "codex_unavailable")
|
||||
return
|
||||
}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
fail(w, 504, "codex_timeout")
|
||||
} else {
|
||||
fail(w, 502, "codex_unavailable")
|
||||
}
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
data, err = io.ReadAll(io.LimitReader(resp.Body, maxOutput+1))
|
||||
if err != nil || len(data) > maxOutput || !json.Valid(data) {
|
||||
fail(w, 502, "invalid_codex_response")
|
||||
return
|
||||
}
|
||||
if retry := resp.Header.Get("Retry-After"); retry != "" {
|
||||
w.Header().Set("Retry-After", retry)
|
||||
}
|
||||
writeJSON(w, resp.StatusCode, json.RawMessage(data))
|
||||
})
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package codex
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestProjectAuthorizationAndForwarding(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
socket := filepath.Join(dir, "worker.sock")
|
||||
listener, err := net.Listen("unix", socket)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
worker := &http.Server{Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/exec" || r.Header.Get("Authorization") != "" {
|
||||
fail(w, 500, "unexpected_worker_request")
|
||||
return
|
||||
}
|
||||
input, ok := readTask(w, r, 6*maxRequest)
|
||||
if ok {
|
||||
writeJSON(w, 200, map[string]string{"result": input.Prompt})
|
||||
}
|
||||
})}
|
||||
go worker.Serve(listener)
|
||||
t.Cleanup(func() { _ = worker.Close() })
|
||||
|
||||
key := make([]byte, 32)
|
||||
token := hex.EncodeToString(key)
|
||||
grants := filepath.Join(dir, "grants.json")
|
||||
writeGrants := func(data string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(grants, []byte(data), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
writeGrants(fmt.Sprintf(`{"%x":"notes"}`, sha256.Sum256(key)))
|
||||
mux := http.NewServeMux()
|
||||
if err := Register(mux, map[string]string{"notes": "", "board": ""}, socket, grants); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
check := func(project, credential, body string, status int, response string) {
|
||||
t.Helper()
|
||||
r := httptest.NewRequest("POST", "/projects/"+project+"/codex/exec", strings.NewReader(body))
|
||||
if credential != "" {
|
||||
r.Header.Set("Authorization", "Bearer "+credential)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
mux.ServeHTTP(w, r)
|
||||
if w.Code != status || !strings.Contains(w.Body.String(), response) {
|
||||
t.Fatalf("got %d %s, want %d containing %q", w.Code, w.Body.String(), status, response)
|
||||
}
|
||||
}
|
||||
check("notes", "", `{"prompt":"hello"}`, 401, "unauthorized")
|
||||
check("board", token, `{"prompt":"hello"}`, 403, "forbidden")
|
||||
check("notes", token, `{"prompt":"hello","sandbox":"danger"}`, 400, "invalid_task")
|
||||
check("notes", token, `{"prompt":"`+strings.Repeat("x", maxRequest)+`"}`, 413, "request_too_large")
|
||||
check("notes", token, `{"prompt":"hello"}`, 200, `"result":"hello"`)
|
||||
writeGrants("{}")
|
||||
check("notes", token, `{"prompt":"hello"}`, 401, "unauthorized")
|
||||
}
|
||||
@@ -0,0 +1,134 @@
|
||||
package codex
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
const executable = "/opt/codex/node_modules/.bin/codex"
|
||||
|
||||
var errOutputLimit = errors.New("Codex output limit exceeded")
|
||||
|
||||
type outputBuffer struct {
|
||||
buffer bytes.Buffer
|
||||
exceeded bool
|
||||
}
|
||||
|
||||
func (b *outputBuffer) Write(p []byte) (int, error) {
|
||||
if len(p) > maxOutput-b.buffer.Len() {
|
||||
b.exceeded = true
|
||||
return 0, errOutputLimit
|
||||
}
|
||||
return b.buffer.Write(p)
|
||||
}
|
||||
|
||||
func ServeWorker(socket string) error {
|
||||
syscall.Umask(0077)
|
||||
if err := os.Remove(socket); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
listener, err := net.Listen("unix", socket)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer listener.Close()
|
||||
if err := os.Chmod(socket, 0600); err != nil {
|
||||
return err
|
||||
}
|
||||
slot := make(chan struct{}, 1)
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("POST /exec", func(w http.ResponseWriter, r *http.Request) {
|
||||
select {
|
||||
case slot <- struct{}{}:
|
||||
defer func() { <-slot }()
|
||||
default:
|
||||
w.Header().Set("Retry-After", "5")
|
||||
fail(w, 429, "codex_busy")
|
||||
return
|
||||
}
|
||||
input, ok := readTask(w, r, 6*maxRequest)
|
||||
if ok {
|
||||
execute(w, r, input.Prompt)
|
||||
}
|
||||
})
|
||||
server := &http.Server{Handler: mux, ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 10 * time.Second, WriteTimeout: timeout + 15*time.Second}
|
||||
return server.Serve(listener)
|
||||
}
|
||||
|
||||
func execute(w http.ResponseWriter, r *http.Request, prompt string) {
|
||||
ctx, cancel := context.WithTimeout(r.Context(), timeout)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, executable, "--ask-for-approval", "never",
|
||||
"--disable", "shell_tool", "--disable", "apps", "--disable", "plugins",
|
||||
"--disable", "multi_agent", "--disable", "hooks", "--disable", "memories",
|
||||
"-c", `web_search="disabled"`, "exec", "--json", "--sandbox", "read-only", "--skip-git-repo-check", "--ephemeral", "-")
|
||||
cmd.Dir = "/work"
|
||||
cmd.Env = []string{"PATH=/usr/local/bin:/usr/bin:/bin", "HOME=/home/node", "CODEX_HOME=/home/node/.codex", "SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt"}
|
||||
cmd.Stdin = strings.NewReader(prompt)
|
||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||
cmd.Cancel = func() error {
|
||||
err := syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
|
||||
if errors.Is(err, syscall.ESRCH) {
|
||||
return os.ErrProcessDone
|
||||
}
|
||||
return err
|
||||
}
|
||||
cmd.WaitDelay = 2 * time.Second
|
||||
var output outputBuffer
|
||||
cmd.Stdout, cmd.Stderr = &output, io.Discard
|
||||
err := cmd.Run()
|
||||
switch {
|
||||
case output.exceeded:
|
||||
fail(w, 502, "codex_output_too_large")
|
||||
case errors.Is(ctx.Err(), context.DeadlineExceeded):
|
||||
fail(w, 504, "codex_timeout")
|
||||
case err != nil:
|
||||
fail(w, 502, "codex_failed")
|
||||
default:
|
||||
var result string
|
||||
var completed bool
|
||||
for _, line := range bytes.Split(bytes.TrimSpace(output.buffer.Bytes()), []byte("\n")) {
|
||||
var event struct {
|
||||
Type string `json:"type"`
|
||||
Item struct {
|
||||
Type string `json:"type"`
|
||||
Text string `json:"text"`
|
||||
} `json:"item"`
|
||||
}
|
||||
if json.Unmarshal(line, &event) != nil {
|
||||
fail(w, 502, "invalid_codex_response")
|
||||
return
|
||||
}
|
||||
if event.Type == "item.completed" && event.Item.Type == "agent_message" {
|
||||
result = event.Item.Text
|
||||
}
|
||||
if event.Type == "turn.failed" || event.Type == "error" {
|
||||
fail(w, 502, "codex_failed")
|
||||
return
|
||||
}
|
||||
if event.Type == "turn.completed" {
|
||||
completed = true
|
||||
}
|
||||
}
|
||||
if !completed || result == "" {
|
||||
fail(w, 502, "codex_failed")
|
||||
return
|
||||
}
|
||||
response, _ := json.Marshal(map[string]string{"result": result})
|
||||
if len(response) >= maxOutput {
|
||||
fail(w, 502, "codex_output_too_large")
|
||||
return
|
||||
}
|
||||
writeJSON(w, 200, json.RawMessage(response))
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user