feat(integration): wire v2 workstreams together (engine policy/budget, sandbox, tenant checks, order route, docs)
Engine honours org policy and monthly budget, legacy runner skips engine tasks, sandbox session provisioned on demand, project fallback for threads, reviewer fallback, tenant checks on project ids, memory reflection respects grants, expiry worker, MCP pipeline_status, login redirect, sharing settings, e2e smoke. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
mjalmoudy94
co-authored by
Claude Sonnet 5.5
Claude Sonnet 5.5
parent
36564e3f2b
commit
fef5a9deab
@@ -31,7 +31,7 @@ for tests).
|
||||
|
||||
## Phase 1 — Docker sandbox adapter
|
||||
|
||||
- [ ] 1.1 `adapters/docker_sandbox.py`, `DockerSandboxAdapter` implementing `SandboxProviderPort` with the CLI in `SANDBOX_DOCKER_BIN` (docker or podman, the same flags):
|
||||
- [x] 1.1 `adapters/docker_sandbox.py`, `DockerSandboxAdapter` implementing `SandboxProviderPort` with the CLI in `SANDBOX_DOCKER_BIN` (docker or podman, the same flags):
|
||||
- `create_workspace`: creates a host dir `SANDBOX_WORKSPACES_ROOT/<org_id>/<project_id>/work` (0700, owned by the platform user).
|
||||
- `start_session`: `docker run -d` the **workspace container**:
|
||||
- image from the app profile language: `node:20-bookworm`, `python:3.13-bookworm`, `php:8.3-cli`, `golang:1.23`, fallback `debian:bookworm`; make the mapping a dict in your config block / domain
|
||||
@@ -44,39 +44,39 @@ for tests).
|
||||
- `read_file` / `write_file`: through the host bind mount with path safety (`resolve()` stays inside the work dir; reject symlinks that escape).
|
||||
- `pause` / `resume` / `destroy`: stop/start/rm. `snapshot`: `docker commit` (tagged per org) or raise a clear "not supported" error, your choice, documented.
|
||||
- `get_preview_url`: delegated to the domain layer (Phase 4).
|
||||
- [ ] 1.2 `registry.py`: `SANDBOX_PROVIDER=docker` → this adapter. The mock stays the default.
|
||||
- [ ] 1.3 The `sandbox_gateway` worker: also reap containers with the `smart-org.*` label that have no live session row (orphans), audited.
|
||||
- [ ] 1.4 Hide infra from non-owners:
|
||||
- [x] 1.2 `registry.py`: `SANDBOX_PROVIDER=docker` → this adapter. The mock stays the default.
|
||||
- [x] 1.3 The `sandbox_gateway` worker: also reap containers with the `smart-org.*` label that have no live session row (orphans), audited.
|
||||
- [x] 1.4 Hide infra from non-owners:
|
||||
- `sandbox.py` command/file endpoints become **owner-only** (or admin+ of the org with a setting off by default; prefer owner-only).
|
||||
- Keep the read-only status endpoints for org members.
|
||||
- Update `components/sandbox-panel.tsx` to show only status to non-owners.
|
||||
- Update existing tests that assumed member access, and explain each change in your report.
|
||||
- [ ] 1.5 Tests `tests/test_docker_sandbox.py`: unit tests with a **fake process runner** (inject the subprocess function) that assert the exact argv (hardening flags present, no secrets, mount of the work dir only), timeout kill, output cap, path escape refused. Integration tests marked `@pytest.mark.docker`, skipped unless `SANDBOX_DOCKER_IT=1`: real start → exec `echo` → write/read file → destroy.
|
||||
- [ ] 1.5 Tests `tests/test_docker_sandbox.py`: unit tests with a **fake process runner** (inject the subprocess function) that assert the exact argv (hardening flags present, no secrets, mount of the work dir only), timeout kill, output cap, path escape refused. Integration tests marked `@pytest.mark.docker`, skipped unless `SANDBOX_DOCKER_IT=1`: real start → exec `echo` → write/read file → destroy. — open: no `@pytest.mark.docker` sandbox integration test (real start, exec, write/read, destroy) exists in tests/test_docker_sandbox.py; the only docker-marked test is test_delivery.py::test_docker_end_to_end_static_app.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Git workspaces (`RepoWorkspacePort`, the platform does git, D6)
|
||||
|
||||
- [ ] 2.1 `adapters/git_workspace.py`, `LocalGitWorkspaceAdapter`. Layout per project under `SANDBOX_WORKSPACES_ROOT/<org>/<project>/`:
|
||||
- [x] 2.1 `adapters/git_workspace.py`, `LocalGitWorkspaceAdapter`. Layout per project under `SANDBOX_WORKSPACES_ROOT/<org>/<project>/`:
|
||||
- `git/`: a bare-ish gitdir, **never mounted**
|
||||
- `work/`: the main working tree for the dev branch, mounted at `/workspace`
|
||||
- `work/.worktrees/<task_id>/`: task worktrees, visible in the container at `/workspace/.worktrees/<task_id>`
|
||||
- `WorktreeRef.sandbox_path` = that in-container path
|
||||
Note: the `.worktrees` dir must be excluded via `info/exclude` in the gitdir, not the user's `.gitignore`.
|
||||
- [ ] 2.2 Every git call is `git --git-dir=<gitdir> --work-tree=<path> -c core.hooksPath=/dev/null -c core.fsmonitor=false -c core.quotePath=false -c protocol.file.allow=never -c safe.directory=*`, with env `GIT_CONFIG_NOSYSTEM=1`, `HOME=<private dir>`, `GIT_TERMINAL_PROMPT=0`. Never trust the `.git` *file* inside a worktree; always pass explicit dirs.
|
||||
- [x] 2.2 Every git call is `git --git-dir=<gitdir> --work-tree=<path> -c core.hooksPath=/dev/null -c core.fsmonitor=false -c core.quotePath=false -c protocol.file.allow=never -c safe.directory=*`, with env `GIT_CONFIG_NOSYSTEM=1`, `HOME=<private dir>`, `GIT_TERMINAL_PROMPT=0`. Never trust the `.git` *file* inside a worktree; always pass explicit dirs.
|
||||
- Status: `status --porcelain=v1 -uall -z`.
|
||||
- One `asyncio.Lock` per project id held for every mutating operation (agybot M14). Remove a stale `index.lock` only if older than 30s.
|
||||
- [ ] 2.3 Remotes:
|
||||
- [ ] 2.3 Remotes: — open: the ssh_key first contact uses StrictHostKeyChecking=accept-new and the key is never stored. test_remote returns no host key (RemoteCheck has no field for it) and nothing writes project_external_git.known_hosts.
|
||||
- **platform** git: the URL and credentials come from the existing Gitea adapter / `project_git_credentials` (decrypt via the existing crypto helper).
|
||||
- **external** git: `project_external_git` (token → `GIT_ASKPASS`; ssh_key → temp key file 0600 + `GIT_SSH_COMMAND="ssh -i <key> -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes -o UserKnownHostsFile=<per-project known_hosts>"`; store `known_hosts` on the first `test_remote` = TOFU, in the existing column).
|
||||
- `test_remote`: `git ls-remote --heads` with a 20s timeout; returns ok, default branch, branch list, and a friendly error (auth failed / not found / unreachable) **without echoing the URL credentials or stderr secrets**.
|
||||
- SSRF check on external hosts (resolve DNS, reject private, loopback and link-local unless `DEPLOY_ALLOW_PRIVATE_HOSTS`); put the check in `domain/platform/rules.py` (yours). It runs inside `test_remote` and before every deploy connection; Agent 1 only does format validation and relies on your check.
|
||||
- [ ] 2.4 Implement every `RepoWorkspacePort` method per PLAN-v2 §5.
|
||||
- [x] 2.4 Implement every `RepoWorkspacePort` method per PLAN-v2 §5.
|
||||
- `ensure_workspace` clones on first use, fetches afterwards. It creates `dev` from the default branch if missing; for platform repos also push `dev`.
|
||||
- `merge_into` does a `--no-ff` merge in the main work tree under the lock and returns the conflict file list on failure, after `merge --abort`. Never leave a half-merge (agybot H3).
|
||||
- `push` returns ok/rejected (non-fast-forward → fetch + report, never force).
|
||||
- [ ] 2.5 Wire it in `container_delivery.py` (setting `GIT_WORKSPACE=local|fake`, fake default in tests).
|
||||
- [ ] 2.6 Tests `tests/test_git_workspace.py` with **real git** and a local bare repo in `tmp_path` as "remote" (no network):
|
||||
- [x] 2.5 Wire it in `container_delivery.py` (setting `GIT_WORKSPACE=local|fake`, fake default in tests).
|
||||
- [x] 2.6 Tests `tests/test_git_workspace.py` with **real git** and a local bare repo in `tmp_path` as "remote" (no network):
|
||||
- the full flow ensure → worktree → write → diff (new dir appears thanks to `-uall`) → commit → sync → merge → push → head
|
||||
- a conflict path leaves a clean tree
|
||||
- hooks are NOT executed (plant a `post-commit` hook that writes a file and assert it is absent)
|
||||
@@ -88,15 +88,15 @@ for tests).
|
||||
|
||||
## Phase 3 — Live CI/CD + demo server
|
||||
|
||||
- [ ] 3.1 Tables (`models_delivery.py`):
|
||||
- [x] 3.1 Tables (`models_delivery.py`):
|
||||
- `pipeline_runs` (id, org, project, trigger `engine|poll|manual`, branch, sha, status `queued|running|succeeded|failed|cancelled|superseded`, started/finished, error_summary)
|
||||
- `pipeline_steps` (id, run_id, index, name, status, started/finished, exit_code, log_tail ≤ 64 KB)
|
||||
- `releases` (id, org, project, run_id, sha, artifact_path, created_at, is_demo_live)
|
||||
- `app_runtimes` (project_id PK, demo_container_id, host_port, demo_host, status, current_release_id, updated_at)
|
||||
- [ ] 3.2 `delivery_service.py` implementing `DeliveryHooks`:
|
||||
- [x] 3.2 `delivery_service.py` implementing `DeliveryHooks`:
|
||||
- `on_app_created`: allocate a demo port in `DEMO_PORT_START..END` (reuse the existing preview port rules), claim the demo host `<org-slug>.<PREVIEW_ROOT_DOMAIN>` through the existing `DomainRouterPort`, create `app_runtimes`, `ensure_workspace`, and queue a first pipeline run if the repo has a `dev` head.
|
||||
- `on_branch_pushed`: queue a run (one active run per project; a newer queued run supersedes an older queued one).
|
||||
- [ ] 3.3 `workers/delivery.py`: loop (`--loop`) that executes queued runs and polls remotes every `DELIVERY_POLL_SECONDS` (`head` vs the last run sha) for pushes made outside the engine (users pushing to their own git).
|
||||
- [x] 3.3 `workers/delivery.py`: loop (`--loop`) that executes queued runs and polls remotes every `DELIVERY_POLL_SECONDS` (`head` vs the last run sha) for pushes made outside the engine (users pushing to their own git).
|
||||
- Steps, each in a **fresh build container** (same image mapping and hardening, work dir = a clean checkout of the sha in a per-run dir):
|
||||
1. `checkout`
|
||||
2. `setup` (setup_command)
|
||||
@@ -107,9 +107,9 @@ for tests).
|
||||
7. `healthcheck`: HTTP GET `http://127.0.0.1:<host_port><healthcheck_path>` with 20 retries × 3s; any 2xx/3xx = ok
|
||||
8. `route`: `configure_reverse_proxy` for the demo host → `127.0.0.1:<host_port>`
|
||||
- **Blue/green:** start the new container on a temporary second port, healthcheck it, then switch the Caddy route, then stop the old one. If any step fails, the old demo keeps running untouched and the run is `failed` with a Persian `error_summary`.
|
||||
- [ ] 3.4 Live logs: every step streams output via the `on_output` callback → batched `pipeline.log` events on `org:<id>` (≤ 1 event / 500 ms, ≤ 200 lines), plus `pipeline.step` and `pipeline.status`; `release.live` when switched. Store the step `log_tail` (last 64 KB). Pass all output through `app/core/redact.py` before publishing or storing.
|
||||
- [ ] 3.5 Router `delivery.py` (org member read, admin+ actions): list runs, run detail with steps and tails, `POST /delivery/runs` (manual re-run of the dev head, admin+), cancel the queued/running run, list releases, rollback the demo to a previous release (admin+, confirm, audit), and the runtime status (demo URL, live release, health).
|
||||
- [ ] 3.6 Tests `tests/test_delivery.py` with the fake runner + `FakeRepoWorkspace` + mock domain router:
|
||||
- [x] 3.4 Live logs: every step streams output via the `on_output` callback → batched `pipeline.log` events on `org:<id>` (≤ 1 event / 500 ms, ≤ 200 lines), plus `pipeline.step` and `pipeline.status`; `release.live` when switched. Store the step `log_tail` (last 64 KB). Pass all output through `app/core/redact.py` before publishing or storing.
|
||||
- [x] 3.5 Router `delivery.py` (org member read, admin+ actions): list runs, run detail with steps and tails, `POST /delivery/runs` (manual re-run of the dev head, admin+), cancel the queued/running run, list releases, rollback the demo to a previous release (admin+, confirm, audit), and the runtime status (demo URL, live release, health).
|
||||
- [x] 3.6 Tests `tests/test_delivery.py` with the fake runner + `FakeRepoWorkspace` + mock domain router:
|
||||
- the happy path step sequence and events
|
||||
- a failed build keeps the old demo
|
||||
- healthcheck failure → no route switch
|
||||
@@ -123,50 +123,50 @@ for tests).
|
||||
|
||||
## Phase 4 — Domains (Caddy) owned by the platform owner
|
||||
|
||||
- [ ] 4.1 `admin_domains.py` (owner only):
|
||||
- [x] 4.1 `admin_domains.py` (owner only):
|
||||
- list all routes (host → org/app → target port → status)
|
||||
- attach an extra host to an org's app: a single label under the root via the existing rules, OR a full custom FQDN when `ALLOW_CUSTOM_DOMAINS=true` (strict FQDN regex, no wildcard, no protected hosts such as `git`, `mcp`, `panel`, `api`, root)
|
||||
- detach (confirm + audit)
|
||||
- re-sync all routes from the DB to Caddy (idempotent)
|
||||
Users cannot add domains (the existing member-facing preview-domain endpoints in `projects.py` become owner-only, and the tests are updated with reasons).
|
||||
- [ ] 4.2 The Caddy adapter keeps validate → revert → reload. Add a `routes_fingerprint` check so re-sync does nothing when nothing changed.
|
||||
- [ ] 4.3 The demo host is created automatically in `on_app_created` (Phase 3). Record the MCP gateway host suggestion (`mcp.<root>` → `MCP_GATEWAY_PORT`) as a protected system route that the owner can enable from this page.
|
||||
- [ ] 4.4 Tests `tests/test_admin_domains.py`: non-owner forbidden; protected hosts refused; invalid FQDNs refused; detach audited; re-sync idempotent (mock router call count).
|
||||
- [x] 4.2 The Caddy adapter keeps validate → revert → reload. Add a `routes_fingerprint` check so re-sync does nothing when nothing changed.
|
||||
- [ ] 4.3 The demo host is created automatically in `on_app_created` (Phase 3). Record the MCP gateway host suggestion (`mcp.<root>` → `MCP_GATEWAY_PORT`) as a protected system route that the owner can enable from this page. — open: the MCP route is only a read-only suggestion (GET /api/admin/domains/suggested, shown as enabled or disabled on the page). There is no enable action, and validate_extra_host refuses the protected label mcp, so the owner cannot enable it from this page.
|
||||
- [x] 4.4 Tests `tests/test_admin_domains.py`: non-owner forbidden; protected hosts refused; invalid FQDNs refused; detach audited; re-sync idempotent (mock router call count).
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Production deploy over SSH / SFTP / FTP / FTPS (always human-triggered)
|
||||
|
||||
- [ ] 5.1 `adapters/deploy_ssh.py` (asyncssh), `DeployTargetTester` + `SshDeployer`:
|
||||
- [x] 5.1 `adapters/deploy_ssh.py` (asyncssh), `DeployTargetTester` + `SshDeployer`:
|
||||
- Test: connect (15s timeout) → host-key TOFU (store the fingerprint on the first success; afterwards a mismatch = hard fail "کلید میزبان تغییر کرده"), check `remote_path` exists and is writable.
|
||||
- Deploy: upload the release artifact → `remote_path/releases/<ts>-<sha7>/` → extract → run `post_deploy_command` with `cwd` there (timeout 600s, streamed output) → atomically switch the `remote_path/current` symlink → keep the last 5 releases.
|
||||
- Rollback = re-point `current` to the previous release + rerun post_deploy.
|
||||
- [ ] 5.2 `adapters/deploy_ftp.py` (aioftp; FTPS via explicit TLS), `DeployTargetTester` + `FtpDeployer`:
|
||||
- [x] 5.2 `adapters/deploy_ftp.py` (aioftp; FTPS via explicit TLS), `DeployTargetTester` + `FtpDeployer`:
|
||||
- Test: login, cwd `remote_path`, write and delete a probe file.
|
||||
- Deploy: upload the build output (the build dir is set by the framework catalog default or `dist/`, `build/`, `public/`, else the whole artifact) into `remote_path` after downloading a **backup listing** of the files it overwrites into `ARTIFACTS_ROOT/.../ftp-backup-<ts>.tar.gz`.
|
||||
- Rollback = re-upload that backup. Document the limits of FTP (no atomic switch).
|
||||
- [ ] 5.3 SSRF check for every target host (the Phase 2.3 helper). A combined `DeployTargetTester` dispatching on `kind`, wired in `container_delivery.py`.
|
||||
- [ ] 5.4 `deploy_service.py` + endpoints in `delivery.py`:
|
||||
- [x] 5.3 SSRF check for every target host (the Phase 2.3 helper). A combined `DeployTargetTester` dispatching on `kind`, wired in `container_delivery.py`.
|
||||
- [x] 5.4 `deploy_service.py` + endpoints in `delivery.py`:
|
||||
- `POST /delivery/production/deploy` with `release_id` (default: the live demo release). Org **owner/admin only**; requires a `confirm: "<org-slug>"` field; one production deploy per project at a time.
|
||||
- Status: `deploy.status` events and a `production_deploys` table (id, release, target, status, started/finished, log_tail, rolled_back_from).
|
||||
- `POST /delivery/production/rollback`.
|
||||
- Every action audited. Credentials decrypted only inside the adapter call.
|
||||
- [ ] 5.5 Tests `tests/test_deploy.py`: fake SSH/FTP clients injected. Cover: the release/symlink sequence, keep-5 pruning, rollback, host-key mismatch refused, SSRF refused, a member (non-admin) refused, the confirm slug mismatch refused, no password in logs/events/DB columns other than encrypted ones. Optional integration (`DEPLOY_IT=1`) against a local `sshd`/`pure-ftpd` container.
|
||||
- [x] 5.5 Tests `tests/test_deploy.py`: fake SSH/FTP clients injected. Cover: the release/symlink sequence, keep-5 pruning, rollback, host-key mismatch refused, SSRF refused, a member (non-admin) refused, the confirm slug mismatch refused, no password in logs/events/DB columns other than encrypted ones. Optional integration (`DEPLOY_IT=1`) against a local `sshd`/`pure-ftpd` container.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — UI: the "Application" page (replaces `/projects`)
|
||||
|
||||
- [ ] 6.1 `(app)/app/page.tsx` (+ components under `(app)/app/_components/`):
|
||||
- [x] 6.1 `(app)/app/page.tsx` (+ components under `(app)/app/_components/`):
|
||||
- **Header:** app name, type/language/framework badges, demo URL (open button), live release sha + time, health dot.
|
||||
- **Pipeline runs:** list with status; clicking one opens the run view: a step list with status and duration, and a **live log console** (subscribe `org:<id>`, filter by run id, ANSI colours stripped, auto-scroll with pause, and a "copy" button).
|
||||
- **Releases:** history; "rollback demo" (admin+, confirm).
|
||||
- **Production:** target summary (kind, host, path, last test result; no secrets), the "استقرار در production" button (admin+, modal requiring the org slug), a deploy live log, production history + rollback.
|
||||
- **Git:** repo link (platform: Gitea URL; external: the URL without credentials), `dev` head sha, last poll time. No branch/PR/port controls for users.
|
||||
- [ ] 6.2 Remove `(app)/projects` (redirect `/projects*` → `/app`) and drop the member-facing port/branch/PR/command controls. Owner-only debug tools move to `/admin/sandbox`.
|
||||
- [ ] 6.3 `/admin/sandbox` (owner): running containers per org (workspace/demo/build), CPU/memory limits per org (existing `sandbox_resource_limits`), stop/restart a container (confirm + audit), and an image mapping view. `/admin/domains` (owner): the Phase 4 UI.
|
||||
- [ ] 6.4 `lib/api-delivery.ts`: typed client for all of the above.
|
||||
- [ ] 6.5 Web tests `src/test/delivery-*.test.tsx`: the log console batching and auto-scroll, the confirm-slug modal, no secret fields rendered, non-admin sees no deploy button.
|
||||
- [x] 6.2 Remove `(app)/projects` (redirect `/projects*` → `/app`) and drop the member-facing port/branch/PR/command controls. Owner-only debug tools move to `/admin/sandbox`.
|
||||
- [ ] 6.3 `/admin/sandbox` (owner): running containers per org (workspace/demo/build), CPU/memory limits per org (existing `sandbox_resource_limits`), stop/restart a container (confirm + audit), and an image mapping view. `/admin/domains` (owner): the Phase 4 UI. — open: no restart action. The backend and the page offer list and stop only (POST /api/admin/sandbox/containers/{id}/stop).
|
||||
- [ ] 6.4 `lib/api-delivery.ts`: typed client for all of the above. — open: /admin/sandbox calls apiRequest directly, and lib/api-delivery.ts has deliveryApi and domainsApi only, with no sandbox client.
|
||||
- [ ] 6.5 Web tests `src/test/delivery-*.test.tsx`: the log console batching and auto-scroll, the confirm-slug modal, no secret fields rendered, non-admin sees no deploy button. — open: no test for log batching or auto-scroll. The non-admin deploy-button case is covered by `hides the deploy actions from an org member who is not an admin` (`apps/web/src/test/delivery-app.test.tsx`, added in T14).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user