feat(delivery): live CI/CD pipeline + demo runtime (v2 phase A3-3)

A push to `dev` now becomes a live demo, and a failed build never takes the
demo down.

- Four tables (pipeline_runs, pipeline_steps, releases, app_runtimes) and their
  migration; alembic upgrade/downgrade/check all clean.
- DeliveryService implements DeliveryHooks: on_app_created provisions the demo
  port + host + first build (idempotently); on_branch_pushed queues a run for
  `dev` and supersedes older *queued* runs, leaving a running one alone.
- The 8-step pipeline. Blue/green is real: the replacement starts on a staging
  port, is healthchecked, and only then does the route move — then the previous
  container is stopped. A failure removes the replacement, releases its port and
  leaves the old container serving.
- The demo port ledger is the DATABASE, not the router adapters' in-memory maps:
  the API and the delivery worker are separate processes.
- Runs are claimed with a conditional UPDATE, so two workers never build the
  same run.
- Live logs are batched (1 event / 500 ms, 200 lines) and redacted as a whole
  tail — per-chunk redaction misses a secret split across pipe reads, and the
  *stored* tail leaked until this was fixed.
- delivery worker: drain the queue, then poll remotes for pushes made outside
  the engine.
- 7 routes; reads need membership, actions need projects.manage and are audited.

Tests: 415 API (+21, 1 docker-marked skipped); ruff, mypy, alembic check clean.

Report: docs/plan/reports/agent-3-phase-3.md
This commit is contained in:
mjalmoudy94
2026-10-14 00:44:02 +03:30
parent d41e00ad10
commit a1b8a28dda
16 changed files with 3770 additions and 11 deletions
@@ -475,6 +475,107 @@ class DockerSandboxAdapter:
# -- demo / build containers (used by the delivery service) --------------
async def run_step(
self,
*,
organization_id: str,
project_id: str,
run_id: str,
image: str,
work_dir: str,
command: str,
timeout_seconds: float,
on_output: Callable[[str], Awaitable[None]] | None = None,
role: str = ROLE_BUILD,
publish_port: int | None = None,
app_port: int | None = None,
keep_alive: bool = False,
name_suffix: str = "",
) -> ProcessResult:
"""Run one pipeline step in a **fresh** container over ``work_dir``.
A fresh container per step is the point: a build must not be able to
inherit a process, a cache or a stray file from the previous step or from
an employee's interactive workspace. ``--rm`` guarantees nothing is left
behind on the host.
``publish_port`` publishes on ``127.0.0.1`` only. ``keep_alive`` starts
the container detached so it outlives this call — that is how the demo
container is created; a build step is run in the foreground so its output
streams straight back.
"""
org = _check_id(organization_id, "سازمان")
project = _check_id(project_id, "پروژه")
safe_run = re.sub(r"[^a-z0-9]", "", (run_id or "").lower())[:12] or "run"
name = f"smartorg-{role}-{org}-{project}-{safe_run}"
if name_suffix:
name = f"{name}-{_SNAPSHOT_NAME.sub('-', name_suffix.lower()).strip('-')}"
name = name[:200]
args: list[str] = ["run"]
if keep_alive:
args.append("-d")
else:
args.extend(["--rm"])
args.extend(
[
"--name",
name,
"--user",
"1000:1000",
"--cap-drop",
"ALL",
"--security-opt",
"no-new-privileges",
"--pids-limit",
"512",
"--cpus",
str(self.default_cpus),
"--memory",
f"{self.default_memory_mb}m",
"--read-only",
"--tmpfs",
"/tmp",
"--tmpfs",
"/home/app",
"-v",
f"{work_dir}:{SANDBOX_WORKSPACE_PATH}",
"-w",
SANDBOX_WORKSPACE_PATH,
]
)
if publish_port is not None:
assert_not_protected_port(int(publish_port))
# Loopback publish only — never 0.0.0.0.
args.extend(
[
"-p",
f"127.0.0.1:{int(publish_port)}:{int(app_port or publish_port)}",
]
)
args.extend(
[
"--label",
f"{LABEL_ORG}={org}",
"--label",
f"{LABEL_PROJECT}={project}",
"--label",
f"{LABEL_ROLE}={role}",
image or FALLBACK_IMAGE,
"sh",
"-lc",
command or _KEEPALIVE,
]
)
return await self._runner(
[self.bin, *args],
timeout_seconds=float(timeout_seconds),
max_output_bytes=self.max_output_bytes,
on_output=on_output,
env={},
)
async def start_demo_container(
self,
*,