smart-org/services/ai-runtime/README.md

@agy/ai-runtime

Drains the queue of tasks nobody has run yet, through the real run engine.

pnpm --filter @agy/ai-runtime run     # one pass, then exit
pnpm --filter @agy/ai-runtime dev     # poll every 30s

Why it exists

The API can start a run inline — that is what the «اجرا» button does. But a platform that only runs work on the request thread has two problems: a slow model call holds a connection open, and a task created by anything other than a button press (an automation, a seed script, a future integration) never runs at all. This worker is the second driver.

What it does

1. Find tasks in new or triaged state, oldest first.
2. Re-read each through WorkforceService.get_task_or_404 — so the tenant check and the "already closed" guard both apply. The worker gets no privileged bypass.
3. Hand it to WorkforceService.start_run — the exact call the router makes.
4. Record the outcome and continue. One poisoned task must not block the queue behind it.

Model routing, usage accounting, the run timeline and the artifact are all the service's business; the worker owns none of it.

Idempotency

A task that has been run is no longer in new/triaged, so a second pass finds nothing. Re-running a completed task would double-bill the customer, which is why the query filters on state rather than on "has a run".

Implementation

apps/api/app/workers/ai_runtime.py. Tests: apps/api/tests/test_workers.py.

Extracting this into its own process

DATABASE_URL=... python -m app.workers ai-runtime --loop --interval 30

Note that the run engine currently executes inline within the worker process. To make runs asynchronous for the customer (return queued, complete later), the next step is to have the API enqueue and this worker claim, rather than having the API call start_run directly. The port boundary for that already exists.