smart-org/docs/plan/reports/agent-2-phase-4.md

Agent 2 — Phase 4 report: the scheduler (one tick → one step → merged code)

Date: 2026-10-14 · Branch: dev · Status: done, all checks green

What was done

4.1 The tick — app/application/engine/scheduler.py (new)

Scheduler.tick() is the engine's driver. One tick:

1. reads every task whose stage waits for a turn (TURN_STAGES) and every task ready to merge (MERGE_STAGES = (merge_request,)),
2. applies the selection caps, then
3. runs one step per selected task — a turn, or the platform's merge.

A step never aborts the tick: _guarded catches, records and continues. A lost optimistic race surfaces as ConflictError and is counted as skipped, because "another worker moved it first" is a normal outcome, not a failure.

Selection caps, not execution caps. _select honours engine_max_turns_per_org (3) and engine_max_concurrent_turns (8) by choosing that many tasks per tick. Turns themselves run one at a time inside a tick: an AsyncSession is not safe to share across concurrent turns, and correctness across worker processes comes from the optimistic advance (two workers ticking the same task both try; exactly one wins). This is a deliberate deviation from a literal reading of "run 8 turns at once" — see Deviations.

TickReport (selected / acted / skipped / failed / merged / paused, decisions, notes) is what the worker logs and the tests assert on.

4.2 Worktree lifecycle

_worktree_for calls RepoWorkspacePort.create_task_worktree(project_id, task_id, base="dev") for any pipeline that requires_worktree (i.e. not analysis_only) and a task with a project_id; otherwise None. The branch is saved onto the task state (engine.state → task_pipeline_state.branch) the moment the developer's diff is accepted, so a merged task records what it merged.

The worktree is removed on completion, cancellation and failure (TODO 4.2): _merge_step removes it after a successful push, and _move calls _release_worktree whenever the target is failed/cancelled. It is kept during error_review, so an error_review retry still has the code to move forward.

4.3 The gate chain — app/application/engine/gates.py (new)

A WORK_REPORT is a claim, not evidence. Four gates check it, and every verdict is a gate_results row and a task.gate event — a gate that only lives in a log is unauditable.

• diff guard — empty diff for a code task; .git/CI paths; symlinks escaping the repo; files over MAX_FILE_BYTES (1 MiB).
• no-progress — hashes the diff; the same hash twice in a row means the agent is spinning, so the task goes back to the developer (and eventually to the manager via max_cycles). no_progress returns the new (hash, count), which the scheduler persists on the state.
• tests — the app profile's test_command runs in the sandbox with engine_tests_timeout_seconds; the failure tail is stored and handed back. No command defined → green (nothing to run is not a failure).
• green — the tester's TEST_PASS is only believed when the tests gate is green, or when the tester waives it with a reason. A waiver is stored on the state (waiver_reason) and in the gate row; it is never inferred.

GateRunner.record(task, outcome) is the public entry the scheduler uses to write a verdict it reasoned about itself (a merge conflict is a diff-guard verdict that comes from git, not from hashing).

_escalate_after_repeated_red: a second red tests row moves the developer onto the strongest granted model (one escalation only; no stronger model granted → left alone).

4.4 Commit / merge / push — the platform owns git

The AI never runs git. After final_review → COMPLETE the task reaches merge_request, and _merge_step does the git:

1. commit in the worktree, author "<employee> <platform@smart-org.local>" (ENGINE_COMMIT_EMAIL, matching the delivery adapter),
2. sync_with_base,
3. merge_into(branch, target="dev"),
4. push(branch="dev") → diff_guard note,
5. move → completed, remove_worktree, DeliveryHooks.on_branch_pushed(source="engine"),
6. MemoryPort.on_task_finished.

A merge conflict is retried by sending the task back; after engine_max_merge_conflicts (3) the manager takes over (error_review). A failed push is ErrorKind.NETWORK → error_review.

4.5 Budget

Before any turn, _budget_reason checks the org's wallet. A wallet present with balance <= 0 pauses the task (Stage.PAUSED, status waiting_approval) with the Persian note BUDGET_PAUSED_MESSAGE — no turn is spent. A missing wallet means no budget was ever configured, and a fresh org must not be frozen out of its first task (so get_for_org, never ensure, and None is not a pause). An optional per-task ceiling (engine_max_task_credits > 0) reads projected spend from TaskSpendRepository.spend_for_task (a read-only join of the usage ledger to the task's runs).

4.6 Flow test — tests/test_engine_flow.py (new)

The end-to-end test: real repositories, real scheduler, real manager, real turn runner; only the model (scripted), the sandbox (scripted) and git (FakeRepoWorkspace) are fakes. The happy path walks manager_review → analysis → implementation → test → final_review → merge → completed one tick at a time and asserts the stage, the coarse status, the gate rows, the git calls (author, target, branch), the removal of the worktree, the delivery hook and the memory callback. Eleven tests cover: the happy path; git never flowing through the model; red tests → back → then green; a forbidden diff never reaching git; a tester failure; a TEST_PASS with no green tests; the analysis_only pipeline (no worktree ever); the budget pause; dependencies; a transient provider failure retried then handed to the manager; and a merge conflict.

4.7 Wiring — container_engine.build_scheduler, the worker, .env.example

• build_scheduler(session) assembles the runner, manager, gate runner, git workspace, sandbox, delivery hooks, wallet and per-task spend repos on the caller's session — the same rule as the bootstrap (container.py is frozen, so the engine wires itself in my file).
• sandbox_session_for reuses the active sandbox session of the task's project (_EngineSandboxSessions). The engine does not create sandboxes; when there is none, the tests gate goes red and sends the work back — never "merge untested because there was no container".
• workers/ai_runtime.py now drives both engines in one pass: the legacy run queue, then one Scheduler.tick(). A broken tick is recorded against the worker and never looks like a clean pass. __main__.py defaults the ai-runtime interval to engine_tick_seconds; the one-pass (cron) default is unchanged.
• .env.example gains the six new ENGINE_* settings.

Bug found and fixed by the tests

forbidden_paths normalised with path.lstrip("./"), which strips all leading . and / characters: .git/config became git/config and .github/workflows/ci.yml became github/..., so the diff guard would have silently allowed changes to git internals and CI configuration. It now strips a single leading ./ prefix. Caught by test_forbidden_paths_flag_git_internals_and_ci.

Other support changes

• ManagerOutcome.error_kind — the manager carries the runner's error vocabulary so the scheduler can tell a transient provider failure from a permanent one without re-classifying a string.
• TaskSpendRepository (repositories/engine.py) — spend_for_task, read-only.
• Scheduler._patch refreshes the caller's snapshot version after each write, so a second patch in the same step locks against the version the first write produced instead of silently losing the write to StaleStageError.
• tests/engine_fakes.py: RecordingMemory now records record and on_task_finished (it previously returned None and asserted nothing).

Tests

tests/test_engine_gates.py (new, 13 tests): pure helpers (diff_hash, forbidden_paths, code_task); the diff guard's four refusals; no-progress; the tests gate with and without a command; the green gate's waiver rule.

tests/test_engine_flow.py (new, 11 tests): see 4.6.

Together +24 tests, all through the real repositories.

Checks (PLAN-v2 §7.4)

cd apps/api && .venv/bin/python -m pytest -q   → 746 passed, 1 skipped
cd apps/api && .venv/bin/ruff check .          → All checks passed!
cd apps/api && .venv/bin/mypy app              → Success: no issues found in 160 source files

No web files were touched; no migration was needed (Phase 1's a2b3c4d5e6f2 already has the engine tables).

Deviations & notes

1. Concurrency is a selection cap, not 8 live turns. A tick runs at most one turn per task, sequentially. An AsyncSession is not concurrency-safe, so running several turns on one session in parallel would be a corruption bug, not a speed feature. engine_max_concurrent_turns / engine_max_turns_per_org therefore bound how many tasks are chosen, and multi-process safety comes from the optimistic advance. Recorded because it is a deliberate reading of 4.1.
2. The engine does not create sandbox workspaces. It reuses the active session of the task's project; creating one automatically is a product decision the plan does not make. Absent session → tests gate red → work sent back.
3. The commit author e-mail is fixed (platform@smart-org.local) and matches the delivery adapter; the name is the employee's, which is what a reviewer reads.
4. no_progress/diff verdicts are persisted on the task state (last_diff_hash, same_diff_count), so progress is judged across ticks and across worker processes, not within one process.

Open questions / CHANGE REQUESTs

• None new. The org automation policy (automation_enabled) is still behind ModelGrantPort as planned.

Next

Phase 5 — the web side: the engine board, the task page (live log, gates, diff), /workforce, /dashboard + /traces, src/test/engine-*.test.tsx, and the final workstream report.