feat(integration): prepaid billing, engine UI, credits console, direct tasks, v2 final
- Prepaid credit system: org wallet, debit on usage, chat gate, admin credit console - Engine UI: /manager, /tasks, /tasks/[id], /workforce, /dashboard, /traces pages - Direct task creation (HTTP POST /tasks, MCP tasks_create) at pending_manager_review - Pause/resume/cancel/answer actions in engine API and UI - Employee cap (7 per org default), review pipeline guard (active reviewer required) - Scheduler: wide window prevents org starvation; archived employees excluded from cap - Tenant isolation: project-id cross-org 404, engine never uses another tenant's sandbox - Order route fix: manager built with pipeline states + employee profiles - Contract extension: ModelGrantPort.get_org_policy with DefaultOrgPolicy - Engine event types in @agy/contracts (task.step, task.gate, run.log, run.tool, team.changed) - api-engine.ts typed client; api-credit.ts client - Alembic migration c4d5e6f7a8b9: prepaid credit + engine UI tables - 859 tests passed, 1 skipped; ruff/mypy/tsc/eslint/vitest all green Refs: PLAN-v2.md §9 (T1–T43), todo-v2-final.md
This commit is contained in:
mjalmoudy94
@@ -38,7 +38,7 @@ You implement: `WorkforceBootstrapPort` (wired in `container_engine.py`).
|
||||
|
||||
Create package `app/application/engine/` (and `app/domain/workforce/` additions where the logic is pure):
|
||||
|
||||
- [ ] 1.1 `domain/workforce/pipeline.py`:
|
||||
- [x] 1.1 `domain/workforce/pipeline.py`:
|
||||
- `Stage` enum: `pending_manager_review`, `pending_analysis`, `pending_implementation`, `pending_review`, `pending_test`, `pending_final_review`, `merge_request`, `completed`, plus side states `pending_user_decision`, `error_review`, `paused`, `cancelled`, `failed`.
|
||||
- `PIPELINES`: `plan_code_test` (default), `plan_code_review_test`, `code_test`, `analysis_only` (no worktree, no merge). Each is an ordered list of stages.
|
||||
- `TRANSITIONS: dict[Stage, frozenset[Stage]]` and `next_stage(pipeline, current, outcome)`.
|
||||
@@ -46,7 +46,7 @@ Create package `app/application/engine/` (and `app/domain/workforce/` additions
|
||||
- `resolve_pipeline(manager_choice, task_kind, app_default, has_active_reviewer)`: priority manager > kind > app default > `plan_code_test`. A review pipeline without an active reviewer falls back to `plan_code_test`.
|
||||
- `TASK_KINDS`: `feature`, `bugfix`, `refactor`, `docs`, `analysis` → default pipeline + risk.
|
||||
- `RoleKey.DEVELOPER` is our "programmer". Keep the existing enum values; do not rename them.
|
||||
- [ ] 1.2 `engine/protocol.py`: parser for `[[TAG attr="v" ...]]body[[/TAG]]` and self-closing `[[TAG attr="v"/]]`. Tolerant of whitespace; rejects unknown tags; returns typed dataclasses. Allowed tags per role/mode:
|
||||
- [x] 1.2 `engine/protocol.py`: parser for `[[TAG attr="v" ...]]body[[/TAG]]` and self-closing `[[TAG attr="v"/]]`. Tolerant of whitespace; rejects unknown tags; returns typed dataclasses. Allowed tags per role/mode:
|
||||
- manager/triage: `ASSIGN`, `REJECT`, `ASK_USER`, `KNOWLEDGE`
|
||||
- manager/final: `COMPLETE`, `REVISE`, `KNOWLEDGE`
|
||||
- manager/error: `RESOLVE_ERROR`, `CANCEL`, `KNOWLEDGE`
|
||||
@@ -57,20 +57,20 @@ Create package `app/application/engine/` (and `app/domain/workforce/` additions
|
||||
- tester: `TEST_PASS`, `TEST_FAIL`
|
||||
- tool tags (all working roles): `READ_FILE path`, `WRITE_FILE path` (body = content), `EDIT_FILE path` (body = `<<<OLD … === … NEW>>>` search/replace blocks), `LIST_DIR path`, `SEARCH pattern path`, `RUN cmd timeout`
|
||||
Disallowed tags are ignored and reported back to the model as an error line. Limits: max 3 `CREATE_TASK` per reply, max 3 team changes per turn, max 3 `KNOWLEDGE` per turn.
|
||||
- [ ] 1.3 Tables in `models_engine.py`:
|
||||
- [x] 1.3 Tables in `models_engine.py`:
|
||||
- `task_pipeline_state` (task_id PK/FK, pipeline, kind, risk, stage, cycle, max_cycles, branch, last_diff_hash, same_diff_count, retry_count, waiver_reason, parent_task_id, depends_on JSON, version int for optimistic updates, updated_at)
|
||||
- `employee_profiles` (employee_id PK/FK, ai_model_id, effort, roles CSV, hired_by `user|manager`, busy_task_id)
|
||||
- `engine_turns` (id, task_id, run_id, employee_id, role, mode, stage, prompt_chars, reply_excerpt ≤ 4000, tags JSON, tool_calls int, ok, error_kind, started_at, finished_at)
|
||||
- `gate_results` (id, task_id, gate `tests|diff_guard|no_progress|green|lint`, ok, detail ≤ 8000, created_at)
|
||||
Map the coarse `tasks.status` from the stage: new / running / waiting_approval / done / failed / cancelled. Existing APIs keep working.
|
||||
- [ ] 1.4 `engine/state.py`: `advance(task_id, expected_from, to)` with an optimistic check (`version`). It fails cleanly when another worker moved the task first. A failed or cancelled **dependency** moves the dependents to `pending_user_decision`, never leaving them stuck (agybot bug M1).
|
||||
- [ ] 1.5 Tests `tests/test_engine_protocol.py`, `tests/test_engine_pipeline.py`: every pipeline walks start→completed on the happy path; every illegal transition is refused; concurrent advance (two sessions) → exactly one wins; the dependency failure rule; tag parsing edge cases (nested brackets in a body, unknown tags, the per-role matrix, limits).
|
||||
- [x] 1.4 `engine/state.py`: `advance(task_id, expected_from, to)` with an optimistic check (`version`). It fails cleanly when another worker moved the task first. A failed or cancelled **dependency** moves the dependents to `pending_user_decision`, never leaving them stuck (agybot bug M1).
|
||||
- [x] 1.5 Tests `tests/test_engine_protocol.py`, `tests/test_engine_pipeline.py`: every pipeline walks start→completed on the happy path; every illegal transition is refused; concurrent advance (two sessions) → exactly one wins; the dependency failure rule; tag parsing edge cases (nested brackets in a body, unknown tags, the per-role matrix, limits).
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Agent turn runner (one employee, one turn, real tools in the sandbox)
|
||||
|
||||
- [ ] 2.1 `engine/prompt.py`, a prompt builder with a **stable prefix** (cache-friendly), in this order:
|
||||
- [x] 2.1 `engine/prompt.py`, a prompt builder with a **stable prefix** (cache-friendly), in this order:
|
||||
1. platform rules (short, yours)
|
||||
2. role protocol (allowed tags + format examples)
|
||||
3. toolkit description
|
||||
@@ -79,7 +79,7 @@ Create package `app/application/engine/` (and `app/domain/workforce/` additions
|
||||
6. task brief (≤ 1,200 chars) + stage-specific context (the planner's spec for the developer; the **diff only** for the reviewer; test output for the tester)
|
||||
7. the conversation so far in this turn (tool results)
|
||||
Hard cap on total characters; truncate oldest tool results first.
|
||||
- [ ] 2.2 `engine/tools.py`: executes tool tags **inside the sandbox** through `SandboxProviderPort`, rooted at `WorktreeRef.sandbox_path`.
|
||||
- [x] 2.2 `engine/tools.py`: executes tool tags **inside the sandbox** through `SandboxProviderPort`, rooted at `WorktreeRef.sandbox_path`.
|
||||
- Path safety: normalise; reject absolute paths outside the root, `..` escapes and anything under `.git`.
|
||||
- `RUN` has a timeout (default 120s, max 600s) and output capped (last 8,000 chars). The command is passed to the sandbox as-is (the sandbox is the security boundary), but `git push`, `git remote` and `git config` are refused with a message ("the platform handles git").
|
||||
- `EDIT_FILE` applies search/replace blocks and fails with a clear message when OLD is not found exactly once.
|
||||
@@ -89,20 +89,20 @@ Create package `app/application/engine/` (and `app/domain/workforce/` additions
|
||||
- Model = the employee's `ai_model_id`, which must be granted (`ModelGrantPort.is_granted`; if not, fail with `error_kind=auth` and tell the manager).
|
||||
- Error kinds `rate_limit`, `unavailable`, `network`, `timeout` are **transient** (retry later, up to 2 retries); `auth`, `model`, `protocol` are not.
|
||||
- Writes `engine_turns`, `run_steps` and usage events, and publishes `run.log` / `run.tool` / `run.status` (batched, PLAN-v2 §5).
|
||||
- Cancellation: `cancel_run` must stop the loop between steps.
|
||||
- [ ] 2.4 Tests `tests/test_engine_runner.py` with the scripted fake model + `FakeRepoWorkspace` + a fake sandbox (dict filesystem + scripted command results; put it in `tests/engine_fakes.py`). Cover: the write → run → report loop; path escape refused; `git push` refused; the tool-call cap; timeout; transient vs fatal errors; usage events counted per model call; an ungranted model refused; no secret from settings ever appears in a prompt (assert on the fake model's recorded prompts).
|
||||
- Cancellation: `cancel_run` must stop the loop between steps. — open: run.status is not published for engine runs (engine/scheduler.py sets the Run row's status without an event); run.log and run.tool are published, and stage moves publish task.step instead.
|
||||
- [x] 2.4 Tests `tests/test_engine_runner.py` with the scripted fake model + `FakeRepoWorkspace` + a fake sandbox (dict filesystem + scripted command results; put it in `tests/engine_fakes.py`). Cover: the write → run → report loop; path escape refused; `git push` refused; the tool-call cap; timeout; transient vs fatal errors; usage events counted per model call; an ungranted model refused; no secret from settings ever appears in a prompt (assert on the fake model's recorded prompts).
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Manager AI (orders → tasks → team)
|
||||
|
||||
- [ ] 3.1 Manager chat: rewrite `post_user_message` so the manager replies **and** may emit `CREATE_TASK` (title, kind, priority, depends_on, optional pipeline), `PAUSE_TASK` / `RESUME_TASK` / `CANCEL_TASK`, and `HIRE` / `UPDATE_EMPLOYEE` / `FIRE`. Tasks created this way get a `task_pipeline_state`, stage `pending_manager_review`, and are linked to the thread. The manager's prompt must ask for **atomic tasks that do not touch the same files**. Count chat usage.
|
||||
- [ ] 3.2 Team management: `HIRE role= tier= name=` picks a granted model matching the tier/capabilities (`ModelGrantPort.granted_models`). Respect max employees (setting + later org policy), max 3 changes per turn, and "the last manager cannot be fired". Publish `team.changed`. Users can still hire and fire from the UI (existing endpoints) but only choose **role + tier**, never a model.
|
||||
- [ ] 3.3 Triage turn (manager, mode triage): decide pipeline + assignee (`ASSIGN employee= pipeline=`), or `REJECT`, or `ASK_USER` (→ `pending_user_decision`, message posted in the thread). Default employee selection: least busy active employee with the needed role; `ASSIGN employee="name"` uses fuzzy name matching inside the org only.
|
||||
- [ ] 3.4 Final turn (manager, mode final): sees the plan, a diff summary, gate results and the test output → `COMPLETE` (→ `merge_request`) or `REVISE` (→ back to implementation, cycle+1). The error turn (mode error) handles failed gates and errors → `RESOLVE_ERROR` (retry with a hint) or `CANCEL`.
|
||||
- [ ] 3.5 `KNOWLEDGE` tags → `MemoryPort.record(kind=fact|rule, source=manager)`.
|
||||
- [ ] 3.6 `WorkforceBootstrapPort.bootstrap_team(org, project)`: creates one manager + one planner + one developer + one tester on the strongest/balanced/fast granted tiers (whatever is granted; if nothing is granted, create the employees "inactive" and post a system message "منتظر تخصیص مدل توسط مالک سامانه"). Idempotent. Wire it in `container_engine.py`.
|
||||
- [ ] 3.7 Tests `tests/test_engine_manager.py`: an order → ≤ 3 tasks with deps; HIRE limits; FIRE the last manager refused; ASK_USER path; idempotent bootstrap; cross-org isolation (the manager of org A can never assign an employee of org B, even by exact name).
|
||||
- [x] 3.1 Manager chat: rewrite `post_user_message` so the manager replies **and** may emit `CREATE_TASK` (title, kind, priority, depends_on, optional pipeline), `PAUSE_TASK` / `RESUME_TASK` / `CANCEL_TASK`, and `HIRE` / `UPDATE_EMPLOYEE` / `FIRE`. Tasks created this way get a `task_pipeline_state`, stage `pending_manager_review`, and are linked to the thread. The manager's prompt must ask for **atomic tasks that do not touch the same files**. Count chat usage.
|
||||
- [x] 3.2 Team management: `HIRE role= tier= name=` picks a granted model matching the tier/capabilities (`ModelGrantPort.granted_models`). Respect max employees (setting + later org policy), max 3 changes per turn, and "the last manager cannot be fired". Publish `team.changed`. Users can still hire and fire from the UI (existing endpoints) but only choose **role + tier**, never a model.
|
||||
- [x] 3.3 Triage turn (manager, mode triage): decide pipeline + assignee (`ASSIGN employee= pipeline=`), or `REJECT`, or `ASK_USER` (→ `pending_user_decision`, message posted in the thread). Default employee selection: least busy active employee with the needed role; `ASSIGN employee="name"` uses fuzzy name matching inside the org only.
|
||||
- [x] 3.4 Final turn (manager, mode final): sees the plan, a diff summary, gate results and the test output → `COMPLETE` (→ `merge_request`) or `REVISE` (→ back to implementation, cycle+1). The error turn (mode error) handles failed gates and errors → `RESOLVE_ERROR` (retry with a hint) or `CANCEL`.
|
||||
- [x] 3.5 `KNOWLEDGE` tags → `MemoryPort.record(kind=fact|rule, source=manager)`.
|
||||
- [x] 3.6 `WorkforceBootstrapPort.bootstrap_team(org, project)`: creates one manager + one planner + one developer + one tester on the strongest/balanced/fast granted tiers (whatever is granted; if nothing is granted, create the employees "inactive" and post a system message "منتظر تخصیص مدل توسط مالک سامانه"). Idempotent. Wire it in `container_engine.py`.
|
||||
- [x] 3.7 Tests `tests/test_engine_manager.py`: an order → ≤ 3 tasks with deps; HIRE limits; FIRE the last manager refused; ASK_USER path; idempotent bootstrap; cross-org isolation (the manager of org A can never assign an employee of org B, even by exact name).
|
||||
|
||||
---
|
||||
|
||||
@@ -112,30 +112,30 @@ Create package `app/application/engine/` (and `app/domain/workforce/` additions
|
||||
- finds tasks whose stage needs a turn and whose deps are completed, and a free employee with the role (one task per employee at a time)
|
||||
- honours org automation (if Agent 1's policy says `automation_enabled=false`, tasks wait in `pending_manager_review`; read it through `ModelGrantPort` when the CHANGE REQUEST lands, until then assume enabled)
|
||||
- runs turns concurrently with a per-org cap (3) and a global cap (setting)
|
||||
`workers/ai_runtime.py` calls `tick()` in its loop every `ENGINE_TICK_SECONDS` (keep the one-pass mode for cron). Safe with two worker processes thanks to the optimistic `advance`.
|
||||
- [ ] 4.2 Worktree lifecycle: on the first developer/planner turn, `RepoWorkspacePort.create_task_worktree(project, task, base="dev")`; the branch is `task/<task_id>`. `analysis_only` gets none. The worktree is removed after completion, cancellation or failure.
|
||||
- [ ] 4.3 Gates after `WORK_REPORT` (each writes a `gate_results` row and publishes `task.gate`):
|
||||
`workers/ai_runtime.py` calls `tick()` in its loop every `ENGINE_TICK_SECONDS` (keep the one-pass mode for cron). Safe with two worker processes thanks to the optimistic `advance`. — open: turns run one after another in tick() (a plain loop over the selected states); the per-org and global caps only bound the selection, there is no concurrent execution.
|
||||
- [x] 4.2 Worktree lifecycle: on the first developer/planner turn, `RepoWorkspacePort.create_task_worktree(project, task, base="dev")`; the branch is `task/<task_id>`. `analysis_only` gets none. The worktree is removed after completion, cancellation or failure.
|
||||
- [x] 4.3 Gates after `WORK_REPORT` (each writes a `gate_results` row and publishes `task.gate`):
|
||||
- **diff guard**: `RepoWorkspacePort.diff`. Fail on changes under `.git`, CI files (`.github/`, `.gitlab-ci.yml`, `.gitea/`), symlinks pointing outside the repo, or files > 1 MB. An empty diff for a code task also fails.
|
||||
- **no-progress**: hash the diff; 2 identical hashes in a row → `error_review`.
|
||||
- **tests**: run the app profile's `test_command` in the sandbox (timeout 600s), store the tail of the output. Red → back to the developer with the failure tail (one escalation to a stronger granted tier on the 2nd red), counting cycles. Green → tester stage.
|
||||
- **green gate**: the tester's `TEST_PASS` is accepted only if the tests gate is green, OR `TEST_PASS waive="reason"` with a non-empty reason (stored in `waiver_reason`, shown in UI).
|
||||
- `max_cycles` reached → `error_review`.
|
||||
- [ ] 4.4 Commit + merge + push on `merge_request`:
|
||||
- [x] 4.4 Commit + merge + push on `merge_request`:
|
||||
- `commit` (message: `task/<id>: <title>`, author = employee display name + platform email), then `sync_with_base`, then re-run the tests if base changed (a semantic conflict is only caught by retests).
|
||||
- Then `merge_into(target="dev")` and `push(branch="dev")`.
|
||||
- A conflict → back to the developer with the conflicting files (max 3 times, then `error_review`).
|
||||
- On success: stage `completed`, `DeliveryHooks.on_branch_pushed(org, project, "dev", sha, source="engine")`, `MemoryPort.on_task_finished(TaskOutcome)` (never let its failure fail the task).
|
||||
- [ ] 4.5 Budget: before each turn, check the org credit wallet (existing billing service public functions; read, do not edit) and the per-task cap. Out of budget → `paused` with a Persian reason.
|
||||
- [ ] 4.6 Tests `tests/test_engine_flow.py`, the end-to-end engine with all fakes. The scripted model plays manager → planner → developer → tester → manager; the fake sandbox makes tests red once and then green. Assert the stage sequence, gate rows, cycles, the commit/merge/push calls on `FakeRepoWorkspace`, `on_branch_pushed` called once with the right sha, `on_task_finished` called once, and usage events. Plus the failure paths: no-progress, max cycles, merge conflict ×3, cancel mid-turn, a dependency failure.
|
||||
- [x] 4.5 Budget: before each turn, check the org credit wallet (existing billing service public functions; read, do not edit) and the per-task cap. Out of budget → `paused` with a Persian reason.
|
||||
- [ ] 4.6 Tests `tests/test_engine_flow.py`, the end-to-end engine with all fakes. The scripted model plays manager → planner → developer → tester → manager; the fake sandbox makes tests red once and then green. Assert the stage sequence, gate rows, cycles, the commit/merge/push calls on `FakeRepoWorkspace`, `on_branch_pushed` called once with the right sha, `on_task_finished` called once, and usage events. Plus the failure paths: no-progress, max cycles, merge conflict ×3, cancel mid-turn, a dependency failure. — open: tests/test_engine_flow.py has no max-cycles → error_review test and only one merge conflict (retry), not three; the blocking no-progress case, cancel mid-turn and the dependency-failure rule are asserted in test_engine_gates.py, test_engine_runner.py and test_engine_pipeline.py.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — UI (orders, live task view, team)
|
||||
|
||||
- [ ] 5.1 `lib/api-engine.ts`: typed client functions for everything new (task detail with pipeline state, turns, gates, team with tier, hire/fire with role+tier).
|
||||
- [ ] 5.2 `/manager`: the chat shows task cards created by the manager inline (title, stage badge, link), live via `org:` + `thread:` channels; an "در انتظار تصمیم شما" banner for `pending_user_decision` with a reply box.
|
||||
- [ ] 5.3 `/tasks`: a board grouped by stage (columns: بررسی مدیر، تحلیل، پیادهسازی، بازبینی، تست، ادغام، انجامشده، نیازمند تصمیم) with a live update; filters; pause/resume/cancel (with confirm).
|
||||
- [ ] 5.4 `/tasks/[id]`:
|
||||
- [x] 5.1 `lib/api-engine.ts`: typed client functions for everything new (task detail with pipeline state, turns, gates, team with tier, hire/fire with role+tier).
|
||||
- [x] 5.2 `/manager`: the chat shows task cards created by the manager inline (title, stage badge, link), live via `org:` + `thread:` channels; an "در انتظار تصمیم شما" banner for `pending_user_decision` with a reply box.
|
||||
- [x] 5.3 `/tasks`: a board grouped by stage (columns: بررسی مدیر، تحلیل، پیادهسازی، بازبینی، تست، ادغام، انجامشده، نیازمند تصمیم) with a live update; filters; pause/resume/cancel (with confirm).
|
||||
- [x] 5.4 `/tasks/[id]`:
|
||||
- a pipeline stepper (stages of its pipeline, current one highlighted)
|
||||
- the turn timeline (who, role, when, outcome)
|
||||
- a **live log** panel (subscribe to `run:<id>`, auto-scroll, pause-scroll)
|
||||
@@ -143,9 +143,9 @@ Create package `app/application/engine/` (and `app/domain/workforce/` additions
|
||||
- a diff viewer (read-only, from the latest diff)
|
||||
- the waiver reason when present
|
||||
- a link to the pipeline run in `/app` once pushed (just a link with the sha; that page is Agent 3's)
|
||||
- [ ] 5.5 `/workforce`: employees with role, tier, status, current task; hire (role + tier) and fire with confirm. **No model names anywhere** for non-owners (test it).
|
||||
- [ ] 5.6 `/dashboard` + `/traces`: show active tasks, a stage distribution and recent completions; keep the existing widgets working.
|
||||
- [ ] 5.7 Web tests `src/test/engine-*.test.tsx`: board grouping, live log append and batching, no model name rendered (render with a fixture that contains a model name and assert it is absent).
|
||||
- [x] 5.5 `/workforce`: employees with role, tier, status, current task; hire (role + tier) and fire with confirm. **No model names anywhere** for non-owners (test it).
|
||||
- [x] 5.6 `/dashboard` + `/traces`: show active tasks, a stage distribution and recent completions; keep the existing widgets working.
|
||||
- [x] 5.7 Web tests `src/test/engine-*.test.tsx`: board grouping, live log append and batching, no model name rendered (render with a fixture that contains a model name and assert it is absent).
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user