docs: add project-specific agent guidelines and memory documentation
This commit is contained in:
mamad
@@ -0,0 +1,59 @@
|
||||
# Copykar Project Memory & Agent Guidelines
|
||||
|
||||
## 1. Project Overview & Architecture
|
||||
Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites.
|
||||
|
||||
### Key Components:
|
||||
- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel.
|
||||
- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles.
|
||||
- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**:
|
||||
- Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`).
|
||||
- Automatic fallback upon rate limits, errors, or timeouts.
|
||||
- Multimodal Vision: Handles image analysis along with text.
|
||||
- Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity.
|
||||
- Semantic Deduplication: Evaluates topic tags and content similarity with previous posts.
|
||||
- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs.
|
||||
- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order.
|
||||
- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management.
|
||||
- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards.
|
||||
|
||||
---
|
||||
|
||||
## 2. Interaction & Workflow Rules
|
||||
- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`).
|
||||
- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code.
|
||||
- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code.
|
||||
- **Git Workflow**:
|
||||
- One logical change per commit.
|
||||
- Stage files strictly by name (never `git add .`).
|
||||
- Never commit `.env`, credentials, secrets, or temporary dumps.
|
||||
- Never create or switch branches unless explicitly requested.
|
||||
|
||||
---
|
||||
|
||||
## 3. Project Invariants & Domain Standards
|
||||
|
||||
### 3.1 Categorization & Navigation (5 Primary Hubs)
|
||||
The bot interface is strictly organized into 5 category hubs:
|
||||
1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`).
|
||||
2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs.
|
||||
3. **`Bots` (کانالها و رباتها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`).
|
||||
4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`).
|
||||
5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs.
|
||||
|
||||
### 3.2 Release Notes Delivery
|
||||
- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk.
|
||||
- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit.
|
||||
- **Direct Admin DMs**: Always send release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`), never to the review channel.
|
||||
|
||||
### 3.3 Target Channel Context History
|
||||
- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts.
|
||||
- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest).
|
||||
|
||||
### 3.4 Multimodal Vision & AGY Bridge
|
||||
- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`.
|
||||
- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters.
|
||||
|
||||
### 3.5 Button Permissions & Channel Callbacks
|
||||
- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`.
|
||||
- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Copykar Project Memory & Agent Guidelines
|
||||
|
||||
## 1. Project Overview & Architecture
|
||||
Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites.
|
||||
|
||||
### Key Components:
|
||||
- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel.
|
||||
- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles.
|
||||
- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**:
|
||||
- Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`).
|
||||
- Automatic fallback upon rate limits, errors, or timeouts.
|
||||
- Multimodal Vision: Handles image analysis along with text.
|
||||
- Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity.
|
||||
- Semantic Deduplication: Evaluates topic tags and content similarity with previous posts.
|
||||
- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs.
|
||||
- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order.
|
||||
- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management.
|
||||
- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards.
|
||||
|
||||
---
|
||||
|
||||
## 2. Interaction & Workflow Rules
|
||||
- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`).
|
||||
- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code.
|
||||
- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code.
|
||||
- **Git Workflow**:
|
||||
- One logical change per commit.
|
||||
- Stage files strictly by name (never `git add .`).
|
||||
- Never commit `.env`, credentials, secrets, or temporary dumps.
|
||||
- Never create or switch branches unless explicitly requested.
|
||||
|
||||
---
|
||||
|
||||
## 3. Project Invariants & Domain Standards
|
||||
|
||||
### 3.1 Categorization & Navigation (5 Primary Hubs)
|
||||
The bot interface is strictly organized into 5 category hubs:
|
||||
1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`).
|
||||
2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs.
|
||||
3. **`Bots` (کانالها و رباتها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`).
|
||||
4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`).
|
||||
5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs.
|
||||
|
||||
### 3.2 Release Notes Delivery
|
||||
- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk.
|
||||
- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit.
|
||||
- **Direct Admin DMs**: Always send release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`), never to the review channel.
|
||||
|
||||
### 3.3 Target Channel Context History
|
||||
- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts.
|
||||
- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest).
|
||||
|
||||
### 3.4 Multimodal Vision & AGY Bridge
|
||||
- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`.
|
||||
- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters.
|
||||
|
||||
### 3.5 Button Permissions & Channel Callbacks
|
||||
- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`.
|
||||
- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages.
|
||||
Reference in New Issue
Block a user