diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index f76e16c..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,74 +0,0 @@ -# Copykar Project Memory & Agent Guidelines - -## 1. Project Overview & Architecture -Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites. - -### Key Components: -- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel. -- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles. -- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**: - - Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`). - - Automatic fallback upon rate limits, errors, or timeouts. - - Multimodal Vision: Handles image analysis along with text. - - Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity. - - Semantic Deduplication: Evaluates topic tags and content similarity with previous posts. -- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs. -- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order. -- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management. -- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards. - ---- - -## 2. Interaction & Workflow Rules -- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`). -- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code. -- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code. -- **Git Workflow**: - - One logical change per commit. - - Stage files strictly by name (never `git add .`). - - Never commit `.env`, credentials, secrets, or temporary dumps. - - Never create or switch branches unless explicitly requested. - ---- - -## 3. Project Invariants & Domain Standards - -### 3.1 Categorization & Navigation (5 Primary Hubs) -The bot interface is strictly organized into 5 category hubs: -1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`). -2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs. -3. **`Bots` (کانال‌ها و ربات‌ها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`). -4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`). -5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs. - -### 3.2 Release Notes Delivery & Environment Configuration -- **Always use app `.env` file**: Always read bot credentials (`BOT_TOKEN`), admin IDs (`ADMIN_USER_IDS`), and configurations directly from `/projects/telegram-bots/copykar/.env`. -- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk. -- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit. -- **Direct Admin DMs Only**: Always dispatch release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`) using the `.env` configuration (and proxy `http://127.0.0.1:10809` if needed), never to any public or review channel. - -### 3.3 Target Channel Context History -- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts. -- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest). - -### 3.4 Multimodal Vision & AGY Bridge -- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`. -- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters. - -### 3.5 Button Permissions & Channel Callbacks -- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`. -- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages. - -### 3.6 Self-Contained Project Memory -- Always record, update, and maintain all rules, architecture notes, domain standards, and operational memories about this project inside its own folder (`AGENTS.md` and `GEMINI.md`). -- This ensures any developer or collaborator receiving this repository has immediate, self-contained access to the project's memory and guidelines. - -### 3.7 Persistent Data Storage Layout (`./data/`) -All service persistent data volumes are centralized within the project's `./data/` directory (git-ignored): -- `data/postgres/`: PostgreSQL database files. Uses `monitoring/postgres/libfakestat_musl.so` for Docker Desktop virtiofs permission compatibility. -- `data/redis/`: Redis RDB persistence and cache data. -- `data/app/`: Downloaded Telegram media and cached files. -- `data/sessions/`: Telethon SQLite session files for userbot and bot accounts. -- `data/prometheus/`: Prometheus metric time-series database. -- `data/grafana/`: Grafana dashboards and SQLite database. -- `data/pgadmin/`: pgAdmin configuration and query history. diff --git a/GEMINI.md b/GEMINI.md deleted file mode 100644 index f76e16c..0000000 --- a/GEMINI.md +++ /dev/null @@ -1,74 +0,0 @@ -# Copykar Project Memory & Agent Guidelines - -## 1. Project Overview & Architecture -Copykar is an automated content ingestion, AI rewriting, admin curation, and paced publishing fleet for Telegram channels and websites. - -### Key Components: -- **Collector Service (`services/collector.py`)**: Telethon Userbot client monitoring source Telegram channels, downloading media to `MEDIA_DIR`, extracting subjects/tags, performing semantic deduplication, and sending raw review cards to the admin review channel. -- **Website Collector Service (`services/website_collector.py` & `services/website_analyzer.py`)**: Automated HTTP crawler for web sources. Uses AI to analyze site DOM/API structures with user-specified custom extraction needs (`custom_instructions`), discovering data endpoints and extracting new articles. -- **AI Processing Pipeline (`core/llm.py` & `services/ai_processor.py`)**: - - Multi-provider resilient chain (OpenAI, Gemini, AGY CLI / local bridge on `host.docker.internal:8088`). - - Automatic fallback upon rate limits, errors, or timeouts. - - Multimodal Vision: Handles image analysis along with text. - - Target Channel Context Ingestion: Injects the last N published messages from the target channel into the AI prompt to preserve tone, style, and narrative continuity. - - Semantic Deduplication: Evaluates topic tags and content similarity with previous posts. -- **Admin Bot Panel (`services/admin_bot.py`)**: Telethon Bot account providing interactive management and instant review cards with 5 primary category hubs. -- **Publisher Service (`services/publisher.py`)**: Paced target delivery worker consuming queues with configurable intervals, sleep schedules, and FIFO/Random dispatch order. -- **Database & Persistence (`db/database.py`, `db/models.py`, `db/repository.py`)**: PostgreSQL with asyncpg, automatic migrations, and JSONB post tracking. Redis for queue management. -- **Observability (`core/metrics.py`)**: Prometheus metrics server on port 8008 + Grafana dashboards. - ---- - -## 2. Interaction & Workflow Rules -- **English Correction Rule**: Every user prompt must be corrected at the very top of each reply in the exact ASCII box format (`┌ ─ ┐ │ └ ┘`). -- **No AI Attribution**: Never mention AI, LLM, Gemini, Claude, Antigravity, or model names in commit messages, comments, PRs, or user-facing code. -- **Minimal Changes**: Smallest possible diff to achieve the goal. Do not refactor untouched code. -- **Git Workflow**: - - One logical change per commit. - - Stage files strictly by name (never `git add .`). - - Never commit `.env`, credentials, secrets, or temporary dumps. - - Never create or switch branches unless explicitly requested. - ---- - -## 3. Project Invariants & Domain Standards - -### 3.1 Categorization & Navigation (5 Primary Hubs) -The bot interface is strictly organized into 5 category hubs: -1. **`Copy` (مدیریت محتوا)**: Unreviewed posts queue (`/pending`), unified add source/target hub (`/add`), and channel topic categories (`/categories`). -2. **`AI` (تنظیمات هوش مصنوعی)**: Provider profiles, model selection, vision toggle, reasoning effort, fallback chain configuration, connectivity tests, and processing logs. -3. **`Bots` (کانال‌ها و ربات‌ها)**: Source Telegram channels (`/sources`), Target Telegram channels (`/targets`), Source websites (`/websites`), and collector userbot authentication (`/request_code`). -4. **`System` (مدیریت سیستم)**: Emergency fleet pause/resume (`/pause`, `/resume`), dynamic release notes (`/changes`), and comprehensive documentation (`/help`). -5. **`Monitor` (مانیتورینگ و آمار)**: Live queue & channel throughput statistics (`/stats`), unresolved error log review & bulk-fix (`/errors`), and Grafana time-range graphs. - -### 3.2 Release Notes Delivery & Environment Configuration -- **Always use app `.env` file**: Always read bot credentials (`BOT_TOKEN`), admin IDs (`ADMIN_USER_IDS`), and configurations directly from `/projects/telegram-bots/copykar/.env`. -- **Never save release notes to a static file**: Do not create or keep `RELEASE_NOTES.md` on disk. -- **Dynamic Generation**: Always generate release notes dynamically strictly describing the changes of the current commit. -- **Direct Admin DMs Only**: Always dispatch release notes via the Telegram bot directly to admin private DMs (`ADMIN_USER_IDS`) using the `.env` configuration (and proxy `http://127.0.0.1:10809` if needed), never to any public or review channel. - -### 3.3 Target Channel Context History -- Target channels support injecting recent published history (`context_message_count`) into AI rewriting prompts. -- History is queried via PostgreSQL JSONB array query against `published_to` matching `{"target_id": target.id}`, sorted chronologically (oldest to newest). - -### 3.4 Multimodal Vision & AGY Bridge -- When images are present, payload is structured with `{"type": "text", "text": ...}` and `{"type": "image_url", ...}`. -- The host-side AGY bridge (`agy_bridge.py` on port 8088) decodes image base64 into temporary files and executes the CLI with multimodal parameters. - -### 3.5 Button Permissions & Channel Callbacks -- Inline callbacks on review cards (`sel_trg:`, `rej:`, `del_msg:`) check `is_admin(event.sender_id)` against `ADMIN_USER_IDS`. -- The bot must have Administrator privileges in the review channel with permissions to edit and delete messages. - -### 3.6 Self-Contained Project Memory -- Always record, update, and maintain all rules, architecture notes, domain standards, and operational memories about this project inside its own folder (`AGENTS.md` and `GEMINI.md`). -- This ensures any developer or collaborator receiving this repository has immediate, self-contained access to the project's memory and guidelines. - -### 3.7 Persistent Data Storage Layout (`./data/`) -All service persistent data volumes are centralized within the project's `./data/` directory (git-ignored): -- `data/postgres/`: PostgreSQL database files. Uses `monitoring/postgres/libfakestat_musl.so` for Docker Desktop virtiofs permission compatibility. -- `data/redis/`: Redis RDB persistence and cache data. -- `data/app/`: Downloaded Telegram media and cached files. -- `data/sessions/`: Telethon SQLite session files for userbot and bot accounts. -- `data/prometheus/`: Prometheus metric time-series database. -- `data/grafana/`: Grafana dashboards and SQLite database. -- `data/pgadmin/`: pgAdmin configuration and query history.