1# AGENTS.md 2 3This file provides guidance to LLM Code Agents when working with code in this repository. 4 5## Overview 6 7This is a DokuWiki plugin that enables AI-powered chat functionality using LLMs (Large Language Models) and RAG (Retrieval-Augmented Generation). The plugin indexes wiki pages as embeddings in a vector database and allows users to ask questions about wiki content. 8 9## Development Commands 10 11### Testing 12```bash 13../../../bin/plugin.php dev test 14``` 15 16### CLI Commands 17The plugin provides a CLI interface via `cli.php`: 18 19```bash 20# Get a list of available commands 21../../../bin/plugin.php aichat --help 22``` 23 24## Architecture 25 26### Core Components 27 28**helper.php (helper_plugin_aichat)** 29- Main entry point for plugin functionality 30- Manages model factory and configuration 31- Handles question answering with context retrieval 32- Prepares messages with chat history and token limits 33- Implements question rephrasing for better context search 34 35**Embeddings.php** 36- Manages the vector embeddings index 37- Splits pages into chunks using TextSplitter 38- Creates and retrieves embeddings via embedding models 39- Performs similarity searches through storage backends 40- Handles incremental indexing (only updates changed pages) 41 42**TextSplitter.php** 43- Splits text into token-sized chunks (configurable, typically ~1000 tokens) 44- Prefers sentence boundaries using Vanderlee\Sentence 45- Handles long sentences by splitting at word boundaries 46- Maintains overlap between chunks (MAX_OVERLAP_LEN = 200 tokens) for context preservation 47 48**ModelFactory.php** 49- Creates and caches model instances (chat, rephrase, embedding) 50- Loads model configurations from Model/*/models.json files 51- Supports multiple providers: OpenAI, Gemini, Anthropic, Mistral, Ollama, Groq, Reka, VoyageAI 52 53### Model System 54 55**Model/AbstractModel.php** 56- Base class for all LLM implementations 57- Handles API communication with retry logic (MAX_RETRIES = 3) 58- Tracks usage statistics (tokens, costs, time, requests) 59- Implements debug mode for API inspection 60- Uses DokuHTTPClient for HTTP requests 61 62**Model Interfaces** 63- `ChatInterface`: For conversational models (getAnswer method) 64- `EmbeddingInterface`: For embedding models (getEmbedding method, getDimensions method) 65- `ModelInterface`: Base interface with token limits and pricing info 66 67**Model Providers** 68Each provider has its own namespace under Model/: 69- OpenAI/, Gemini/, Anthropic/, Mistral/, Ollama/, Groq/, Reka/, VoyageAI/ 70- Each contains ChatModel.php and/or EmbeddingModel.php 71- Model info (token limits, pricing, dimensions) defined in models.json 72 73### Storage Backends 74 75**Storage/AbstractStorage.php** 76- Abstract base for vector storage implementations 77- Defines interface for chunk storage and similarity search 78 79**Available Implementations:** 80- SQLiteStorage: Local SQLite database 81- ChromaStorage: Chroma vector database 82- PineconeStorage: Pinecone cloud service 83- QdrantStorage: Qdrant vector database 84 85### Data Flow 86 871. **Indexing**: Pages → TextSplitter → Chunks → EmbeddingModel → Vector Storage 882. **Querying**: Question → EmbeddingModel → Vector → Storage.getSimilarChunks() → Filtered Chunks 893. **Chat**: Question + History + Context Chunks → ChatModel → Answer 90 91### Key Features 92 93**Question Rephrasing** 94- Converts follow-up questions into standalone questions using chat history 95- Controlled by `rephraseHistory` config (number of history entries to use) 96- Only applied when rephraseHistory > chatHistory to avoid redundancy 97 98**Context Management** 99- Chunks include breadcrumb trail (namespace hierarchy + page title) 100- Token counting uses tiktoken-php for accurate limits 101- Respects model's max input token length 102- Filters chunks by ACL permissions and similarity threshold 103 104**Language Support** 105- `preferUIlanguage` setting controls language behavior: 106 - LANG_AUTO_ALL: Auto-detect from question 107 - LANG_UI_ALL: Always use UI language 108 - LANG_UI_LIMITED: Use UI language and limit sources to that language 109 110### AJAX Integration 111 112**action.php** 113- Handles `AJAX_CALL_UNKNOWN` event for 'aichat' calls 114- Processes questions with chat history 115- Returns JSON with answer (as rendered Markdown), sources, and similarity scores 116- Implements access restrictions via helper->userMayAccess() 117- Optional logging of all interactions 118 119### Frontend 120- **script/**: JavaScript for UI integration 121- **syntax/**: DokuWiki syntax components 122- **renderer.php**: Custom renderer for AI chat output 123 124## Configuration 125 126Plugin configuration is in `conf/`: 127- **default.php**: Default config values 128- **metadata.php**: Config field definitions and validation 129 130Key settings: 131- Model selection: chatmodel, rephrasemodel, embedmodel 132- Storage: storage backend type 133- API keys: openai_apikey, gemini_apikey, etc. 134- Chunk settings: chunkSize, contextChunks, similarityThreshold 135- History: chatHistory, rephraseHistory 136- Access: restrict (user/group restrictions) 137- Indexing filters: skipRegex, matchRegex 138 139## Testing 140 141Tests are in `_test/` directory: 142- Extends DokuWikiTest base class 143- Uses @group plugin_aichat annotation 144 145## Important Implementation Notes 146 147- All token counting uses TikToken encoder for rough estimates 148- Chunk IDs are calculated as: pageID * 100 + chunk_sequence (pageIDs come from DokuWiki's internal search index) 149- Models are cached in ModelFactory to avoid re-initialization 150- API retries use exponential backoff (sleep for retry count seconds) 151- Breadcrumb trails provide context to AI without requiring full page content 152- Storage backends handle similarity search differently but provide unified interface 153- UTF-8 handling is critical for text splitting (uses dokuwiki\Utf8\PhpString) 154