Changes in Embeddings.php

80dbccf334d874a6ca34fb040db3a629869300bf - Fix chunksize calculation

Wed, 18 Mar 2026 11:59:36 +0000

Fix chunksize calculation List of files: /plugin/aichat/Embeddings.php

bdf0ac5445aa1ff823b70743fb995bd12a7fe360 - make fullpagecontext a numeric setting

Tue, 17 Mar 2026 10:42:54 +0000

make fullpagecontext a numeric settingYou now can define how many pages should be sent, regardless of thenumber of matching chunks.Pages are still identified via the chunks, so this number can't belarger than the chunks. List of files: /plugin/aichat/Embeddings.php

2d02fff5bb8245df792cb940eaffb67df2e9387c - avoid deleting non-existant chunks. fixes #46

Mon, 19 Jan 2026 15:24:37 +0000

avoid deleting non-existant chunks. fixes #46 List of files: /plugin/aichat/Embeddings.php

ae2d01b1d365015c724499895640641092b1a634 - Merge branch 'sentencesplit' into partner

Mon, 06 Oct 2025 20:02:47 +0000

Merge branch 'sentencesplit' into partner* sentencesplit: add tests for text splitting make overlap a class member for easier testing Agents: make clearer how to run tests added an AGENTS.md file for LLM based work split sentences by token, not bytes. handle UTF-8 move text splitting into it's own class Some enhancements on the subsentence splitting Squashed commit of the following: List of files: /plugin/aichat/Embeddings.php

072e009990858d649f31eceb61c1bc980d28f40c - move text splitting into it's own class

Mon, 06 Oct 2025 13:20:56 +0000

move text splitting into it's own class List of files: /plugin/aichat/Embeddings.php

3daef465a3ad57db8396fa8d6dee5f8a398b8c8f - Some enhancements on the subsentence splitting

Mon, 06 Oct 2025 12:58:47 +0000

Some enhancements on the subsentence splittingWhen a sentence is longer than a chunk, it should be split forcefully insmaller parts - these parts should NOT be the size of a full chunk sincewe still want to do some overlap with previous and following texts. Ichose to split into a quarter of a chunk.This also ensures that whitespace is kept for the split sentences,because they may be joined with follow up texts. List of files: /plugin/aichat/Embeddings.php

867b7752e6a464a7e454d06581773f5de49ac143 - Squashed commit of the following:

Mon, 06 Oct 2025 12:45:19 +0000

Squashed commit of the following:commit 4e0adf2a8d810e55db6d37ccc87c76d95ddcfd8dAuthor: Henry Date: Mon Feb 3 22:04:17 2025 +0100 Updated splitLongSentence()commit 9883844f1db6df9e11051c4c7b034e68baaca0beAuthor: Henry Date: Mon Feb 3 22:03:25 2025 +0100 Updated splitLongSentence()commit 6f737f6fe4da25fa438211d5c00605c2df9c81baAuthor: Henry Date: Mon Feb 3 21:43:16 2025 +0100 array_unshift($sentences, ...$this->splitLongSentence($sentence, $tiktok));commit 21966eab02f87f632e82ad0055f9bc2aadb92053Author: Henry Date: Mon Feb 3 21:23:40 2025 +0100 Updated splitIntoChunks method Push split sentences to the front of the queue with array_unshift($sentences, ...$this->splitLongSentence($sentence, $tiktok)); List of files: /plugin/aichat/Embeddings.php

9634d7345e88e8177bbba1e0ecb312352866df1d - add option to always send full page context

Wed, 21 May 2025 11:48:33 +0000

add option to always send full page context List of files: /plugin/aichat/Embeddings.php

7be8078ef9026e317a5c01f90a94183276bbbbd2 - allow models to have a zero token limit

Tue, 15 Apr 2025 13:15:03 +0000

allow models to have a zero token limitThis allows for configuring completely unknown models. For these modelsno token limit is known and we will simply do not apply any. Instead wetrust that the model will be either large enough to handle our input orat least throw useful error messages. List of files: /plugin/aichat/Embeddings.php

ed47fd87135eb149e159ef170f0ca8a6c136c9cc - new UI with option to chat about the current page

Thu, 27 Mar 2025 12:49:25 +0000

new UI with option to chat about the current page List of files: /plugin/aichat/Embeddings.php

aa6bbe75d15853a1d8cbdfee301c7e380f672a69 - added "similar" endpoint to the remote api

Wed, 12 Mar 2025 09:59:50 +0000

added "similar" endpoint to the remote api List of files: /plugin/aichat/Embeddings.php

c2f550810a36b17b39023fb61665f572a18e27b0 - show used query when doing similarity queries

Mon, 22 Jul 2024 09:41:02 +0000

show used query when doing similarity queries List of files: /plugin/aichat/Embeddings.php

661701ee9039525690adc9c537cc48f68a48903b - Use custom renderer when creating embeddings

Tue, 25 Jun 2024 17:56:01 +0000

Use custom renderer when creating embeddingsRendering makes plugin output available and and handles includes. Itmight also help with #15.The renderer uses markdown like output since all LLMs seem to be veryfamiliar with it's syntax. This might help them to understand thedocument structure better.This also adds a breadcrumb trail at the top of each chunk which mighthelp with contextulization as well. List of files: /plugin/aichat/Embeddings.php

303d0c59f1b12d14fa2d9b91127b31f070c16911 - gracefully handle render errors

Mon, 17 Jun 2024 16:06:48 +0000

gracefully handle render errorsplugins may act up during text rendering, this should not abort thewhole indexing. Instead we fall back to the page source List of files: /plugin/aichat/Embeddings.php

8c08cb3f6b0f30c35f378fd151abfb219b75b92e - auto style fixes

Wed, 27 Mar 2024 14:01:02 +0000

auto style fixes List of files: /plugin/aichat/Embeddings.php

ab1f8dde36106432cc0a6f320220da5fae6971fe - emit the INDEXER_PAGE_ADD event

Tue, 26 Mar 2024 10:21:11 +0000

emit the INDEXER_PAGE_ADD eventThis allows plugins that add data to the fulltext index to add the samedata to the embeddings. This improves embedding searches with structdata for example. List of files: /plugin/aichat/Embeddings.php

720bb43f9ac252f6e0b09e7b06804dec7c547a47 - make threshold configurable

Mon, 25 Mar 2024 13:49:07 +0000

make threshold configurable List of files: /plugin/aichat/Embeddings.php

2071dced6f96936ea7b9bf5dbe8a117eef598448 - automatic stylefixes

Thu, 21 Mar 2024 14:50:48 +0000

automatic stylefixes List of files: /plugin/aichat/Embeddings.php

5f71c9bbe31dfcc1db5ab9659debc4833c4ec6eb - small adjustments

Thu, 21 Mar 2024 13:06:02 +0000

small adjustments List of files: /plugin/aichat/Embeddings.php

c2b7a1f7fd0f6c6579c9ee46f0437ff89c2fc4b3 - various refactoring and introduction of a simulate command

Thu, 21 Mar 2024 11:50:49 +0000

various refactoring and introduction of a simulate commandThe new command makes it easier to run the same chat questions againstmultiple models and compare the results in a spreadsheet List of files: /plugin/aichat/Embeddings.php