make fullpagecontext a numeric setting

You now can define how many pages should be sent, regardless of the
number of matching chunks.
Pages are still identified via the chunks, so this number can't

make fullpagecontext a numeric setting

You now can define how many pages should be sent, regardless of the
number of matching chunks.
Pages are still identified via the chunks, so this number can't be
larger than the chunks.

show more ...

make ollama auth optional

move api endpoint init to AbstractModel

This should finally solve the chicken-egg issue. This also renames the
ollama url parameter from baseurl to apiurl and sets a sensible default

add option to always send full page context

introduce a "Generic" model

This is basically an OpenAI implementation, which then can be reused by
most other providers since they usually implement a somewhat compatible
API. The API URL can be se

introduce a "Generic" model

This is basically an OpenAI implementation, which then can be reused by
most other providers since they usually implement a somewhat compatible
API. The API URL can be set manually so it should be easy to add new
unsupported providers.

show more ...

mark some more settings that need reindexing

exclude page parts from indexing using a regex

Merge branch 'pr/19' into ollama

* pr/19:
Add missing field for "Ollama" model configuration

support the Google Gemini API

Add missing field for "Ollama" model configuration

added configurable custom prompt

This makes it easier to add to the existing internal prompt instead of
completely overwriting it using the localization mechanism.

added support for groq

Added support for reka.ai

make threshold configurable

separate the rephrasing model from the chat model

Rephrasing can be done with faster, simpler models as there is not much
reasoning needed.

added Voyage AI for embeddings

clean up of the config options

Emojis are used to make the different options easier to distinguish

Regular expressions to limit the indexed pages. Implements #5

Both regular expressions (when set) need to apply at the same time. Eg a
page MUST match the matchRegex and MUST NOT match the skipRegex

Regular expressions to limit the indexed pages. Implements #5

Both regular expressions (when set) need to apply at the same time. Eg a
page MUST match the matchRegex and MUST NOT match the skipRegex to be
applied.

The regular expressions are applied when running the `embed` command
line command. Pages no longer adhering to a changed regex setup will be
removed from the vector store.

For the sqlite storage it is recommended to re-cluster the index when
the reges are changed by running the `maintenance` command.

show more ...

Added Qdrant Storage

Added ChromaDB vector storage support

optionally search one language only

option to prefer a configured language instead of guessing

refactoring to make models selectable

This makes it much easier to add new models. Models can now be selected
via the configuration

allow to restrict access to certain users and groups

add optional conversation logging

This might be useful to debug and monitor the real world usage of the
bot and analyze abuse.