Skip to content

FEATURE: configure a default LLM model for all features #1496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from

Conversation

keegangeorge
Copy link
Member

@keegangeorge keegangeorge commented Jul 9, 2025

πŸ” Overview

This update adds a setting to the configure a default LLM model to be used for all features (unless overridden) and removes the custom: prefix when defining LLM models

βž• More information

For all features, the feature's assigned AI Persona's default_llm_id will be used, however, if it doesn't exist, the SiteSetting.ai_default_llm_model will be used instead.

This update also adds a few migrations which migrates the old model settings for Helper and Summarization into their respective personas. That is:

  • SiteSetting.ai_helper_model's existing value on sites will be copied into:

    AiPersona.where(id: [-18, -19, -20, -21, -22, -23, -24, -25, -26]).default_llm_id
  • SiteSetting.ai_summarization_model's existing value on sites will be copied into

    AiPersona.where(id: [-11, -12]).default_llm_id

Eventually, we will follow-up in a separate commit to remove the following settings:

SiteSetting.ai_helper_model
SiteSetting.ai_helper_image_caption_model
SiteSetting.ai_summarization_model
SiteSetting.ai_embeddings_semantic_search_hyde_model

By default SiteSetting.ai_default_llm_model will have default: "" in it's yaml file, however, this update also adds a migration which will ensure that the last created LLM model is used to fill the setting.

πŸ“Έ Screenshots

Screenshot 2025-07-09 at 15 27 57 Screenshot 2025-07-09 at 15 27 44

This update adds a setting to the configure a default LLM model to be used for all features (unless overridden).
@keegangeorge keegangeorge marked this pull request as draft July 9, 2025 22:33
@keegangeorge keegangeorge marked this pull request as ready for review July 15, 2025 18:17
Copy link
Member

@romanrizzi romanrizzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do ai_translation_model too?

@keegangeorge keegangeorge requested a review from romanrizzi July 17, 2025 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants