FEATURE: mentionable personas and random picker tool, context limits #466
Conversation
Random picker lets llms pick 1. Random numbers from min to max 2. An element in a string list The tool allows the LLM to make multiple choices at once (like roll a bunch of dice)
SamSaffron
changed the title
Ensure PMs use the LLM defined in the OP
(streaming is somewhat expensive, lets keep it on PMs for now)
the amount of context the llm gets for completions Very important for mentionables cause we often do not want to feed in the entire backlog
SamSaffron
changed the title
There was a problem hiding this comment.
Didn't see anything block-worthy 👍 Might be good to wait for @martin-brennan if he has time.
There was a problem hiding this comment.
From a UI perspective, these list items with "(not configured)" are a bit confusing. What happens if I choose one? Is there some validation to prevent me from doing it? Would be nice if select kit supports it (not sure if it does) to disable these items in the list at the least:
Also, when I toggled on Mentionable then didn't choose the "Default Language Model", I got a "Persona saved!" toast, but the mentionable toggle silently went back to false. IMO we should show a validation error if the user toggles on Mentionable but doesn't choose a "Default Language Model".
Also, even if I choose a language model, saving the page always toggles Mentionable back to false, so there is a bug here somewhere.
The actual mentioning and switching worked nicely :) The "let's welcome them to the community" message will not display with bot users, so if we change to using a bot that would be good. Also the list of avatars in the top left of the PM doesn't change until the page refreshes, not a huge deal:
| @@ -108,6 +108,16 @@ en: | |||
| ai_persona: | |||
| name: Name | |||
| description: Description | |||
| no_llm_selected: "No language model selected" | |||
| max_context_posts: "Max Context Posts" | |||
| max_context_posts_help: "The maximum number of posts to use as context for the AI when responding to a user. (empty for default)" | |||
There was a problem hiding this comment.
Could be helpful to mention the default is 40, the only problem then is that you have to keep the translation and the server const in sync :D
There was a problem hiding this comment.
We might be able to pass it down via the Site serializer and then use that to pass to the translation/the placeholder here:
There was a problem hiding this comment.
hmm tricky... yeah maybe this should be a site setting? I can see people who want to save money set it lower ... bot probably operates fine at 10 context and you would save a big pile of money, especially considering giant convos @tgxworld tends to have :)
There was a problem hiding this comment.
Yeah site setting could be fine as well, at least as the default, because then you can use that on the client too for the input and translation.
| end | ||
|
|
||
| post_types = [Post.types[:regular]] | ||
| post_types << Post.types[:whisper] if post.post_type == Post.types[:whisper] |
There was a problem hiding this comment.
| post_types << Post.types[:whisper] if post.post_type == Post.types[:whisper] | |
| post_types << Post.types[:whisper] if post.whisper? |
Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Martin Brennan <martin@discourse.org>
Co-authored-by: Martin Brennan <martin@discourse.org>
- fix, always use "bot" ids for auto created persona users - dev improve routes so they are ai-personas and not ai_personas
was a bug... all sorted |
Co-authored-by: Martin Brennan <martin@discourse.org>
| # find the first id smaller than FIRST_USER_ID that is not taken | ||
| id = nil | ||
|
|
||
| id = DB.query_single(<<~SQL, FIRST_PERSONA_USER_ID, FIRST_PERSONA_USER_ID - 200).first |
There was a problem hiding this comment.
I guess we don't worry about this for now...but what if > 200 persona users are made 🤔 Might be good to make a site-wide persona count limit to deal with this. Anyway, not holding up this current PR 👍
There was a problem hiding this comment.
oh there is a fallback afterwards just in case... we take minimum ... very unlikely to hit it, but it is handled.
|
|
||
| id = DB.query_single(<<~SQL, FIRST_PERSONA_USER_ID, FIRST_PERSONA_USER_ID - 200).first | ||
| WITH seq AS ( | ||
| SELECT generate_series(?, ?, -1) AS id |
This rather large PR adds support for the following: