-
-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flex conf #15
Flex conf #15
Conversation
I was as well thinking about the limitation of this approach. |
notebook/use_guide.clj
Outdated
@@ -91,7 +91,9 @@ Playwright: This is a synopsis for the above play: | |||
(def synopsis | |||
(gen/complete-template | |||
synopsis-template | |||
{:title "Mr. X" :genre "crime"})) | |||
{:title "Mr. X" :genre "crime"} | |||
{:api-key "sk-xxxxx"} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a usage example of the complete-template function.
The user can the "code" whatever he want to get the key, bosquet does not need to prescribe this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One alternative would be to allow to "configure" any "tag" in this way, so from a user perspective:
(gen/complete-template
synopsis-template
{:title "Mr. X" :genre "crime"}
{:llm-generate
{:api-key "sk-xxxxx"}}
this would allow to keep the same structure in case of more tags
(but still assuming that all usages of the same tag in a multi-tag complex prompt get configured in the same way.)
Basically saying that all llm-generate
tags in a complex prompt need to use the same model. (at least when I want to externally configure it)
By "hardcoding" api-keys inside prompt definition this could still be done differently.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A completely different way to achieve the same (external configuration of models), could be by using "global vars" inside a prompt definition, maye in this way:
(def synopsis-template
"You are a playwright. Given the play's title and it's genre
it is your job to write synopsis for that play.
Title: {{title}}
Genre: {{genre}}
Playwright: This is a synopsis for the above play:
{% llm-generate model=text-davinci-003
var-name=play
api-key={{my-global-var-with-the-api-key}}
%}")
I think Selmer supports this already
Disadvantage here: The prompt definition would decide which models opts to use, which might be different "per model". So the prompt definitions are less model agnostic.
@@ -22,7 +25,7 @@ | |||
and set it as required inputs for the resolver. | |||
For the output check if themplate is producing generated content | |||
anf if so add a key for it into the output" | |||
[the-key template] | |||
[the-key template model-opts] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the new map of model-opts which will be merged with the model opts defined in template definition
In practice these are "llm-generate-opts" as used for any "llm-generate" tag.
I am new to Selmer, so maybe there is already all we need in Selmer itself.
|
I am wondering if and how we should be able to configure externally both calls to completion see here: differently (so using a different model, different key and so on) without specifying this inside the prompt definition. |
Probably a convention to "configure" nested llm-generate tags in the form of (gen/complete
template
{:title "Mr. X" :genre "crime"}
{"review.llm-generate"
{:impl :openai
:api-key "sk-xxxxx"}
"synopsis.llm-generate"
{:impl :azure
:api-key "azure key"}} so "prompt-map-key.tag-name" to inject options into the llm-generate call at a certain place in the template (and merge with ev. existing options) could work. This is in some way what I would like to achieve.. use different models (and their configuration) at different places of a prompt template, without need to configure this inside the prompt template. |
Maybe we want as well address the case of the same tag multiple times in a template, by allow to index: (gen/complete
template
{:title "Mr. X" :genre "crime"}
{"review.llm-generate.1"
{:impl :openai
:api-key "sk-xxxxx"}
"review.llm-generate.2"
{:impl :azure
:api-key "azure key"}} |
I am know quite happy with the PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good update! Nice separation of concerns for who and where manages config
In this PR I "moved" the configuration of the model "up" to the function "gen/copmplete-template" and "complete".
(so it merges the opts from inside the prompt definition and the external opts), external overrides
This has in my view 2 advantages:
and it can be overriden from the prompt definition user (without touching teh prompt definition)
(this should make the prompt definition completely model agnostic, so allow to use the same prompt definition with any model
I fixed as well the Azure OpenAI issue. (#9 ) Know the code works again with OpenAI and Azure OpenAI, provided that the correct model options are given to "complete-template" (or a specified inside the template)