-
-
Notifications
You must be signed in to change notification settings - Fork 124
[Platform] Add new OpenRouter models, ModelApiCatalog and openrouter/auto + presets #915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Platform] Add new OpenRouter models, ModelApiCatalog and openrouter/auto + presets #915
Conversation
|
Or do we prefer to integrate an external model selection directly into the catalog (like here: https://github.com/symfony/ai/pull/910/files )? I try to avoid external API calls in the model catalog... :-/ |
|
Can we move it to the platform in bin/ folder? |
should be even part of the OpenRouter bridge i'd say |
|
we'd need another solution in the long run here - i would prefer some kind of catalog that is dynamic with a cache. updating with the code base doesn't really solve the issue here - even if we'd make it part of the pipeline to do it automatically on release or something. the change frequency requires decoupling lib updates and catalog updates. not scope of this PR maybe, but we'll face that issue over and over again - with multiple bridges. |
yeah, thats what I meant |
|
We could have libs just for the models getting regular automated updates, but that's future :-) |
|
OK.... thanks for your feedback @OskarStark & @chr-hertel . how could I adapt the PR for this future need? My thoughts on this: I will drop the example script, I will drop the static model list in the ModelCatalog, move the fetch logic for the models and embeddings to the ModelCatalog and add some kind of PSR-6 compatible cache logic to keep the requests to the API endpoint low. I will rework the PR today... |
|
you can have a look at the ollama api catalog. next step would be to add proper caching to those api catalogs |
|
So... I created a ModelApiCatalog that fetch the models and embeddings via API (incl. capabilities). The default catalog is still the default ModelCatalog that contains only the Embeddings (because they are created with another Model class) but create the models "on the fly" without capabilities (because they are not used in the ModelClient/ResultConverter). I also switch from json_decode functions to Serializer->JsonDecoder. There is no cache currently, but I think we should create some general interface (or even abstract integration incl. cache logic) for "remote fetched model information", because several bridges need this (Ollama, AiMlApi, Albert etc.) and every Bridge integrated this in another way currently... |
|
I will check the broken pipelines shortly... |
chr-hertel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one thing open for me in the ModelApiCatalog before merging it:
the constructor cannot to http calls - that's too much of a workload for a new => i know it's a bit annoying with that abstraction, but we need to shift that to a later point in time => getters maybe?
|
Pleas update PR title + body, thanks |
Done. Move the loading of the model list to getModel/getModels incl. a flag to avoid double loading of the list.
Also done. |
chr-hertel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's goooo! thanks @lochmueller 🙏
b536116 to
435f5f7
Compare
|
Thank you @lochmueller. |
Features...