Enabled configuration of remote APIs by moving model name parameter from main.ts to config.ts #8
Conversation
Moving models parameter from hardcoded in main.ts to config.ts this is rarely needed by local models but required to access most APIs
Moved model name parameter from main.ts to config.ts. This parameter needs to be configured for remote APIs that support completions, added note about model name and top-k value change sometimes required, Tested successfully using APIs from together.ai and anyscale.com, failed test using groq.com API because no completions provided only chat/completions.
|
@p-e-w hello, I'm afraid I'm not a dev and I've never actually suggested a pull request before, so apologies if my etiquette is off in any way. I'm a big fan of Arrows but I found that I couldn't use various APIs until I'd found out where the model parameter was identified, when I found it and configured to identify the intended model, I could suddenly use any API which supported completions. I tried out the above edits to simply move where the model parameter is identified onto the config page with the other parameters for convenience, as this is something I have to configure often, for example to use NeMo via the Openrouter API, because I have a very limited local llm capacity. I've tried to submit this suggested change here in the hopes that it will enable more people to access and enjoy using Arrows but I'm afraid I don't really know what I'm doing with git so I don't know if this request has reached you or if there's a good reason not to make the update. So this suggestion is offered with thanks from a user who's enjoying using arrows for their own writing practice. Thank you again for open sourcing Arrows, it's great! |
|
This has been brought up several times before and I have laid out my reason for not having the model name as a parameter. Namely, with the actual OpenAI API, the model can not be freely changed because the newer OpenAI models don't support the text completion API. I am currently testing several experimental changes on my local Arrows instance, and once I have integrated those, I plan to reconsider how the ability to select the model for alternative providers can best be reconciled with the constraints OpenAI places on that parameter. Incidentally, the |
Moved model name parameter from main.ts to config.ts.
This parameter is rarely needed for local APIs but needs to be configured for remote APIs that support completions, added note about format of model name and top-k value change sometimes required.
Tested successfully using APIs from together.ai and anyscale.com, failed test using groq.com API because completions aren't supported only chat/completions.