[Roadmap] Custom OPEN AI Endpoint

[gabelul]

Allows users to call your custom endpoint in the OpenAI ChatCompletion format. I saw someone mentioning litellm

[enricoros]

@gabelul Is it possible already by going to OpenAI > Advanced Options?

[gabelul]

@enricoros, that's what I thought at first, but I tried with the same settings that work with others, and it didn't work

[enricoros]

I need some more details to reproduce this. This is a very frequent use case, and we have no reports of malfunction.

Can you attach a screenshot that shows the error or walk through the reproduction steps?

[enricoros]

Hi @gabelul please provide more details around the use case or the issue - it's not clear to me. Otherwise I'll have to close the request.

[gabelul]

Hey @enricoros, it does seem to work with some of the APIs that way. Not sure how can I send you a PM with the ones that don't so you can take a look ( they are private API )

[enricoros]

You can join discord and send me a pm. And attach screenshots of what's not working, maybe I can improve the error messages.

[shiribailem]

Dropping this here because I think it's related: ideally there would be functionality to have multiple instances of the OpenAI interface, just because someone might want to use both OpenAI directly and one or more systems that share it's API.

[shiribailem]

I'd imagine putting it in as a separate "OpenAI Compatible" model feature that can be added multiple times (or if that's not worth the effort, maybe just copy it 5 times as "OpenAI Compatible (1)" through "OpenAI Compatible (5)"

[enricoros]

@shiribailem You can do this today by changing this 1 line of code: https://github.com/enricoros/big-agi/blob/main/src%2Fmodules%2Fllms%2Fvendors%2Fopenai%2Fopenai.vendor.ts#L42

It's set to a single instance right now just to simplify the UX, but definitely you may create as many OpenAI compatible sources as you'd like.

[enricoros]

@shiribailem added the option for 1-5 OpenAI compatible endpoints (see below).
@shiribailem please let me know what is not working as expected in the OpenAI compatibility (which should be okay).

Screenshot1: adding OpenAI 4

Screenshot 2: