Custom endpoints defined in librechat.yaml are added to the model selection even if the ENDPOINTS configuration does not include "custom"

[jegraham-amplify]

What happened?

Expected behavior

Per the documentation, the ENDPOINTS environment variable is meant to customize the available endpoints in LibreChat. If custom is not present in the ENDPOINTS environment variable, I would expect that custom endpoints should not be loaded into the model selection dropdown.

Actual behavior

If any custom endpoints are defined in the librechat.yaml file, they are loaded into the model selection dropdown even if custom is NOT present in the ENDPOINTS environment variable

Why is this a problem?

It is easy to make the argument that "if you defined the custom endpoint, obviously you must want to make it available". However, this does not account for the case of using an LLM gateway.

In my organization, we only want a few models to be available on production, and with specific settings. To achieve this, we can use the modelSpecs parameter in the librechat.yaml file.

We also use LiteLLM as a gateway. This means that we need to define a custom endpoint in the librechat.yaml file, so that the definitions in modelSpecs can access the models.

We do NOT want these endpoints to show outside of how we've defined them in modelSpecs. We only want them to be available in the specific way that we've defined them. Our environment file has ENDPOINTS=agents - we want to include agents in the model selection list, but we do NOT want to include the endpoint that we defined (remember, we only defined it so that we could use it with modelSpecs).

Technical details

api / server / services / Config / getEndpointsConfig.js > getEndpointsConfig - line 19 (at the time of writing this) loads the custom endpoints REGARDLESS of whether custom is included in the ENDPOINTS environment variable.

Suggested fix

Only load call loadConfigEndpoints if custom is present in the ENDPOINTS environment variable configuration.

Current code:

 1: const { CacheKeys, EModelEndpoint, orderEndpointsConfig } = require('librechat-data-provider');
...
18: const defaultEndpointsConfig = await loadDefaultEndpointsConfig(req);
19: const customConfigEndpoints = await loadConfigEndpoints(req);

Potential fix:

 1: const { CacheKeys, EModelEndpoint, getEnabledEndpoints, orderEndpointsConfig } = require('librechat-data-provider');
...
18: const defaultEndpointsConfig = await loadDefaultEndpointsConfig(req);
19: const enabledEndpoints = getEnabledEndpoints();
20: const customConfigEndpoints = {};
21: if (enabledEndpoints.includes(EmodelEndpoint.custom)) {
22:   customConfigEndpoints = await loadConfigEndpoints(req);
23: }

NOTE: While this does work, it results in getEnabledEndpoints() being called twice (it is already called once in loadDefaultEndpointsConfig). A more elegant solution may be to update loadDefaultEndpointsConfig to either take in the list of enabled endpoints, or perhaps to include custom in its returned values if custom is present in the ENDPOINTS environment variable.

Version Information

The last source commit we have in our mirror-pushed repository is commit c4f1da2, related to PR #6687. However, I have verified that the main branch of the LibreChat repository still contains the problematic code.

Steps to Reproduce

1. In your environment file, configure your ENDPOINTS variable with a value that does not include custom.
2. In your librechat.yaml file, define a custom endpoint.
3. Run LibreChat

What browsers are you seeing the problem on?

No response

Relevant log output

[Logs on page load]
Navigated to https://chat.amplify.com/c/76628434-604f-47a0-b479-7f4f30d7547a
10:34:39.771 vendor.CsAktuwn.js:19 [Violation] 'setTimeout' handler took 79ms
10:34:39.822 vendor.CsAktuwn.js:19 [Violation] 'setTimeout' handler took 50ms
10:34:40.100 vendor.CsAktuwn.js:19 [Violation] 'setTimeout' handler took 226ms
10:34:40.368 vendor.CsAktuwn.js:19 [Violation] 'setTimeout' handler took 141ms
10:34:40.498 index.BWeDnsZu.js:1 Parameters processed successfully


There are no additional logs when opening up the model selection dropdown.

Screenshots

.env file:

librechat.yaml (individual. model specs collapsed for the sake of a smaller screenshot):

UI:

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

I moved this to "Troubleshooting" because a possible solution has already been implemented.

You can utilize the new addedEndpoints field to limit the endpoint selections from the dropdown to whatever you set:

Config:

modelSpecs:
  enforce: false
  prioritize: true
  addedEndpoints: ["agents"]
  list:
  - name: "Sonnet 3.7"
    label: "Sonnet 3.7"
    iconURL: "anthropic"
    description: "Smart, Efficient Conversational AI"
    default: false
    preset:
      endpoint: "OpenRouter"
      model: "anthropic/claude-3.7-sonnet"
      modelLabel: "Sonnet 3.7"

Note, I am using OpenRouter for my model spec, and I have only allowed "Agents" to be another option.

More info: https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/model_specs#addedendpoints

---

Furthermore, you could limit the providers available to agents, in case you didn't want your custom endpoint to be used with Agents (in this case, OpenRouter is not included):

  agents:
    allowedProviders: ["openAI", "azureOpenAI", "anthropic", "google"]

You could limit the model selections for your endpoints by defining them explicitly.

More info: https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/agents#allowedproviders

[jegraham-amplify]

This does work for the very specific problem of not wanting to show the custom endpoint in the models list 👍