NotFoundError: 404 status code (no body) error, when hosted inference URL is used inside OpenAI

[BakingBrains]

const model = new OpenAI({
    temperature: 0.9,
    openAIApiKey: "xyz",
    configuration: {
      baseURL: "http://localhost:PORT/v1",
    },
});


const res = await model.call(
  "Tell me a story about a boy"
);
console.log({ res });```


**ERROR facing**

```NotFoundError: 404 status code (no body)
    at APIError.generate (file:///D:/pycharmprojects/API_SERVICES/DUMP/node_modules/openai/error.mjs:49:20)
    at OpenAI.makeStatusError (file:///D:/pycharmprojects/API_SERVICES/DUMP/node_modules/openai/core.mjs:256:25)
    at OpenAI.makeRequest (file:///D:/pycharmprojects/API_SERVICES/DUMP/node_modules/openai/core.mjs:299:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async file:///D:/pycharmprojects/API_SERVICES/DUMP/node_modules/@langchain/openai/dist/llms.js:472:29
    at async RetryOperation._fn (D:\pycharmprojects\API_SERVICES\DUMP\node_modules\p-retry\index.js:50:12)```


Any suggestions here?

Thank you

[dosubot]

🤖

Based on the error message you've shared, it seems like the server is unable to find the resource you're requesting. This could be due to an incorrect endpoint or the model you're trying to use is not available.

In your code, you're trying to use a baseURL of "http://localhost:PORT/v1". Please ensure that you have a server running on this URL and that it's correctly configured to handle the requests from the OpenAI model. If you're using a local server for testing, replace "PORT" with the actual port number where your server is running.

If you're trying to use a hosted model from OpenAI, you should use the correct baseURL for OpenAI's API. Here's an example of how to do it:

import { OpenAI } from "@langchain/openai";

const model = new OpenAI({
    temperature: 0.9,
    openAIApiKey: "xyz",
    configuration: {
      baseURL: "https://api.openai.com/v1",
    },
});

const res = await model.call(
  "Tell me a story about a boy"
);
console.log({ res });

In this example, the baseURL is set to "https://api.openai.com/v1", which is the correct baseURL for OpenAI's API.

Also, please ensure that you're using the correct OpenAI API key. The key should be passed as the openAIApiKey parameter when creating a new instance of the OpenAI class.

If you're still encountering the issue after checking these points, it would be helpful if you could provide more details about your setup and the exact steps you're taking when the error occurs. This will help in identifying the root cause of the issue and finding a suitable solution.

For more information, you can refer to the LangChainJS documentation and the OpenAI API documentation.

I hope this helps! If you have any more questions, feel free to ask.

Sources

• How do I use AzureOpenAI with Custom Agent Properly?
• examples/src/models/llm/openai_userid.ts
• examples/src/models/chat/integration_openai_custom_base.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[BakingBrains]

I am using huggingface text-generation inference endpoints. so the URL is "http://x.x.x.x:PORT/v1"