[Bug]: openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: xxxxx********************xxxx

[lenka1]

Bug Description

I am calling aevaluate method for anwerrelevancy keep getting this message openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: xxxxx********************xxxx. The same key works for other OPENAI calls. Please help.

judges["answer_relevancy"] = AnswerRelevancyEvaluator(
service_context=ServiceContext.from_defaults(
llm=OpenAI(temperature=0, model="gpt-3.5-turbo")

eval_tasks.append(
await judges["answer_relevancy"].aevaluate(
query=query,
response=answer,
sleep_time_in_seconds=1.0,

Version

llama-index==0.9.44

Steps to Reproduce

None

Relevant Logs/Tracbacks

LOG BELOW
/bin/python3 /home/tlmalla/answerrelevancy.py
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 401 Unauthorized"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 401 Unauthorized"
Traceback (most recent call last):
  File "/home/tlmalla/answerrelevancy.py", line 90, in <module>
    asyncio.run(extract_json(json_file, output_file))
  File "/home/tlmalla/.local/lib/python3.10/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
  File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/home/tlmalla/answerrelevancy.py", line 76, in extract_json
    await judges["answer_relevancy"].evaluate(
  File "/home/tlmalla/.local/lib/python3.10/site-packages/llama_index/evaluation/base.py", line 62, in evaluate
    return asyncio.run(
  File "/home/tlmalla/.local/lib/python3.10/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
  File "/usr/lib/python3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/llama_index/evaluation/answer_relevancy.py", line 119, in aevaluate
    eval_response = await self._service_context.llm.apredict(
  File "/home/tlmalla/.local/lib/python3.10/site-packages/llama_index/llms/llm.py", line 280, in apredict
    chat_response = await self.achat(messages)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/llama_index/llms/base.py", line 57, in wrapped_async_llm_chat
    f_return_val = await f(_self, messages, **kwargs)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/llama_index/llms/openai.py", line 494, in achat
    return await achat_fn(messages, **kwargs)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/llama_index/llms/openai.py", line 538, in _achat
    response = await aclient.chat.completions.create(
  File "/home/tlmalla/.local/lib/python3.10/site-packages/openai/resources/chat/completions.py", line 1322, in create
    return await self._post(
  File "/home/tlmalla/.local/lib/python3.10/site-packages/openai/_base_client.py", line 1725, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
  File "/home/tlmalla/.local/lib/python3.10/site-packages/openai/_base_client.py", line 1428, in request
    return await self._request(
  File "/home/tlmalla/.local/lib/python3.10/site-packages/openai/_base_client.py", line 1519, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: xxx********************xxx. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

[dosubot]

🤖

Hey @lenka1! 👋 I'm Dosu, a bot here to help you solve bugs, answer questions, and get you on the path to becoming a contributor. While we're waiting for a human maintainer, I'm here to assist. I'm taking a look at your issue now and will provide you with a full answer shortly. Stay tuned!

[logan-markewich]

service_context=ServiceContext.from_defaults(llm=OpenAI(temperature=0, model="gpt-3.5-turbo"))
print(service_context.llm.complete("Hello!"))

Does that work for you?

[lenka1]

Thank you for a quick response. It does not work for me. Still gives the same error.

[logan-markewich]

what if you do this with your API key

service_context=ServiceContext.from_defaults(llm=OpenAI(temperature=0, model="gpt-3.5-turbo", api_key="sk-..."))
print(service_context.llm.complete("Hello!"))

If that still doesn't work, I think you need to generate a new API key

[lenka1]

I am using this key generated by my organization. It works for other calls as shown below. Is there any other reason why it could fail in the above case?

Log Below:
response = client.chat.completions.create(
model="gpt-35-turbo"
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Does Azure OpenAI support customer managed keys?"},
{"role": "assistant", "content": "Yes, customer managed keys are supported by Azure OpenAI."},
{"role": "user", "content": "Do other Azure AI services support this too?"}
]
)
INFO:root:This will get logged
INFO:httpx:HTTP Request: POST https://xxxxxxxxxc-dev.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2023-05-15 "HTTP/1.1 200 OK"
Yes, many Azure AI services support customer managed keys. Some examples include:

• Azure Cognitive Services (Text Analytics, Machine Learning, etc.)
• Azure Bot Service
• Azure Databricks (for AI and machine learning workflows)
• Azure Stream Analytics (for real-time analytics and data processing)

However, it's important to note that not all Azure AI services have the same level of support for customer managed keys. You should always check the documentation for the specific service you're interested in to confirm whether this feature is supported and to understand any limitations or requirements.

[dosubot]

🤖

Based on the information you've provided, it seems like you're using the Azure OpenAI service, which has a different authentication mechanism compared to the standard OpenAI API. The Azure OpenAI service uses Azure AD for token authentication, and the token might have expired, leading to the "Incorrect API key provided" error.

In the LlamaIndex codebase, the AzureOpenAIMultiModal class handles the API key authentication for OpenAI calls. If use_azure_ad is set to True, the _get_credential_kwargs method refreshes the Azure AD token and sets it as the API key. If the token has expired and is not refreshed properly, it could lead to this error.

Here is the relevant code:

def _get_credential_kwargs(self, **kwargs: Any) -> Dict[str, Any]:
    if self.use_azure_ad:
        self._azure_ad_token = refresh_openai_azuread_token(self._azure_ad_token)
        self.api_key = self._azure_ad_token.token

    return {
        "api_key": self.api_key,
        "max_retries": self.max_retries,
        "timeout": self.timeout,
        "azure_endpoint": self.azure_endpoint,
        "azure_deployment": self.azure_deployment,
        "api_version": self.api_version,
        "default_headers": self.default_headers,
        "http_client": self._http_client,
        **kwargs,
    }

You can find this method in the llama_index/llms/azure_openai.py file.

Ensure that use_azure_ad is set to True only if you are using Azure AD for token authentication and that the token is being refreshed correctly. If you are not using Azure AD for token authentication, make sure use_azure_ad is set to False and the correct API key is provided.

If you're still facing issues, could you please provide more details about your setup? Specifically, it would be helpful to know how you're initializing the AzureOpenAIMultiModal instance and whether use_azure_ad is set to True or False.

Sources

• llama_index/llms/azure_openai.py
• llama_index/multi_modal_llms/azure_openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[logan-markewich]

@lenka1 ohhhh you are using azure!

You should follow the azure setup here
https://docs.llamaindex.ai/en/stable/examples/customization/llms/AzureOpenAI.html

[lenka1]

Thank you for the information. Unfortunately I am still getting the 401 error with the call

File "/home/tlmalla/.local/lib/python3.10/site-packages/openai/_base_client.py", line 959, in _request
response.raise_for_status()
File "/home/tlmalla/.local/lib/python3.10/site-packages/httpx/_models.py", line 759, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://xxxxxxxxxx-dev.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-06-01-preview'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401
Encountered httpx.HTTPStatusError
Traceback (most recent call last):
File "/home/tlmalla/.local/lib/python3.10/site-packages/openai/_base_client.py", line 959, in _request
response.raise_for_status()
File "/home/tlmalla/.local/lib/python3.10/site-packages/httpx/_models.py", line 759, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://xxxxxxxxx.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2023-06-01-preview'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401
DEBUG:openai._base_client:Not retrying
Not retrying
DEBUG:openai._base_client:Re-raising status error
Re-raising status error
WARNING:llama_index.llms.openai_utils:Retrying llama_index.embeddings.openai.get_embeddings in 0.6291118746022746 seconds as it raised AuthenticationError: Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.

[lenka1]

llm = AzureChatOpenAI(
temperature=temperature,
api_key=api_key,
azure_endpoint=api_base,
api_version=api_version,
openai_api_type=api_type,
azure_deployment='gpt-35-turbo'.
)

[logan-markewich]

Did you follow the guide exactly? Setup the llm and embedding model, and set the global service context?

The error message above is for embeddings

[tslmy]

@lenka1 , I'm sad to see that we aren't aligned after multiple rounds of clarification.

To ensure we can be on the same page, please share a complete script. The script should:

1. set up your LlamaIndex LLM,
2. call client.chat.completions.create so that it returns a normal response, and then
3. call await judges["answer_relevancy"].aevaluate which would result in that 401 error.

Clarifications:

• Just let the script fail (i.e., return a non-zero code; don't catch that exception) at the end upon that 401 error.
• Make sure that the script can run from start to finish when someone provides an Azure managed API key and run python script.py, nothing else.
• Refrain from leaving out setup steps that people have to do verbally describing them, as humans are awfully bad at being precise or accurate.
• Also, please use a code fence. Reading code in a non-fixed-width font hurts eyes.
• As always, do not provide the API key in the script, but you have been doing great on that front.

[YanZhangADS]

I encounter similar issue.

llama-index==0.9.48
openai==1.12.0

llm = AzureOpenAI(
engine="gpt-4-32k",
model="gpt-4-32k",
temperature=0,
api_key=openai.api_key,
azure_endpoint=openai.api_base,
api_version=openai.api_version
)

#use following code to test out llm working fine
response = llm.complete("The sky is a beautiful blue and")
print(response)

#However, when I use agent (OpenAIAgent or ReActAgent), it shows the error message.
agent = OpenAIAgent.from_tools(tools, llm=llm, verbose=True, context=context)
prompt = "xxxxxx" #my prompt
result = agent.query(prompt) #error occurs here

#error message
Got output: Error: Error code: 401 - {'error': {'message': 'Incorrect API key provided: be456993********************395a. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

[logan-markewich]

@YanZhangADS what tools are you using? Are they query engines? Did you properly setup azure for any them?

See the general azure setup here (note it's for v0.10.x syntax, before that you would set a global service context)
https://docs.llamaindex.ai/en/stable/examples/customization/llms/AzureOpenAI.html

[YanZhangADS]

@YanZhangADS what tools are you using? Are they query engines? Did you properly setup azure for any them?

See the general azure setup here (note it's for v0.10.x syntax, before that you would set a global service context) https://docs.llamaindex.ai/en/stable/examples/customization/llms/AzureOpenAI.html

Thanks for youre reply! This tool is a PandasQueryEngine object (from llama_index.query_engine import PandasQueryEngine). It extract information from a local csv file. No embedding or indexing is used. "Did you properly setup azure for any them?" -- No, I did not setup azure on this tool.

[logan-markewich]

@YanZhangADS the pandas query engine uses an LLM to write pandas queries. You'll need to pass in the LLM if you want to use it with azure 👍

[YanZhangADS]

Thanks @logan-markewich for pointing out the direction! My issue is resolved.

For llama-index==0.9.48, if you use AzureOpenAI you need to add following. Reference: https://docs.llamaindex.ai/en/latest/module_guides/supporting_modules/service_context_migration.html#
However, because the code is slightly different for various versions, I made following code working for me.

from llama_index import ServiceContext, set_global_service_context
service_context = ServiceContext.from_defaults(
llm=AzureOpenAI(engine="gpt-4-32k", model="gpt-4-32k"),
)
set_global_service_context(service_context)

[logan-markewich]

I'm going to close this issue. I think there's more than enough details for @lenka1 to get it straightened out.

The docs I shared above give an exact example when using the latest version, otherwise @YanZhangADS gave a good example too.