langchain.embeddings.OpenAIEmbeddings is not working with AzureOpenAI

[JonAtDocuWare]

When using the AzureOpenAI LLM the OpenAIEmbeddings are not working. After reviewing source, I believe this is because the class does not accept any parameters other than an api_key. A "Model deployment name" parameter would be needed, since the model name alone is not enough to identify the engine. I did, however, find a workaround. If you name your deployment exactly "text-embedding-ada-002" then OpenAIEmbeddings will work.

edit: my workaround is working on version .088, but not the current version.

[aajn88]

I'm having the same issue. However, in my case, my deployment is also called text-embedding-ada-002 too but it's failing with the following error:

InvalidRequestError: Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.

I printed the name of the model right before the call was done in the source code and it's correct, so no sure what's happening

[aajn88]

Actually, by reading the error again, it seems like it's not related to the name of the model and azure just throttling.

So, in order to configure langchain embeddings properly, I have to replace this part with my desired chunk_size:

embeddings = embedding.embed_documents(texts, 1)

Of course, ideally this should be sent by parameter

[floleuerer]

I ran into the same issue with the chunk_size and Embeddings in Azure OpenAI Services and provided a fix. The easiest way is to initialize your OpenAIEmbeddings with chunk_size=1 - it works in other helper functions too when you can't pass the chunk_size e.g. .from_documents()...

embeddings = OpenAIEmbeddings(chunk_size=1)

For the deployment name, it should work with names != text-embedding-ada-002:

embeddings = OpenAIEmbeddings(engine=<your deployment name>)

[aajn88]

@floleuerer wait, there's no AzureOpenAIEmbeddings, is there? Were you referring to AzureOpenAI instead?

[floleuerer]

Ah sorry my bad! it' s OpenAIEmbeddings of course.

[aajn88]

Thanks for confirming. Got excited to think they had a version for Azure! Although it doesn't make a big difference as it's just a config matter.

I just got to my laptop and tried. It fails with the following message:

pydantic.error_wrappers.ValidationError: 1 validation error for OpenAIEmbeddings
chunk_size
  extra fields not permitted (type=value_error.extra)

Maybe I have the wrong version? I forked version 0.0.100 - could you confirm yours?

[floleuerer]

The fix should be in 0.0.106.

[aajn88]

Gotcha! I'll try to fork the new version this week.

[deb007]

Is this issue fixed? Am on 0.0.107 version, but still getting below error with trying to use OpenAIEmbeddings with the azure openai service. Tried both FAISS as well as Pinecone vector stores. from_documents() fails with
openai.error.InvalidRequestError: Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions

[aajn88]

I ran into the same issue with the chunk_size and Embeddings in Azure OpenAI Services and provided a fix. The easiest way is to initialize your OpenAIEmbeddings with chunk_size=1 - it works in other helper functions too when you can't pass the chunk_size e.g. .from_documents()...

embeddings = OpenAIEmbeddings(chunk_size=1)

For the deployment name, it should work with names != text-embedding-ada-002:

embeddings = OpenAIEmbeddings(engine=<your deployment name>)

@deb007 did you try @floleuerer solution?

[deb007]

I ran into the same issue with the chunk_size and Embeddings in Azure OpenAI Services and provided a fix. The easiest way is to initialize your OpenAIEmbeddings with chunk_size=1 - it works in other helper functions too when you can't pass the chunk_size e.g. .from_documents()...

embeddings = OpenAIEmbeddings(chunk_size=1)

For the deployment name, it should work with names != text-embedding-ada-002:

embeddings = OpenAIEmbeddings(engine=<your deployment name>)

@deb007 did you try @floleuerer solution?

Yes, I tried OpenAIEmbeddings(document_model_name="<deployment_name>", chunk_size=1) - it gives error -
openai.error.InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.

OpenAIEmbeddings(document_model_name="<deployment_name>") gives error -
openai.error.InvalidRequestError: Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.

OpenAIEmbeddings(engine="<deployment_name>") gives below error:
extra fields not permitted (type=value_error.extra)

[aajn88]

Actually, by reading the error again, it seems like it's not related to the name of the model and azure just throttling.

So, in order to configure langchain embeddings properly, I have to replace this part with my desired chunk_size:

embeddings = embedding.embed_documents(texts, 1)

Of course, ideally this should be sent by parameter

@deb007 this 👆 worked for me.

[tomasfernandez1212]

embeddings = OpenAIEmbeddings(chunk_size=1)

This also worked for me. Thanks!

However, what are the implications of setting chunk size to 1?

[Huyueeer]

Unfortunately, I have tasted all the above methods and they have not worked, including deployment_name is set to text-davinci-003... Or change the version. I'm confused about openai.api_base and openai.api_type doesn't work. When I comment out the code, I find that there is no difference, and I still report an error as follows:

openai.error.InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.

#1560 (comment) This solves my problem

[aajn88]

embeddings = OpenAIEmbeddings(chunk_size=1)

This also worked for me. Thanks!

However, what are the implications of setting chunk size to 1?

I think latency (costs should be the same as it’s based on tokens and not API calls). If I’m not mistaken, you can add multiple texts in one call. I remember reading this somewhere but I couldn’t find it in their docs anymore. However, the error seems align with what I’m saying.

[csiebler]

I'm on langchain-0.0.117 and as long as I use OpenAIEmbeddings() without any parameters, it works smoothly with Azure OpenAI Service, but requires the model deployment to be named text-embedding-ada-002.

[hr1sh]

@floleuerer @aajn88

Setting the chunk_size explicity while initializing the OpenAIEmbeddings() embeds worked for me.
Setting the deployment name explicitly did not help to overcome the max_input error.

langchain v=0.0.117 (latest)

[csiebler]

@floleuerer @aajn88

Setting the chunk_size explicity while initializing the OpenAIEmbeddings() embeds worked for me. Setting the deployment name explicitly did not help to overcome the max_input error.

langchain v=0.0.117 (latest)

Agreed, I was not able to get it to work with setting the model name manually.

[germanpinzon807]

I'm on langchain=0.0.119 but OpenAIEmbeddings() throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persists

[Granine]

I'm on langchain=0.0.119 but OpenAIEmbeddings() throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persists

For some reason, the suggested implementation on the documentation (by setting environmental variables) does not work. However, a workaround is before openAIEmbedding is ever called, import openai package and set parameters manually
eg:

import openai
openai.api_base = "www.x.com/" 
openai.api_type = 'azure'
openai.api_version = "2022-12-01" 
# optionally, set key

then

OpenAIEmbeddings(document_model_name="MODEL_NAME", chunk_size=1)

[Granine]

A straight forward yet messy solution seems to be updating
validate_enviroment validator function in langchain/embeddings/openai.py by adding a check-and-fix. As this is where openai seem to be first provoked.

[emigre459]

I'm suffering from these same issues, although my error message is a bit different (likely since I'm using LangChain v0.0.123:

2023-03-25 19:47:29.827 INFO    openai: error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
2023-03-25 19:47:29.831 WARNING langchain.embeddings.openai: Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit..

It seems to me like the wait time between retries (which have clearly been handled in multiple places in LangChain) may not be the issues so much as the number of async workers sent to Azure in the space of a minute (since the limit is something like 300 per minute for embeddings). Does anyone know where in the LangChain code the number of max async workers is defined such that we could throttle it down to avoid this error perhaps?

[Granine]

I'm suffering from these same issues, although my error message is a bit different (likely since I'm using LangChain v0.0.123:

2023-03-25 19:47:29.827 INFO    openai: error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False
2023-03-25 19:47:29.831 WARNING langchain.embeddings.openai: Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 5 seconds. Please contact Azure support service if you would like to further increase the default rate limit..

It seems to me like the wait time between retries (which have clearly been handled in multiple places in LangChain) may not be the issues so much as the number of async workers sent to Azure in the space of a minute (since the limit is something like 300 per minute for embeddings). Does anyone know where in the LangChain code the number of max async workers is defined such that we could throttle it down to avoid this error perhaps?

Don't really think this is the same issue, Maybe open a new issue page? I did encounter the same problem tho. My novel solution is to add a sleep inside the langchain library source code (I do not require high performance). I did not find a usage or performance limiter available to use.

[emigre459]

Apologies, I copied the wrong error (this is a related one, but caused when you try to use embeddings in a vector store using llama-index). My actual LangChain-only error is Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions.' error_param=None error_type=invalid_request_error message='OpenAI API error received' stream_error=False

@Granine where did you put the sleep call (at the top of the openai.py module perhaps; or maybe in the embedding loop)? I'm (currently) fine with something like that, but it seems like that just repeats the retry logic doesn't it? But if it works for you, I'm willing to give it a whirl!

[germanpinzon807]

I'm on langchain=0.0.119 but OpenAIEmbeddings() throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persists

For some reason, the suggested implementation on the documentation (by setting environmental variables) does not work. However, a workaround is before openAIEmbedding is ever called, import openai package and set parameters manually eg:

import openai
openai.api_base = "www.x.com/" 
openai.api_type = 'azure'
openai.api_version = "2022-12-01" 
# optionally, set key

then

OpenAIEmbeddings(document_model_name="MODEL_NAME", chunk_size=1)

I'm on langchain=0.0.119 but OpenAIEmbeddings() throws an AuthenticationError: Incorrect API key provided... it seems that it tries to authenticate through the OpenAI API instead of the AzureOpenAI service, even when I configured the OPENAI_API_TYPE and OPENAI_API_BASE previously. Does anyone have the same problem?... tried with version 0.0.117 but the problem persists

For some reason, the suggested implementation on the documentation (by setting environmental variables) does not work. However, a workaround is before openAIEmbedding is ever called, import openai package and set parameters manually eg:

import openai
openai.api_base = "www.x.com/" 
openai.api_type = 'azure'
openai.api_version = "2022-12-01" 
# optionally, set key

then

OpenAIEmbeddings(document_model_name="MODEL_NAME", chunk_size=1)

it worked! thank you! @Granine

[EAYoshi]

I've spent the last 5 hours or so trying to work this ❗💲 #️⃣ ⚡er out, which has certainly put a damper on my hackathon spirits :)

I'm using:

• Openai => 0.27.2
• langchain => 0.0.127

and I was just trying to following the simple indexing getting started guide

Setting chunk_size=1 against the OpenAIEmbeddings initializer is indeed one part of the solve, at least until Azure opens up more chunks. It seems they have this as a filter in the router at your deployment level (or a part of the model ingestion handler) as you'll get this error The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again. with or without chunk_size=1, but you'll only get Too many inputs for model None. The max number of inputs is 1. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions. if you end up on a valid embedding model.

Embedding model is the key point here, if you try to use a model that outputs text rather than embedding vectors you'll get openai.error.InvalidRequestError: The embeddings operation does not work with the specified model, text-davinci-002. Please choose different model and try again. You can learn more about which models can be used with each operation here: https://go.microsoft.com/fwlink/?linkid=2197993.

So actually, if you are receiving Too many inputs for model None. The max number of inputs is 1. ... then it's a sign your deployment is set up correctly in Azure and you're actually hitting it. Just put in chunk_size=1 and you'll hit our final boss: openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.completion.Completion'>

As a brief tangent, the easiest way to actually get this to work up until this point is to just make a deployment called text-embedding-ada-002 in Azure and deploy the model with the same name to it, then you're work with the langchain defaults as of the time of writing.
If you want to use another model then pass in OpenAIEmbeddings(model="<your embeddings deployment name>"), (note not model_name nor document_model_name/query_model_name unless you specifically want different models for upsert vs query, just model. See here)
Hopefully MS adds a front door model attribute => model router in the near future so things are API compatible with the openAI API.
Now, I digress. Onto M. Bison

In the particular case of the getting started guide I was using, the actual problem was that things were not being set up correctly to bring in a text LLM for the LLMChain that is behind the operations. I confirmed this by adding "engine":"text-davinci-003" on the line below this (as well as having text-davinci-003 deployed to Azure with a deployment of the same name), which made everything work.

To 'properly' fix this you need to do as follows (assuming you have the right Azure env vars in .env):

from dotenv import load_dotenv
from langchain.document_loaders import DirectoryLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.indexes import VectorstoreIndexCreator
from langchain.llms import OpenAI

load_dotenv()

# Instantiate a Langchain OpenAI class, but give it a default engine
llm = OpenAI(model_kwargs={'engine':'text-davinci-003'})

# Load docs
docs = DirectoryLoader(os.path.join(os.path.dirname(__file__), 'my_docs_directory'))

# Sort out the chunking issue
embedding_model = OpenAIEmbeddings(chunk_size=1)

# Load things into your index, this all works fine so long as you do the above. Things are auto instantiated to use Azure correctly, so long as you have a deployment of text-embedding-ada-002 with the same model in your Azure instance
index = VectorstoreIndexCreator(embedding=embedding_model).from_loaders([my_files])

# This is where the fix is. You can pass in the pre-instantiated LLM with the default text model set, so your LLMChain behind the query action actually works
index.query(ym_query, llm=llm)

But as you can see, this is case by case. I've poked a lot but I don't see a way to set the text LLM model against the high level abstractions (i.e. VectorstoreIndexCreator) in a way that propagates. Hopefully it's there and I'm just missing it.

So the easy solution is.... well, there's no easy solution. The problem is actually the default instantiation of the Azure based openAI LLM takes into account that it's running on Azure, but doesn't have nice defaults or ways to pass through that you need to change the engine as well/instead of the model. Really, this is just a huge pain for the library writers due to the decision to have 'deployments' on Azure and not adhere to the model attribute like the OpenAI API

[LewisLebentz]

@EAYoshi I've been trying to play around with the code here: https://github.com/alphasecio/llama-index/blob/main/index.py to get it working on Azure, changed it to use from langchain.llms import AzureOpenAI but not having any luck. Getting the same error as you, tried setting chunk size to 1 but all I get is this:

error_code=DeploymentNotFound error_message='The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.' error_param=None error_type=None message='OpenAI API error received' stream_error=False

Do you think this is related?

[EAYoshi]

Likely, yes.

First, make sure these environment variables are set correctly in the context of your code

OPENAI_API_BASE=<azure OpenAI base URL without deployment>
OPENAI_API_KEY=<azure key>
OPENAI_API_TYPE=azure

Then, make sure you have an Azure deployment named text-davinci-003 that is set to use the text-davinci-003 model

Finally, try changing this line to be

llm_predictor = LLMPredictor(llm=OpenAI(model_kwargs={'engine':'text-davinci-003'}))

then play around with the temperature once it works, I'm not sure where that parameter should go

[JonAtDocuWare]

well it's clear that a base endpoint for your azure services is not defined.

what do you expect this line of code to do:

openai.api_base = os.getenv('')

[JonAtDocuWare]

@sai-kiran-adusumilli you also exposed your API key in your first post. You edited it, but any user can view the edit history. You need to trash that key now and get a new one... unless you want a huge Azure bill.

[JonAtDocuWare]

if i delete the comment ? can users still see it

well I get email notifications for comments and that is where I first saw you exposed your API key.... deleting the comment here isn't going to delete the email in my inbox.

[zioproto]

@JonAtDocuWare I proposed #3711 to address this issue.

According to the docs:
https://learn.microsoft.com/en-us/azure/cognitive-services/openai/reference#embeddings

The POST request should look like:

POST https://{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id}/embeddings?api-version={api-version}

However my current PR is not working because it generates

https://xxxxxx.openai.azure.com/openai/engines/text-embedding-ada-002/embeddings

I am currently debugging this problem.

[simhallq]

Explicitly passing texts to embed as "input" resovled this for me. On openai==0.27.6

openai.Embedding.create(input=texts, model="text-embedding-ada-002")

[fms-santos]

I was having the same issue but using OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1) fixed it.
Now I'm getting the INFO:openai:error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False error .

[That1Panda]

openai.Embedding.create(input=texts, model="text-embedding-ada-002")

This worked for me but had to add deployment_id On openai==0.27.1
embedding =openai.Embedding.create(input="input text", model="text-embedding-ada-002",deployment_id="text-embedding-ada-002")

[zioproto]

I was having the same issue but using OpenAIEmbeddings(model="text-embedding-ada-002", chunk_size=1) fixed it. Now I'm getting the INFO:openai:error_code=429 error_message='Requests to the Embeddings_Create Operation under Azure OpenAI API version 2022-12-01 have exceeded call rate limit of your current OpenAI S0 pricing tier. Please retry after 1 second. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit.' error_param=None error_type=None message='OpenAI API error received' stream_error=False error .

The 429 error is not a bug in your code, is the API throttling you. Which Azure region are you using ? You could try using Azure OpenAI in a different region.

[fms-santos]

Thanks! I'm in west europe. I'll try that...

Edit: it worked on South Central US with no issues. Seems that in West Europe the API is throttling me indeed.

[kingname]

The prefect way to solve this problem is:

from langchain.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator
from langchain.embeddings import OpenAIEmbeddings

loader = TextLoader('text.txt')
embeddings = OpenAIEmbeddings(model='text-embedding-ada-002',
                              deployment='your deployment name',
                              openai_api_base='https://your-resource-name.openai.azure.com',
                              openai_api_type='azure',
                              openai_api_key='your key',
                              chunk_size=1)
index = VectorstoreIndexCreator(embedding=embeddings).from_loaders([loader])

[JonAtDocuWare]

has been resolved, closing this issue.

[parth-patel2023]

I am getting error :-

Retrying langchain.embeddings.openai.embed_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details..

I am using this code :-

embeddings = OpenAIEmbeddings(
    openai_api_key=OPENAI_API_KEY)

version

 langchain==0.0.113

[DSgUY]

you only need this:

embeddings: OpenAIEmbeddings = OpenAIEmbeddings(
    openai_api_base= f"https://{AZURE_OPENAI_SERVICE}.openai.azure.com",
    openai_api_type='azure',
    deployment='text-embedding-ada-002',
    openai_api_key=AZURE_OPENAI_API_KEY,
    chunk_size=1,
)

query_result = embeddings.embed_query("is this issue solve?")

My deployment name and model name are the same: text-embedding-ada-002

[pietz]

I had a few issues yesterday without knowing about this thread. Let me share some updates and additional information that I haven't seen here:

• The maximum chunk_size is still 1 which makes the functionality almost useless. Using Azure OpenAI to embed 1000 chunks takes 170 seconds while OpenAI with chunk size 1000 takes 12 seconds. That's 14x slower.
• It's not a langchain problem but an Azure limitation. Same behavior with the OpenAI library.
• Naming your deployment text-embedding-ada-002 results in not having to set the deployment parameter in OpenAIEmbeddings because that's the default value. You can name your deployment anything you want.
• Using environment variables works just fine if 1) you stick to the official naming convention (like the parameters but in caps) 2) best don't set the openai_api_version (default works for now) 3) No trailing slash / in the openai_api_base URL.
• The code sample from @DSgUY works if your deployment has the same name as the model

langchain 0.0.235
openai 0.27.8

[DSgUY]

@pietz I'm also having this problem:#7841

[gclem]

Same here

Too many inputs. The max number of inputs is 16. We hope to increase the number of inputs per request soon. Please contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 for further questions

[babsy3]

@gclem I'm getting the same error, and none of the workarounds above work for me! Were you able to resolve your error?

[JordanRex]

classic microsoft. with 16 its bit more bearable but still infeasible for larger embeddings. probably best to have local bert embeddings even than to use the azure openai for now until they can increase it around 100 at the least

[gclem]

Found a workarround :

const embeddings = new OpenAIEmbeddings({
  batchSize : 16
});

batchSize since the same of chunksize, for langchainJS