community: update AzureSearch class to work with azure-search-documents=11.4.0

[lz-chen]

• Description: Updates libs/community/langchain_community/vectorstores/azuresearch.py to support the stable version azure-search-documents=11.4.0
• Issue: Stable release 11.4.0 of azure.search.documents.models not compatible with latest langchain version -> class Vector gone #14534, AzureSearch Bug -- langchain.vectorstores.azuresearch  #15039, AzureSearch semantic_hybrid_search fails due to hardcoding of metadata fields #15355
• Dependencies: azure-search-documents>=11.4.0

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Feb 12, 2024 11:40pm

[lz-chen]

Hello, there is a previous PR #13472 that provided some partial fix to the AzureSearch class but did not fully update the dependency. Now calling class as vector store still doesn't work in my case, with the same problems encontered by others in the issues mentioned in the description.
I'd like to contribute to update the dependency to azure-search-documents=11.4.0. However I am not sure if that means updating all the pyproject.toml and poetry.lock files that has azure-search-documents? What is the best way to do that? @efriis @hwchase17 Thanks in advance 😊

[efriis]

Hey @lz-chen ! You actually can't do this with a community integration, as it's not a concrete dependency. Unfortunately, people need to manage their other python package versions themselves.

A good addition would be a note in the documentation to upgrade that package!

[lz-chen]

@efriis Thanks for the reply 😊 What does that mean for this PR? Should I keep the changes I made in libs/community/langchain_community/vectorstores/azuresearch.py without updating the dependencies in libs/langchain/pyproject.toml and libs/langchain/poetry.lock?
Or follow what's done in PR #13472 to keep support for both azure-search-documents>=11.4.0 and azure-search-documents<11.4.0?

[lz-chen]

@efriis Could you give me some pointers on what I should do with this? As of now, the AzureSearch class in libs/community/langchain_community/vectorstores/azuresearch.py doesn't work with azure-search-documents version 11.4.0, which has been released for 2 months now. The older version 11.4.0b8 that langchain pinned to has some bugs and I can't keep using that.
If I can't make change in libs/langchain/pyproject.toml, should I make changes somewhere else? For example making contribution to langchain-community?

[HeidiSteen]

@mattgotteiner for subject matter expertise on Azure Search SDK migration to 11.4 stable library. Agree with @lz-chen that the existing vector store integration module doesn't work with 11.4.0.

[mattgotteiner]

@lz-chen thank you very much for doing this work.

@efriis Please let us know if there's any additional work required to get the PR merged.

[Skar0]

Hi @lz-chen,

Thanks for merging my PR into your update! I reviewed the original changes and tested them with azure-search-documents==11.4.0, and everything seems to work just fine :) My PR aimed to address a specific bug related to the removal of query_language in azure-search-documents==11.4.0. I'm happy to discuss any further details or assist with additional improvements if needed.

[pamelafox]

Just looking at the discussion here about pyproject.toml-

It looks like the main langchain pyproject.toml doesn't reference azure-search-documents, which makes sense as this in community. However, I notice that the langchain-community pyproject.toml also doesn't reference azure-search-documents. I believe that it probably should, just based off similar recent integration updates, like this PR for Azure Document Intelligence:
#14389

The file to change:

langchain/libs/community/pyproject.toml

Line 86 in ac970c9

azure-ai-documentintelligence = {version = "^1.0.0b1", optional = true}

Notably it'd be marked as optional=True, like the other community dependencies.

That's my suggestion after looking through how the packages are structured. Perhaps @efriis could confirm. Thanks!

[baskaryan]

Just looking at the discussion here about pyproject.toml-

It looks like the main langchain pyproject.toml doesn't reference azure-search-documents, which makes sense as this in community. However, I notice that the langchain-community pyproject.toml also doesn't reference azure-search-documents. I believe that it probably should, just based off similar recent integration updates, like this PR for Azure Document Intelligence: #14389

The file to change:

langchain/libs/community/pyproject.toml

Line 86 in ac970c9

azure-ai-documentintelligence = {version = "^1.0.0b1", optional = true}

Notably it'd be marked as optional=True, like the other community dependencies.
That's my suggestion after looking through how the packages are structured. Perhaps @efriis could confirm. Thanks!

Optional dependencies only need to be included in pyproject if they're required for unit tests, so no need to add in this case!

[pamelafox]

@baskaryan Ah, thanks for clarifying, that explains why azure-document-intelligence was added in that PR, as it had a short unit test.

[mattgotteiner]

apologies - but I think we need to follow up here. SearchField is still using vector_search_configuration when it should be vector_search_profile_name

[mattgotteiner]

We are also missing a from azure.search.documents.indexes.models import VectorSearch

[mattgotteiner]

the import is only performed with typing.TYPE_CHECKING is true

[kristapratico]

@mattgotteiner noticed these issues today as well and opened a PR to fix here: #17599

[baskaryan]

hey @lz-chen @mattgotteiner, was there a specific reason we removed FIELDS_ID in this pr?

opening issue to track regression: #22827