Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: weaviate handles lowercase index names #1711

Merged
merged 2 commits into from
Jul 18, 2023
Merged

Conversation

jupyterjazz
Copy link
Contributor

WeaviateDocumentIndex capitalizes the index name when creating an index, so if we pass lowercase index name, there's a mismatch leading to various errors

from docarray import BaseDoc, DocList
from docarray.index import WeaviateDocumentIndex
from docarray.typing import NdArray
from pydantic import Field
import numpy as np

# Define the document schema.
class MyDoc(BaseDoc):
    title: str
    embedding: NdArray[128] = Field(is_embedding=True)

# Create dummy documents.
docs = DocList[MyDoc](MyDoc(title=f'title #{i}', embedding=np.random.rand(128)) for i in range(10))

# Initialize a new WeaviateDocumentIndex instance and add the documents to the index.
doc_index = WeaviateDocumentIndex[MyDoc](index_name='zdzd')
doc_index.index(docs)

# Perform a vector search.
query = np.ones(128)
retrieved_docs = doc_index.find(query, limit=10)

File "/Users/jinaai/Desktop/docarray/docarray/index/backends/weaviate.py", line 354, in find
docs, scores = self._find(
File "/Users/jinaai/Desktop/docarray/docarray/index/backends/weaviate.py", line 408, in _find
results["data"]["Get"][index_name], score_name
KeyError: 'zdzd'

Solution: add a post_init function that capitalizes the first letter of the index name

Signed-off-by: jupyterjazz <saba.sturua@jina.ai>
@codecov
Copy link

codecov bot commented Jul 18, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (b306c80) 85.51% compared to head (00147c6) 85.51%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1711   +/-   ##
=======================================
  Coverage   85.51%   85.51%           
=======================================
  Files         132      132           
  Lines        8303     8308    +5     
=======================================
+ Hits         7100     7105    +5     
  Misses       1203     1203           
Flag Coverage Δ
docarray 85.51% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
docarray/index/backends/hnswlib.py 95.33% <100.00%> (+0.05%) ⬆️
docarray/index/backends/weaviate.py 95.04% <100.00%> (+0.02%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Signed-off-by: jupyterjazz <saba.sturua@jina.ai>
@jupyterjazz jupyterjazz requested a review from JoanFM July 18, 2023 13:53
@github-actions
Copy link

📝 Docs are deployed on https://ft-fix-weaviate-index-name--jina-docs.netlify.app 🎉

@JoanFM JoanFM merged commit 15e3ed6 into main Jul 18, 2023
25 checks passed
@JoanFM JoanFM deleted the fix-weaviate-index-name branch July 18, 2023 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: WeaviateDocumentIndex fails if index name is not capitalized
2 participants