fix: SentenceWindowRetriever convert metadata fields back to int#8304
fix: SentenceWindowRetriever convert metadata fields back to int#8304davidsbatista wants to merge 3 commits intomainfrom
SentenceWindowRetriever convert metadata fields back to int#8304Conversation
Pull Request Test Coverage Report for Build 10594089336Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
SentenceWindowRetriever convert metadata fields back to int
|
Since this change is related to how Pinecone handles metadata, WDYT? |
|
It's a good suggestion, but we can't just blindly convert everything back to integers. To make this generic, we need to know which type the values were originally, i.e.: before being stored in Pinecone; and I don't know where to store that information. |
|
Yes, I understand. I'm simply suggesting that we move the Unrelated: I have the impression that the fact that our |
|
``
ah ok, now I understand what you suggested - the only thing is that for the tests we will need to import Pinecone/add them to the CI |
No, I suggest modifying the Pinecone Document Store in core-integrations. |
|
moving to core-integrations |
Proposed Changes:
Pinecone converts all meta numbers in the meta field to float (https://docs.pinecone.io/guides/data/filter-with-metadata). This causes the
SentenceWindowRetrieverto crash completely.This PR checks if the metadata values are floats and converts them back to integers making
PineConesupported by theSentenceWindowRetrieverHow did you test it?
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:.