This repository has been archived by the owner on May 10, 2024. It is now read-only.
Multimodal #157
Merged
Multimodal #157
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
4 tasks
atroyn
added a commit
to chroma-core/chroma
that referenced
this pull request
Nov 7, 2023
## Description of changes This PR adds URIs and DataLoaders into Chroma. - `DataLoader` works like `EmbeddingFunction`, except it takes a `URIs` and outputs the specified datatype. - A `DataLoader` using `pillow` for image file loading. - Adds `uris` as a field on `add`, `query`, as well as an `include` field - Adds `data` as an `include` field URIs specify a place where data can be loaded from, and can be used to load data for embedding, or as the result of retrieval. This makes multimodal retrieval with data stored externally as files seamless and extensible. This PR is stacked on #1293 ## Test Integration tests pass. A new unit test for data loaders: https://github.com/chroma-core/chroma/blob/c71c3efa15d2a9252db26470b9ff1cdb10a2b681/chromadb/test/data_loader/test_data_loader.py Try the notebook: https://github.com/chroma-core/chroma/blob/c71c3efa15d2a9252db26470b9ff1cdb10a2b681/examples/multimodal/multimodal_retrieval.ipynb ## Documentation Documentation for this and #1293 chroma-core/docs#157 ## TODOs - [x] Concurrent Loading - [x] Tests - [x] Wiring through FastAPI - [x] Documentation
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds documentation about Chroma's new Multi-modal functionality.
The idea is to keep all the multi-modal specific information in one easy-to-reference place, rather than spreading it out throughout the API reference.