Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update default_embedding_functions.md #744

Merged
merged 1 commit into from
Dec 26, 2023
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions docs/src/embeddings/default_embedding_functions.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
There are various Embedding functions available out of the box with lancedb. We're working on supporting other popular embedding APIs.
There are various Embedding functions available out of the box with LanceDB. We're working on supporting other popular embedding APIs.

## Text Embedding Functions
Here are the text embedding functions registered by default.
Embedding functions have inbuilt rate limit handler wrapper for source and query embedding function calls that retry with exponential standoff.
Each `EmbeddingFunction` implementation automatically takes `max_retries` as an argument which has the deafult value of 7.
Embedding functions have an inbuilt rate limit handler wrapper for source and query embedding function calls that retry with exponential standoff.
Each `EmbeddingFunction` implementation automatically takes `max_retries` as an argument which has the default value of 7.

### Sentence Transformers
Here are the parameters that you can set when registering a `sentence-transformers` object, and their default values:
Expand Down Expand Up @@ -69,15 +69,15 @@ print(actual.text)
```

### Instructor Embeddings
Instructor is an instruction-finetuned text embedding model that can generate text embeddings tailored to any task (e.g., classification, retrieval, clustering, text evaluation, etc.) and domains (e.g., science, finance, etc.) by simply providing the task instruction, without any finetuning
Instructor is an instruction-finetuned text embedding model that can generate text embeddings tailored to any task (e.g. classification, retrieval, clustering, text evaluation, etc.) and domains (e.g. science, finance, etc.) by simply providing the task instruction, without any finetuning.

If you want to calculate customized embeddings for specific sentences, you may follow the unified template to write instructions:

Represent the `domain` `text_type` for `task_objective`:

* `domain` is optional, and it specifies the domain of the text, e.g., science, finance, medicine, etc.
* `text_type` is required, and it specifies the encoding unit, e.g., sentence, document, paragraph, etc.
* `task_objective` is optional, and it specifies the objective of embedding, e.g., retrieve a document, classify the sentence, etc.
* `domain` is optional, and it specifies the domain of the text, e.g. science, finance, medicine, etc.
* `text_type` is required, and it specifies the encoding unit, e.g. sentence, document, paragraph, etc.
* `task_objective` is optional, and it specifies the objective of embedding, e.g. retrieve a document, classify the sentence, etc.

More information about the model can be found here - https://github.com/xlang-ai/instructor-embedding

Expand Down Expand Up @@ -119,10 +119,10 @@ tbl.add(texts)
```

## Multi-modal embedding functions
Multi-modal embedding functions allow you query your table using both images and text.
Multi-modal embedding functions allow you to query your table using both images and text.

### OpenClipEmbeddings
We support CLIP model embeddings using the open souce alternbative, open-clip which support various customizations. It is registered as `open-clip` and supports following customizations.
We support CLIP model embeddings using the open source alternative, open-clip which supports various customizations. It is registered as `open-clip` and supports the following customizations:


| Parameter | Type | Default Value | Description |
Expand Down Expand Up @@ -205,4 +205,4 @@ print(actual.label)

```

If you have any questions about the embeddings API, supported models, or see a relevant model missing, please raise an issue.
If you have any questions about the embeddings API, supported models, or see a relevant model missing, please raise an issue.