From 178c04a81c085939cd780d1b6a4eb495b03c698a Mon Sep 17 00:00:00 2001 From: Paul Cornell Date: Tue, 28 Jan 2025 10:50:56 -0800 Subject: [PATCH 1/4] Platform: Add Amazon Bedrock models for Embedder node --- platform/embedding.mdx | 8 ++++++++ platform/workflows.mdx | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/platform/embedding.mdx b/platform/embedding.mdx index aa8cfb97..19fc684a 100644 --- a/platform/embedding.mdx +++ b/platform/embedding.mdx @@ -71,6 +71,14 @@ To generate embeddings, choose one of the following embedding providers and mode [Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings). +- **Amazon Bedrock**: Use [Amazon Bedrock](https://aws.amazon.com/bedrock) to generate embeddings. Also, choose the model to use: + + - **Titan Text Embeddings V2**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). + - **Titan Embeddings G1 - Text**, with 1536 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). + - **Titan Multimodal Embeddings G1**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html). + - **Cohere Embed English**, with 1024 embeddings. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed Multilingual**, with 1024 embedding. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **TogetherAI**: Use [TogetherAI](https://www.together.ai/) to generate embeddings. Also, choose the model to use: - **M2-BERT-80M-2K-Retrieval**, with 768 dimensions. diff --git a/platform/workflows.mdx b/platform/workflows.mdx index 72c66ab2..fa0e2eac 100644 --- a/platform/workflows.mdx +++ b/platform/workflows.mdx @@ -298,6 +298,14 @@ There are two ways to create a custom workflow: [Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings). + - **Amazon Bedrock**: Use [Amazon Bedrock](https://aws.amazon.com/bedrock) to generate embeddings. Also, choose the model to use: + + - **Titan Text Embeddings V2**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). + - **Titan Embeddings G1 - Text**, with 1536 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). + - **Titan Multimodal Embeddings G1**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html). + - **Cohere Embed English**, with 1024 embeddings. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed Multilingual**, with 1024 embedding. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **TogetherAI**: Use TogetherAI to generate embeddings. Also, choose the model to use: - **M2-BERT-80M-2K-Retrieval**, with 768 dimensions. From 80d47d29cbb320bf8324324ef6885fb6c2fa4896 Mon Sep 17 00:00:00 2001 From: Paul Cornell Date: Tue, 28 Jan 2025 15:44:48 -0800 Subject: [PATCH 2/4] Add Bedrock embedder details to Embedding node section, too. --- platform/workflows.mdx | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/platform/workflows.mdx b/platform/workflows.mdx index fa0e2eac..7fd92ccc 100644 --- a/platform/workflows.mdx +++ b/platform/workflows.mdx @@ -502,6 +502,14 @@ There are two ways to create a custom workflow: [Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings). + - **Amazon Bedrock**: Use Amazon Bedrock to generate embeddings. Also, choose the model to use: + + - **Titan Text Embeddings V2**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). + - **Titan Embeddings G1 - Text**, with 1536 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). + - **Titan Multimodal Embeddings G1**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html). + - **Cohere Embed English**, with 1024 embeddings. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed Multilingual**, with 1024 embedding. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **TogetherAI**: Use TogetherAI to generate embeddings. Also, choose the model to use: - **M2-BERT-80M-2K-Retrieval**, with 768 dimensions. From 0a66967f05411b1b58cd9d957c41ef19996f4550 Mon Sep 17 00:00:00 2001 From: Paul Cornell Date: Tue, 28 Jan 2025 15:46:24 -0800 Subject: [PATCH 3/4] Typo: 1024 embeddings -> 1024 dimensions --- platform/workflows.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/platform/workflows.mdx b/platform/workflows.mdx index 7fd92ccc..eb63728d 100644 --- a/platform/workflows.mdx +++ b/platform/workflows.mdx @@ -303,8 +303,8 @@ There are two ways to create a custom workflow: - **Titan Text Embeddings V2**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). - **Titan Embeddings G1 - Text**, with 1536 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). - **Titan Multimodal Embeddings G1**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html). - - **Cohere Embed English**, with 1024 embeddings. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). - - **Cohere Embed Multilingual**, with 1024 embedding. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed English**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed Multilingual**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). - **TogetherAI**: Use TogetherAI to generate embeddings. Also, choose the model to use: @@ -507,8 +507,8 @@ There are two ways to create a custom workflow: - **Titan Text Embeddings V2**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). - **Titan Embeddings G1 - Text**, with 1536 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). - **Titan Multimodal Embeddings G1**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-multiemb-models.html). - - **Cohere Embed English**, with 1024 embeddings. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). - - **Cohere Embed Multilingual**, with 1024 embedding. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed English**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). + - **Cohere Embed Multilingual**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). - **TogetherAI**: Use TogetherAI to generate embeddings. Also, choose the model to use: From 81be56f88ff7c8731a9d6ba9c7a4730d5d3a7896 Mon Sep 17 00:00:00 2001 From: Paul-Cornell Date: Thu, 30 Jan 2025 10:52:48 -0800 Subject: [PATCH 4/4] Update embedding.mdx --- platform/embedding.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/platform/embedding.mdx b/platform/embedding.mdx index 19fc684a..09cf0df9 100644 --- a/platform/embedding.mdx +++ b/platform/embedding.mdx @@ -59,11 +59,11 @@ on Hugging Face: ## Generate embeddings -To generate embeddings, choose one of the following embedding providers and models in the **Providers** section of an **Embedder** node in a workflow: +To generate embeddings, choose one of the following embedding providers and models in the **Select Embedding Model** section of an **Embedder** node in a workflow: You can change a workflow's preconfigured provider only through [Custom](/platform/workflows#create-a-custom-workflow) workflow settings. -- **Azure OpenAI**: Use [Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) to generate embeddings. Also, choose the model to use: +- **Azure OpenAI**: Use [Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) to generate embeddings with one of the following models: - **text-embedding-3-small**, with 1536 dimensions. - **text-embedding-3-large**, with 3072 dimensions. @@ -71,7 +71,7 @@ To generate embeddings, choose one of the following embedding providers and mode [Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings). -- **Amazon Bedrock**: Use [Amazon Bedrock](https://aws.amazon.com/bedrock) to generate embeddings. Also, choose the model to use: +- **Amazon Bedrock**: Use [Amazon Bedrock](https://aws.amazon.com/bedrock) to generate embeddings with one of the following models: - **Titan Text Embeddings V2**, with 1024 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). - **Titan Embeddings G1 - Text**, with 1536 dimensions. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). @@ -79,10 +79,10 @@ To generate embeddings, choose one of the following embedding providers and mode - **Cohere Embed English**, with 1024 embeddings. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). - **Cohere Embed Multilingual**, with 1024 embedding. [Learn more](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). -- **TogetherAI**: Use [TogetherAI](https://www.together.ai/) to generate embeddings. Also, choose the model to use: +- **TogetherAI**: Use [TogetherAI](https://www.together.ai/) to generate embeddings with one of the following models: - **M2-BERT-80M-2K-Retrieval**, with 768 dimensions. - **M2-BERT-80M-8K-Retrieval**, with 768 dimensions. - **M2-BERT-80M-32K-Retrieval**, with 768 dimensions. - [Learn more](https://docs.together.ai/docs/serverless-models#embedding-models). \ No newline at end of file + [Learn more](https://docs.together.ai/docs/serverless-models#embedding-models).