diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/aws/amazon-bedrock.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/aws/amazon-bedrock.md index 83ba05a0cd1..4906202a665 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/aws/amazon-bedrock.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/aws/amazon-bedrock.md @@ -107,7 +107,7 @@ You can follow a similar approach to implement any of the other operations in ** ### Chatting with Large Language Models using the ChatCompletions Operation -A common use case of the Amazon Bedrock Connector is the development of chatbots and chat solutions. The **ChatCompletions (without history / with history)** operations offer an easy way to connect to most of the text-generation models available on Amazon Bedrock. The ChatCompletions operations are built on top of Bedrock's Converse API, allowing you to talk to different models without the need of a model-specific implementation. For more information on the ChatCompletion operations, see [GenAI Commons: Chat Completions](/appstore/modules/genai/commons/#genai-generate). +A common use case of the Amazon Bedrock Connector is the development of chatbots and chat solutions. The **ChatCompletions (without history / with history)** operations offer an easy way to connect to most of the text-generation models available on Amazon Bedrock. The ChatCompletions operations are built on top of Bedrock's Converse API, allowing you to talk to different models without the need of a model-specific implementation. For more information on the ChatCompletion operations, see [GenAI Commons: Chat Completions](/appstore/modules/genai/genai-for-mx/commons/#genai-generate). For an overview of supported models and model-specific capabilities and limitations, see [Amazon Bedrock Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html#conversation-inference-supported-models-features) in the AWS documentation. @@ -187,14 +187,14 @@ To invoke a Bedrock agent for your Mendix app, do the following steps: ### Token Usage {#tokenusage} -[Token usage](/appstore/modules/genai/commons/#token-usage) monitoring is now possible for the following operations: +[Token usage](/appstore/modules/genai/genai-for-mx/commons/#token-usage) monitoring is now possible for the following operations: * Chat Completions with History * Chat Completion without History * Embeddings with Cohere Embed * Embeddings with Amazon Titan Embeddings -For more information about using this feature, refer to the [GenAI commons documentation](/appstore/modules/genai/commons/#token-usage). +For more information about using this feature, refer to the [GenAI commons documentation](/appstore/modules/genai/genai-for-mx/commons/#token-usage). ## Technical Reference {#technical-reference} @@ -213,20 +213,20 @@ For additional information about available operations, refer to the sections bel #### ChatCompletions (With History) and ChatCompletions (Without History) {#chat-completions} -The [ChatCompletions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history) and [ChatCompletions (without history)](/appstore/modules/genai/commons/#chat-completions-without-history) activities can be used with a variety of supported LLMs. +The [ChatCompletions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history) and [ChatCompletions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history) activities can be used with a variety of supported LLMs. Some capabilities of the chat completions operations are currently only available for specific models: -* **Function Calling** - You can use function calling in all chat completions operations using a [supported model](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html) by adding a `ToolCollection` with a `Tool` via the [Tools: Add Function to Request](/appstore/modules/genai/commons/#add-function-to-request) operation. For more information about function calling, see the [Function Calling Documentation](/appstore/modules/genai/function-calling/). +* **Function Calling** - You can use function calling in all chat completions operations using a [supported model](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html) by adding a `ToolCollection` with a `Tool` via the [Tools: Add Function to Request](/appstore/modules/genai/genai-for-mx/commons/#add-function-to-request) operation. For more information about function calling, see the [Function Calling Documentation](/appstore/modules/genai/function-calling/). **Function calling microflows**: A microflow used as a tool for function calling must satisfy the following conditions: 1. One input parameter of type String or no input parameter. 2. Return value of type String. -* **Vision** - This operation supports the *vision* capability for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). With vision, you can send image prompts, in addition to the traditional text prompts. You can use vision by adding a `FileCollection` with a `File` to the `Message` using the [Files: Initialize Collection with File](/appstore/modules/genai/commons/#initialize-filecollection) or the [Files: Add to Collection](/appstore/modules/genai/commons/#add-file-to-collection) operation. Make sure to set the `FileType` attribute to **image**. +* **Vision** - This operation supports the *vision* capability for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). With vision, you can send image prompts, in addition to the traditional text prompts. You can use vision by adding a `FileCollection` with a `File` to the `Message` using the [Files: Initialize Collection with File](/appstore/modules/genai/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/appstore/modules/genai/genai-for-mx/commons/#add-file-to-collection) operation. Make sure to set the `FileType` attribute to **image**. -* **Document Chat** - This operation supports the ability to chat with documents for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). To send a document to the model add a `FileCollection` with a `System.FileDocument` to the `Message` using the [Files: Initialize Collection with File](/appstore/modules/genai/commons/#initialize-filecollection) or the [Files: Add to Collection](/appstore/modules/genai/commons/#add-file-to-collection) operation. For Document Chat, it is not supported to create a `FileContent` from an URL using the above mentioned operations; Please use the `System.FileDocument` option. Make sure to set the `FileType` attribute to **document**. +* **Document Chat** - This operation supports the ability to chat with documents for [supported models](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-supported-models-features.html). To send a document to the model add a `FileCollection` with a `System.FileDocument` to the `Message` using the [Files: Initialize Collection with File](/appstore/modules/genai/genai-for-mx/commons/#initialize-filecollection) or the [Files: Add to Collection](/appstore/modules/genai/genai-for-mx/commons/#add-file-to-collection) operation. For Document Chat, it is not supported to create a `FileContent` from an URL using the above mentioned operations; Please use the `System.FileDocument` option. Make sure to set the `FileType` attribute to **document**. #### RetrieveAndGenerate {#retrieve-and-generate} @@ -270,17 +270,17 @@ The history can be enabled using the `SessionId` parameter on the RetrieveAndGen This activity was introduced in Amazon Bedrock Connector version 3.1.0. {{% /alert %}} -The [Generate Image](/appstore/modules/genai/commons/#generate-image) operation can be used to generate one or more images. Currently *Amazon Titan Image Generator G1* is the only supported model for image generation of the Amazon Bedrock Connector. +The [Generate Image](/appstore/modules/genai/genai-for-mx/commons/#generate-image) operation can be used to generate one or more images. Currently *Amazon Titan Image Generator G1* is the only supported model for image generation of the Amazon Bedrock Connector. -`GenAICommons.ImageOptions` can be an empty object. If provided, it allows you to set additional options for Image Generation and can be created by using the [Image: Create Options](/appstore/modules/genai/commons/#imageoptions-create) operation of GenAI Commons. +`GenAICommons.ImageOptions` can be an empty object. If provided, it allows you to set additional options for Image Generation and can be created by using the [Image: Create Options](/appstore/modules/genai/genai-for-mx/commons/#imageoptions-create) operation of GenAI Commons. -To retrieve actual image objects from the response, you can use the [Image: Get Generated Image (Single)](/appstore/modules/genai/commons/#image-get-single) or [Image: Get Generated Images (List)](/appstore/modules/genai/commons/#image-get-list) helper operations from GenAI Commons. +To retrieve actual image objects from the response, you can use the [Image: Get Generated Image (Single)](/appstore/modules/genai/genai-for-mx/commons/#image-get-single) or [Image: Get Generated Images (List)](/appstore/modules/genai/genai-for-mx/commons/#image-get-list) helper operations from GenAI Commons. For Titan Image models, the `Image Generation: Add Titan Image Extension` operation can be used to configure Titan image-specific values (currently only *NegativeText*). #### Generate Embeddings (String) {#embeddings-single-string} -The [Generate Embeddings (String)](/appstore/modules/genai/commons/#embeddings-string) activity can be used to generate an embedding vector for a given input string with one of the Cohere Embed models or Titan Embeddings v2. +The [Generate Embeddings (String)](/appstore/modules/genai/genai-for-mx/commons/#embeddings-string) activity can be used to generate an embedding vector for a given input string with one of the Cohere Embed models or Titan Embeddings v2. For Cohere Embed and Titan Embeddings, the request can be associated to their respective EmbeddingsOptions extension object which can be created with the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) or [Embeddings Options: Add Titan Embeddings Extension](#add-titan-embeddings-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. @@ -288,7 +288,7 @@ Currently, embeddings are available for the Cohere Embed family and or Titan Emb #### Generate Embeddings (Chunk Collection) {#embeddings-chunk-collection} -The [Generate Embeddings (Chunk Collection)](/appstore/modules/genai/commons/#embeddings-chunk-collection) activity can be used to generate a collection of embedding vectors for a given collection of text chunks with one of the Cohere Embed models or Titan Embeddings v2. +The [Generate Embeddings (Chunk Collection)](/appstore/modules/genai/genai-for-mx/commons/#embeddings-chunk-collection) activity can be used to generate a collection of embedding vectors for a given collection of text chunks with one of the Cohere Embed models or Titan Embeddings v2. For each model family, the request can be associated to an extension of the EmbeddingsOptions object which can be created with either the [Embeddings Options: Add Cohere Embed Extension](#add-cohere-embed-extension) or the [Embeddings Options: Add Titan Embeddings Extension](#add-titan-embeddings-extension) operation. Through this extension, it is possible to tailor the operation to more specific needs. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/_index.md index 5f62dd5c247..23f02cd0d3f 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/_index.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/_index.md @@ -2,7 +2,7 @@ title: "Get Started with Generative AI" url: /appstore/modules/genai/get-started/ linktitle: "Get Started with GenAI" -weight: 10 +weight: 20 description: "Describes the concepts behind generative AI and what you might implement with it." aliases: - /appstore/modules/genai/using-gen-ai/ diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/prompt-engineering.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/prompt-engineering.md index 11a9be51323..c6ff720bb3e 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/prompt-engineering.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/prompt-engineering.md @@ -37,9 +37,9 @@ A user prompt is another fundamental type. It is the user’s input, question, o ### Context Prompt -Depending on the project or use case, adding contextual information to the model may be necessary. Normally, this information, called context prompt or conversation history, is sent in the same interaction as the system and user prompt. It captures the historical information of the conversation to maintain coherence with the end user and be context aware. In the Mendix app chatbot setup, developers configure this within their application, and it is included in the request sent to the LLM using the [Chat Completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history) operation. +Depending on the project or use case, adding contextual information to the model may be necessary. Normally, this information, called context prompt or conversation history, is sent in the same interaction as the system and user prompt. It captures the historical information of the conversation to maintain coherence with the end user and be context aware. In the Mendix app chatbot setup, developers configure this within their application, and it is included in the request sent to the LLM using the [Chat Completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history) operation. -To understand this concept, imagine a user interacting with a chatbot while asking, *How should I start?*. If in previous interactions, the user asked about Mendix, the LLM will understand that the question refers to the Mendix apps. In cases where the context is not needed, such as in command-based interactions where the inquiry could be: *Turn on the lights* and the LLM does not need any historical conversation, developers can use operations like [Chat Completions (without history)](/appstore/modules/genai/commons/#chat-completions-without-history). +To understand this concept, imagine a user interacting with a chatbot while asking, *How should I start?*. If in previous interactions, the user asked about Mendix, the LLM will understand that the question refers to the Mendix apps. In cases where the context is not needed, such as in command-based interactions where the inquiry could be: *Turn on the lights* and the LLM does not need any historical conversation, developers can use operations like [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history). ## Typical Components of a Prompt diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/rag-example-implementation.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/rag-example-implementation.md index 79662a9ec69..a4f9b47fac6 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/rag-example-implementation.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/concepts/rag-example-implementation.md @@ -25,7 +25,7 @@ Every LLM will have its algorithm for generating vectors, but the convention is #### Chunk -In the context of GenAI Commons in a Mendix app, embedding vectors are generated using a [Chunk](/appstore/modules/genai/commons/#chunk-entity). Each object represents a discrete piece of information and contains its original string representation, as well as (after the embedding operation) the vector representation of that string according to the LLM of choice. +In the context of GenAI Commons in a Mendix app, embedding vectors are generated using a [Chunk](/appstore/modules/genai/genai-for-mx/commons/#chunk-entity). Each object represents a discrete piece of information and contains its original string representation, as well as (after the embedding operation) the vector representation of that string according to the LLM of choice. #### Knowledge base @@ -35,11 +35,11 @@ In the context of GenAI Commons in a Mendix app, we use the [PgVector Knowledge #### Knowledge base chunk -In most use cases, more information needs to be stored than just the original input string and its vector representation. A [KnowledgeBaseChunk](/appstore/modules/genai/commons/#knowledgebasechunk-entity) is an extension of [Chunk](/appstore/modules/genai/commons/#chunk-entity) that can hold additional information that is typically required for useful insertion and retrieval from a Mendix application. +In most use cases, more information needs to be stored than just the original input string and its vector representation. A [KnowledgeBaseChunk](/appstore/modules/genai/genai-for-mx/commons/#knowledgebasechunk-entity) is an extension of [Chunk](/appstore/modules/genai/genai-for-mx/commons/#chunk-entity) that can hold additional information that is typically required for useful insertion and retrieval from a Mendix application. #### Metadata -If additional conventional filtering is needed during similarity searches, such additional data can be stored in the knowledge base as well. [Metadata](/appstore/modules/genai/commons/#metadata-entity) objects are key-value pairs that are inserted along with the chunks and contain this additional information. The filtering is applied on an exact string-match basis for the key-value pair. Records are only retrieved if they match all records of the metadata in the collection provided as part of the search step. +If additional conventional filtering is needed during similarity searches, such additional data can be stored in the knowledge base as well. [Metadata](/appstore/modules/genai/genai-for-mx/commons/#metadata-entity) objects are key-value pairs that are inserted along with the chunks and contain this additional information. The filtering is applied on an exact string-match basis for the key-value pair. Records are only retrieved if they match all records of the metadata in the collection provided as part of the search step. {{% alert color="info" %}}The example described in the remainder of this document does not include the more advanced use case of metadata filtering nor does it cover the construction of complex input strings. If you want to see how this can work in practice, take a look at the *RAG with Semantic Search on Historical Data* example in the [GenAI Showcase app](https://marketplace.mendix.com/link/component/220475). {{% /alert %}} @@ -69,7 +69,7 @@ In summary, in the first step, you need to provide the private knowledge base, s Before you start experimenting with the end-to-end process, make sure that you have access to a (remote) PostgreSQL database with the [pgvector](https://github.com/pgvector/pgvector) extension available. If you do not have one yet, [learn more](/appstore/modules/genai/pgvector-setup/) about how a PostgreSQL vector database can be set up to explore use cases with knowledge bases. -{{% alert color="info" %}}If you have access to an Amazon Web Services (AWS) account or Microsoft Azure account, Mendix recommends you use a setup described in the [Creating a PostgreSQL Database with Amazon RDS](/appstore/modules/genai/pgvector-setup/#aws-database-create) or [Managing a PostgreSQL Database with Microsoft Azure](/appstore/modules/genai/pgvector-setup/#azure-database) section. This is convenient, since these PostgreSQL databases in the cloud have the required pgvector extension available by default.{{% /alert %}} +{{% alert color="info" %}}If you have access to an Amazon Web Services (AWS) account or Microsoft Azure account, Mendix recommends you use a setup described in the [Creating a PostgreSQL Database with Amazon RDS](/appstore/modules/genai/reference-guide/external-connectors/pgvector-setup/#aws-database-create) or [Managing a PostgreSQL Database with Microsoft Azure](/appstore/modules/genai/reference-guide/external-connectors/pgvector-setup/#azure-database) section. This is convenient, since these PostgreSQL databases in the cloud have the required pgvector extension available by default.{{% /alert %}} ### Steps {#steps} diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/Mx GenAI Connector.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/Mx GenAI Connector.md similarity index 79% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/Mx GenAI Connector.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/Mx GenAI Connector.md index 33f8e002655..7107fd025c0 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/Mx GenAI Connector.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/Mx GenAI Connector.md @@ -1,9 +1,11 @@ --- title: "Mendix Cloud GenAI Connector" -url: /appstore/modules/genai/MxGenAI/ +url: /appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/ linktitle: "Mendix Cloud GenAI Connector" -description: "Describes the configuration and usage of the Mendix Cloud GenAI Connector, which allows you to utilize Mendix Cloud GenAI Resource Packs directly within your Mendix application." -weight: 60 +description: "Describes the configuration and usage of the Mendix Cloud GenAI Connector, enabling you to integrate Mendix Cloud GenAI Resource Packs directly into your Mendix application." +weight: 20 +aliases: + - /appstore/modules/genai/MxGenAI/ --- ## Introduction @@ -101,7 +103,7 @@ After following the general setup above, you are ready to use the chat completio These microflows expect a `DeployedModel` as input to determine the connection details. -In chat completions, system prompts and user prompts are two key components that help guide the language model in generating relevant and contextually appropriate responses. For more information on prompt engineering, see the [Read More](#readmore) section. Different exposed microflow activities may require different prompts and logic for how the prompts must be passed, as described in the following sections. For more information on message roles, see the [ENUM_MessageRole](/appstore/modules/genai/commons/#enum-messagerole) enumeration in *GenAI Commons*. +In chat completions, system prompts and user prompts are two key components that help guide the language model in generating relevant and contextually appropriate responses. For more information on prompt engineering, see the [Read More](#readmore) section. Different exposed microflow activities may require different prompts and logic for how the prompts must be passed, as described in the following sections. For more information on message roles, see the [ENUM_MessageRole](/appstore/modules/genai/genai-for-mx/commons/#enum-messagerole) enumeration in *GenAI Commons*. Apart from `Retrieve and Generate (MxCloud, without history)`, the chat completion operations support [Function Calling](#function-calling), [Vision](#vision), and [Document Chat](#document-chat). @@ -109,21 +111,21 @@ For more inspiration or guidance on how to use the above-mentioned microflows in #### Chat Completions (without History) -The microflow activity [Chat Completions (without history)](/appstore/modules/genai/commons/#chat-completions-without-history) supports scenarios where there is no need to send a list of (historic) messages comprising the conversation so far as part of the request. +The microflow activity [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history) supports scenarios where there is no need to send a list of (historic) messages comprising the conversation so far as part of the request. #### Chat Completions (with History) -The microflow activity [Chat completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history) supports more complex use cases where a list of (historical) messages (for example, the conversation or context so far) is sent as part of the request to the LLM. +The microflow activity [Chat completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history) supports more complex use cases where a list of (historical) messages (for example, the conversation or context so far) is sent as part of the request to the LLM. #### Chat Completions (Retrieve & Generate) -The microflow activity `Retrieve and Generate (MxCloud, without history)` simplifies `Retrieve and Generate` use cases without history. By providing a user prompt, the knowledge base is searched for similar knowledge chunks, which are then passed to the model. The model is instructed to base its response on the retrieved knowledge while referring to the source used to generate the response. This operation requires a [Request](/appstore/modules/genai/commons/#request) which is associated to a `RetrieveAndGenerateRequest_Extension` pointing to a `MxKnowledgebaseConnection` object. Please use the flow shown below as orientation when setting up your logic to make sure that all is implemented as required: +The microflow activity `Retrieve and Generate (MxCloud, without history)` simplifies `Retrieve and Generate` use cases without history. By providing a user prompt, the knowledge base is searched for similar knowledge chunks, which are then passed to the model. The model is instructed to base its response on the retrieved knowledge while referring to the source used to generate the response. This operation requires a [Request](/appstore/modules/genai/genai-for-mx/commons/#request) which is associated to a `RetrieveAndGenerateRequest_Extension` pointing to a `MxKnowledgebaseConnection` object. Please use the flow shown below as orientation when setting up your logic to make sure that all is implemented as required: {{< figure src="/attachments/appstore/platform-supported-content/modules/genai/mxgenAI-connector/MxGenAIConnector_ConfigureRAG.png" >}} A `SystemPrompt` can be provided through the `Request` and other filter options can be set when initializing the `RetrieveAndGenerateRequest_Extension` (for example through metadata). -The returned `Response` includes [References](/appstore/modules/genai/commons/#reference) if the model used them to generate its response. In some cases, a knowledge chunk consists of two texts: one for the semantic search step and another for the generation step. For example, when solving a problem based on historical solutions, the semantic search identifies similar problems using their descriptions, while the generation step produces a solution based on the corresponding historical solutions. In those cases, you can add [MetaData](/appstore/modules/genai/commons/#chunkcollection-add-knowledgebasechunk) with the key `knowledge` to the chunks during the insertion stage, allowing the model to base its response on the specified metadata rather than the input text. +The returned `Response` includes [References](/appstore/modules/genai/genai-for-mx/commons/#reference) if the model used them to generate its response. In some cases, a knowledge chunk consists of two texts: one for the semantic search step and another for the generation step. For example, when solving a problem based on historical solutions, the semantic search identifies similar problems using their descriptions, while the generation step produces a solution based on the corresponding historical solutions. In those cases, you can add [MetaData](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-knowledgebasechunk) with the key `knowledge` to the chunks during the insertion stage, allowing the model to base its response on the specified metadata rather than the input text. Additionally, to utilize the `Source` attribute of the references, you can include `MetaData` with the key `sourceUrl`. Finally, the `HumanReadableId` of a chunk is used to display the reference's title in the response. @@ -133,7 +135,7 @@ Function calling enables LLMs to connect with external tools to gather informati The model does not call the function but rather returns a tool called JSON structure that is used to build the input of the function (or functions) so that they can be executed as part of the chat completions operation. Functions in Mendix are essentially microflows that can be registered within the request to the LLM​. The connector takes care of handling the tool call response and executing the function microflows until the API returns the assistant's final response. -Function microflows take a single input parameter of type string or no input parameter and must return a string. Currently, adding a [ToolChoice](/appstore/modules/genai/commons/#set-toolchoice) for function calling is not supported by the Mendix Cloud GenAI Connector. +Function microflows take a single input parameter of type string or no input parameter and must return a string. Currently, adding a [ToolChoice](/appstore/modules/genai/genai-for-mx/commons/#set-toolchoice) for function calling is not supported by the Mendix Cloud GenAI Connector. {{% alert color="warning" %}} Function calling is a highly effective capability and should be used with caution. Function microflows run in the context of the current user, without enforcing entity access. You can use `$currentUser` in XPath queries to ensure that you retrieve and return only information that the end-user is allowed to view; otherwise, confidential information may become visible to the current end-user in the assistant's response. @@ -141,22 +143,22 @@ Function calling is a highly effective capability and should be used with cautio Mendix also strongly advises that you build user confirmation logic into function microflows that have a potential impact on the world on behalf of the end-user. Some examples of such microflows include sending an email, posting online, or making a purchase. {{% /alert %}} -You can use function calling in all chat completions operations by adding a `ToolCollection` with a `Function` via the [Tools: Add Function to Request](/appstore/modules/genai/commons/#add-function-to-request) operation. +You can use function calling in all chat completions operations by adding a `ToolCollection` with a `Function` via the [Tools: Add Function to Request](/appstore/modules/genai/genai-for-mx/commons/#add-function-to-request) operation. For more information, see [Function Calling](/appstore/modules/genai/function-calling/). #### Vision{#vision} -Vision enables the model to interpret and analyze images, allowing them to answer questions and perform tasks related to visual content. This integration of computer vision and language processing enhances the model's comprehension and makes it valuable for tasks involving visual information. To ensure vision inside the connector, an optional [FileCollection](/appstore/modules/genai/commons/#filecollection) containing one or multiple images must be sent with a single message. +Vision enables the model to interpret and analyze images, allowing them to answer questions and perform tasks related to visual content. This integration of computer vision and language processing enhances the model's comprehension and makes it valuable for tasks involving visual information. To ensure vision inside the connector, an optional [FileCollection](/appstore/modules/genai/genai-for-mx/commons/#filecollection) containing one or multiple images must be sent with a single message. -For [Chat Completions (without history)](/appstore/modules/genai/commons/#chat-completions-without-history), `OptionalFileCollection` is an optional input parameter. For [Chat completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history), a `FileCollection` can optionally be added to individual user messages using [Add Message to Request](/appstore/modules/genai/commons/#chat-add-message-to-request). +For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history), `OptionalFileCollection` is an optional input parameter. For [Chat completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history), a `FileCollection` can optionally be added to individual user messages using [Add Message to Request](/appstore/modules/genai/genai-for-mx/commons/#chat-add-message-to-request). In the entire conversation, you can pass up to 20 images that are smaller than 3.75 MB each and with a height and width of a maximum 8000 pixels. The following types are accepted: PNG, JPEG, JPG, GIF, and WebP. #### Document Chat{#document-chat} -Document chat enables the model to interpret and analyze documents, such as PDFs or Excel files, allowing them to answer questions and perform tasks related to the content. To use document chat, an optional [FileCollection](/appstore/modules/genai/commons/#filecollection) containing one or multiple documents must be sent along with a single message. +Document chat enables the model to interpret and analyze documents, such as PDFs or Excel files, allowing them to answer questions and perform tasks related to the content. To use document chat, an optional [FileCollection](/appstore/modules/genai/genai-for-mx/commons/#filecollection) containing one or multiple documents must be sent along with a single message. -For [Chat Completions (without history)](/appstore/modules/genai/commons/#chat-completions-without-history), `OptionalFileCollection` is an optional input parameter. For [Chat completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history), a `FileCollection` can optionally be added to individual user messages using [Add Message to Request](/appstore/modules/genai/commons/#chat-add-message-to-request). +For [Chat Completions (without history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-without-history), `OptionalFileCollection` is an optional input parameter. For [Chat completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history), a `FileCollection` can optionally be added to individual user messages using [Add Message to Request](/appstore/modules/genai/genai-for-mx/commons/#chat-add-message-to-request). In the entire conversation, you can pass up to five documents that are smaller than 4.5 MB each. The following file types are accepted: PDF, CSV, DOC, DOCX, XLS, XLSX, HTML, TXT, and MD. @@ -168,7 +170,7 @@ Note that the model uses the file name when analyzing documents, which could mak ### Knowledge Base Operations -To implement knowledge base logic into your Mendix application, you can use the actions in the **USE_ME** > **Knowledge Base** folder or under the **GenAI Knowledge Base (Content)** or **Mendix Cloud Knowledge Base** categories in the **Toolbox**. These actions require a specialized [Connection](/appstore/modules/genai/commons/#connection) of type `MxKnowledgeBaseConnection` that determines the model and endpoint to use. Additionally, the collection name must be passed when creating the object and it must be associated with a `Configuration` object. Please note that for Mendix Cloud a knowledge base resource may contain several collections (tables). +To implement knowledge base logic into your Mendix application, you can use the actions in the **USE_ME** > **Knowledge Base** folder or under the **GenAI Knowledge Base (Content)** or **Mendix Cloud Knowledge Base** categories in the **Toolbox**. These actions require a specialized [Connection](/appstore/modules/genai/genai-for-mx/commons/#connection) of type `MxKnowledgeBaseConnection` that determines the model and endpoint to use. Additionally, the collection name must be passed when creating the object and it must be associated with a `Configuration` object. Please note that for Mendix Cloud a knowledge base resource may contain several collections (tables). Dealing with knowledge bases involves two main stages: @@ -181,7 +183,7 @@ You do not need to manually add embeddings to a chunk, as the connector handles ##### Data Chunks -To add data to the knowledge base, you need discrete pieces of information and create knowledge base chunks for each one. Use the GenAICommons operations to first [initialize a ChunkCollection object](/appstore/modules/genai/commons/#chunkcollection-create), and then [add a KnowlegdebaseChunk](/appstore/modules/genai/commons/#chunkcollection-add-knowledgebasechunk) object to it for each piece of information. Both can be found in the **Toolbox** inside of the **GenAI Knowledge Base (Content)** category. +To add data to the knowledge base, you need discrete pieces of information and create knowledge base chunks for each one. Use the GenAICommons operations to first [initialize a ChunkCollection object](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-create), and then [add a KnowlegdebaseChunk](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-knowledgebasechunk) object to it for each piece of information. Both can be found in the **Toolbox** inside of the **GenAI Knowledge Base (Content)** category. ##### Chunking Strategy @@ -193,9 +195,9 @@ The chunk collection can then be stored in the knowledge base using one of the f Use the following toolbox actions inside the **Mendix Cloud Knowledge Base** toolbox category to populate knowledge data into the knowledge base: -1. `Embed & Insert` embeds a list of chunks (passed via a [ChunkCollection](/appstore/modules/genai/commons/#chunkcollection)) and inserts them into the knowledge base. +1. `Embed & Insert` embeds a list of chunks (passed via a [ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection)) and inserts them into the knowledge base. 2. `Embed & repopulate KB` is similar to the `Embed & Insert`, but deletes all existing chunks from the knowledge base before inserting the new chunks. -3. `Embed & Replace` replaces existing chunks in the knowledge base that match the associated Mendix object which was passed via [Add KnowledgeBaseChunk to ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-add-knowledgebasechunk) action at the insertion stage. +3. `Embed & Replace` replaces existing chunks in the knowledge base that match the associated Mendix object which was passed via [Add KnowledgeBaseChunk to ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-knowledgebasechunk) action at the insertion stage. Additionally, use the following toolbox actions to delete chunks: @@ -218,7 +220,7 @@ The following toolbox actions can be used to retrieve knowledge data from the kn {{% alert color="info" %}}You must define your entity specialized from `KnowledgeBaseChunk`, which is associated to the entity that was used to pass a MendixObject during the [insertion stage](#knowledge-base-insertion). {{% /alert %}} -3. `Embed & Retrieve Nearest Neighbors` retrieves a list of type [KnowledgeBaseChunk](/appstore/modules/genai/commons/#knowledgebasechunk-entity) from the knowledge base that are most similar to a given `Content` by calculating the cosine similarity of its vectors. +3. `Embed & Retrieve Nearest Neighbors` retrieves a list of type [KnowledgeBaseChunk](/appstore/modules/genai/genai-for-mx/commons/#knowledgebasechunk-entity) from the knowledge base that are most similar to a given `Content` by calculating the cosine similarity of its vectors. 4. `Embed & Retrieve Nearest Neighbors & Associate` combines the above actions `Retrieve & Associate` and `Embed & Retrieve Nearest Neighbors`. ### Embedding Operations @@ -227,15 +229,15 @@ If you are working directly with embedding vectors for specific use cases that d To implement embeddings into your Mendix application, you can use the microflows in the **Knowledge Bases & Embeddings** folder inside of the GenAICommons module. Both microflows for embeddings are exposed as microflow actions under the **GenAI (Generate)** category in the **Toolbox** in Mendix Studio Pro. -These microflows require a [DeployedModel](/appstore/modules/genai/commons/#deployed-model) that determines the model and endpoint to use. Depending on the selected operation, an `InputText` String or a [ChunkCollection](/appstore/modules/genai/commons/#chunkcollection) needs to be provided. +These microflows require a [DeployedModel](/appstore/modules/genai/genai-for-mx/commons/#deployed-model) that determines the model and endpoint to use. Depending on the selected operation, an `InputText` String or a [ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection) needs to be provided. #### Embeddings (String) -The microflow activity [Generate Embeddings (String)](/appstore/modules/genai/commons/#embeddings-string) supports scenarios where the vector embedding of a single string must be generated. This input string can be passed directly as the `TextInput` parameter of this microflow. Note that the parameter [EmbeddingsOptions](/appstore/modules/genai/commons/#embeddingsoptions-entity) is optional. Use the exposed microflow [Embeddings: Get First Vector from Response](/appstore/modules/genai/commons/#embeddings-get-first-vector) to retrieve the generated embeddings vector. +The microflow activity [Generate Embeddings (String)](/appstore/modules/genai/genai-for-mx/commons/#embeddings-string) supports scenarios where the vector embedding of a single string must be generated. This input string can be passed directly as the `TextInput` parameter of this microflow. Note that the parameter [EmbeddingsOptions](/appstore/modules/genai/genai-for-mx/commons/#embeddingsoptions-entity) is optional. Use the exposed microflow [Embeddings: Get First Vector from Response](/appstore/modules/genai/genai-for-mx/commons/#embeddings-get-first-vector) to retrieve the generated embeddings vector. #### Embeddings (ChunkCollection) -The microflow activity [Generate Embeddings (ChunkCollection)](/appstore/modules/genai/commons/#embeddings-chunk-collection) supports the more complex scenario where a collection of [Chunk](/appstore/modules/genai/commons/#chunkcollection) objects is vectorized in a single API call, such as when converting a collection of text strings (chunks) from a private knowledge base into embeddings. Instead of calling the API for each string, executing a single call for a list of strings can significantly reduce HTTP overhead. The embedding vectors returned after a successful API call will be stored as an `EmbeddingVector` attribute in the same `Chunk` object. Use the exposed microflows of GenAI Commons [Chunks: Initialize ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-create), [Chunks: Add Chunk to ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-add-chunk), or [Chunks: Add KnowledgeBaseChunk to ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-add-knowledgebasechunk) to construct the input. +The microflow activity [Generate Embeddings (ChunkCollection)](/appstore/modules/genai/genai-for-mx/commons/#embeddings-chunk-collection) supports the more complex scenario where a collection of [Chunk](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection) objects is vectorized in a single API call, such as when converting a collection of text strings (chunks) from a private knowledge base into embeddings. Instead of calling the API for each string, executing a single call for a list of strings can significantly reduce HTTP overhead. The embedding vectors returned after a successful API call will be stored as an `EmbeddingVector` attribute in the same `Chunk` object. Use the exposed microflows of GenAI Commons [Chunks: Initialize ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-create), [Chunks: Add Chunk to ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-chunk), or [Chunks: Add KnowledgeBaseChunk to ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-knowledgebasechunk) to construct the input. To create embeddings, it does not matter whether the ChunkCollection contains Chunks or its specialization KnowledgeBaseChunks. Note that the knowledge base operations handle the embedding generation themselves internally. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/_index.md new file mode 100644 index 00000000000..9f8aad6c83d --- /dev/null +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/_index.md @@ -0,0 +1,26 @@ +--- +title: "Mendix Cloud GenAI" +url: /appstore/modules/genai/mx-cloud-genai/ +linktitle: "Mendix Cloud GenAI" +weight: 30 +description: "Provides guidance on how to navigate through the Mendix Cloud GenAI Resource Packs." +no_list: false +--- + +## Introduction + +In order to help developers integrate GenAI capabilities into custom applications, Mendix Cloud provides GenAI Resource Packs. These packs offer access to Large Language Models (for text generation and text embeddings) and knowledge bases, enabling seamless implementation of common GenAI patterns in a low-code environment. They simplify the process of leveraging GenAI technologies for Mendix customers and partners by abstracting complex provisioning processes and reducing configuration to just a few clicks within the platform experience. Feel free to contact [genai-resource-packs@mendix.com](mailto:genai-resource-packs@mendix.com) to learn more. + +## Resources Overview + +The GenAI Resources portal provides an overview of all the resources you have access to. You can also request new GenAI Resources through the portal. + +There are three different types of resources: + +* Compute – Text Generation: generates human-like text based on given inputs, essential for applications requiring natural language generation. + +* Knowledge Base: A knowledge base can be used to upload your data which then can be used by a text generation resource. + +* Compute – Embeddings Generation: converts text into vector representations. An embeddings resource is required to uploading data to your Knowledge Base. + +## Relevant Sources diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/mendix-cloud-grp.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/mendix-cloud-grp.md new file mode 100644 index 00000000000..b63be760887 --- /dev/null +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/mendix-cloud-genai/mendix-cloud-grp.md @@ -0,0 +1,87 @@ +--- +title: "Mendix Cloud GenAI Resource Packs" +url: /appstore/modules/genai/mx-cloud-genai/resource-packs +linktitle: "Mendix Cloud GenAI Resource Packs" +description: "Provides an overview of Mendix Cloud GenAI Resource Packs, including their capabilities, limitations, and frequently asked questions (FAQ)" +weight: 10 +--- + +## Introduction + +Mendix Cloud GenAI Resource Packs provide turn-key access to Generative AI technology, delivered through Mendix Cloud. + +* Model Resource Packs offer customers access to large language model capacity. Each resource pack includes an allocation of input/output tokens for Anthropic's Claude and Cohere's Embed. Support for additional models will be introduced in the future. + +* Knowledge Base Resource Packs provide an OpenSearch-based vector database to support Retrieval-Augmented Generation (RAG), Semantic Search, and other Generative AI use cases. + +Developers can use the Mendix Portal to manage their Mendix Cloud GenAI resources and seamlessly integrate model and knowledge base capabilities into their Mendix applications using the [Mendix Cloud GenAI Connector](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/). Optimized for high performance and low latency, Mendix Cloud GenAI Resource Packs provide the easiest and fastest way to deliver end-to-end Generative AI solutions on a single platform. + +### Limited Availability + +Mendix Cloud GenAI Resource Packs are currently available under limited availability. Mendix is collaborating with early adopters—including customers, partners, and ISVs—to drive successful project outcomes. Access to GenAI Resource Packs is evaluated on a case-by-case basis. To learn more about the conditions for access during the limited availability phase, contact [genai-resource-packs@mendix.com](mailto:genai-resource-packs@mendix.com) until the general availability release. + +## Models + +Mendix Cloud Model Resource Packs provide customers with a monthly quota of input and output tokens for Anthropic's Claude and Cohere's Embed models. This allows customers to implement typical Generative AI use cases like: + +### Supported models + +Mendix Cloud provides access to the following models: + +* Anthropic Claude v3.5 Sonnet v1 +* Cohere Embed v3 (English & multilingual options) + +The models are available through the Mendix Cloud, leveraging AWS's highly secure Amazon Bedrock multi-tenant architecture. This architecture employs advanced logical isolation techniques to effectively segregate customer data, requests, and responses, ensuring a level of data protection that aligns with global security compliance requirements. Customer prompts, requests, and responses are neither stored nor used for model training. Your data remains your data. + +Customers looking to leverage other models in addition to the above can also take advantage of Mendix's [(Azure) OpenAI Connector](/appstore/modules/genai/reference-guide/external-connectors/openai/) and Amazon [Bedrock Connector](/appstore/modules/genai/reference-guide/external-connectors/bedrock/) to integrate numerous other models into their apps. + +## Knowledge Bases + +Mendix Cloud Knowledge Base Resource Packs provide customers with an elastic, logically isolated vector database, to use for standard Generative AI architectural patterns such as Retrieval-Augmented Generation (RAG), semantic similarity search, and other Generative AI use cases. The Knowledge Bases on Mendix Cloud are based on AWS's highly secure Amazon Bedrock Knowledge Bases capability, combined with AWS' OpenSearch Serverless database— a widely adopted standard infrastructure for Generative AI Knowledge Bases on AWS, ensuring fast & accurate information retrieval. + +Knowledge bases enable you to bring your own data for RAG, semantic similarity search, and other generative AI use cases: + +* Make your app's data available through integration +* Connect to third-party information sources +* Manage knowledge base content and add metadata labels + +Knowledge Bases are based on elastically scaling, serverless OpenSearch vector databases, to ensure high performance under load. The database is set up as a highly available cluster to ensure business continuity. Customer data is stored in logical isolation from other customers and is not used for model training, ensuring data security and privacy in compliance with industry standards. + +## Mendix Portal + +The Mendix Portal allows easy access to manage the resources, through the GenAI Resources section in the portal. + +* Get insight into the consumption of input/output tokens against entitlements for Models +* Manage content for Knowledge Bases +* Manage team access to all resources +* Create and manage connection keys to connect your apps with all resources +* Track activity logs for team access and connection key management + +## Mendix Cloud GenAI Connector + +The [Mendix Cloud GenAI connector](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/) lets you utilize Mendix Cloud GenAI resource packs directly within your Mendix application. It allows you to integrate generative AI by dragging and dropping common operations from its toolbox. + +## Regional Availability + +Mendix Cloud GenAI Resource Packs are available in the following regions of Mendix Cloud: + +* Europe (Frankfurt) - eu-central-1 + +## FAQ + +### What Happens to Data Processed by Mendix Cloud GenAI Services? + +For Mendix Cloud GenAI Model Resources using Anthropic’s Claude and Cohere’s Embed, neither Mendix nor its partners (Amazon, Anthropic, and Cohere) store any requests (prompts) or responses (answers, embeddings). Your data is not used for model training. + +Data stored in GenAI Knowledge Base Resources resides in a logically isolated database, accessible only to you—the customer—via keys you can generate in the Portal. + +### How does Mendix Cloud GenAI service Store and Use Data Sent to It? + +Requests (prompts) sent to and responses (answers, embeddings) received from the models are not stored and not used for training. Only metadata—such as token input/output counts—is collected for logging, monitoring, metering, billing, product improvement, and maintenance purposes. + +Data sent to the Knowledge Base (vectors, chunks) is stored in a logically isolated, fully secure vector database, following industry-standard practices. This data is exclusively accessible to you and not used by Mendix. Similar to model requests, only metadata about Knowledge Base usage is collected for logging, monitoring, metering, billing, product improvement, and maintenance purposes. + +### Read More + +* [Enrich your Mendix app with GenAI capabilities](/appstore/modules/genai/) +* [Build a Chatbot Using the AI Bot Starter App](/appstore/modules/genai/using-genai/starter-template/) diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/_index.md new file mode 100644 index 00000000000..913108b0fcf --- /dev/null +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/_index.md @@ -0,0 +1,14 @@ +--- +title: "Reference Guide" +url: /appstore/modules/genai/reference-guide/ +linktitle: "Reference Guide" +weight: 40 +description: "Provides references of Mendix's GenAI Modules and Tools." +no_list: false +--- + +## Introduction {#introduction} + +This guide provides comprehensive information on the tools and modules available within the Mendix platform. It helps you explore how to enhance your applications by integrating Generative AI and how each tool supports this process. Additionally, it includes technical reference guides to ensure you have all the information needed for effective implementation and optimization. + +## Documents in This Category diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/_index.md new file mode 100644 index 00000000000..4b3c30a89fa --- /dev/null +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/_index.md @@ -0,0 +1,14 @@ +--- +title: "Connectors to External Platforms" +url: /appstore/modules/genai/reference-guide/external-connectors/ +linktitle: "Connectors to External Platforms" +weight: 20 +description: "Provides information on connectors that enable seamless integration between Mendix applications and external platforms." +no_list: false +--- + +## Introduction + +The Mendix platform provides seamless integration with various external platforms through specialized connectors. These connectors enable you to extend the functionality of your applications by leveraging external services and data sources. This section introduces the connectors available for [Snowflake Cortex](/appstore/modules/genai/snowflake-cortex/), [OpenAI](/appstore/modules/genai/openai/), [Amazon Bedrock](/appstore/modules/genai/bedrock/), and [PGVector Knowledge Base](/appstore/modules/genai/pgvector/), providing a high-level overview of their capabilities. + +## Connectors \ No newline at end of file diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/bedrock.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/bedrock.md similarity index 88% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/bedrock.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/bedrock.md index 28523564fe6..d5bc5af557c 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/bedrock.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/bedrock.md @@ -1,9 +1,10 @@ --- title: "Amazon Bedrock" -url: /appstore/modules/genai/bedrock/ -weight: 40 +url: /appstore/modules/genai/reference-guide/external-connectors/bedrock/ +weight: 10 description: "Describes the Amazon Bedrock GenAI service." - +aliases: + - /appstore/modules/genai/bedrock/ --- ## Introduction diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/openai/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/openai.md similarity index 85% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/openai/_index.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/openai.md index 6e4df4f6de8..dd71438c21b 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/openai/_index.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/openai.md @@ -1,11 +1,12 @@ --- title: "OpenAI" -url: /appstore/modules/genai/openai/ +url: /appstore/modules/genai/reference-guide/external-connectors/openai/ linktitle: "OpenAI" description: "Describes the configuration and usage of the OpenAI Connector, which allows you to integrate generative AI into your Mendix app." -weight: 60 +weight: 20 aliases: - /appstore/connectors/openai-connector/ + - /appstore/modules/genai/openai/ --- ## Introduction {#introduction} @@ -114,10 +115,6 @@ The following inputs are required for the OpenAI configuration: | Endpoint | This is the API endpoint (for example, `https://api.openai.com/v1`) | | Token | This is the access token to authorize your API call.
To get an API, follow these steps:
  1. Create an account and sign in at [OpenAI](https://platform.openai.com/).
  2. Go to the [API key page](https://platform.openai.com/account/api-keys) to create a new secret key.
  3. Copy the API key and save this somewhere safe.
| -{{% alert color="info" %}} -If you have signed up for an OpenAI account and are using free trial credits, note that the credits are only valid for three months after the account is created (not after the API key is created). To continue using the OpenAI API with an account that is more than three months old, you must top up your account balance with credit and create a new API key. For more details, see the [OpenAI API reference](https://platform.openai.com/docs/api-reference/authentication). -{{% /alert %}} - #### Azure OpenAI Configuration {#azure-openai-configuration} The following inputs are required for the Azure OpenAI configuration: @@ -148,7 +145,7 @@ The following inputs are required for the Azure OpenAI configuration: #### Configuring the OpenAI Deployed Models -A [Deployed Model](/appstore/modules/genai/commons/#deployed-model) represents a GenAI model instance that can be used by the app to generate text, embeddings, or images. For every model you want to invoke from your app, you need to create a `OpenAIDeployedModel` record, a specialization of `DeployedModel`. In addition to the model display name and a technical name/identifier, an OpenAI deployed model contains a reference to the additional connection details as configured in the previous step. For OpenAI, a set of common models will be prepopulated automatically upon saving the configuration. If you want to use additional models that are made available by OpenAI you need to configure additional OpenAI deployed models in your Mendix app. For Azure OpenAI no deployed models are created by default. The technical model names depend on the deployment names that were chosen while deploying the models in the [Azure Portal](https://oai.azure.com/resource/deployments). Therefore in this case you always need to configure the deployed models manually in your Mendix app. +A [Deployed Model](/appstore/modules/genai/genai-for-mx/commons/#deployed-model) represents a GenAI model instance that can be used by the app to generate text, embeddings, or images. For every model you want to invoke from your app, you need to create a `OpenAIDeployedModel` record, a specialization of `DeployedModel`. In addition to the model display name and a technical name/identifier, an OpenAI deployed model contains a reference to the additional connection details as configured in the previous step. For OpenAI, a set of common models will be prepopulated automatically upon saving the configuration. If you want to use additional models that are made available by OpenAI you need to configure additional OpenAI deployed models in your Mendix app. For Azure OpenAI no deployed models are created by default. The technical model names depend on the deployment names that were chosen while deploying the models in the [Azure Portal](https://oai.azure.com/resource/deployments). Therefore in this case you always need to configure the deployed models manually in your Mendix app. 1. If needed, click the three dots for an OpenAI configuration to open the "Manage Deployed Models" pop-up. 2. For every additional model, add a record. The following fields are required: @@ -164,20 +161,20 @@ A [Deployed Model](/appstore/modules/genai/commons/#deployed-model) represents a ### Using GenAI Commons Operations {#genai-commons-operations} -After following the general setup above, you are all set to use the microflow actions under the **GenAI (Generate)** category from the toolbox. These operations are part of GenAI Commons. Since OpenAI is compatible with the principles of GenAI Commons, you can pass an `OpenAIDeployedModel` to all GenAI Commons operations that expect the generalization `DeployedModel`. All actions under **GenAI (Generate)** will take care of executing the right provider-specific logic, based on the type of specialization passed, in this case OpenAI. From an implementation perspective, it is not needed to inspect the inner workings of this operation. The input, output, and behavior are as described in the [GenAICommons documentation](/appstore/modules/genai/commons/#microflows). Applicable operations and some OpenAI-specific aspects are listed below. +After following the general setup above, you are all set to use the microflow actions under the **GenAI (Generate)** category from the toolbox. These operations are part of GenAI Commons. Since OpenAI is compatible with the principles of GenAI Commons, you can pass an `OpenAIDeployedModel` to all GenAI Commons operations that expect the generalization `DeployedModel`. All actions under **GenAI (Generate)** will take care of executing the right provider-specific logic, based on the type of specialization passed, in this case OpenAI. From an implementation perspective, it is not needed to inspect the inner workings of this operation. The input, output, and behavior are as described in the [GenAICommons documentation](/appstore/modules/genai/genai-for-mx/commons/#microflows). Applicable operations and some OpenAI-specific aspects are listed below. For more inspiration or guidance on how to use the microflow actions in your logic, Mendix recommends downloading our [GenAI Showcase App](https://marketplace.mendix.com/link/component/220475), which demonstrates a variety of examples that cover all the operations mentioned. #### Chat Completions -Operations for chat completions focus on the generation of text based on a certain input. In this context, system prompts and user prompts are two key components that help guide the language model in generating relevant and contextually appropriate responses. For more information on the type of prompts and message roles, see the [ENUM_MessageRole](/appstore/modules/genai/commons/#enum-messagerole) enumeration. To learn more about how to create the right prompts for your use case, see the prompt engineering links in the [Read More](#read-more) section. +Operations for chat completions focus on the generation of text based on a certain input. In this context, system prompts and user prompts are two key components that help guide the language model in generating relevant and contextually appropriate responses. For more information on the type of prompts and message roles, see the [ENUM_MessageRole](/appstore/modules/genai/genai-for-mx/commons/#enum-messagerole) enumeration. To learn more about how to create the right prompts for your use case, see the prompt engineering links in the [Read More](#read-more) section. -The `OpenAIDeployedModel` is compatible with the two [Chat Completions operations from GenAI Commons](/appstore/modules/genai/commons/#genai-generate). While developing your custom microflow, you can drag and drop the following operations from the toolbox in Studio Pro, see category **GenAI (Generate)**: +The `OpenAIDeployedModel` is compatible with the two [Chat Completions operations from GenAI Commons](/appstore/modules/genai/genai-for-mx/commons/#genai-generate). While developing your custom microflow, you can drag and drop the following operations from the toolbox in Studio Pro, see category **GenAI (Generate)**: * Chat Completions (with history) * Chat Completions (without history) -You can use the GenAI Commons toolbox actions to [create the required Request](/appstore/modules/genai/commons/#genai-request-building) and [handle the Response](/appstore/modules/genai/commons/#genai-response-handling) for your use case. +You can use the GenAI Commons toolbox actions to [create the required Request](/appstore/modules/genai/genai-for-mx/commons/#genai-request-building) and [handle the Response](/appstore/modules/genai/genai-for-mx/commons/#genai-response-handling) for your use case. The internal chat completion logic within the OpenAI connector supports [JSON mode](#chatcompletions-json-mode), [function calling](#chatcompletions-functioncalling), and [vision](#chatcompletions-vision). Make sure to check the actual compatibility of the available models with these functionalities, as this changes over time. Any specific OpenAI microflow actions from the toolbox are listed below. @@ -191,7 +188,7 @@ Function calling enables LLMs to connect with external tools to gather informati OpenAI does not call the function. The model returns a tool called JSON structure that is used to build the input of the function (or functions) so that they can be executed as part of the chat completions operation. Functions in Mendix are essentially microflows that can be registered within the request to the LLM​. The OpenAI connector takes care of handling the tool call response as well as executing the function microflows until the API returns the assistant's final response. -This is all part of the implementation that is executed by the GenAI Commons chat completions operations mentioned before. As a developer, you have to make the system aware of your functions and what these do by registering the function(s) to the request. This is done using the GenAI Commons operation [Tools: Add Function to Request](/appstore/modules/genai/commons/#add-function-to-request) once per function before passing the request to the chat completions operation. +This is all part of the implementation that is executed by the GenAI Commons chat completions operations mentioned before. As a developer, you have to make the system aware of your functions and what these do by registering the function(s) to the request. This is done using the GenAI Commons operation [Tools: Add Function to Request](/appstore/modules/genai/genai-for-mx/commons/#add-function-to-request) once per function before passing the request to the chat completions operation. Currently, the connector supports the calling of Function microflows that take a single input parameter of type string or no input parameter and return a string. @@ -205,11 +202,11 @@ For more information, see [Function Calling](/appstore/modules/genai/function-ca #### Vision {#chatcompletions-vision} -Vision enables models like GPT-4o and GPT-4 Turbo to interpret and analyze images, allowing them to answer questions and perform tasks related to visual content. This integration of computer vision and language processing enhances the model's comprehension and makes it valuable for tasks involving visual information. To make use of vision inside the OpenAI connector, an optional [FileCollection](/appstore/modules/genai/commons/#filecollection) containing one or multiple images must be sent along with a single message. +Vision enables models like GPT-4o and GPT-4 Turbo to interpret and analyze images, allowing them to answer questions and perform tasks related to visual content. This integration of computer vision and language processing enhances the model's comprehension and makes it valuable for tasks involving visual information. To make use of vision inside the OpenAI connector, an optional [FileCollection](/appstore/modules/genai/genai-for-mx/commons/#filecollection) containing one or multiple images must be sent along with a single message. For `Chat Completions without History`, `FileCollection` is an optional input parameter. -For `Chat Completions with History`, `FileCollection` can optionally be added to individual user messages using [Chat: Add Message to Request](/appstore/modules/genai/commons/#chat-add-message-to-request). +For `Chat Completions with History`, `FileCollection` can optionally be added to individual user messages using [Chat: Add Message to Request](/appstore/modules/genai/genai-for-mx/commons/#chat-add-message-to-request). Use the two OpenAI-specific microflow actions from the toolbox [Files: Initialize Collection with OpenAI File](#initialize-filecollection) and [Files: Add OpenAIFile to Collection](#add-file) to construct the input with either `FileDocuments` (for vision, it needs to be of type `Image`) or `URLs`. There are similar file operations exposed by the GenAI commons module that can be used for vision requests with the OpenAIConnector; however, these generic operations do not support the optional OpenAI-specific `Detail` attribute. @@ -223,37 +220,37 @@ For more information on vision, see [OpenAI](https://platform.openai.com/docs/gu #### Image Generations {#image-generations-configuration} -OpenAI also provides image generation capabilities which can be invoked using this connector module. The `OpenAIDeployedModel` entity is compatible with the [image generation operation from GenAI Commons](/appstore/modules/genai/commons/#generate-image). +OpenAI also provides image generation capabilities which can be invoked using this connector module. The `OpenAIDeployedModel` entity is compatible with the [image generation operation from GenAI Commons](/appstore/modules/genai/genai-for-mx/commons/#generate-image). To implement image generation into your Mendix application, you can use the Image generation microflow action from GenAI Commons directly. When developing your microflow, you can drag and drop it from the toolbox: find it under the **GenAI (Generate)** category in the **Toolbox** in Mendix Studio Pro: * Generate Image -When you drag this operation into your app microflow logic, use the `user prompt` to describe the desired image, and for the `DeployedModel` pass the relevant `OpenAIDeployedModel` that supports image generation. Additional parameters like the height and the width can be configured using [Image Generation: Create ImageOptions](/appstore/modules/genai/commons/#imageoptions-create). To configure OpenAI-specific options, like quality and style an extension to the ImageOptions can be added using [Image Generation: Set ImageOptions Extension](#set-imageoptions-extension). +When you drag this operation into your app microflow logic, use the `user prompt` to describe the desired image, and for the `DeployedModel` pass the relevant `OpenAIDeployedModel` that supports image generation. Additional parameters like the height and the width can be configured using [Image Generation: Create ImageOptions](/appstore/modules/genai/genai-for-mx/commons/#imageoptions-create). To configure OpenAI-specific options, like quality and style an extension to the ImageOptions can be added using [Image Generation: Set ImageOptions Extension](#set-imageoptions-extension). -A generated image needs to be stored in a custom entity that inherits from the `System.Image` entity. The `Response` from the single image operation can be processed using [Get Generated Image (Single)](/appstore/modules/genai/commons/#image-get-single) to store the image in your custom `Image` entity. +A generated image needs to be stored in a custom entity that inherits from the `System.Image` entity. The `Response` from the single image operation can be processed using [Get Generated Image (Single)](/appstore/modules/genai/genai-for-mx/commons/#image-get-single) to store the image in your custom `Image` entity. #### Embeddings Generation {#embeddings-configuration} -OpenAI also provides vector embedding generation capabilities which can be invoked using this connector module. The `OpenAIDeployedModel` entity is compatible with the [knowledge base operations](/appstore/modules/genai/commons/#genai-knowledgebase-content) from the GenAI Commons. +OpenAI also provides vector embedding generation capabilities which can be invoked using this connector module. The `OpenAIDeployedModel` entity is compatible with the [knowledge base operations](/appstore/modules/genai/genai-for-mx/commons/#genai-knowledgebase-content) from the GenAI Commons. In order to implement embeddings generation into your Mendix application, you can use the Embedding generation microflow actions from GenAI Commons directly. When developing your microflow, you can drag and drop the one you need from the toolbox: find it under the **GenAI (Generate)** category in the **Toolbox** in Mendix Studio Pro: * Generate Embeddings (String) * Generate Embeddings (Chunk Collection) -Depending on the operation you use in the microflow, an `InputText` String or a [ChunkCollection](/appstore/modules/genai/commons/#chunkcollection) needs to be provided. The current version of this operation only supports the float representation of the resulting vector. +Depending on the operation you use in the microflow, an `InputText` String or a [ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection) needs to be provided. The current version of this operation only supports the float representation of the resulting vector. -The microflow action `Generate Embeddings (String)` supports scenarios where the vector embedding of a single string must be generated, e.g. to use for a nearest neighbor search across an existing knowledge base. This input string can be passed directly as the `InputText` parameter of this microflow. Additionally, [EmbeddingsOptions](/appstore/modules/genai/commons/#embeddingsoptions-entity) is optional and can be instantiated using [Embeddings: Create EmbeddingsOptions](/appstore/modules/genai/commons/#embeddingsoptions-create) from GenAI Commons. Use the GenAI Commons toolbox action [Embeddings: Get First Vector from Response](/appstore/modules/genai/commons/#embeddings-get-first-vector) to retrieve the generated embeddings vector. Both mentioned operations can be found under **GenAI Knowledge Base (Content)** in the **Toolbox** in Mendix Studio Pro. +The microflow action `Generate Embeddings (String)` supports scenarios where the vector embedding of a single string must be generated, e.g. to use for a nearest neighbor search across an existing knowledge base. This input string can be passed directly as the `InputText` parameter of this microflow. Additionally, [EmbeddingsOptions](/appstore/modules/genai/genai-for-mx/commons/#embeddingsoptions-entity) is optional and can be instantiated using [Embeddings: Create EmbeddingsOptions](/appstore/modules/genai/genai-for-mx/commons/#embeddingsoptions-create) from GenAI Commons. Use the GenAI Commons toolbox action [Embeddings: Get First Vector from Response](/appstore/modules/genai/genai-for-mx/commons/#embeddings-get-first-vector) to retrieve the generated embeddings vector. Both mentioned operations can be found under **GenAI Knowledge Base (Content)** in the **Toolbox** in Mendix Studio Pro. -The microflow action `Generate Embeddings (Chunk Collection)` supports the more complex scenario where a collection of string inputs is vectorized in a single API call, such as when converting a collection of texts (chunks) into embeddings to be inserted into a knowledge base. Instead of calling the API for each string, executing a single call for a list of strings can significantly reduce HTTP overhead. Use the exposed microflows of GenAI Commons [Chunks: Initialize ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-create) to create the wrapper and [Chunks: Add Chunk to ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-add-chunk), or [Chunks: Add KnowledgeBaseChunk to ChunkCollection](/appstore/modules/genai/commons/#chunkcollection-add-knowledgebasechunk) to construct the input. The resulting embedding vectors returned after a successful API call will be stored in the `EmbeddingVector` attribute in the same `Chunk` object. \ +The microflow action `Generate Embeddings (Chunk Collection)` supports the more complex scenario where a collection of string inputs is vectorized in a single API call, such as when converting a collection of texts (chunks) into embeddings to be inserted into a knowledge base. Instead of calling the API for each string, executing a single call for a list of strings can significantly reduce HTTP overhead. Use the exposed microflows of GenAI Commons [Chunks: Initialize ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-create) to create the wrapper and [Chunks: Add Chunk to ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-chunk), or [Chunks: Add KnowledgeBaseChunk to ChunkCollection](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-knowledgebasechunk) to construct the input. The resulting embedding vectors returned after a successful API call will be stored in the `EmbeddingVector` attribute in the same `Chunk` object. \ Purely to generate embeddings, it does not matter whether the ChunkCollection contains Chunks or its specialization KnowledgeBaseChunks. However, if the end goal is to store the generated embedding vectors in a knowledge base (e.g. using the [PgVector Knowledge Base](/appstore/modules/pgvector-knowledge-base/) module), then Mendix recommends adding `KnowledgeBaseChunks` to the `ChunkCollection` and using these as an input for the embeddings operations, so they can afterward directly be used to populate the knowledge base with. Note that currently, the OpenAI connector does not support knowledge base interaction (e.g. inserting or retrieving chunks). For more information on possible ways to work with knowledge bases when using the OpenAI Connector for embedding generation, read more about [PgVector Knowledge Base](/appstore/modules/pgvector-knowledge-base/) and [setting up a Vector Database](/appstore/modules/genai/pgvector-setup/). ### Exposed Microflow Actions for (Azure) OpenAI {#exposed-microflows} -OpenAI-specific exposed microflow actions to construct requests via drag-and-drop are listed below. These microflows can be found in the **Toolbox** in Studio Pro. Note that using these flows is only required if you need to add options to the request that are specific to OpenAI. For the generic part can use the GenAI Commons toolbox actions to [create the required Request](/appstore/modules/genai/commons/#genai-request-building) and [handle the Response](/appstore/modules/genai/commons/#genai-response-handling), which can be found under the **GenAI (Request Building)** and **GenAI (Response Handling)** categories in the Toolbox. +OpenAI-specific exposed microflow actions to construct requests via drag-and-drop are listed below. These microflows can be found in the **Toolbox** in Studio Pro. Note that using these flows is only required if you need to add options to the request that are specific to OpenAI. For the generic part can use the GenAI Commons toolbox actions to [create the required Request](/appstore/modules/genai/genai-for-mx/commons/#genai-request-building) and [handle the Response](/appstore/modules/genai/genai-for-mx/commons/#genai-response-handling), which can be found under the **GenAI (Request Building)** and **GenAI (Response Handling)** categories in the Toolbox. #### Set Response Format {#set-responseformat-chat} @@ -269,7 +266,7 @@ This microflow adds a new `FileDocument` or URL to an existing `FileCollection`. #### Image Generation: Set ImageOptions Extension {#set-imageoptions-extension} -This microflow adds a new `OpenAIImageOptions_Extension` to an [ImageOptions](/appstore/modules/genai/commons/#imageoptions-entity) object to specify additional configurations for the image generation operation. The object will be used inside of the image generation operation if the same `ImageOptions` are passed. The parameters are optional. +This microflow adds a new `OpenAIImageOptions_Extension` to an [ImageOptions](/appstore/modules/genai/genai-for-mx/commons/#imageoptions-entity) object to specify additional configurations for the image generation operation. The object will be used inside of the image generation operation if the same `ImageOptions` are passed. The parameters are optional. ## Technical Reference {#technical-reference} diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/pg-vector-knowledge-base/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/pg-vector-knowledge-base/_index.md similarity index 98% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/pg-vector-knowledge-base/_index.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/pg-vector-knowledge-base/_index.md index e4ff6a285ae..dedf4689e80 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/pg-vector-knowledge-base/_index.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/pg-vector-knowledge-base/_index.md @@ -1,11 +1,12 @@ --- title: "PgVector Knowledge Base" -url: /appstore/modules/genai/pgvector/ +url: /appstore/modules/genai/reference-guide/external-connectors/pgvector/ linktitle: "PgVector Knowledge Base" description: "Describes the configuration and usage of the PgVector Knowledge Base module from the Mendix Marketplace. This module allows developers to integrate PostgreSQL databases with pgvector installed as knowledge bases into their Mendix app." weight: 70 aliases: - /appstore/modules/pgvector-knowledge-base/ + - /appstore/modules/genai/pgvector/ --- ## Introduction {#introduction} @@ -62,7 +63,7 @@ You must perform the following steps to integrate a Mendix app integrate a PgVec 1. Add the module role **PgVectorKnowledgeBase.Administrator** to your Administrator user role in the security settings of your app. Optionally, map **GenAICommons.User** to any user roles that need read access directly on retrieved entities. 2. Add the **DatabaseConfiguration_Overview** page (**USE_ME > Configuration**) to your navigation, or add the **Snippet_DatabaseConfigurations** to a page that is already part of your navigation. -3. Set up your database configurations at runtime. For more information, see the [Configuring the Database Connection Details](/appstore/modules/genai/pgvector-setup/#configure-database-connection) section in *Setting up a Vector Database*. +3. Set up your database configurations at runtime. For more information, see the [Configuring the Database Connection Details](/appstore/modules/genai/reference-guide/external-connectors/pgvector-setup/#configure-database-connection) section in *Setting up a Vector Database*. {{% alert color="info" %}} It is possible to have multiple knowledge bases in the same database in parallel by providing different knowledge base names in combination with the same `DatabaseConfiguration`. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/pg-vector-knowledge-base/vector-database-setup.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/pg-vector-knowledge-base/vector-database-setup.md similarity index 99% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/pg-vector-knowledge-base/vector-database-setup.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/pg-vector-knowledge-base/vector-database-setup.md index 833d44332ea..29148135a09 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/pg-vector-knowledge-base/vector-database-setup.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/pg-vector-knowledge-base/vector-database-setup.md @@ -1,9 +1,11 @@ --- title: "Setting up a Vector Database" -url: /appstore/modules/genai/pgvector-setup/ +url: /appstore/modules/genai/reference-guide/external-connectors/pgvector-setup/ linktitle: "Vector Database Setup" weight: 5 description: "Describes how to set up a vector database to store and manage vector embeddings for a knowledge base" +aliases: + - /appstore/modules/genai/pgvector-setup/ --- ## Introduction {#introduction} diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/snowflake-cortex.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/snowflake-cortex.md similarity index 100% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/snowflake-cortex.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/external-platforms/snowflake-cortex.md diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/_index.md new file mode 100644 index 00000000000..c39005deb1a --- /dev/null +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/_index.md @@ -0,0 +1,22 @@ +--- +title: "GenAI For Mendix" +url: /appstore/modules/genai/genai-for-mx/ +linktitle: "GenAI For Mendix" +weight: 10 +description: "Provides information on the GenAI For Mendix Module." +no_list: false +--- + +## Introduction {#introduction} + +The [GenAI For Mendix](https://marketplace.mendix.com/link/component/227931) bundle provides comprehensive guidance on enhancing your Mendix applications with Generative AI. It contains [GenAI Commons](/appstore/modules/genai/commons/), [Conversational UI](/appstore/modules/genai/conversational-ui/), and the [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/MxGenAI/) offering a comprehensive overview of the technical aspects. + +By leveraging the tools and features provided in this bundle, you can: + +* Integrate with any LLM and knowledge base using the included [GenAI Commons](/appstore/modules/genai/commons/) module. +* Create a [Conversational UI](/appstore/modules/genai/conversational-ui/) with prebuilt pages and snippets to facilitate user-friendly interactions. +* Perform [Prompt Engineering](/appstore/modules/genai/prompt-engineering/) at runtime to optimize your AI-generated results via [Prompt Management](/appstore/modules/genai/genai-for-mx/prompt-management/). +* Monitor your token consumption with the Token Consumption Monitor, ensuring efficient use of resources. +* Integrate directly with Mendix Cloud GenAI Models and Knowledge Bases via [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/MxGenAI/), enhancing your application’s intelligence and responsiveness. + +## Documents in This Category diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/conversational-ui.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/conversational-ui.md similarity index 92% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/conversational-ui.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/conversational-ui.md index 6a2528809f9..492aac4e5bc 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/conversational-ui.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/conversational-ui.md @@ -1,9 +1,11 @@ --- title: "Conversational UI" -url: /appstore/modules/genai/conversational-ui/ +url: /appstore/modules/genai/genai-for-mx/conversational-ui/ linktitle: "Conversational UI" weight: 20 description: "Describes the Conversational UI marketplace module that assists developers in implementing conversational use cases such as an AI Bot." +aliases: + - /appstore/modules/genai/conversational-ui/ --- ## Introduction {#introduction} @@ -31,7 +33,7 @@ The Conversational UI module provides the following functionalities: * Snippets that you can use directly on your pages, for example, to display messages or a history sidebar * A floating button for opening a pop-up chat * Pages, snippets, and logic to display and export token usage data (if enabled in GenAI Commons and supported by the GenAI Connector of choice) - * Pages, snippets, and logic to experiment with prompts. An admin, developer, or data scientist can test multiple versions at runtime, define variables, and decide which prompt version should be applied when end-users use the app. See [Prompt Management](/appstore/modules/genai/prompt-management/) for more information. + * Pages, snippets, and logic to experiment with prompts. An admin, developer, or data scientist can test multiple versions at runtime, define variables, and decide which prompt version should be applied when end-users use the app. See [Prompt Management](/appstore/modules/genai/genai-for-mx/prompt-management/) for more information. * Operations to set up your context, interact with the model, and add the data to be displayed in the UI * Domain model to store the chat conversations and additional information @@ -54,7 +56,7 @@ You must also ensure you have the other prerequisite modules that Conversational * [Nanoflow Commons](https://marketplace.mendix.com/link/component/109515) * [Web Actions](https://marketplace.mendix.com/link/component/114337) -Finally, you must also set up a connector that is compatible with [GenAI Commons](/appstore/modules/genai/commons/). One option is to use the Mendix Cloud GenAI connector, which is part of the above Marketplace listing package. For more information on how to configure this connector, see the [Configuration](/appstore/modules/genai/MxGenAI/#configuration) section of *Mendix Cloud GenAI connector*. Additionally, Mendix offers platform-supported integration with [(Azure) OpenAI](/appstore/modules/genai/openai/) and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/). If desired, you need to download these integrations manually from the Marketplace. Alternatively, you can integrate with custom models by creating your own connector and making its operations and object structure compatible with the [GenAI Commons](/appstore/modules/genai/commons/) `Request` and `Response`. +Finally, you must also set up a connector that is compatible with [GenAI Commons](/appstore/modules/genai/commons/). One option is to use the Mendix Cloud GenAI connector, which is part of the above Marketplace listing package. For more information on how to configure this connector, see the [Configuration](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/#configuration) section of *Mendix Cloud GenAI connector*. Additionally, Mendix offers platform-supported integration with [(Azure) OpenAI](/appstore/modules/genai/openai/) and [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/). If desired, you need to download these integrations manually from the Marketplace. Alternatively, you can integrate with custom models by creating your own connector and making its operations and object structure compatible with the [GenAI Commons](/appstore/modules/genai/commons/) `Request` and `Response`. ## Installation {#installation} @@ -78,7 +80,7 @@ The main entities are shown for reference in the diagram below. For technical do ### Configuring the Roles {#module-roles} -Make sure that the module role `User` is part of the user roles that are intended to chat with the model. Optionally, you can grant the `_addOn_ReadAll` role to admin roles, so that users with that role can read all messages. A role for usage monitoring is related only to the [Token consumption monitor snippets](#snippet-token-monitor). Additionally, a separate role is applicable for the [Prompt Management functionality](/appstore/modules/genai/prompt-management/). Both do not relate to the conversational snippets. +Make sure that the module role `User` is part of the user roles that are intended to chat with the model. Optionally, you can grant the `_addOn_ReadAll` role to admin roles, so that users with that role can read all messages. A role for usage monitoring is related only to the [Token consumption monitor snippets](#snippet-token-monitor). Additionally, a separate role is applicable for the [Prompt Management functionality](/appstore/modules/genai/genai-for-mx/prompt-management/). Both do not relate to the conversational snippets. | Module role | Description | | --- | --- | @@ -148,7 +150,7 @@ If you need custom attributes or settings in your action microflow required for Depending on the implementation, you can create this object using a microflow that opens the page or using a datasource microflow on the page itself. The following are the operations in the toolbox for creating the ChatContext: -* `New Chat` creates a new `ChatContext` and a new `ProviderConfig`. The `ProviderConfig` is added to the `ChatContext` and set to active. Additionally, the action microflow of the new `ProviderConfig` is set. A [DeployedModel](/appstore/modules/genai/commons/#deployed-model) needs to be passed in order to access the right model. Via the association `ProviderConfig_DeployedModel` the DeployedModel can be retrieved and used to pass to the [Chat Completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history) later in the Action Microflow. +* `New Chat` creates a new `ChatContext` and a new `ProviderConfig`. The `ProviderConfig` is added to the `ChatContext` and set to active. Additionally, the action microflow of the new `ProviderConfig` is set. A [DeployedModel](/appstore/modules/genai/genai-for-mx/commons/#deployed-model) needs to be passed in order to access the right model. Via the association `ProviderConfig_DeployedModel` the DeployedModel can be retrieved and used to pass to the [Chat Completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history) later in the Action Microflow. * `New Chat with Existing Config` creates a new `ChatContext` and sets a given `ProviderConfig` to active. * `New Chat with Additional Configs` creates a new `ChatContext`, adds a `ProviderConfig` to the `ChatContext`, and sets it to active. In addition, a list of `ProviderConfig` can be added to the `ChatContext` (non-active, but selectable in the UI). @@ -176,7 +178,7 @@ Add the action microflow to an existing `ProviderConfig` by using the **Set Chat A typical action microflow is responsible for the following: * Convert the `ChatContext` with user input to a `Request` structure for the chat completions operation. This module provides the **Default Preprocessing** toolbox action to take care of that in basic cases; for more advanced or custom cases you need to create your own logic based on this. -* Execute the [Chat Completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history) operation. To pass a [DeployedModel](/appstore/modules/genai/commons/#deployed-model), you can use the `ProviderConfig_DeployedModel` association of the active `ProviderConfig` for the `ChatContext`. +* Execute the [Chat Completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history) operation. To pass a [DeployedModel](/appstore/modules/genai/genai-for-mx/commons/#deployed-model), you can use the `ProviderConfig_DeployedModel` association of the active `ProviderConfig` for the `ChatContext`. * Update the `ChatContext` structure based on the response so that the user can see the result in the UI. This module provides the **Update Assistant Response** microflow action in the toolbox. It is only required to execute this logic in successful model interactions, make sure to pass the response object. In the case of an unhappy scenario, the action microflow should return false and the module logic will take care of setting the applicable error status and no response object is needed. The example action microflow in this module, to be found in the **USE_ME > ConversationalUI > Action microflow examples** folder follows this basic structure. @@ -193,14 +195,14 @@ If you want to create your custom action microflow, keep the following considera The following operations can be found in the toolbox for changing the [ChatContext](#chat-context) in a (custom) action microflow: * `Set Topic` sets the `Topic` of the `ChatContext`. This attribute can be used in the **History** sidebar while making historical chats visible to users. -* `Default Preprocessing` sets a default `Topic` for `ChatContext` and creates a sample [Request](/appstore/modules/genai/commons/#request). +* `Default Preprocessing` sets a default `Topic` for `ChatContext` and creates a sample [Request](/appstore/modules/genai/genai-for-mx/commons/#request). * `Set ConversationID` sets the ConversationID on the `ChatContext`. Storing the ConversationID is needed for a chat with history within [Retrieve and Generate with Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/#retrieve-and-generate). ##### Request Operations {#request-operations} The following operations are used in a (custom) action microflow: -* `Create Request with Chat History` creates a [Request](/appstore/modules/genai/commons/) object that is used as an input parameter in a [Chat Completions (with history)](/appstore/modules/genai/commons/#chat-completions-with-history) operation as part of the [action microflow](#action-microflow). +* `Create Request with Chat History` creates a [Request](/appstore/modules/genai/commons/) object that is used as an input parameter in a [Chat Completions (with history)](/appstore/modules/genai/genai-for-mx/commons/#chat-completions-with-history) operation as part of the [action microflow](#action-microflow). * `Get Current User Prompt` gets the current user prompt. It can be used in the [action microflow](#action-microflow) because the `CurrentUserPrompt` from the chat context is no longer available. * `Update Assistant Response` processes the response of the model and adds the new message and any sources to the UI. This is typically one of the last steps of the logic in an [action microflow](#action-microflow). It only needs to be included at the end of the happy flow of an action microflow. Make sure to pass the response object. @@ -258,14 +260,14 @@ You may need to use the following classes when building a more complex custom pa ### Token Consumption Monitor Snippets {#snippet-token-monitor} -A separate set of snippets has been made available to display and export token usage information in the running application. This is applicable for LLM connectors that follow the principles of [GenAI Commons](/appstore/modules/genai/commons/#token-usage) and as a result store token usage information. The following snippets can be added to (admin) pages independently from the conversation logic described in earlier sections. +A separate set of snippets has been made available to display and export token usage information in the running application. This is applicable for LLM connectors that follow the principles of [GenAI Commons](/appstore/modules/genai/genai-for-mx/commons/#token-usage) and as a result store token usage information. The following snippets can be added to (admin) pages independently from the conversation logic described in earlier sections. * **Snippet_TokenMonitor** - This snippet can be used to display token usage information in charts and contains several other snippets that you can use to build your token consumption monitor dashboard. To display the token usage data, users will need the `UsageMonitoring` user role. * **Snippet_TokenMonitor_Export** - This snippet can be used to display token usage information in a grid and export it as *.xlsx*. ### Prompt Management {#prompt-management} -The Conversational UI module comes with pages and snippets that provide the Prompt Management functionality. For more details, see [Prompt Management](/appstore/modules/genai/prompt-management/). +The Conversational UI module comes with pages and snippets that provide the Prompt Management functionality. For more details, see [Prompt Management](/appstore/modules/genai/genai-for-mx/prompt-management/). ## Technical Reference {#technical-reference} diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/genai-commons.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/genai-commons.md similarity index 99% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/genai-commons.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/genai-commons.md index ba53d76b3fa..3d97224febd 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/genai-commons.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/genai-commons.md @@ -1,11 +1,12 @@ --- title: "GenAI Commons" -url: /appstore/modules/genai/commons/ +url: /appstore/modules/genai/genai-for-mx/commons/ linktitle: "GenAI Commons" description: "Describes the purpose, configuration and usage of the GenAI Commons module from the Mendix Marketplace that allows developers to integrate GenAI common principles and patterns into their Mendix app." weight: 50 aliases: - /appstore/modules/genai-commons/ + - /appstore/modules/genai/commons/ --- ## Introduction {#introduction} diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/prompt-management.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/prompt-management.md similarity index 96% rename from content/en/docs/appstore/use-content/platform-supported-content/modules/genai/prompt-management.md rename to content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/prompt-management.md index ac3e8411d4a..bdca1461b06 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/prompt-management.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/reference-guide/genai-for-mendix.md/prompt-management.md @@ -1,6 +1,6 @@ --- title: "Prompt Management" -url: /appstore/modules/genai/prompt-management +url: /appstore/modules/genai/genai-for-mx/prompt-management/ linktitle: "Prompt Management" weight: 20 description: "Describes the Prompt Management functionality that assists developers and data scientists in implementing prompts in their GenAI Mendix applications use cases." @@ -32,7 +32,7 @@ The Prompt Management functionality provides the following: ### Prerequisites -The prerequisites of the [Conversational UI module](/appstore/modules/genai/conversational-ui/#prerequisites) apply here. +The prerequisites of the [Conversational UI module](/appstore/modules/genai/genai-for-mx/conversational-ui/#prerequisites) apply here. ## Installation @@ -69,9 +69,9 @@ For example, download and run the [GenAI Showcase App](https://marketplace.mendi You need at least one GenAI connector that follows the principles of GenAI commons to interact with LLMs from the Prompt Management logic. To test a prompt, you must configure at least one Deployed Model for your chosen connector. Refer to the specific connector’s documentation for detailed setup instructions on configuring the Deployed Model. -* For [Mendix Cloud GenAI](https://marketplace.mendix.com/link/component/227931), included by default, importing the **Key** from the Mendix portal automatically creates a MxCloud Deployed Model. This is part of the [configuration](/appstore/modules/genai/MxGenAI/#configuration). +* For [Mendix Cloud GenAI](https://marketplace.mendix.com/link/component/227931), included by default, importing the **Key** from the Mendix portal automatically creates a MxCloud Deployed Model. This is part of the [configuration](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/#configuration). * For [Amazon Bedrock](https://marketplace.mendix.com/link/component/215042), the creation of Bedrock Deployed Models is part of the [model synchronization mechanism](/appstore/modules/aws/amazon-bedrock/#sync-models). -* For [OpenAI](https://marketplace.mendix.com/link/component/220472), the configuration of OpenAI Deployed Models is part of the [configuration](/appstore/modules/genai/openai/#general-configuration). +* For [OpenAI](https://marketplace.mendix.com/link/component/220472), the configuration of OpenAI Deployed Models is part of the [configuration](/appstore/modules/genai/reference-guide/external-connectors/openai/#general-configuration). ### Write the Prompt {#write-prompt} @@ -95,7 +95,7 @@ New prompts will be created in the draft status by default, meaning they are st For a Single-Call type prompt, use `Get Prompt for Context Object`, which can be found in the **Toolbox** in Studio Pro while editing a microflow, under the category **GenAI (Request Building)**. This operation returns both a system prompt and a user prompt strings, on a combined `PromptToUse` object. These string attributes can be passed to the chat completions operation. Retrieve the prompt (e.g. by name) and pass it with your custom context object to the operation. For an example of this pattern, see the product description generation example in the [GenAI Showcase app](https://marketplace.mendix.com/link/component/220475). -For a conversational prompt, the chat context can be created based on the prompt in one operation. Use the `New Chat for Prompt` operation from the **Toolbox** under the **Conversational UI** category. Retrieve the prompt (e.g. by name) and pass it with your custom context object to the operation. Note that this sets the system prompt for the chat context, making it applicable to the entire (future) conversation. Similar to other chat context operations, an [action microflow needs to be selected](/appstore/modules/genai/conversational-ui/#action-microflow) for this microflow action. +For a conversational prompt, the chat context can be created based on the prompt in one operation. Use the `New Chat for Prompt` operation from the **Toolbox** under the **Conversational UI** category. Retrieve the prompt (e.g. by name) and pass it with your custom context object to the operation. Note that this sets the system prompt for the chat context, making it applicable to the entire (future) conversation. Similar to other chat context operations, an [action microflow needs to be selected](/appstore/modules/genai/genai-for-mx/conversational-ui/#action-microflow) for this microflow action. With this microflow logic, the prompt version is ready to be tested from the end-user flow (in a local or test environment). The prompt can be exported/imported for transport to other environments if needed. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/_index.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/_index.md index d00be2fdcd9..23bb32ce962 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/_index.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/_index.md @@ -28,7 +28,7 @@ Generative Artificial Intelligence (GenAI) transforms business applications, emp ### Additional Resources * Basic documentation on [Get Started with Generative AI](/appstore/modules/genai/get-started/) is an essential resource for anyone beginning their GenAI journey. -* The [GenAI For Mendix](https://marketplace.mendix.com/link/component/227931) is a bundle containing [GenAI Commons](/appstore/modules/genai/commons/), [Conversational UI](/appstore/modules/genai/conversational-ui/), and the [Mendix Cloud GenAI Resources Packs](/appstore/modules/genai/MxGenAI/) offering a comprehensive overview of the technical aspects. +* The [GenAI For Mendix](https://marketplace.mendix.com/link/component/227931) is a bundle containing [GenAI Commons](/appstore/modules/genai/commons/), [Conversational UI](/appstore/modules/genai/conversational-ui/), and the [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/MxGenAI/) offering a comprehensive overview of the technical aspects. * The [OpenAI](/appstore/modules/genai/openai/) provides essential information about the OpenAI connector. * The [Amazon Bedrock](/appstore/modules/aws/amazon-bedrock/) provides key information about the AWS Bedrock connector. * The [PGVector Knowledge Base](/appstore/modules/genai/pgvector/) offers the option for a private knowledge base outside of the LLM infrastructure. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/integrate_function_calling.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/integrate_function_calling.md new file mode 100644 index 00000000000..f71723d9def --- /dev/null +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/integrate_function_calling.md @@ -0,0 +1,159 @@ +--- +title: "Integrate Function Calling into Your Mendix App" +url: /appstore/modules/genai/using-genai/howto-functioncalling/ +linktitle: "Integrating Function Calling" +weight: 30 +description: "This document guides you through integrating and implementing function calling in your Mendix application to enhance functionality." +--- + +## Introduction + +This document explains how to use function calling in your smart app. To do this, you can use your existing app or or follow the [Build a Smart App from a Blank GenAI App](/appstore/modules/genai/using-genai/blank-app/) guide to start from scratch, as demonstrated in the sections below. + +Through this document, you will: + +* Understand how to implement function calling within your Mendix application. +* Learn to integrate GenAI capabilities to address specific business requirements effectively. + +### Prerequisites {#prerequisites} + +Before integrating function calling into your app, make sure you meet the following requirements: + +* An existing app: To simplify your first use case, start building from a preconfigured set up [Blank GenAI Starter App](https://marketplace.mendix.com/link/component/227934). For more information, see [Build a Chatbot from Scratch Using the Blank GenAI App](/appstore/modules/genai/using-genai/blank-app/). + +* Installation: Install the [GenAI For Mendix](https://marketplace.mendix.com/link/component/227931) bundle from the Mendix marketplace. If you start with the Blank GenAI App, skip this installation. + +* Intermediate knowledge of the Mendix platform: Familiarity with Mendix Studio Pro, microflows, and modules. + +* Basic understanding of GenAI concepts: Review the [Enrich Your Mendix App with GenAI Capabilities](/appstore/modules/genai/) page for foundational knowledge and familiarize yourself with the [concepts](/appstore/modules/genai/using-gen-ai/). + +* Understanding Function Calling and Prompt Engineering: Learn about [Function Calling](/appstore/modules/genai/function-calling/) and [Prompt Engineering](/appstore/modules/genai/get-started/#prompt-engineering) to use them within the Mendix ecosystem. + +## Function Calling Use Case {#use-case} + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/structure_functioncalling.png" >}} + +In this example, two functions will be implemented with the following purposes: + +1. Retrieving the display name of the user when an email is requested in a chatbot, allows the information to be automatically filled for the end user. +2. Extracting bank holidays in the Netherlands using an API. For this example, a public API from [Open Holidays API](https://www.openholidaysapi.org/en/) is used. + +### Choosing the Infrastructure {#infrastructure} + +Selecting the infrastructure for integrating GenAI into your Mendix application is the first step. Depending on your use case and preferences, you can choose from the following options: + +* [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/MxGenAI/): The Mendix Cloud GenAI Connector is part of the [GenAI For Mendix](https://marketplace.mendix.com/link/component/227931) bundle on the marketplace, allowing you to utilize Mendix Cloud GenAI Resource Packs directly within your Mendix application. + +* [OpenAI](/appstore/modules/genai/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports both OpenAI’s platform and Azure’s OpenAI service. + +* [Amazon Bedrock](/appstore/modules/genai/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. + +* Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the Mendix Cloud GenAI, OpenAI, and Amazon Bedrock connectors, as they offer comprehensive support and ease of use to get started. + +{{% alert color="info" %}} +Not all models support function calling. Ensure that your preferred GenAI provider is set up in your Mendix app and that a compatible model is available. Mendix provides an [overview of models and their capabilities](https://docs.mendix.com/appstore/modules/genai/#models). +{{% /alert %}} + +### Customizing Microflows {#microflows} + +To make the functions work, create and adjust certain microflows as shown below. These microflows will handle the logic required for gathering the display name of the user and extracting the bank holidays from the Netherlands in 2025 using an API. + +1. Locate the pre-built microflow `ChatContext_ChatWithHistory_ActionMicroflow` in the **ConversationalUI** > **USE_ME** > **Conversational UI** > **Action microflow examples** folder and copy it into your `MyFirstBot` module. + +2. Locate the `New Chat` action in the `ACT_FullScreenChat_Open` microflow. Inside this action, change the `Action microflow` input parameter to your new `MyFirstBot.ChatContext_ChatWithHistory_ActionMicroflow` from your `MyFirstBot` module. + +To call a function, create a microflow per function to extract the necessary information. + +#### Function: Extracting the User Name {#function-username} + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetCurrentUserName_Function.jpg" >}} + +Create a new microflow with the name `GetCurrentUserName_Function`. + +1. Start with the `Retrieve` action, where you can use the following modifications as an example: + + * Source: `From database` + * Entity: `Administration.Account` + * Range: `First` + * XPath constraint: `[id = $currentUser]` + * Object name: `Account` + +2. Include a decision where: + * Caption: for example, `Found?` + * Decision Type: `Expression` + * Expression: `$Account != empty` + + 1. If the decision is `false`, an end event of type `String` is added where the return value can be set to `Mendix Administrator Chat User`. + + 2. If the decision is `true`, an end event of type `String` is added where the return value is `$Account/FullName`. + +#### Function: Getting Bank Holidays in the Netherlands 2025 {#function-bankholidays} + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetBankHolidays_Function.jpg" >}} + + For this example, call the new microflow `GetBankHolidays_Function`. + +1. Start with the `Call REST service` action, where you can use the following modifications as an example: + + General tab: + + * Location: `https://openholidaysapi.org/PublicHolidays?countryIsoCode=NL&validFrom=2025-01-01&validTo=2025-12-31&languageIsoCode=EN` + * HTTP method: `GET` + * Use timeout on request: `Yes` + * Timeout (s): You can choose the value, here we set it to `300` + * The rest can be set to default. + + Response tab: + + * Response handling: `Store in a string` + * Store in variable: `Yes` + * Variable name: `HolidayJSON` + +2. Right-click on the `Call REST` action and select `Set $HolidayJSON` as the return value. + +### Calling the Functions {#calling-the-functions} + +Now, the following steps will focus exclusively on the `ChatContext_ChatWithHistory_ActionMicroflow` from your `MyFirstBot` module. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/CallingFunctions_Microflow.jpg" >}} + +As shown in the image, two key steps must be completed to enable the execution of both functions. + +#### Adding Functions to the Request {#add-to-request} + +1. After the outgoing `Request found` equals `true` decision, add the `Tools: Add Function to Request` toolbox action for the first function with the following settings: + + * Request: `$Request` + * Tool name: `get-current-user-name` + * Tool description: `This function has no input, and returns a string containing the name of the user using the chat. It can be used to generate texts on behalf of the user, for example, the signature of an email, "Best regards, [user's name]".` + * Function microflow: select the `GetCurrentUserName_Function` microflow created in the previous step. + * Use return value: No + +2. Following this action, continue with the second function by adding the `Tools: Add Function to Request` action with the following settings: + + * Request: `$Request` + * Tool name: `get-bank-holidays-2025` + * Tool description: `This function has no input, and returns a JSON containing the bank holidays in the Netherlands for the year 2025.` + * Function microflow: select the `GetBankHolidays_Function` microflow created in the previous step. + * Use return value: No + +### Optional: Changing the System Prompt {#edit-systemprompt} + +Optionally, you can change the system prompt to provide the model additional instructions, for example, the tone of voice. Therefore, follow a similar approach described in the [Build a Chatbot from Scratch Using the Blank GenAI App](/appstore/modules/genai/using-genai/blank-app/#changing-system-prompt). + +1. Open the copied `ACT_FullScreenChat_Open` microflow from your `MyFirstBot` module. +2. Locate the **New Chat** action. +3. Inside this action, find the `System prompt` parameter, which has by default an empty value. +4. Update the `System prompt` value to reflect your desired behavior. For example, *`Answer like a Gen Z person. Always keep your answers short.`* +5. Save the changes. + +## Testing and Troubleshooting {#testing-troubleshooting} + +Before testing, ensure that you have completed the Mendix Cloud GenAI, OpenAI, or Bedrock configuration as described in the [Build a Chatbot from Scratch Using the Blank GenAI App](/appstore/modules/genai/using-genai/blank-app/), particularly the [Infrastructure Configuration](/appstore/modules/genai/using-genai/blank-app/#config) section. + +To test the Chatbot, go to the **Home** icon to open the chatbot interface. Start interacting with your chatbot by typing in the chat box. +For example, type—`Write a message to my colleague Max asking about a meeting to discuss the content for our next GenAI how-to.` or `How many bank holidays do I have in December?` + +Congratulations! Your chatbot is now ready to use. + +If an error occurs, check the **Console** in Studio Pro for detailed information to assist in resolving the issue. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_a_starter_app.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_a_starter_app.md index 8b8ba3fe185..463c6d6df24 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_a_starter_app.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_a_starter_app.md @@ -44,11 +44,7 @@ Selecting the infrastructure for integrating GenAI into your Mendix application * [OpenAI](/appstore/modules/genai/openai/): The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) supports OpenAI’s platform and Azure’s OpenAI service. -{{% alert color="info" %}} -To start, you can sign up for a free trial with OpenAI and receive credits valid for three months from the account creation date. For more details, see the [OpenAI API reference](https://platform.openai.com/docs/api-reference/authentication). -{{% /alert %}} - -* [Amazon Bedrock](/appstore/modules/genai/bedrock/): The [Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. +* [Amazon Bedrock](/appstore/modules/genai/bedrock/): The [Amazon Bedrock Connector](https://marketplace.mendix.com/link/component/215042) allows you to leverage Amazon Bedrock’s fully managed service to integrate foundation models from Amazon and leading AI providers. * Your Own Connector: Optionally, if you prefer a custom connector, you can integrate your chosen infrastructure. However, this document focuses on the OpenAI and Bedrock connectors, as they offer comprehensive support and ease of use to get started. @@ -58,7 +54,7 @@ Download the [AI Bot Starter App](https://marketplace.mendix.com/link/component/ #### Mendix Cloud GenAI Configuration -Follow these steps to configure the Mendix Cloud GenAI Resources Packs for your application and for more background information, look at the [Mendix Cloud GenAI Configuration](/appstore/modules/genai/MxGenAI/#configuration) documentation: +Follow these steps to configure the Mendix Cloud GenAI Resources Packs for your application and for more background information, look at the [Mendix Cloud GenAI Configuration](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/#configuration) documentation: 1. Run the application locally. @@ -72,7 +68,7 @@ Follow these steps to configure the Mendix Cloud GenAI Resources Packs for your #### OpenAI Configuration -Follow the steps below to configure OpenAI for your application. For more information, see the [Configuration](/appstore/modules/genai/openai/#configuration) section of the *OpenAI*. +Follow the steps below to configure OpenAI for your application. For more information, see the [Configuration](/appstore/modules/genai/reference-guide/external-connectors/openai/#configuration) section of the *OpenAI*. 1. Run the application locally. diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_blank_app.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_blank_app.md index 46e581ad2f8..6ed3fd48ea5 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_blank_app.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/genai/using_genai/start_from_blank_app.md @@ -80,16 +80,17 @@ Since the **ConversationalUI_FullScreenChat** page contains a **Data View** usin To tailor your application's behavior, you can customize the [System Prompt](/appstore/modules/genai/prompt-engineering/#system-prompt) to make it more specific to your use case: -##### Changing the System Prompt +##### Changing the System Prompt {#changing-system-prompt} -{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_mf.jpg" >}} +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_systemprompt.png" >}} 1. Open the copied `ACT_FullScreenChat_Open` microflow from your `MyFirstBot` module. 2. Locate the **ChatContext** action. -3. Inside this action, find the `System prompt` parameter, which has default an emprty value. +3. Inside this action, find the `System prompt` parameter, which has default an empty value. 4. Update the `System prompt` value to reflect your desired behavior. For example: * For a customer service chatbot: *'You are a helpful customer service assistant providing answers to common product questions.'* * For a travel advisor assistant: *'You are a travel advisor assistant providing travel tips and destination information.'* + * Or keep it simple with *'You are an assistant.'* 5. Save the changes. #### Navigation Configuration @@ -100,11 +101,11 @@ Click **Home** and select the `ACT_FullScreenChat_Open` microflow from your `MyF You may encounter an error about allowed roles. To resolve this, go to the page **Properties** and update the **Navigation > Visible for** setting to include the appropriate user roles. {{% /alert %}} -### Infrastructure Configuration +### Infrastructure Configuration {#config} #### Mendix Cloud GenAI Configuration -Follow these steps to configure the Mendix Cloud GenAI Resources Packs for your application and more background information, look at the [Mendix Cloud GenAI Configuration](/appstore/modules/genai/MxGenAI/#configuration) documentation: +Follow these steps to configure the Mendix Cloud GenAI Resources Packs for your application and more background information, look at the [Mendix Cloud GenAI Configuration](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/#configuration) documentation: 1. Run the application locally. @@ -118,7 +119,7 @@ Follow these steps to configure the Mendix Cloud GenAI Resources Packs for your #### OpenAI Configuration -Follow the steps below to configure OpenAI for your application. For more information, see the [Configuration](/appstore/modules/genai/openai/#configuration) section of the *OpenAI*. +Follow the steps below to configure OpenAI for your application. For more information, see the [Configuration](/appstore/modules/genai/reference-guide/external-connectors/openai/#configuration) section of the *OpenAI*. 1. Run the application locally. @@ -150,7 +151,7 @@ Follow the steps below to configure OpenAI for your application. For more inform * In the **Test configuration**, select the deployed model and press **Test**. * If an error occurs, check the **Mendix Console** for more details on resolving the issue. -#### Bedrock Configuration +#### Amazon Bedrock Configuration Follow the steps below to configure Amazon Bedrock for your application: diff --git a/content/en/docs/appstore/use-content/platform-supported-content/modules/mobile-sso.md b/content/en/docs/appstore/use-content/platform-supported-content/modules/mobile-sso.md index f915319b465..6c78d24cda1 100644 --- a/content/en/docs/appstore/use-content/platform-supported-content/modules/mobile-sso.md +++ b/content/en/docs/appstore/use-content/platform-supported-content/modules/mobile-sso.md @@ -265,6 +265,10 @@ After adding the IdP details to the web page, the IdP sign in tab is visible on Click **IdP login** to redirect to the IdP sign in page. After successful authentication, you will be directed to the user landing page, indicating successful integration of the Mobile SSO module. +### Sign-in Failure Error + +When tryin to sign in multiple times, you may receive a sign-in failure error message. To resolve this error, check your network connection and try signing in again. + ## Troubleshooting ### Sign-in Failure Error diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_mf.jpg b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/[excluded]blank_genai_mf.jpg similarity index 100% rename from static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_mf.jpg rename to static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/[excluded]blank_genai_mf.jpg diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_systemprompt.png b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_systemprompt.png new file mode 100644 index 00000000000..917a7cccfde Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-blankapp/blank_genai_systemprompt.png differ diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/CallingFunctions_Microflow.jpg b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/CallingFunctions_Microflow.jpg new file mode 100644 index 00000000000..f762c462188 Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/CallingFunctions_Microflow.jpg differ diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetBankHolidays_Function.jpg b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetBankHolidays_Function.jpg new file mode 100644 index 00000000000..4ad24a24791 Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetBankHolidays_Function.jpg differ diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetCurrentUserName_Function.jpg b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetCurrentUserName_Function.jpg new file mode 100644 index 00000000000..adca14a4472 Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/GetCurrentUserName_Function.jpg differ diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/structure_functioncalling.png b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/structure_functioncalling.png new file mode 100644 index 00000000000..c7328863058 Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/genai-howto-functioncalling/structure_functioncalling.png differ