diff --git a/content/en/docs/marketplace/genai/concepts/_index.md b/content/en/docs/marketplace/genai/concepts/_index.md index ccd275c71d9..3b7f479019c 100644 --- a/content/en/docs/marketplace/genai/concepts/_index.md +++ b/content/en/docs/marketplace/genai/concepts/_index.md @@ -23,13 +23,49 @@ One of the most important components for generative AI is the Large Language Mod For example, you can use an LLM to do: * **Text Generation:** Creating coherent and contextually relevant text from a prompt. -* **Content Creation:** Assisting in writing articles, stories, and marketing copy. -* **Translation:** Translating text from one language to another. -* **Summarization:** Condensing long documents into concise summaries. -* **Conversational Interaction:** Powering chatbots and virtual assistants to interact naturally with users. * **Data Generation:** Generating Mendix objects by combining JSON generation and [Import mappings](/refguide/import-mapping-action/). +### Typical LLM Use Cases {#llm-use-cases} + +#### Text Generation + +* Develop interactive AI chatbots and virtual assistants that can carry out conversations naturally and engagingly. +* By using text generation models, you can build applications with features such as: + + * Draft documents (for example, content creation: Assist in writing articles, stories, and marketing copy) + * Write computer code + * Answer questions about a knowledge base or (large) texts + * Analyze texts + * Give software a natural language interface + * Tutor in a range of subjects + * Translate languages + * Simulate characters for games + Some LLMs, such as [Anthropic Claude](/appstore/modules/aws/amazon-bedrock/) and [GPT-4o](/appstore/modules/genai/openai/), can also use one or more images as input, allowing you to ask questions about images for use cases such as object recognition, image to text (OCR), and validating whether an image is as intended. + +#### Embeddings Generation + +Convert strings into vector embeddings for various purposes based on the relatedness of texts. + +Embeddings are commonly used for the following: + +* Semantic Search (Meaning-based Similarity) +* Clustering +* Recommendations +* Anomaly detection +* Diversity measurement +* Classification + +You can combine embeddings with text generation capabilities and leverage specific sources of information to create a smart chat functionality tailored to your knowledge base. + +#### Knowledge Base + +Adding knowledge bases helps to tailor response generation to specific contexts by grounding them in data. Knowledge bases can store discrete data (commonly called chunks) and relevant information for end-user actions or application processes can be retrieved from them. + +Knowledge bases are often used for: + +1. [Retrieval Augmented Generation (RAG)](/appstore/modules/genai/rag/) retrieves relevant knowledge from the knowledge base, incorporates it into a prompt, and sends it to the model to generate a response. +2. Semantic search enables advanced search capabilities by considering the semantic meaning of the text, going beyond exact and approximate matching. It allows the knowledge base to be searched for similar chunks effectively. ### What is an LLM Not? diff --git a/content/en/docs/marketplace/genai/mendix-cloud-genai/Mx GenAI Connector.md b/content/en/docs/marketplace/genai/mendix-cloud-genai/Mx GenAI Connector.md index 9a96888fc3b..a38cee15ab3 100644 --- a/content/en/docs/marketplace/genai/mendix-cloud-genai/Mx GenAI Connector.md +++ b/content/en/docs/marketplace/genai/mendix-cloud-genai/Mx GenAI Connector.md @@ -10,69 +10,23 @@ aliases: ## Introduction -The [Mendix Cloud GenAI connector](https://marketplace.mendix.com/link/component/239449) lets you utilize Mendix Cloud GenAI resource packs directly within your Mendix application. It allows you to integrate generative AI by dragging and dropping common operations from its toolbox. +The [Mendix Cloud GenAI connector](https://marketplace.mendix.com/link/component/239449) lets you utilize [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/mx-cloud-genai/resource-packs/) directly within your Mendix application. It allows you to integrate generative AI by dragging and dropping common operations from its toolbox. -### Typical Use Cases - -The Mendix Cloud GenAI Connector is commonly used for text generation, embeddings generation, and knowledge bases. These use cases are described in more detail below: - -#### Text Generation - -* Develop interactive AI chatbots and virtual assistants that can carry out conversations naturally and engagingly. -* Use state-of-the-art large language models (LLMs) by providers like Anthropic for text comprehension and analysis use cases such as summarization, synthesis, and answering questions about large amounts of text. -* By using text generation models, you can build applications with features such as: - - * Draft documents - * Write computer code - * Answer questions about a knowledge base - * Analyze texts - * Give the software a natural language interface - * Tutor in a range of subjects - * Translate languages - * Simulate characters for games - * Image to text - -#### Embeddings Generation +### Features -Convert strings into vector embeddings for various purposes based on the relatedness of texts. +In the current version, Mendix supports text generation (including function/tool calling, chat with images, and chat with documents), vector embedding generation, knowledge base storage, and retrieval of knowledge base chunks. -Embeddings are commonly used for the following: +Typical use cases for generative AI are described in more detail in the [Typical LLM Use Cases](/appstore/modules/genai/get-started/#llm-use-cases) section of the *GenAI Concepts*. -* Search -* Clustering -* Recommendations -* Anomaly detection -* Diversity measurement -* Classification +### Prerequisites -You can combine embeddings with text generation capabilities and leverage specific sources of information to create a smart chat functionality tailored to your knowledge base. +To use this connector, you need configuration keys to authenticate to the Mendix Cloud GenAI services. You can generate keys in the [Mendix Cloud GenAI Portal](https://genai.home.mendix.com) or ask someone with access to either generate them for you or add you to their team so you can generate keys yourself. {{% alert color="info" %}} -The Mendix Cloud GenAI Connector module generates embeddings internally when interacting with a knowledge base. Pure embedding operations are only required if additional processes, such as using the generated vectors instead of text, are needed. For example, a similar search algorithm could use vector distances to calculate relatedness. -{{% /alert %}} - -#### Knowledge Base - -The module enables tailoring generated responses to specific contexts by grounding them in data inside of a collection belonging to a Mendix Cloud GenAI knowledge base resource. This allows for the secure use of private company data or other non-public information when interacting with GenAI models within the Mendix app. It provides a low-code solution to store discrete data (commonly called chunks) in the knowledge base and retrieves relevant information for end-user actions or application processes. - -Knowledge bases are often used for: - -1. [Retrieval Augmented Generation (RAG)](/appstore/modules/genai/rag/) retrieves relevant knowledge from the knowledge base, incorporates it into a prompt, and sends it to the model to generate a response. -2. Semantic search enables advanced search capabilities by considering the semantic meaning of the text, going beyond exact and approximate matching. It allows the knowledge base to be searched for similar chunks effectively. -If you are looking for a step-by-step guide on how to get your application data into a Mendix Cloud Knowledge Base, refer [Grounding Your Large Language Model in Data – Mendix Cloud GenAI](/appstore/modules/genai/how-to/howto-groundllm/). Note that the Mendix Portal also provides options for importing data into your knowledge base, such as file uploads. For more information, see [Navigate through the Mendix Cloud GenAI Portal](/appstore/modules/genai/mx-cloud-genai/Navigate-MxGenAI/). This documentation focuses solely on adding data from an application using the connector. +The Mendix Cloud GenAI Connector module generates embeddings internally when interacting with a knowledge base. This means that you do not need to create embedding keys yourself when interacting with a Mendix Cloud knowledge base. Direct embedding operations are only required if additional processes, such as using the generated vectors instead of text, are needed. For example, a similar search algorithm could use vector distances to calculate relatedness. -##### Architecture - -A Knowledge Base resource can comprise several collections. Each collection is specifically designed to hold numerous documents, serving as a logical grouping for related information based on its shared domain, purpose, or thematic focus. While collections provide a mechanism for data separation, with each corresponding to a [DeployedKnowledgebase](/appstore/modules/genai/genai-for-mx/commons/#deployed-knowledge-base), it is not best practice to create a large number of collections within a single Knowledge Base resource. A more performant and practical approach for achieving fine-grained data separation is through the strategic use of metadata. To learn more, see [Retrieve and Generate](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/#retrieve-and-generate). - -### Features - -In the current version, Mendix supports text generation (including function/tool calling, chat with images, and chat with documents), vector embedding generation, knowledge base storage, and retrieval of knowledge base chunks. - -### Prerequisites - -To use this connector, you need configuration keys to authenticate to the Mendix Cloud GenAI services. You can generate keys in the [developer portal](https://genai.home.mendix.com) or ask someone with access to either generate them for you or be added to the team to generate keys yourself. +{{% /alert %}} ### Dependencies {#dependencies} @@ -93,7 +47,11 @@ Follow the steps below to get started: * Make sure to configure the [Encryption module](/appstore/modules/encryption/#configuration) before you connect your app to Mendix Cloud GenAI. * Add the module role `MxGenAIConnector.Administrator` to your Administrator **User roles** in the **Security** settings of your app. * Add the `Configuration_Overview` page (**USE_ME** > **Configuration**) to your navigation, or add the `Snippet_Configuration` to a page that is already part of your navigation. Alternatively, you can register your key by using the `Configuration_RegisterByString` microflow. -* Complete the runtime setup of Mendix Cloud GenAI configuration by navigating to the page mentioned above. Import a key generated in the [portal](https://genai.home.mendix.com) or provided to you and click **Test Key** to validate its functionality. Note that this key establishes a connection between the Mendix Cloud resources and your application. It contains all the information required to set up the connection. +* Complete the runtime setup of the Mendix Cloud GenAI configuration by navigating to the page mentioned above. Import a key generated in the [Mendix Cloud GenAI Portal](https://genai.home.mendix.com) or provided to you and click **Test Key** to validate its functionality. Note that this key establishes a connection between the Mendix Cloud resources and your application. It contains all the information required to set up the connection. + +{{% alert color="info" %}} +When using an Embeddings Model Resource together with a Knowledge Base Resource, you do not need to import both keys. Importing the Knowledge Base Resource key automatically generates the connection details for the embeddings generation model. +{{% /alert %}} ## Operations @@ -101,9 +59,15 @@ Follow the steps below to get started: Configuration keys are stored persistently after they are imported (either via the UI or the exposed microflow). There are three different types of configurations that reflect the use cases this service supports. The specific operations are described below. -To use the operations, either a `DeployedModel` (text, embeddings) or a `DeployedKnowledgeBase` must always be passed as input. The DeployedModel will be created automatically when importing keys at runtime and needs to be retrieved from the database. To initialize a knowledge base operation, use the `DeployedKnowledgeBase: Get` toolbox action to retrieve the DeployedKnowledgeBase object for a specified collection. It requires the collection's Name (string) as input. +To use the operations, either a `DeployedModel` (text, embeddings) or a `DeployedKnowledgeBase` must always be passed as input. + +### How to get the `DeployedModel` in scope + +The `DeployedModel` object will be created automatically when importing keys at runtime and needs to be retrieved from the database. -In Mendix Cloud GenAI, a single knowledge base resource (MxCloudKnowledgeBaseResource) can contain multiple collections (tables). As a result, several DeployedKnowledgeBase objects may belong to the same resource. +### How to get the `DeployedKnowledgeBase` in scope + +In Mendix Cloud GenAI, a single knowledge base resource (`MxCloudKnowledgeBaseResource`) can contain multiple `DeployedKnowledgeBase` objects (tables, referred to as 'collections'). As a result, several collections may belong to the same resource. You can use the `DeployedKnowledgeBase: Get` toolbox action to retrieve the right collection and initialize a knowledge base operation. It requires the `Collection.Name` (string) as input (which is usually different from the `Collection.DisplayName` attribute). ### Chat Completions Operation @@ -176,6 +140,48 @@ In the entire conversation, you can pass up to five documents that are smaller t The model uses the file name when analyzing documents, which may introduce a potential vulnerability to prompt injection. To reduce this risk, consider modifying file names before including them in the request. {{% /alert %}} +### About Knowledge Bases + +#### Data Separation with Collections and Metadata + +##### Collections + +A Knowledge Base resource can comprise several collections. Each collection is specifically designed to hold numerous documents, serving as a logical grouping for related information based on its shared domain, purpose, or thematic focus. + +Below is a diagram showing how resources are organized into separate collections. This approach allows multiple use cases to share a common resource while the option to only add the required collections to the conversation context is preserved. For example, both employee onboarding and IT ticket support require information about IT setup and equipment. However, only onboarding needs knowledge about the company culture and values, while only IT support requires access to historical support ticket data. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKnowledgeBaseResource.png" >}} + +While collections provide a mechanism for data separation, it is not best practice to create a large number of collections within a single Knowledge Base resource. A more performant and practical approach for achieving fine-grained data separation is through the strategic use of metadata. + +##### Metadata + +Metadata is additional information that can be attached to data in a GenAI knowledge base. Unlike the actual content, metadata provides structured details that help in organizing, searching, and filtering information more efficiently. It helps manage large datasets by allowing the retrieval of relevant data based on specific attributes rather than relying solely on similarity-based searches. + +Metadata consists of key-value pairs and serves as additional information connected to the data, though it is not part of the vectorization itself. + +In the employee onboarding and IT ticket support example, instead of having two different collections, such as IT setup, and equipment and historical support tickets, there could be one named 'Company IT'. To retrieve tickets only and no other information from this collection, add the metadata below during insertion. + +```text +key: `Category`, value: `Ticket` +``` + +The model then generates its response using the specified metadata instead of solely the input text. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKBMetadataSeparation.png" >}} + +Using metadata, even more fine-grained filtering becomes feasible. Each ticket may have associated metadata, such as + +* key: `Ticket Type`, value: `Bug` +* key: `Status`, value: `Solved` +* key: `Priority`, value: `High` + +Instead of relying solely on similarity-based searches of ticket descriptions, users can then filter for specific tickets, such as 'Bug' tickets with the status set to 'Solved'. You can add [MetaData](/appstore/modules/genai/genai-for-mx/commons/#chunkcollection-add-knowledgebasechunk) with the respective key to each chunk during insertion. + +#### How to get data into a knowledge base + +If you are looking for a step-by-step guide on how to get your application data into a collection inside of a Mendix Cloud Knowledge Base Resource, refer to [Grounding Your Large Language Model in Data – Mendix Cloud GenAI](/appstore/modules/genai/how-to/howto-groundllm/). Note that the Mendix Portal also provides options for importing data into your knowledge base, such as file uploads. For more information, see [Navigate through the Mendix Cloud GenAI Portal](/appstore/modules/genai/mx-cloud-genai/Navigate-MxGenAI/). This documentation focuses solely on adding data from inside a Mendix application and using the connector. + ### Knowledge Base Operations To implement knowledge base logic into your Mendix application, you can use the actions in the **USE_ME** > **Knowledge Base** folder or under the **GenAI Knowledge Base (Content)** or **Mendix Cloud Knowledge Base** categories in the **Toolbox**. These actions require a specialized [DeployedKnowledgeBase](/appstore/modules/genai/genai-for-mx/commons/#deployed-knowledge-base) of type `Collection` that determines the model and endpoint to use. Additionally, the collection name must be passed when creating the object and it must be associated with a `Configuration` object. Please note that for Mendix Cloud GenAI a knowledge base resource may contain several collections (tables). @@ -185,7 +191,7 @@ Dealing with knowledge bases involves two main stages: 1. [Insertion of knowledge](#knowledge-base-insertion) 2. [Retrieval of knowledge (Nearest neighbor)](#knowledge-base-retrieval) -You do not need to manually add embeddings to a chunk, as the connector handles this internally. To see all existing knowledge bases for a configuration, go to the **Knowledge Base** tab on the [Mendix Cloud GenAI Configuration](#configuration) page and refresh the view on the right. Alternatively, use the `Get Collections` action to retrieve a synchronized list of collections inside of your knowledge base resource to include in your module. Lastly, you can delete a collection using the `Delete Collection` action. +You do not need to manually add embeddings to a chunk, as the connector handles this internally. To see all existing collections for a knowledge base configuration, go to the **Knowledge Base** tab on the [Mendix Cloud GenAI Configuration](#configuration) page and refresh the view on the right. Alternatively, use the `Get Collections` action to retrieve a synchronized list of collections inside your knowledge base resource to include in your module. Lastly, you can delete a collection using the `Delete Collection` action. {{% alert color="warning" %}} The knowledge chunks are stored in an AWS OpenSearch Serverless database to ensure scalable and high-performance vector calculations—for example, retrieving the nearest neighbors of a given input. Inserted or modified chunks are only available for read operations (retrieval) in the knowledge base within 60-120 seconds. For more information, see [AWS documentation](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-vector-search.html#serverless-vector-limitations). diff --git a/content/en/docs/marketplace/genai/mendix-cloud-genai/_index.md b/content/en/docs/marketplace/genai/mendix-cloud-genai/_index.md index 9f8aad6c83d..5f49d7de81a 100644 --- a/content/en/docs/marketplace/genai/mendix-cloud-genai/_index.md +++ b/content/en/docs/marketplace/genai/mendix-cloud-genai/_index.md @@ -13,14 +13,18 @@ In order to help developers integrate GenAI capabilities into custom application ## Resources Overview -The GenAI Resources portal provides an overview of all the resources you have access to. You can also request new GenAI Resources through the portal. - There are three different types of resources: * Compute – Text Generation: generates human-like text based on given inputs, essential for applications requiring natural language generation. * Knowledge Base: A knowledge base can be used to upload your data which then can be used by a text generation resource. -* Compute – Embeddings Generation: converts text into vector representations. An embeddings resource is required to uploading data to your Knowledge Base. +* Compute – Embeddings Generation: converts text into vector representations. An embeddings resource is required to uploading data to your Knowledge Base. + +## Getting started + +1. Learn about GenAI Resource Packs and how to acquire them in the [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/mx-cloud-genai/resource-packs/) document. +2. Once you have access to GenAI resources, log in to the [Mendix Cloud GenAI portal](https://genai.home.mendix.com/) to generate access keys for your resources. This portal provides an overview of all the resources you have access to and you can also request new GenAI Resources there. For more information, see [Navigate through the Mendix Cloud GenAI Portal](https://docs.mendix.com/appstore/modules/genai/mx-cloud-genai/Navigate-MxGenAI/). +3. Use a starter app containing the [Mendix Cloud GenAI Connector](https://marketplace.mendix.com/link/component/239449) (for example, the [BlankGenAI starter app](https://marketplace.mendix.com/link/component/227934)) or implement the connector in the Mendix application according to its documentation. Once you have imported access key in its configuration overview, you are connected to Mendix Cloud GenAI and can access available resources within your application. ## Relevant Sources diff --git a/content/en/docs/marketplace/genai/mendix-cloud-genai/mendix-cloud-grp.md b/content/en/docs/marketplace/genai/mendix-cloud-genai/mendix-cloud-grp.md index d989e2ea541..a9387064823 100644 --- a/content/en/docs/marketplace/genai/mendix-cloud-genai/mendix-cloud-grp.md +++ b/content/en/docs/marketplace/genai/mendix-cloud-genai/mendix-cloud-grp.md @@ -35,7 +35,7 @@ The Mendix Cloud GenAI Resource Packs provide access to the following models: The models are available through the Mendix Cloud, leveraging AWS's highly secure Amazon Bedrock multi-tenant architecture. This architecture employs advanced logical isolation techniques to effectively segregate customer data, requests, and responses, ensuring a level of data protection that aligns with global security compliance requirements. Customer prompts, requests, and responses are neither stored nor used for model training. Your data remains your data. -Customers looking to leverage other models in addition to the above can also take advantage of Mendix's [(Azure) OpenAI Connector](/appstore/modules/genai/reference-guide/external-connectors/openai/) and Amazon [Bedrock Connector](/appstore/modules/genai/reference-guide/external-connectors/bedrock/) to integrate numerous other models into their apps. +Customers looking to leverage other models in addition to the above can also take advantage of Mendix's [(Azure) OpenAI Connector](/appstore/modules/genai/reference-guide/external-connectors/openai/), Amazon [Bedrock Connector](/appstore/modules/genai/reference-guide/external-connectors/bedrock/), and [Mistral Connector](/appstore/modules/genai/reference-guide/external-connectors/mistral/) to integrate numerous other models into their apps. ### Technical Details for Model Resource Packs @@ -65,24 +65,6 @@ Knowledge Bases are based on elastically scaling, serverless OpenSearch vector d | Memory | Elastic | | Disk Space | 10 GB | -## Mendix Portal - -The Mendix Portal allows easy access to manage the resources, through the GenAI Resources section in the portal. - -* Get insight into the consumption of input/output tokens against entitlements for Models -* Manage content for Knowledge Bases -* Manage team access to all resources -* Create and manage connection keys to connect your apps with all resources -* Track activity logs for team access and connection key management - -## Mendix Cloud GenAI Connector - -The [Mendix Cloud GenAI connector](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/) lets you utilize Mendix Cloud GenAI resource packs directly within your Mendix application. It allows you to integrate generative AI by dragging and dropping common operations from its toolbox. Note that any versions older than the ones listed below are no longer functional: - -* GenAI for Mendix bundle v2.4.1 (Mendix 9) (contains Mendix Cloud GenAI connector) or -* Mendix Cloud GenAI connector v3.1.1 (no `DeployedKnowledgeBase` support) or -* Mendix Cloud GenAI connector v4.4.0 (`DeployedKnowledgeBase` support). - ## Regional Availability Mendix Cloud GenAI Resource Packs are available in the following regions of Mendix Cloud: @@ -90,6 +72,8 @@ Mendix Cloud GenAI Resource Packs are available in the following regions of Mend * Europe (Frankfurt) - eu-central-1 * Canada (Montreal) - ca-central-1 +The availability of model versions depends on the selected region. + ## Understanding Third-Party Requirements Mendix AI services are powered by third-party technologies, including AWS Bedrock, Anthropic, and Cohere. To help you succeed with your implementation, here is what to do next: @@ -109,6 +93,28 @@ Save these links for future reference. Always review the terms before starting d Compliance with these terms is mandatory to maintain access to the services. {{% /alert %}} +## More resources + +### Mendix Cloud GenAI Portal + +The [Mendix Cloud GenAI Portal](https://genai.home.mendix.com/) allows easy access to manage the resources through the GenAI Resources section of the portal. + +* Get insight into the consumption of input/output tokens for Text and Embeddings Generation Resources. +* Manage content for Knowledge Bases. +* Manage team access to all resources. +* Create and manage connection keys to connect your apps with all resources. +* Track activity logs for team access and connection key management. + +For more information, see [Navigate through the Mendix Cloud GenAI Portal](/appstore/modules/genai/mx-cloud-genai/Navigate-MxGenAI/). + +### Mendix Cloud GenAI Connector + +The [Mendix Cloud GenAI connector](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/) lets you utilize Mendix Cloud GenAI resource packs directly within your Mendix application. It allows you to integrate generative AI by dragging and dropping common operations from its toolbox. Note that any versions older than the ones listed below are no longer functional: + +* GenAI for Mendix bundle v2.4.1 (Mendix 9) (contains Mendix Cloud GenAI connector) or +* Mendix Cloud GenAI connector v3.1.1 (no `DeployedKnowledgeBase` support) or +* Mendix Cloud GenAI connector v4.4.0 (`DeployedKnowledgeBase` support). + ## FAQ ### What Happens to Data Processed by Mendix Cloud GenAI Services? diff --git a/content/en/docs/marketplace/genai/mendix-cloud-genai/navigate_mxgenai.md b/content/en/docs/marketplace/genai/mendix-cloud-genai/navigate_mxgenai.md index da7c5ca3c27..e2ba353b2d5 100644 --- a/content/en/docs/marketplace/genai/mendix-cloud-genai/navigate_mxgenai.md +++ b/content/en/docs/marketplace/genai/mendix-cloud-genai/navigate_mxgenai.md @@ -8,39 +8,33 @@ weight: 30 ## Introduction -The [Mendix Cloud GenAI portal](https://genai.home.mendix.com/) is an online platform that provides access to Mendix Cloud GenAI resource packs. These resource packs on Mendix Cloud enable seamless integration with Generative AI technology, provisioned and hosted by Mendix: +The [Mendix Cloud GenAI portal](https://genai.home.mendix.com/) is the part of the Mendix portal that provides access to [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/mx-cloud-genai/resource-packs/). After logging in, you can navigate to the overview of all resources. You can see all resources, that you are a team member of and access their details. -* GenAI Model Resource Packs provide access to model resources: Anthropic's Claude and Cohere's Embed. -* GenAI Knowledge Base Resource Packs provide the infrastructure to deliver retrieval-augmented generation (RAG) architecture and other GenAI use cases requiring a vector database. +## Resource Details -GenAI resource packs accelerate the delivery of complete generative AI solutions within Mendix apps that seamlessly integrate with GenAI technology. Learn more by following these links to [Mendix Cloud GenAI Resource Packs](/appstore/modules/genai/mx-cloud-genai/resource-packs/) and the [Mendix Cloud GenAI Connector](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/). To start with a GenAI-infused Mendix application, see [creating a chatbot using the AI Bot Starter App](/appstore/modules/genai/using-genai/starter-template/) or [building a GenAI app from scratch with the Blank GenAI App](/appstore/modules/genai/using-genai/blank-app/). +After clicking on a specific resource, you land on its details page, offering shortcut to consumption insights, key generation, team management, and helpful documentation. -## Resource Details +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_Details.png" >}} ### Settings -{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_Settings.png" >}} The **Settings** tab contains the details of a GenAI resource. It shows the following: * **Display Name**: indicates the name of the resource. * **ID**: indicates the resource ID. * **Region(s)**: the region where the resource is hosted. -* **Cross Region Inference (CRI)**: shows if the model supports cross-region inference. +* **Cross Region Inference (CRI)**: shows if the model supports cross-region inference*. * **Cloud Provider**: indicates the cloud provider, for example, AWS. * **Type**: this is the type of resource, for example, Text Generation, Embedding, Knowledge Base, etc. * **Model**: indicates which model is used, for example, Anthropic Claude Sonnet 3.5. * **Plan**: indicates the subscription plan used for compute resources (for example, embedding or text generation resources). * **Environment**: shows which environment is used, for example, test, acceptance, or production. -When you are looking at the knowledge base resource settings, you will also see details of the associated embeddings resource and vice versa. To learn more about embeddings, see the [Embedding vector](/appstore/modules/genai/rag/#embedding-vector) section of *RAG in a Mendix App*. - -#### Cross-Region Inference (CRI) - -Cross-region inference (CRI) allows a model to redirect requests to another region, helping to distribute the load across multiple regions. Connecting to a cross-region inference profile does not change how the request is sent; the redirection happens on the server side, determining the region to handle the request to get the fastest response. +\* Cross-region inference (CRI) allows a model to redirect requests to another region, helping to distribute the load across multiple regions within the same area. So, EU requests always stay within EU regions. Connecting to a cross-region inference profile does not change how the request is sent; the redirection happens on the server side, determining the region to handle the request to get the fastest response. For more information, see [Increase throughput with cross-Region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html). If applicable, CRI profiles are selected during provisioning of a model resource. -For more details on how CRI can help increase throughput, see [Increase throughput with cross-Region inference](https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html). +#### Additional Details for Knowledge Base Resources -Currently, CRI is available for a limited number of models and can only be enabled for new resources. In some cases, certain models are available only through cross-region inference. +For knowledge base resources, you can also see details of the associated embeddings resource and vice versa. To learn more about embeddings, see the [Embedding vector](/appstore/modules/genai/rag/#embedding-vector) section of *RAG in a Mendix App*. ### Team @@ -64,7 +58,11 @@ Make sure to store it securely, as it will only be shown once. {{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_KeyGeneration.png" >}} -Once created, the key can be used in the Mendix application via the Mendix Cloud GenAI Connector. When you create a key for a knowledge base, an embedding key is automatically generated and marked accordingly in the keys overview. +Once created, the key can be used in the Mendix application via the Mendix Cloud GenAI Connector. + +#### Additional Information for Knowledge Base Resource Keys + +When you create a key for a knowledge base, an embeddings resource key is automatically generated for the selected embeddings model and marked accordingly in the keys overview. To configure a knowledge base connection from a Mendix application, you only need to import the knowledge base resource key. The connection details for the embeddings model are created automatically. ### Content (Only for Knowledge Bases) @@ -85,33 +83,55 @@ When you select the **Add Files Like .TXT or .PDF** option, you can upload docum {{% alert color="info" %}} Only TXT and PDF files are supported. {{% /alert %}} -Before uploading, you can choose to upload the data to a new collection, the default collection, or another existing collection within the resource. A Knowledge Base resource can comprise several collections. Each collection is specifically designed to hold numerous documents, serving as a logical grouping for related information based on its shared domain, purpose, or thematic focus. While collections provide a mechanism for data separation, with each corresponding to a [DeployedKnowledgebase](/appstore/modules/genai/genai-for-mx/commons/#deployed-knowledge-base), it is not best practice to create a large number of collections within a single Knowledge Base resource. A more performant and practical approach for achieving fine-grained data separation is through the strategic use of metadata. +Before uploading, you can choose to upload the data to a new collection, the default collection, or another existing collection within the resource. A Knowledge Base resource can comprise several collections. Each collection is specifically designed to hold numerous documents, serving as a logical grouping for related information based on its shared domain, purpose, or thematic focus. Below is a diagram showing how resources are organized into separate collections. This approach allows multiple use cases to share a common resource while the option to only add the required collections to the conversation context is preserved. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKnowledgeBaseResource.png" >}} + +{{% alert color="info" %}} While collections provide a mechanism for data separation, it is not best practice to create a large number of collections within a single Knowledge Base resource. A more performant and practical approach for achieving fine-grained data separation is through the strategic use of [Metadata](#metadata). {{% /alert %}} ##### Metadata {#metadata} -Metadata is additional information that can be attached to data in a GenAI knowledge base. Unlike the actual content, metadata provides structured details that help in organizing, searching, and filtering information more efficiently. It helps manage large datasets by allowing to retrieve of relevant data based on specific attributes rather than relying solely on similarity-based searches. +Metadata is additional information that can be attached to data in a GenAI knowledge base. Unlike the actual content, metadata provides structured details that help in organizing, searching, and filtering information more efficiently. It helps manage large datasets by allowing the retrieval of relevant data based on specific attributes rather than relying solely on similarity-based searches. Metadata consists of key-value pairs and serves as additional information connected to the data, though it is not part of the vectorization itself. -For example, a GenAI Knowledge Base could be used to store customer support tickets. Each ticket may have associated metadata such as Ticket Type, Status, and Priority. Instead of relying solely on similarity-based searches of ticket descriptions, users can filter for specific tickets, such as 'bug' tickets with the status set to 'solved'. +In the employee onboarding and IT ticket support example, instead of having two different collections, such as IT setup, and equipment and historical support tickets, there could be one named 'Company IT'. To retrieve tickets only and no other information from this collection, add the metadata below during insertion. + +```text +key: `Category`, value: `Ticket` +``` + +The model then generates its response using the specified metadata instead of solely the input text. + +{{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKBMetadataSeparation.png" >}} + +Using metadata, even more fine-grained filtering becomes feasible. Each ticket may have associated metadata, such as + +* key: `Ticket Type`, value: `Bug` +* key: `Status`, value: `Solved` +* key: `Priority`, value: `High` + +Instead of relying solely on similarity-based searches of ticket descriptions, users can then filter for specific tickets, such as 'Bug' tickets with the status set to 'Solved'. #### Add Data from a Mendix Application You can upload data directly from Mendix to the Knowledge Base. To do so, several operations of the Mendix Cloud GenAI Connector are required. For a detailed guide on this process, see the [Add Data Chunks to Your Knowledge Base](/appstore/modules/genai/mx-cloud-genai/MxGenAI-connector/#add-data-chunks-to-your-knowledge-base) section of **Mendix Cloud GenAI Connector**. -## Token Consumption Monitor +### Consumption (Only for Text and Embeddings Generation Resources) + +{{% alert color="info" %}} The **Consumption** tab is available for Model resources only.{{% /alert %}} -The **Token Consumption Monitor** shows detailed graphs of the token consumption used by the GenAI resource. Use this overview to see the current usage, insights on the usage per day, and to compare the current month with previous months. +The **Consumption** section provides outcomes of token consumption monitoring for each GenAI resource in a graphical way. Use this overview to see the current usage, insights on the usage per day, and to compare the current month with previous months. Note that months represent bundle months here, which is the period during which token consumption is tracked, beginning on the date of your last GenAI Resource plan entitlement reset and ending on the next reset date. This creates a recurring monthly cycle based on your plan activation date, not the calendar month. {{< figure src="/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_TokenConsumptionMonitor.png" >}} -### What Are Tokens? +#### What Are Tokens? Tokens are what you pay for when consuming large language model services. In order for a large language model to understand text input, the text is first ‘tokenized’: broken down into smaller pieces where each piece represents a token with its unique ID. A good rule of thumb is that 100 tokens are around 75 English words, however there are always differences depending on the model or the language used. After tokenization, each token will be assigned an embeddings vector. The tokens required to feed the input prompt to the model are called ‘input tokens’. The tokens required to transform the model output vectors into, for example, text or images are called ‘output tokens’. -### When Are Tokens Consumed? +#### When Are Tokens Consumed? Text generation resources consume both input and output tokens (text sent to the model and generated by the model). @@ -119,6 +139,6 @@ Embeddings resources only consume input tokens. This is because only the generat Knowledge base resources do not consume tokens as they only store embedding vectors. Uploading a document to a knowledge base connected to an Embeddings resource will consume tokens in the embeddings resource. -### Exporting Token Consumption Data +#### Exporting Token Consumption Data Click **Export** to export consumption data in CSV format. The export contains basic information about input tokens, output tokens, and dates. Days with no consumption are not exported. diff --git a/content/en/docs/marketplace/genai/reference-guide/external-platforms/mistral.md b/content/en/docs/marketplace/genai/reference-guide/external-platforms/mistral.md index ae0492297cf..c5484e207fc 100644 --- a/content/en/docs/marketplace/genai/reference-guide/external-platforms/mistral.md +++ b/content/en/docs/marketplace/genai/reference-guide/external-platforms/mistral.md @@ -11,29 +11,11 @@ weight: 20 The [Mistral Connector](https://marketplace.mendix.com/link/component/248276) allows you to integrate generative AI capabilities into your Mendix application. Since the Mistral API is compatible with [OpenAI API](https://platform.openai.com/), this module mainly focuses on Mistral specific UI while reusing the operations inside of the OpenAI connector. -The current scope covers text generation use cases based on the [Chat Completions API](https://docs.mistral.ai/api/endpoint/chat) and embedding use cases based on the [Embeddings API](https://docs.mistral.ai/api/endpoint/embeddings). - -### Typical Use Cases - -The Mistral Connector is commonly used for text generation and embeddings. These use cases are described in more detail below. - -#### Text Generation {#use-cases-text} +### Features {#features} -* Develop interactive AI chatbots and virtual assistants that can carry out conversations in a natural and engaging manner. -* Use Mistral’s large language models (LLMs) for text comprehension and analysis use cases such as summarization, synthesis, and answering questions about large amounts of text. -* Tailor the Mistral models on a specific task or domain by adding custom data to improve their performance. -* Integrate more easily with Mistral’s platform. By providing text generation models, this allows you to build applications with the following features: - * Draft documents - * Write computer code - * Answer questions about a knowledge base - * Analyze texts - * Give software a natural language interface - * Tutor in a range of subjects - * Translate languages - * Simulate characters for games - * Image to text +The Mistral Connector is commonly used for text generation based on the [Chat Completions API](https://docs.mistral.ai/api/endpoint/chat) and embeddings generation with the [Embeddings API](https://docs.mistral.ai/api/endpoint/embeddings). Typical use cases for generative AI are described in the [Typical LLM Use Cases](/appstore/modules/genai/get-started/#llm-use-cases). -For more information, see [Mistral models](https://docs.mistral.ai/getting-started/models). +For more information about the models, see [Mistral models](https://docs.mistral.ai/getting-started/models). #### Image Generation {#use-cases-images} @@ -43,10 +25,6 @@ Mistral does not currently offer image generation models out of the box. It is p The Mistral connector supports Knowledge bases from providers such as pgVector, Mendix Cloud, Amazon Bedrock, and Azure AI Search to be added to a conversation. -### Features {#features} - -With the current version, Mendix supports the Chat Completions API for [text generation](https://docs.mistral.ai/api/endpoint/chat) and the Embeddings API for [vector embeddings](https://docs.mistral.ai/api/endpoint/embeddings). - ### Prerequisites To use this connector, you need to sign up for a Mistral account and create an API key. For more information, see the [Quickstart guide](https://docs.mistral.ai/getting-started/quickstart). @@ -63,7 +41,7 @@ To use this connector, you need to sign up for a Mistral account and create an A Install all required modules from the Mendix Marketplace as listed in the [Dependencies](#dependencies) section above. -To import the [Mistral Connector](https://marketplace.mendix.com/link/component/248276) into your app, follow the instructions in [How to Use Marketplace Content](/appstore/use-content/). +To import the [Mistral Connector](https://marketplace.mendix.com/link/component/248276) and the other modules into your app, follow the instructions in [How to Use Marketplace Content](/appstore/use-content/). ## Configuration {#configuration} @@ -74,7 +52,7 @@ After you install the Mistral and OpenAI connector, you can find them in the **M 1. Add the module roles `OpenAIConnector.Administrator` and `MistralConnector.Administrator` to your Administrator **User roles** in the **Security** settings of your app. 2. Add the **MistralConfiguration_Overview** page from the Mistral connector module (**USE_ME > MistralConfiguration**) to your navigation, or add the `Snippet_MistralConfigurations` to a page that is already part of your navigation. 3. Continue setting up your Mistral configuration at runtime. For more information, follow the instructions in the [Mistral Configuration](#mistral-configuration) section below. -4. Configure the models you need to use for your use case. +4. Configure the models you need for your use case. #### Mistral Configuration {#mistral-configuration} @@ -151,7 +129,7 @@ Mistral does not directly connect to the knowledge resources. The model returns This functionality is part of the implementation executed by the GenAI Commons Chat Completions operations mentioned earlier. As a developer, you need to make the system aware of your indexes and their purpose by registering them with the request. This is done using the GenAI Commons operation [Tools: Add Knowledge Base](/appstore/modules/genai/genai-for-mx/commons/#add-knowledge-base-to-request), which must be called once per knowledge resource before passing the request to the Chat Completions operation. -Note that the retrieval process is independent of the model provider and can be used with any model that supports function calling. +Note that the retrieval process is independent of the model provider and can be used with any model that supports function calling, as it relies on the generalized `GenAICommons.DeployedKnowledgeBase` input parameter. #### Vision {#chatcompletions-vision} diff --git a/content/en/docs/marketplace/genai/reference-guide/external-platforms/openai.md b/content/en/docs/marketplace/genai/reference-guide/external-platforms/openai.md index c8699f86ad0..df90a235bfd 100644 --- a/content/en/docs/marketplace/genai/reference-guide/external-platforms/openai.md +++ b/content/en/docs/marketplace/genai/reference-guide/external-platforms/openai.md @@ -13,29 +13,7 @@ aliases: The [OpenAI Connector](https://marketplace.mendix.com/link/component/220472) allows you to integrate generative AI into your Mendix app. It is compatible with [OpenAI's platform](https://platform.openai.com/) as well as [Azure's OpenAI service](https://oai.azure.com/). -The current scope covers text generation use cases based on the [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat), image generation use cases based on the [Image Generations API](https://platform.openai.com/docs/api-reference/images), and embedding use cases based on the [Embeddings API](https://platform.openai.com/docs/api-reference/embeddings). Furthermore, indexes via [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/) can be used for knowledge base retrieval. - -Mendix provides dual-platform support for both OpenAI and Azure OpenAI. - -### Typical Use Cases {#use-cases} - -The OpenAI Connector is commonly used for text generation, image generation, and embeddings. These use cases are described in more detail below. - -#### Text Generation {#use-cases-text} - -* Develop interactive AI chatbots and virtual assistants that can carry out conversations in a natural and engaging manner. -* Use OpenAI’s large language models (LLMs) for text comprehension and analysis use cases such as summarization, synthesis, and answering questions about large amounts of text. -* Fine-tune the OpenAI models on a specific task or domain by training them on custom data to improve their performance. -* Integrate more easily with OpenAI’s platform. By providing text generation models, this allows you to build applications with the following features: - * Draft documents - * Write computer code - * Answer questions about a knowledge base - * Analyze texts - * Give software a natural language interface - * Tutor in a range of subjects - * Translate languages - * Simulate characters for games - * Image to text +### Features {#features} OpenAI provides market-leading LLM capabilities with GPT-4: @@ -43,38 +21,15 @@ OpenAI provides market-leading LLM capabilities with GPT-4: * Creativity – Generate, edit, and iterate with end-users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning an end-user’s writing style. * Longer context – GPT-4 can handle over 25,000 words of text, allowing for use cases like long-form content creation, extended conversations, and document search and analysis. -#### Image Generation {#use-cases-images} - -Generate one or more completely new, original images and art from a text description. Powered by the OpenAI DALL-E models, the connector enables developers to generate these images by combining concepts, attributes, and styles. - -#### Embeddings {#use-cases-embeddings} - -Convert strings into vector embeddings for various purposes based on the relatedness of texts. - -Embeddings are commonly used for the following: - -* Search -* Clustering -* Recommendations -* Anomaly detection -* Diversity measurement -* Classification +Mendix provides dual-platform support for both [OpenAI](https://platform.openai.com/) and [Azure OpenAI](https://oai.azure.com/). -Combine embeddings with text generation capabilities and leverage specific sources of information to create a smart chat functionality tailored to your own knowledge base. +With the current version, Mendix supports the Chat Completions API for [text generation](https://platform.openai.com/docs/guides/text-generation), the Image Generations API for [images](https://platform.openai.com/docs/guides/images), the Embeddings API for [vector embeddings](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings), and indexes via [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/) for knowledge base retrieval. -{{% alert color="info" %}} -For more information on how to set up a vector database, see [Retrieval Augmented Generation (RAG)](/appstore/modules/genai/rag/). Also, check out the [GenAI Showcase App](https://marketplace.mendix.com/link/component/220475) from the Marketplace for an example implementation. -{{% /alert %}} +Typical use cases for generative AI are described in the [Typical LLM Use Cases](/appstore/modules/genai/get-started/#llm-use-cases). #### Knowledge Base -By integrating Azure AI Search, the OpenAI Connector allows for knowledge base retrieval from Azure datsources. The most common use case is retrieval augmented generation (RAG) to retrieve relevant knowledge from the knowledge base, incorporating it into a prompt, and sending it to the model to generate a response. - -### Features {#features} - -Mendix provides dual-platform support for both [OpenAI](https://platform.openai.com/) and [Azure OpenAI](https://oai.azure.com/). - -With the current version, Mendix supports the Chat Completions API for [text generation](https://platform.openai.com/docs/guides/text-generation), the Image Generations API for [images](https://platform.openai.com/docs/guides/images), the Embeddings API for [vector embeddings](https://platform.openai.com/docs/guides/embeddings/what-are-embeddings), and indexes via [Azure AI Search](https://learn.microsoft.com/en-us/azure/search/) for knowledge base retrieval. +By integrating Azure AI Search, the OpenAI Connector enables knowledge base retrieval from Azure data sources. For Retrieval Augmented Generation (RAG) scenarios, chat completions with (Azure) OpenAI can also be combined with knowledge bases by other provider such as Mendix Cloud. ### Prerequisites {#prerequisites} @@ -228,7 +183,7 @@ OpenAI does not directly connect to the Azure AI Search resource. The model retu This functionality is part of the implementation executed by the GenAI Commons Chat Completions operations mentioned earlier. As a developer, you need to make the system aware of your indexes and their purpose by registering them with the request. This is done using the GenAI Commons operation [Tools: Add Knowledge Base](/appstore/modules/genai/genai-for-mx/commons/#add-knowledge-base-to-request), which must be called once per index before passing the request to the Chat Completions operation. -Note that the retrieval process is independent of the model provider and can be used with any model that supports function calling. +Note that the retrieval process is independent of the model provider and can be used with any model that supports function calling, as it relies on the generalized `GenAICommons.DeyploedKnowledgeBase`entity. #### Vision {#chatcompletions-vision} diff --git a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md index 16f3f2b51c0..464fcc9db5a 100644 --- a/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md +++ b/content/en/docs/marketplace/platform-supported-content/modules/aws/amazon-bedrock.md @@ -21,6 +21,8 @@ Amazon Bedrock is a fully managed service that makes foundation models (FMs) fro * Generating images based on text prompts and displaying them in the Mendix app. * Generating embedding vectors for text inputs. +Typical use cases for generative AI are described in the [Typical LLM Use Cases](/appstore/modules/genai/get-started/#llm-use-cases). + ### Prerequisites {#prerequisites} The Amazon Bedrock connector requires Mendix Studio Pro version 10.24.0 or above. diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKBMetadataSeparation.png b/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKBMetadataSeparation.png new file mode 100644 index 00000000000..241a6dba351 Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKBMetadataSeparation.png differ diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKnowledgeBaseResource.png b/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKnowledgeBaseResource.png new file mode 100644 index 00000000000..bb70ab01e04 Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIKnowledgeBaseResource.png differ diff --git a/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_Details.png b/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_Details.png new file mode 100644 index 00000000000..55d90d7233f Binary files /dev/null and b/static/attachments/appstore/platform-supported-content/modules/genai/navigate_mxgenai/GenAIResource_Details.png differ