From 1a536d11a2dea039813898a31c7bfffec026fc4e Mon Sep 17 00:00:00 2001 From: Brian Munkholm Date: Mon, 16 Jun 2025 22:50:31 +0200 Subject: [PATCH 1/3] Fix TOC for Integrations --- docs/cluster/{integrations.md => integrations/index.md} | 6 +++--- docs/index.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) rename docs/cluster/{integrations.md => integrations/index.md} (93%) diff --git a/docs/cluster/integrations.md b/docs/cluster/integrations/index.md similarity index 93% rename from docs/cluster/integrations.md rename to docs/cluster/integrations/index.md index 984ff57..b77a3c4 100644 --- a/docs/cluster/integrations.md +++ b/docs/cluster/integrations/index.md @@ -10,17 +10,17 @@ Fully managed by CrateDB Cloud, integrations eliminate the need for manual setup or maintenance of separate ETL pipelines. -```{figure} ../_assets/img/integrations-example.png +```{figure} ../../_assets/img/integrations-example.png :width: 600px :align: center :alt: Integration in CrateDB Cloud ``` :::{toctree} -:maxdepth: 1 +:titlesonly: :hidden: -MongoDB CDC (Preview) +MongoDB CDC (Preview) ::: --- diff --git a/docs/index.md b/docs/index.md index c36392b..b5e2c45 100644 --- a/docs/index.md +++ b/docs/index.md @@ -104,7 +104,7 @@ A command-line tool to operate your managed clusters. :::: -Do you want to learn about key database drivers and client applications for +Do you want to learn about key database drivers and client applications for CrateDB, such as CrateDB Admin UI, crash, psql, DataGrip, and DBeaver? Discover how to configure these tools and explore CrateDB's compatibility with analytics, ETL, BI, and monitoring solutions. @@ -141,7 +141,7 @@ for improvements], and [source code contributions], are always welcome. {fab}`gi :::{toctree} -:maxdepth: 1 +:titlesonly: :hidden: Quick Start @@ -149,7 +149,7 @@ Services Import Console Automation -Integrations +Integrations Export Backups Manage Cluster From 8ff557f748e46a21456c8510665a0e06cd82dfce Mon Sep 17 00:00:00 2001 From: Brian Munkholm Date: Mon, 16 Jun 2025 22:50:51 +0200 Subject: [PATCH 2/3] whitespace fix --- docs/cluster/integrations/mongo-cdc.md | 142 ++++++++++++------------- 1 file changed, 71 insertions(+), 71 deletions(-) diff --git a/docs/cluster/integrations/mongo-cdc.md b/docs/cluster/integrations/mongo-cdc.md index fd0beff..692946c 100644 --- a/docs/cluster/integrations/mongo-cdc.md +++ b/docs/cluster/integrations/mongo-cdc.md @@ -1,7 +1,7 @@ (integrations-mongo-cdc)= # MongoDB CDC (Preview) -CrateDB Cloud enables continuous data ingestion from MongoDB using Change Data +CrateDB Cloud enables continuous data ingestion from MongoDB using Change Data Capture (CDC), providing seamless, real-time synchronization of your data. :::{caution} @@ -11,31 +11,31 @@ For more information, please [contact us](https://cratedb.com/contact). ## Key Concepts -The MongoDB CDC integration in CrateDB Cloud allows you to keep your data -synchronized between your MongoDB Atlas cluster and your CrateDB Cloud cluster -in real-time. +The MongoDB CDC integration in CrateDB Cloud allows you to keep your data +synchronized between your MongoDB Atlas cluster and your CrateDB Cloud cluster +in real-time. ### How It Works The integration functions in two main stages: -1. **Initial Sync:** - The integration performs a complete scan of your MongoDB collections, +1. **Initial Sync:** + The integration performs a complete scan of your MongoDB collections, importing all existing data into your CrateDB Cloud cluster. -2. **Continuous Sync:** - The integration uses MongoDB Change Streams to monitor changes in your - MongoDB collections and syncs these updates to your CrateDB Cloud cluster +2. **Continuous Sync:** + The integration uses MongoDB Change Streams to monitor changes in your + MongoDB collections and syncs these updates to your CrateDB Cloud cluster in real-time, ensuring that your data remains current. ### Data Consistency and Mode -For continuous sync, CrateDB Cloud uses MongoDB's **full document mode** to -ensure data consistency. This mode guarantees that MongoDB returns the latest -majority-committed version of the updated document. +For continuous sync, CrateDB Cloud uses MongoDB's **full document mode** to +ensure data consistency. This mode guarantees that MongoDB returns the latest +majority-committed version of the updated document. -While receiving partial deltas is more efficient, full document mode provides -robust functionality. Support for partial deltas may be added in the future to +While receiving partial deltas is more efficient, full document mode provides +robust functionality. Support for partial deltas may be added in the future to enhance performance and flexibility. --- @@ -55,81 +55,81 @@ Supported authentication methods: The following steps should be performed in the MongoDB Atlas UI. #### Step 1: Create a Custom Role -1. **Navigate to Database Access** - Go to **Database Access** in the MongoDB Atlas UI for the cluster you want to +1. **Navigate to Database Access** + Go to **Database Access** in the MongoDB Atlas UI for the cluster you want to connect to CrateDB Cloud. -2. **Add a Custom Role** +2. **Add a Custom Role** Under **Custom Roles**, click **Add New Custom Role**. -3. **Set Up Read-Only Access** +3. **Set Up Read-Only Access** Assign the following actions or roles to the custom role: - `find` - `changeStream` - `collStats` - Specify the databases and collections for these actions. You can update + Specify the databases and collections for these actions. You can update access permissions in the MongoDB Atlas UI later if needed. #### Step 2: Create a User -Depending on whether you plan to use SCRAM or X.509 authentication, create a +Depending on whether you plan to use SCRAM or X.509 authentication, create a database user with one of the following methods: :::{tab} SCRAM Auhentication -1. **Navigate to Database Access** - In the MongoDB Atlas UI, go to **Database Access** and click **Add New +1. **Navigate to Database Access** + In the MongoDB Atlas UI, go to **Database Access** and click **Add New Database User**. -2. **Set Authentication Method** - Choose **Password** as the authentication method and enter a username and +2. **Set Authentication Method** + Choose **Password** as the authentication method and enter a username and password for the database user. -3. **Assign the Role** +3. **Assign the Role** Under **Database User Privileges**, select the custom role created in Step 1. -4. **Copy User Credentials** - Click **Add User**, and make sure to record the username and password. These +4. **Copy User Credentials** + Click **Add User**, and make sure to record the username and password. These credentials will be used later in the CrateDB Cloud Console. ::: :::{tab} x.509 Authentication -1. **Navigate to Database Access** - In the MongoDB Atlas UI, go to **Database Access** and click **Add New +1. **Navigate to Database Access** + In the MongoDB Atlas UI, go to **Database Access** and click **Add New Database User**. -2. **Set Authentication Method** +2. **Set Authentication Method** Choose **Certificate** as the authentication method. -3. **Assign the Role** +3. **Assign the Role** Under **Database User Privileges**, select the custom role created in Step 1. -4. **Save the Certificate** - Click **Add User**, and store the certificate securely. This will be required +4. **Save the Certificate** + Click **Add User**, and store the certificate securely. This will be required later in the CrateDB Cloud Console. ::: #### Step 3: Configure IP Access -To allow CrateDB Cloud to access your MongoDB Atlas cluster, you must add the +To allow CrateDB Cloud to access your MongoDB Atlas cluster, you must add the CrateDB Cloud IP addresses to the IP Access List in MongoDB Atlas. -1. **Navigate to Network Access** +1. **Navigate to Network Access** In the MongoDB Atlas UI, go to **Network Access** from the left navigation. -2. **Add IP Address** - Click **Add IP Address** and choose an IP address or range to allow access. - For testing purposes, you can select **Allow Access from Anywhere**, but for +2. **Add IP Address** + Click **Add IP Address** and choose an IP address or range to allow access. + For testing purposes, you can select **Allow Access from Anywhere**, but for production, it is recommended to specify only the required IPs. :::{note} -The specific IP addresses depend on the region of your CrateDB Cloud cluster. -These IP addresses can also be found in the **Connection Details** section of the -CrateDB Cloud Console, just before you click **Test Connection** during the +The specific IP addresses depend on the region of your CrateDB Cloud cluster. +These IP addresses can also be found in the **Connection Details** section of the +CrateDB Cloud Console, just before you click **Test Connection** during the setup process. **Outbound IP Addresses**: @@ -156,17 +156,17 @@ out to our support team. You'll need to provide the connection string for your MongoDB Atlas cluster so that CrateDB Cloud can connect to it. -1. **Navigate to Your Cluster** +1. **Navigate to Your Cluster** In the MongoDB Atlas UI, navigate to the cluster you want to connect to CrateDB Cloud. -2. **Click "Connect"** +2. **Click "Connect"** From the cluster view, click on **Connect**. -3. **Select "Connect Your Application"** +3. **Select "Connect Your Application"** Choose **Connect your application** as the connection method. -4. **Copy the Connection String** - Copy the connection string provided in the MongoDB Atlas UI. It will look like this: +4. **Copy the Connection String** + Copy the connection string provided in the MongoDB Atlas UI. It will look like this: ``` mongodb+srv://:@/?retryWrites=true&w=majority @@ -193,55 +193,55 @@ in CrateDB Cloud. Follow these steps in the CrateDB Cloud Console to set up the MongoDB CDC integration: -#### Step 1: Create an Integration -1. Navigate to the **Import** section in the CrateDB Cloud Console. +#### Step 1: Create an Integration +1. Navigate to the **Import** section in the CrateDB Cloud Console. 2. Click **Create Integration** and select **MongoDB** as the source type. -#### Step 2: Configure Connection +#### Step 2: Configure Connection 1. Choose **Create New Connection** or select an existing one. 2. Fill in the following details: :::{tab} SCRAM Auhentication - - **Connection Name**: Provide a unique name for the connection. - - **Connection String**: Paste the connection string from MongoDB Atlas. - - **Username**: Enter the database username (required for SCRAM). + - **Connection Name**: Provide a unique name for the connection. + - **Connection String**: Paste the connection string from MongoDB Atlas. + - **Username**: Enter the database username (required for SCRAM). - **Password**: Enter the database password (required for SCRAM). - **Default Database**: Specify the default database to use for this connection. ::: :::{tab} X.509 Auhentication - - **Connection Name**: Provide a unique name for the connection. - - **Connection String**: Paste the connection string from MongoDB Atlas. + - **Connection Name**: Provide a unique name for the connection. + - **Connection String**: Paste the connection string from MongoDB Atlas. - **Certificate**: Upload the X.509 certificate file. - **Default Database**: Specify the default database to use for this connection. ::: -#### Step 3: Test the Connection +#### Step 3: Test the Connection Click **Test Connection** to verify CrateDB Cloud can connect to your MongoDB Atlas cluster. Resolve any issues if the test fails. -#### Step 4: Select Collection +#### Step 4: Select Collection Enter the database and collection name from your MongoDB Atlas cluster, that you want to sync with CrateDB Cloud. -#### Step 5: Select Target Table -1. Specify the target table in your CrateDB Cloud cluster where the data will be synced. -2. MongoDB records will be inserted into an object column called `document`. -3. Select the object type for the column: - - **`dynamic`**: Allows indexing and columnar storage for faster querying. - - **`ignored`**: Prevents type conflicts in CrateDB if your source data lacks a strict schema. +#### Step 5: Select Target Table +1. Specify the target table in your CrateDB Cloud cluster where the data will be synced. +2. MongoDB records will be inserted into an object column called `document`. +3. Select the object type for the column: + - **`dynamic`**: Allows indexing and columnar storage for faster querying. + - **`ignored`**: Prevents type conflicts in CrateDB if your source data lacks a strict schema. :::{note} - If your source data doesn't follow a strict schema, select `ignored` to avoid type conflicts. - However, selecting `dynamic` provides faster query performance by utilizing indexes and columnar storage. + If your source data doesn't follow a strict schema, select `ignored` to avoid type conflicts. + However, selecting `dynamic` provides faster query performance by utilizing indexes and columnar storage. ::: -#### Step 6: Configure Integration Settings -1. Enter a name for the integration. -2. Select the integration mode: - - **Full Load Only**: Imports the data once but doesn’t sync changes. - - **Full Load and CDC**: Imports the data and syncs changes in real-time. +#### Step 6: Configure Integration Settings +1. Enter a name for the integration. +2. Select the integration mode: + - **Full Load Only**: Imports the data once but doesn’t sync changes. + - **Full Load and CDC**: Imports the data and syncs changes in real-time. - **CDC Only**: Syncs only new changes in real-time without importing existing data. -#### Step 7: Create the Integration +#### Step 7: Create the Integration Click **Create Integration** to finalize the setup. CrateDB Cloud will now sync your MongoDB data based on the selected settings. @@ -275,5 +275,5 @@ CDC integration: value. - **Binary data types** other than UUIDs, which are converted to `TEXT` and **vectors**, which are converted to `ARRAY`s of numbers. -- The `Decimal128` data type is not supported and is converted to a string, as +- The `Decimal128` data type is not supported and is converted to a string, as CrateDB does not support a decimal data type. From 91aa94e1f1ab0c86a3cb5e348552f8602dfe8d21 Mon Sep 17 00:00:00 2001 From: Brian Munkholm Date: Wed, 18 Jun 2025 08:41:36 +0200 Subject: [PATCH 3/3] Add TOC to reference --- docs/reference/index.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/reference/index.md b/docs/reference/index.md index f037a24..9e91ffe 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -8,8 +8,10 @@ clusters in the cloud – without needing to worry about database management. CrateDB is a distributed, open-source database that combines the performance of NoSQL with the power and simplicity of standard SQL. +:::{rubric} Table of contents +::: + :::{toctree} -:hidden: :maxdepth: 1 User Roles