From 965ac7b23a70349574014b3c1584d9f83e4b0991 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 03:04:16 +0000 Subject: [PATCH 1/6] Initial plan From 7e3c4b84bd46f7ecd4ae499dc08b1568af4da3df Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 03:09:37 +0000 Subject: [PATCH 2/6] Update SQL Server sink docs to use JDBC connector Co-authored-by: WanYixian <150207222+WanYixian@users.noreply.github.com> --- integrations/destinations/sql-server.mdx | 55 ++++++++++++++++++------ 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/integrations/destinations/sql-server.mdx b/integrations/destinations/sql-server.mdx index 57633137..520df816 100644 --- a/integrations/destinations/sql-server.mdx +++ b/integrations/destinations/sql-server.mdx @@ -1,10 +1,10 @@ --- -title: "Sink data from RisingWave to SQL Server" +title: "Sink data from RisingWave to SQL Server with the JDBC connector" sidebarTitle: Microsoft SQL Server -description: This guide describes how to sink data from RisingWave to Microsoft SQL Server. +description: This guide describes how to sink data from RisingWave to Microsoft SQL Server using the JDBC connector. --- -You can test out this process on your own device by using the `sqlserver_sink.slt` demo in the [integration\_test directory](https://github.com/risingwavelabs/risingwave/tree/main/integration%5Ftests) of the RisingWave repository. +You can test out this process on your own device by using the `sqlserver_sink.slt` demo in the [e2e\_test directory](https://github.com/risingwavelabs/risingwave/tree/main/e2e%5Ftest/sink) of the RisingWave repository. **PREMIUM FEATURE** @@ -19,13 +19,17 @@ Before sinking data from RisingWave to SQL Server, please ensure the following: * The SQL Server table you want to sink to is accessible from RisingWave. * You have an upstream materialized view or table in RisingWave that you can sink data from. +### Notes about running RisingWave from binaries + +If you are running RisingWave locally from binaries and intend to use the JDBC sink connector, make sure you have [JDK 11](https://openjdk.org/projects/jdk/11/) or later versions installed in your environment. + ## Syntax ```sql CREATE SINK [ IF NOT EXISTS ] sink_name [FROM sink_from | AS select_query] WITH ( - connector='sqlserver', + connector='jdbc', connector_parameter = 'value', ... ); ``` @@ -34,15 +38,42 @@ WITH ( | Parameter Names | Description | | :------------------ | :--------------------------------------------------------------------------------------------------------------------------------- | -| type | **Required**. Allowed values: append-only and upsert. | -| force\_append\_only | **Optional**. If true, forces the sink to be append-only, even if it cannot be. | +| connector | **Required**. Sink connector type must be `jdbc` for SQL Server sink. | +| jdbc.url | **Required**. The JDBC URL of the SQL Server database. Format: `jdbc:sqlserver://:;databaseName=;user=;password=;trustServerCertificate=true` | +| table.name | **Required**. The SQL Server table you want to sink to. | +| schema.name | **Optional**. The SQL Server schema name. If not specified, the default schema will be used. | +| type | **Required**. Allowed values: `append-only` and `upsert`. | +| force\_append\_only | **Optional**. If true, forces the sink to be append-only, even if it cannot be. | | primary\_key | **Conditional**. The primary keys of the sink. Use ',' to delimit the primary key columns. Primary keys are required for upsert sinks. | -| sqlserver.host | **Required**. The SQL Server host. | -| sqlserver.port | **Required**. The SQL Server port. | -| sqlserver.user | **Required**. The user for SQL Server access. | -| sqlserver.password | **Required**. The password for SQL Server access. | -| sqlserver.database | **Required**. The SQL Server database you want to sink to. | -| sqlserver.table | **Required**. The SQL Server table you want to sink to. | +| jdbc.query.timeout | **Optional**. Specifies the timeout for the operations to downstream. If not set, the default is 60s. | +| jdbc.auto.commit | **Optional**. Controls whether to automatically commit transactions for JDBC sink. If not set, the default is false. | + +## Example + +The following example demonstrates how to create a SQL Server sink using the JDBC connector. The example uses the upsert type to ensure data consistency. + +```sql Example +-- Create a table in RisingWave +CREATE TABLE t_many_data_type_rw ( + k1 integer, + k2 varchar, + -- other columns +); + +-- Create a sink to SQL Server +CREATE SINK s_many_data_type FROM t_many_data_type_rw WITH ( + connector = 'jdbc', + type = 'upsert', + jdbc.url='jdbc:sqlserver://sqlserver-server:1433;databaseName=SinkTest;user=SA;password=YourPassword;trustServerCertificate=true', + primary_key = 'k1,k2', + schema.name='test_schema', + table.name = 't_many_data_type' +); +``` + + +Replace the connection parameters in `jdbc.url` with your actual SQL Server host, port, database name, username, and password. + ## Data type mapping From 0b0ef5c5f680a00f73b4b781979c8974c6919f65 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 03:12:02 +0000 Subject: [PATCH 3/6] Address code review feedback: enhance security notes Co-authored-by: WanYixian <150207222+WanYixian@users.noreply.github.com> --- integrations/destinations/sql-server.mdx | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/integrations/destinations/sql-server.mdx b/integrations/destinations/sql-server.mdx index 520df816..c7f52693 100644 --- a/integrations/destinations/sql-server.mdx +++ b/integrations/destinations/sql-server.mdx @@ -6,6 +6,10 @@ description: This guide describes how to sink data from RisingWave to Microsoft You can test out this process on your own device by using the `sqlserver_sink.slt` demo in the [e2e\_test directory](https://github.com/risingwavelabs/risingwave/tree/main/e2e%5Ftest/sink) of the RisingWave repository. + +The demo has been updated to use the JDBC connector instead of the legacy SQL Server connector. + + **PREMIUM FEATURE** @@ -72,7 +76,7 @@ CREATE SINK s_many_data_type FROM t_many_data_type_rw WITH ( ``` -Replace the connection parameters in `jdbc.url` with your actual SQL Server host, port, database name, username, and password. +**Security Best Practice**: Replace the connection parameters in `jdbc.url` with your actual SQL Server host, port, database name, username, and password. For production environments, avoid hardcoding credentials directly in the JDBC URL. Instead, use environment variables, secret management systems, or RisingWave's [secret management](/operate/manage-secrets) feature to securely store and reference credentials. ## Data type mapping From 46f2c606d533912b6ffb8e498f16deffdd497941 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 03:13:07 +0000 Subject: [PATCH 4/6] Update SQL Server connector reference in delivery overview Co-authored-by: WanYixian <150207222+WanYixian@users.noreply.github.com> --- delivery/overview.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/delivery/overview.mdx b/delivery/overview.mdx index dbe81988..98eb5b8a 100644 --- a/delivery/overview.mdx +++ b/delivery/overview.mdx @@ -36,7 +36,7 @@ Currently, RisingWave supports the following sink connectors: | [Snowflake](/integrations/destinations/snowflake) | `connector = 'snowflake'` | | [Snowflake v2](/integrations/destinations/snowflake-v2) | `connector = 'snowflake_v2'` | | [StarRocks](/integrations/destinations/starrocks) | `connector = 'starrocks'` | -| [Microsoft SQL Server](/integrations/destinations/sql-server) | `connector = 'sqlserver'` | +| [Microsoft SQL Server](/integrations/destinations/sql-server) | `connector = 'jdbc'` | ## Sink decoupling From d9807675e18762ca26e8d7fd2af3b956a45cfa23 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 5 Dec 2025 05:38:46 +0000 Subject: [PATCH 5/6] Document both JDBC and native SQL Server connectors Co-authored-by: WanYixian <150207222+WanYixian@users.noreply.github.com> --- delivery/overview.mdx | 2 +- integrations/destinations/sql-server.mdx | 78 +++++++++++++++++++----- 2 files changed, 65 insertions(+), 15 deletions(-) diff --git a/delivery/overview.mdx b/delivery/overview.mdx index 98eb5b8a..2a5ba3f7 100644 --- a/delivery/overview.mdx +++ b/delivery/overview.mdx @@ -36,7 +36,7 @@ Currently, RisingWave supports the following sink connectors: | [Snowflake](/integrations/destinations/snowflake) | `connector = 'snowflake'` | | [Snowflake v2](/integrations/destinations/snowflake-v2) | `connector = 'snowflake_v2'` | | [StarRocks](/integrations/destinations/starrocks) | `connector = 'starrocks'` | -| [Microsoft SQL Server](/integrations/destinations/sql-server) | `connector = 'jdbc'` | +| [Microsoft SQL Server](/integrations/destinations/sql-server) | `connector = 'jdbc'` or `'sqlserver'` | ## Sink decoupling diff --git a/integrations/destinations/sql-server.mdx b/integrations/destinations/sql-server.mdx index c7f52693..adce455a 100644 --- a/integrations/destinations/sql-server.mdx +++ b/integrations/destinations/sql-server.mdx @@ -1,14 +1,14 @@ --- -title: "Sink data from RisingWave to SQL Server with the JDBC connector" +title: "Sink data from RisingWave to SQL Server" sidebarTitle: Microsoft SQL Server -description: This guide describes how to sink data from RisingWave to Microsoft SQL Server using the JDBC connector. +description: This guide describes how to sink data from RisingWave to Microsoft SQL Server. --- -You can test out this process on your own device by using the `sqlserver_sink.slt` demo in the [e2e\_test directory](https://github.com/risingwavelabs/risingwave/tree/main/e2e%5Ftest/sink) of the RisingWave repository. +This guide covers two connector options for sinking data to SQL Server: +- **JDBC connector** (`connector='jdbc'`) - Recommended for better performance +- **SQL Server native connector** (`connector='sqlserver'`) - Legacy option - -The demo has been updated to use the JDBC connector instead of the legacy SQL Server connector. - +You can test out this process on your own device by using the `sqlserver_sink.slt` demo in the [e2e\_test directory](https://github.com/risingwavelabs/risingwave/tree/main/e2e%5Ftest/sink) of the RisingWave repository. **PREMIUM FEATURE** @@ -27,22 +27,30 @@ Before sinking data from RisingWave to SQL Server, please ensure the following: If you are running RisingWave locally from binaries and intend to use the JDBC sink connector, make sure you have [JDK 11](https://openjdk.org/projects/jdk/11/) or later versions installed in your environment. -## Syntax +## Create a sink + +RisingWave supports two connector types for SQL Server sinks. The JDBC connector is recommended for better performance. + +### Syntax ```sql CREATE SINK [ IF NOT EXISTS ] sink_name [FROM sink_from | AS select_query] WITH ( - connector='jdbc', + connector='jdbc' | 'sqlserver', connector_parameter = 'value', ... ); ``` -## Parameters +### Parameters (JDBC connector) + + +The JDBC connector (`connector='jdbc'`) offers better performance compared to the native SQL Server connector. + | Parameter Names | Description | | :------------------ | :--------------------------------------------------------------------------------------------------------------------------------- | -| connector | **Required**. Sink connector type must be `jdbc` for SQL Server sink. | +| connector | **Required**. Sink connector type. Use `jdbc` for the JDBC connector. | | jdbc.url | **Required**. The JDBC URL of the SQL Server database. Format: `jdbc:sqlserver://:;databaseName=;user=;password=;trustServerCertificate=true` | | table.name | **Required**. The SQL Server table you want to sink to. | | schema.name | **Optional**. The SQL Server schema name. If not specified, the default schema will be used. | @@ -52,11 +60,28 @@ WITH ( | jdbc.query.timeout | **Optional**. Specifies the timeout for the operations to downstream. If not set, the default is 60s. | | jdbc.auto.commit | **Optional**. Controls whether to automatically commit transactions for JDBC sink. If not set, the default is false. | -## Example +### Parameters (SQL Server native connector) + +| Parameter Names | Description | +| :------------------ | :--------------------------------------------------------------------------------------------------------------------------------- | +| connector | **Required**. Sink connector type. Use `sqlserver` for the native SQL Server connector. | +| type | **Required**. Allowed values: `append-only` and `upsert`. | +| force\_append\_only | **Optional**. If true, forces the sink to be append-only, even if it cannot be. | +| primary\_key | **Conditional**. The primary keys of the sink. Use ',' to delimit the primary key columns. Primary keys are required for upsert sinks. | +| sqlserver.host | **Required**. The SQL Server host. | +| sqlserver.port | **Required**. The SQL Server port. | +| sqlserver.user | **Required**. The user for SQL Server access. | +| sqlserver.password | **Required**. The password for SQL Server access. | +| sqlserver.database | **Required**. The SQL Server database you want to sink to. | +| sqlserver.table | **Required**. The SQL Server table you want to sink to. | + +## Examples + +### Example: JDBC connector (recommended) -The following example demonstrates how to create a SQL Server sink using the JDBC connector. The example uses the upsert type to ensure data consistency. +The following example demonstrates how to create a SQL Server sink using the JDBC connector. The JDBC connector offers better performance. -```sql Example +```sql Example: JDBC connector -- Create a table in RisingWave CREATE TABLE t_many_data_type_rw ( k1 integer, @@ -64,7 +89,7 @@ CREATE TABLE t_many_data_type_rw ( -- other columns ); --- Create a sink to SQL Server +-- Create a sink to SQL Server using JDBC CREATE SINK s_many_data_type FROM t_many_data_type_rw WITH ( connector = 'jdbc', type = 'upsert', @@ -79,6 +104,31 @@ CREATE SINK s_many_data_type FROM t_many_data_type_rw WITH ( **Security Best Practice**: Replace the connection parameters in `jdbc.url` with your actual SQL Server host, port, database name, username, and password. For production environments, avoid hardcoding credentials directly in the JDBC URL. Instead, use environment variables, secret management systems, or RisingWave's [secret management](/operate/manage-secrets) feature to securely store and reference credentials. +### Example: SQL Server native connector + +The following example demonstrates how to create a SQL Server sink using the native SQL Server connector. + +```sql Example: Native connector +-- Create a table in RisingWave +CREATE TABLE t_data ( + id integer, + name varchar +); + +-- Create a sink to SQL Server using native connector +CREATE SINK s_data FROM t_data WITH ( + connector = 'sqlserver', + type = 'upsert', + sqlserver.host = 'sqlserver-server', + sqlserver.port = 1433, + sqlserver.user = 'SA', + sqlserver.password = 'YourPassword', + sqlserver.database = 'SinkTest', + sqlserver.table = 't_data', + primary_key = 'id' +); +``` + ## Data type mapping The following table shows the corresponding data types between RisingWave and SQL Server that should be specified when creating a sink. For details on native RisingWave data types, see [Overview of data types](/sql/data-types/overview). From 547bcb3a4fe7e92649886aec0626ec529ed074b0 Mon Sep 17 00:00:00 2001 From: WanYixian Date: Fri, 5 Dec 2025 15:49:39 +0800 Subject: [PATCH 6/6] language review for #855 --- delivery/overview.mdx | 2 +- integrations/destinations/sql-server.mdx | 75 +++++++----------------- 2 files changed, 22 insertions(+), 55 deletions(-) diff --git a/delivery/overview.mdx b/delivery/overview.mdx index 2a5ba3f7..d752bfd4 100644 --- a/delivery/overview.mdx +++ b/delivery/overview.mdx @@ -26,7 +26,7 @@ Currently, RisingWave supports the following sink connectors: | [Elasticsearch](/integrations/destinations/elasticsearch) | `connector = 'elasticsearch'` | | [Google BigQuery](/integrations/destinations/bigquery) | `connector = 'bigquery'` | | [Google Pub/Sub](/integrations/destinations/google-pub-sub) | `connector = 'google_pubsub'` | -| JDBC: [MySQL](/integrations/destinations/mysql) \| [PostgreSQL](/integrations/destinations/postgresql) \| [TiDB](/integrations/destinations/tidb) | `connector = 'jdbc'` | +| JDBC: [MySQL](/integrations/destinations/mysql), [PostgreSQL](/integrations/destinations/postgresql), [SQL Server](/integrations/destinations/sql-server), [TiDB](/integrations/destinations/tidb)| `connector = 'jdbc'` | | [Kafka](/integrations/destinations/apache-kafka) | `connector = 'kafka'` | | [MQTT](/integrations/destinations/mqtt) | `connector = 'mqtt'` | | [NATS](/integrations/destinations/nats-and-nats-jetstream) | `connector = 'nats'` | diff --git a/integrations/destinations/sql-server.mdx b/integrations/destinations/sql-server.mdx index adce455a..bcf01591 100644 --- a/integrations/destinations/sql-server.mdx +++ b/integrations/destinations/sql-server.mdx @@ -6,7 +6,7 @@ description: This guide describes how to sink data from RisingWave to Microsoft This guide covers two connector options for sinking data to SQL Server: - **JDBC connector** (`connector='jdbc'`) - Recommended for better performance -- **SQL Server native connector** (`connector='sqlserver'`) - Legacy option +- **SQL Server native connector** (`connector='sqlserver'`) You can test out this process on your own device by using the `sqlserver_sink.slt` demo in the [e2e\_test directory](https://github.com/risingwavelabs/risingwave/tree/main/e2e%5Ftest/sink) of the RisingWave repository. @@ -20,12 +20,9 @@ This is a premium feature. For a comprehensive overview of all premium features Before sinking data from RisingWave to SQL Server, please ensure the following: -* The SQL Server table you want to sink to is accessible from RisingWave. -* You have an upstream materialized view or table in RisingWave that you can sink data from. - -### Notes about running RisingWave from binaries - -If you are running RisingWave locally from binaries and intend to use the JDBC sink connector, make sure you have [JDK 11](https://openjdk.org/projects/jdk/11/) or later versions installed in your environment. +- The SQL Server table you want to sink to is accessible from RisingWave. +- You have an upstream materialized view or table in RisingWave that you can sink data from. +- If you are running RisingWave locally from binaries and intend to use the JDBC sink connector, make sure you have [JDK 11](https://openjdk.org/projects/jdk/11/) or later versions installed in your environment. ## Create a sink @@ -44,14 +41,10 @@ WITH ( ### Parameters (JDBC connector) - -The JDBC connector (`connector='jdbc'`) offers better performance compared to the native SQL Server connector. - - | Parameter Names | Description | | :------------------ | :--------------------------------------------------------------------------------------------------------------------------------- | | connector | **Required**. Sink connector type. Use `jdbc` for the JDBC connector. | -| jdbc.url | **Required**. The JDBC URL of the SQL Server database. Format: `jdbc:sqlserver://:;databaseName=;user=;password=;trustServerCertificate=true` | +| jdbc.url | **Required**. The JDBC URL of the destination database necessary for the driver to recognize and connect to the database. | | table.name | **Required**. The SQL Server table you want to sink to. | | schema.name | **Optional**. The SQL Server schema name. If not specified, the default schema will be used. | | type | **Required**. Allowed values: `append-only` and `upsert`. | @@ -68,64 +61,38 @@ The JDBC connector (`connector='jdbc'`) offers better performance compared to th | type | **Required**. Allowed values: `append-only` and `upsert`. | | force\_append\_only | **Optional**. If true, forces the sink to be append-only, even if it cannot be. | | primary\_key | **Conditional**. The primary keys of the sink. Use ',' to delimit the primary key columns. Primary keys are required for upsert sinks. | -| sqlserver.host | **Required**. The SQL Server host. | -| sqlserver.port | **Required**. The SQL Server port. | -| sqlserver.user | **Required**. The user for SQL Server access. | -| sqlserver.password | **Required**. The password for SQL Server access. | -| sqlserver.database | **Required**. The SQL Server database you want to sink to. | -| sqlserver.table | **Required**. The SQL Server table you want to sink to. | +| sqlserver.host | **Required**. The SQL Server host. | +| sqlserver.port | **Required**. The SQL Server port. | +| sqlserver.user | **Required**. The user for SQL Server access. | +| sqlserver.password | **Required**. The password for SQL Server access. | +| sqlserver.database | **Required**. The SQL Server database you want to sink to. | +| sqlserver.table | **Required**. The SQL Server table you want to sink to. | ## Examples -### Example: JDBC connector (recommended) - -The following example demonstrates how to create a SQL Server sink using the JDBC connector. The JDBC connector offers better performance. - -```sql Example: JDBC connector --- Create a table in RisingWave -CREATE TABLE t_many_data_type_rw ( - k1 integer, - k2 varchar, - -- other columns -); - --- Create a sink to SQL Server using JDBC -CREATE SINK s_many_data_type FROM t_many_data_type_rw WITH ( +```sql jdbc as connector +CREATE SINK s_many_data_type from t_many_data_type_rw WITH ( connector = 'jdbc', type = 'upsert', - jdbc.url='jdbc:sqlserver://sqlserver-server:1433;databaseName=SinkTest;user=SA;password=YourPassword;trustServerCertificate=true', + jdbc.url='jdbc:sqlserver://sqlserver-server:1433;databaseName=SinkTest;user=SA;password=SomeTestOnly@SA;trustServerCertificate=true', primary_key = 'k1,k2', schema.name='test_schema', - table.name = 't_many_data_type' + table.name = 't_many_data_type', ); ``` - -**Security Best Practice**: Replace the connection parameters in `jdbc.url` with your actual SQL Server host, port, database name, username, and password. For production environments, avoid hardcoding credentials directly in the JDBC URL. Instead, use environment variables, secret management systems, or RisingWave's [secret management](/operate/manage-secrets) feature to securely store and reference credentials. - - -### Example: SQL Server native connector - -The following example demonstrates how to create a SQL Server sink using the native SQL Server connector. - -```sql Example: Native connector --- Create a table in RisingWave -CREATE TABLE t_data ( - id integer, - name varchar -); - --- Create a sink to SQL Server using native connector -CREATE SINK s_data FROM t_data WITH ( +```sql sqlserver as connector +CREATE SINK s_many_data_type from t_many_data_type_rw WITH ( connector = 'sqlserver', type = 'upsert', sqlserver.host = 'sqlserver-server', sqlserver.port = 1433, sqlserver.user = 'SA', - sqlserver.password = 'YourPassword', + sqlserver.password = 'SomeTestOnly@SA', sqlserver.database = 'SinkTest', - sqlserver.table = 't_data', - primary_key = 'id' + sqlserver.schema = 'test_schema', + sqlserver.table = 't_many_data_type', + primary_key = 'k1,k2', ); ```