Skip to content

Latest commit

 

History

History
827 lines (671 loc) · 98.7 KB

mssql.md

File metadata and controls

827 lines (671 loc) · 98.7 KB

Microsoft SQL Server (MSSQL)

Airbyte's certified MSSQL connector offers the following features:

⚠️ Please note the minimum required platform version is v0.58.0 to run source-mssql 4.0.18 and above.

Features

Feature Supported Notes
Full Refresh Sync Yes
Incremental Sync - Append Yes
Replicate Incremental Deletes Yes
CDC (Change Data Capture) Yes
SSL Support Yes
SSH Tunnel Connection Yes
Microsoft Entra ID Auth Yes Service principal
Namespaces Yes Enabled by default

The MSSQL source does not alter the schema present in your database. Depending on the destination connected to this source, however, the schema may be altered. See the destination's documentation for more details.

Getting Started

Requirements

  1. MSSQL Server Azure SQL Database, Azure Synapse Analytics, Azure SQL Managed Instance, SQL Server 2022, SQL Server 2019, SQL Server 2017, SQL Server 2016, SQL Server 2014, SQL Server 2012, PDW 2008R2 AU34.
  2. Create a dedicated read-only Airbyte user with access to all tables needed for replication
  3. If you want to use CDC, please see the relevant section below for further setup requirements

1. Make sure your database is accessible from the machine running Airbyte

This is dependent on your networking setup. The easiest way to verify if Airbyte is able to connect to your MSSQL instance is via the check connection tool in the UI. If you use Airbyte Cloud, your firewall or security group must allow the Airbyte Cloud IP addresses and the connection must be encrypted; see Airbyte Cloud below.

2. Create a dedicated read-only user with access to the relevant tables (Recommended but optional)

This step is optional but highly recommended to allow for better permission control and auditing. Alternatively, you can use Airbyte with an existing user in your database.

  • Create a login and a database user for Airbyte, then add the user to the db_datareader role. Membership in db_datareader grants SELECT on all current and future tables in the database:

    USE {database name};
    CREATE LOGIN {user name} WITH PASSWORD = '{password}';
    CREATE USER {user name} FOR LOGIN {user name};
    ALTER ROLE db_datareader ADD MEMBER {user name};
    

    ALTER ROLE ... ADD MEMBER replaces the deprecated sp_addrolemember stored procedure, which Microsoft recommends avoiding in new work.

  • If you prefer to scope access to specific schemas rather than the whole database, skip the db_datareader role and instead grant SELECT on each schema you want to replicate from. Re-run this command for each schema:

    USE {database name};
    GRANT SELECT ON SCHEMA :: {schema name} TO {user name};
    

Use the username and password you created here when configuring the MSSQL source in Airbyte. If you plan to use CDC, this user also needs the additional CDC-related permissions described in Setting up CDC for MSSQL.

3. Your database user should now be ready for use with Airbyte!

Airbyte Cloud

On Airbyte Cloud, only secured connections to your MSSQL instance are supported in source configuration. Airbyte Cloud never connects to a database without encryption: the connection must either use TLS/SSL, or go through an SSH tunnel, which encrypts the hop between Airbyte and your bastion host. In practice:

Setup Supported on Airbyte Cloud
Encryption set to Encrypted (trust server certificate) or Encrypted (verify certificate) Yes, with or without an SSH tunnel.
Encryption set to Unencrypted with an SSH tunnel Yes. The SSH tunnel provides the encryption.
Encryption set to Unencrypted without an SSH tunnel No. The connection test fails with Connection from Airbyte Cloud requires SSL encryption or an SSH tunnel.
Database on localhost or on a private network (not reachable from the public internet) Only through an SSH tunnel to a bastion host that can reach the database, which also provides the encryption Airbyte Cloud requires. With No Tunnel, localhost, 127.0.0.1 or 0.0.0.0 fails the connection test with Airbyte Cloud cannot reach a database on your own machine. Use a hostname reachable from the internet, or configure an SSH tunnel.
A URL, credentials or host:port pasted into Host No. The connection test fails with Enter only the hostname in Host; put the port in the Port field. Enter only the hostname or IP address in Host and the port in Port.

On Airbyte Cloud, the Encryption option defaults to Encrypted (trust server certificate).

See Connection to MSSQL via an SSH Tunnel for how to set up the tunnel, and IP allow list if your network restricts inbound connections.

Authentication with Microsoft Entra ID

This connector supports Microsoft Entra ID (formerly Azure Active Directory) authentication using a service principal, as an alternative to SQL Server username and password authentication. This is the recommended authentication mode for Azure SQL Database and Azure SQL Managed Instance.

Prerequisites

  1. An Azure SQL Database, Azure SQL Managed Instance, or SQL Server instance that is configured for Microsoft Entra authentication. For setup instructions, see Configure and manage Microsoft Entra authentication with Azure SQL.
  2. A Microsoft Entra ID app registration (service principal) with a client secret.
  3. A database user created for the service principal, with SELECT access to the tables you want to replicate. For instructions, see Create Microsoft Entra users using service principals.
  4. If you plan to use CDC, the service principal user also needs the CDC-related permissions described in Setting up CDC for MSSQL.

Configuration

In the source configuration form, fill in the following fields under Microsoft Entra ID:

Field Description
Entra ID Client ID The application (client) ID of the service principal.
Entra ID Client Secret The client secret generated for the service principal.

When both fields are provided, the connector authenticates with ActiveDirectoryServicePrincipal mode through the Microsoft JDBC driver, and the Username and Password fields are ignored. If either Entra ID field is empty, the connector falls back to username and password authentication.

:::note Entra ID authentication requires an encrypted connection. Set Encryption to Encrypted (trust server certificate) or Encrypted (verify certificate). The connector fails the configuration check if encryption is disabled while Entra ID fields are set. :::

MSSQL Replication Modes

Incremental Syncs

MSSQL datetime2(7) type stores timestamps with up to 7 decimal places, but since most destinations don't support the extra precision, Airbyte truncates them to 6 (microseconds). Because of that, the saved cursor can land just behind the newest rows in your table. To make sure none of them get skipped, each incremental sync reads everything above the saved cursor, then saves the new max value as the cursor for the next sync.

:::note

Because each sync picks up everything newer than the last saved point, you may see some duplicate rows. If possible, we do recommend using Incremental - Append + Deduped for supported syncs. For more information, please visit our Sync Mode page.

:::

Change Data Capture (CDC)

We use SQL Server's change data capture feature with transaction logs to capture row-level INSERT, UPDATE and DELETE operations that occur on CDC-enabled tables.

Some extra setup requiring at least db_owner permissions on the database(s) you intend to sync from will be required (detailed below).

Please read the CDC docs for an overview of how Airbyte approaches CDC.

Should I use CDC for MSSQL?

  • If you need a record of deletions and can accept the limitations posted below, CDC is the way to go!
  • If your data set is small and/or you just want a snapshot of your table in the destination, consider using Full Refresh replication for your table instead of CDC.
  • If the limitations below prevent you from using CDC and your goal is to maintain a snapshot of your table in the destination, consider using non-CDC incremental and occasionally reset the data and re-sync.
  • If your table has a primary key but doesn't have a reasonable cursor field for incremental syncing (i.e. updated_at), CDC allows you to sync your table incrementally.

CDC Limitations

  • Make sure to read our CDC docs to see limitations that impact all databases using CDC replication.
  • hierarchyid and sql_variant types are not processed in CDC migration type (not supported by Debezium). For more details please check this ticket
  • CDC is only available for SQL Server 2016 Service Pack 1 (SP1) and later.
  • db_owner (or higher) permissions are required to perform the necessary setup for CDC.
  • On Linux, CDC is not supported on versions earlier than SQL Server 2017 CU18 (SQL Server 2019 is supported).
  • Change data capture cannot be enabled on tables with a clustered columnstore index. (It can be enabled on tables with a non-clustered columnstore index).
  • The SQL Server CDC feature processes changes that occur in user-created tables only. You cannot enable CDC on the SQL Server master database.
  • Using variables with partition switching on databases or tables with change data capture (CDC) is not supported for the ALTER TABLE ... SWITCH TO ... PARTITION ... statement.
  • CDC incremental syncing is only available for tables with at least one primary key. Tables without primary keys can still be replicated by CDC but only in Full Refresh mode. For more information on CDC limitations, refer to our CDC Limitations doc.
  • Our CDC implementation uses at least once delivery for all change records.
  • Read more on CDC limitations in the Microsoft docs.

Setting up CDC for MSSQL

1. Enable CDC on database and tables

MS SQL Server provides some built-in stored procedures to enable CDC.

  • To enable CDC, a SQL Server administrator with the necessary privileges (db_owner or sysadmin) must first run a query to enable CDC at the database level.

    USE {database name}
    GO
    EXEC sys.sp_cdc_enable_db
    GO
    
  • The administrator must then enable CDC for each table that you want to capture. Here's an example:

    USE {database name}
    GO
    
    EXEC sys.sp_cdc_enable_table
    @source_schema = N'{schema name}',
    @source_name   = N'{table name}',
    @role_name     = N'{role name}',  [1]
    @filegroup_name = N'{filegroup name}', [2]
    @supports_net_changes = 0 [3]
    GO
    
    • [1] Specifies a role which will gain SELECT permission on the captured columns of the source table. We suggest putting a value here so you can use this role in the next step but you can also set the value of @rolename to NULL to allow only _sysadmin and db_owner to have access. Be sure that the credentials used to connect to the source in Airbyte align with this role so that Airbyte can access the cdc tables.
    • [2] Specifies the filegroup where SQL Server places the change table. We recommend creating a separate filegroup for CDC but you can leave this parameter out to use the default filegroup.
    • [3] If 0, only the support functions to query for all changes are generated. If 1, the functions that are needed to query for net changes are also generated. If supports_net_changes is set to 1, index_name must be specified, or the source table must have a defined primary key.
  • (For more details on parameters, see the Microsoft doc page for this stored procedure).

  • If you have many tables to enable CDC on and would like to avoid having to run this query one-by-one for every table, this script might help!

For further detail, see the Microsoft docs on enabling and disabling CDC.

:::note Google Cloud SQL for SQL Server

On Google Cloud SQL for SQL Server, Google does not grant customers the sysadmin server role, so you cannot run sys.sp_cdc_enable_db to enable CDC at the database level. Instead of the database-level command shown above, use Cloud SQL's dedicated stored procedure, which enables CDC without sysadmin:

EXEC msdb.dbo.gcloudsql_cdc_enable_db 'YOUR_DATABASE_NAME'

To disable CDC at the database level later, use the corresponding EXEC msdb.dbo.gcloudsql_cdc_disable_db 'YOUR_DATABASE_NAME' procedure.

Only the database-level enablement differs on Cloud SQL. Enabling CDC on individual tables still uses the standard sys.sp_cdc_enable_table procedure, and the snapshot isolation and user/permission steps below are unchanged. For the full Google-provided procedure, see Configure CDC for a Cloud SQL for SQL Server source.

:::

2. Enable snapshot isolation
  • When a sync runs for the first time using CDC, Airbyte performs an initial consistent snapshot of your database. To avoid acquiring table locks, Airbyte uses snapshot isolation, allowing simultaneous writes by other database clients. This must be enabled on the database like so:

    ALTER DATABASE {database name}
      SET ALLOW_SNAPSHOT_ISOLATION ON;
    
3. Create a user and grant appropriate permissions
  • Rather than use sysadmin or db_owner credentials, we recommend creating a new user with the relevant CDC access for use with Airbyte. First let's create the login and user and add to the db_datareader role:

    USE {database name};
    CREATE LOGIN {user name} WITH PASSWORD = '{password}';
    CREATE USER {user name} FOR LOGIN {user name};
    ALTER ROLE db_datareader ADD MEMBER {user name};
    
    • Add the user to the role specified earlier when enabling cdc on the table(s):

      ALTER ROLE {role name} ADD MEMBER {user name};
      
    • ALTER ROLE ... ADD MEMBER replaces the deprecated sp_addrolemember stored procedure, which Microsoft recommends avoiding in new work.

    • This should be enough access, but if you run into problems, try also directly granting the user SELECT access on the cdc schema:

      USE {database name};
      GRANT SELECT ON SCHEMA :: [cdc] TO {user name};
      
    • If feasible, granting this user 'VIEW SERVER STATE' permissions will allow Airbyte to check whether or not the SQL Server Agent is running. This is preferred as it ensures syncs will fail if the CDC tables are not being updated by the Agent in the source database.

      USE master;
      GRANT VIEW SERVER STATE TO {user name};
      
4. Extend the retention period of CDC data
  • In SQL Server, by default, only three days of data are retained in the change tables. Unless you are running very frequent syncs, we suggest increasing this retention so that in case of a failure in sync or if the sync is paused, there is still some bandwidth to start from the last point in incremental sync. Airbyte recommends retaining at least 7 days of CDC data.

  • These settings can be changed using the stored procedure sys.sp_cdc_change_job as below:

    -- Airbyte recommends at least 10080 minutes (7 days) as the retention period
    EXEC sp_cdc_change_job @job_type='cleanup', @retention = 10080
    
  • After making this change, a restart of the cleanup job is required:

  EXEC sys.sp_cdc_stop_job @job_type = 'cleanup';

  EXEC sys.sp_cdc_start_job @job_type = 'cleanup';
  • If you are using Transaction Replication, the retention has to be changed using the following scripts:

EXEC sp_changedistributiondb
  @database = 'distribution',
  @property = 'max_distretention',
  @value = 10080 -- 10080 minutes (7 days)

EXEC sp_changedistributiondb
  @database = 'distribution',
  @property = 'history_retention',
  @value = 10080 -- 10080 minutes (7 days)

USE [msdb]
GO
EXEC msdb.dbo.sp_update_jobstep @job_name=N'Distribution clean up: distribution', @step_id=1 ,
		@command=N'EXEC dbo.sp_MSdistribution_cleanup @min_distretention = 0, @max_distretention = 10800'
GO

5. Ensure the SQL Server Agent is running
  • MSSQL uses the SQL Server Agent to run the jobs necessary for CDC. It is therefore vital that the Agent is operational in order for CDC to work effectively. You can check the status of the SQL Server Agent as follows:
  EXEC xp_servicecontrol 'QueryState', N'SQLServerAGENT';
  • If you see something other than 'Running.' please follow the Microsoft docs to start the service.

Connection to MSSQL via an SSH Tunnel

Airbyte has the ability to connect to a MSSQL instance via an SSH Tunnel. The reason you might want to do this because it is not possible (or against security policy) to connect to the database directly (e.g. it does not have a public IP address).

When using an SSH tunnel, you are configuring Airbyte to connect to an intermediate server (a.k.a. a bastion server) that does have direct access to the database. Airbyte connects to the bastion and then asks the bastion to connect directly to the server. The Host and Port of the source are then the address of the database as seen from the bastion (for example localhost and 1433 when SQL Server runs on the bastion itself).

:::note Airbyte Cloud Airbyte Cloud never connects to a database without encryption. An SSH tunnel satisfies this requirement because the hop between Airbyte and the bastion is encrypted, so on Airbyte Cloud a tunnel is the only way to use the Unencrypted encryption method, and the only way to reach a database on localhost or on a private network. Without a tunnel, Airbyte Cloud requires an encrypted SSL method; see Airbyte Cloud. :::

Using this feature requires additional configuration, when creating the source. We will talk through what each piece of configuration means.

  1. Configure all fields for the source as you normally would, except SSH Tunnel Method.

  2. SSH Tunnel Method defaults to No Tunnel (meaning a direct connection). If you want to use an

    SSH Tunnel choose SSH Key Authentication or Password Authentication.

    1. Choose Key Authentication if you will be using an RSA private key as your secret for

      establishing the SSH Tunnel (see below for more information on generating this key).

    2. Choose Password Authentication if you will be using a password as your secret for establishing

      the SSH Tunnel.

  3. SSH Tunnel Jump Server Host refers to the intermediate (bastion) server that Airbyte will connect to. This should

    be a hostname or an IP Address.

  4. SSH Connection Port is the port on the bastion server with which to make the SSH connection. The default port for

    SSH connections is 22, so unless you have explicitly changed something, go with the default.

  5. SSH Login Username is the username that Airbyte should use when connecting to the bastion server. This is NOT the

    MSSQL username.

  6. If you are using Password Authentication, then SSH Login Username should be set to the

    password of the User from the previous step. If you are using SSH Key Authentication leave this

    blank. Again, this is not the MSSQL password, but the password for the OS-user that Airbyte is

    using to perform commands on the bastion.

  7. If you are using SSH Key Authentication, then SSH Private Key should be set to the RSA

    private Key that you are using to create the SSH connection. This should be the full contents of

    the key file starting with -----BEGIN RSA PRIVATE KEY----- and ending

    with -----END RSA PRIVATE KEY-----.

Generating an SSH Key Pair

The connector expects an RSA key in PEM format. To generate this key:

ssh-keygen -t rsa -m PEM -f myuser_rsa

This produces the private key in pem format, and the public key remains in the standard format used by the authorized_keys file on your bastion host. The public key should be added to your bastion host to whichever user you want to use with Airbyte. The private key is provided via copy-and-paste to the Airbyte connector configuration screen, so it may log in to the bastion.

Data type mapping

MSSQL data types are mapped to the following data types when synchronizing data. You can check the test values examples here. If you can't find the data type you are looking for or have any problems feel free to add a new test!

MSSQL Type Resulting Type Notes
bigint integer
binary binary
bit boolean
char string
date date
datetime timestamp
datetime2 timestamp
datetimeoffset timestamp with timezone
decimal number / integer maps to integer when the column scale is 0
int integer
float number
geography string
geometry string
money number
numeric number / integer maps to integer when the column scale is 0
ntext string
nvarchar string
nvarchar(max) string
real number
smalldatetime timestamp
smallint integer
smallmoney number
sql_variant string
uniqueidentifier string
text string
time time
tinyint integer
varbinary binary
varchar string
varchar(max) COLLATE Latin1_General_100_CI_AI_SC_UTF8 string
hierarchyid string Non-CDC only
xml string

If you do not see a type in this list, assume that it is coerced into a string. We are happy to take feedback on preferred mappings.

User-defined alias types

SQL Server lets you create alias types based on a built-in type, for example CREATE TYPE alias_timestamp FROM datetime. The connector doesn't map alias types by name. Instead, it uses the JDBC type that the SQL Server driver reports for the column, which comes from the alias's base type.

The connector maps aliases over the following base types the same way as the base type itself, including in CDC syncs:

Alias base type JDBC type Resulting Type
date DATE date
time TIME time
datetime TIMESTAMP timestamp
datetime2 TIMESTAMP timestamp
smalldatetime TIMESTAMP timestamp

Aliases over base types not listed above may cause mapping issues, because their JDBC type may not map the same way as the built-in type.

We recommend using built-in SQL Server types instead of alias types for columns you sync with Airbyte.

Upgrading to version 4.3.0 and above

Version 4.3.0 introduces a migration from the legacy CDK to the new CDK architecture. This migration includes:

  • Automatic State Migration: The connector automatically migrates legacy version 2 state formats to the new version 3 format. This includes:
    • OrderedColumnLoadStatus (primary key-based initial sync) → version 3 primary_key state type
    • CursorBasedStatus (cursor-based incremental) → version 3 cursor_based state type
  • Backward Compatibility: Existing connections will continue to work seamlessly without any manual intervention

Upgrading from 0.4.17 and older versions to 0.4.18 and newer versions

There is a backwards incompatible spec change between Microsoft SQL Source connector versions 0.4.17 and 0.4.18. As part of that spec change replication_method configuration parameter was changed to object from string.

In Microsoft SQL source connector versions 0.4.17 and older, replication_method configuration parameter was saved in the configuration database as follows:

"replication_method": "STANDARD"

Starting with version 0.4.18, replication_method configuration parameter is saved as follows:

"replication_method": {
    "method": "STANDARD"
}

After upgrading Microsoft SQL Source connector from 0.4.17 or older version to 0.4.18 or newer version you need to fix source configurations in the actor table in Airbyte database. To do so, you need to run two SQL queries. Follow the instructions in Airbyte documentation to run SQL queries on Airbyte database.

If you have connections with Microsoft SQL Source using Standard replication method, run this SQL:

update public.actor set configuration =jsonb_set(configuration, '{replication_method}', '{"method": "STANDARD"}', true)
WHERE actor_definition_id ='b5ea17b1-f170-46dc-bc31-cc744ca984c1' AND (configuration->>'replication_method' = 'STANDARD');

If you have connections with Microsoft SQL Source using Logical Replication (CDC) method, run this SQL:

update public.actor set configuration =jsonb_set(configuration, '{replication_method}', '{"method": "CDC"}', true)
WHERE actor_definition_id ='b5ea17b1-f170-46dc-bc31-cc744ca984c1' AND (configuration->>'replication_method' = 'CDC');

IP allow list

If you use Airbyte Cloud and your organization restricts access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

When Airbyte cannot open a TCP connection to your server, the connection test and syncs fail with:

SQL Server Connection Error: Airbyte could not reach SQL Server at the configured host and port.
Check the host and port, that the instance accepts remote TCP connections, and that your firewall
or security group allows Airbyte's IP addresses.

This covers a wrong host or port, a host name that does not resolve, an instance that does not accept remote TCP connections, and a firewall or security group that does not allow the Airbyte Cloud IP addresses. A database on localhost or on a private network is not reachable from Airbyte Cloud without an SSH tunnel; Airbyte Cloud rejects localhost with No Tunnel before connecting, with Airbyte Cloud cannot reach a database on your own machine. Use a hostname reachable from the internet, or configure an SSH tunnel. A URL or host:port pasted into Host is rejected with Enter only the hostname in Host; put the port in the Port field.

Changelog

Expand to review
Version Date Pull Request Subject
5.1.5 2026-11-09 89073 Fix CDC sync failure for columns using user-defined alias types over date, time, datetime, datetime2, or smalldatetime.
5.1.4 2026-11-02 73349 Escape spaces and $ in Debezium message keys, and quote schema names in the Debezium schema include list.
5.1.3 2026-10-21 89634 Upgrade to Bulk CDK 1.2.3: fix critical CVEs in Apache MINA (mina-core 2.0.31), MINA SSHD 2.20.0 and BouncyCastle 1.86; Kafka 4.1.2, Jetty 12.0.40, plexus-utils 3.6.2.
5.1.2 2026-10-19 85353 Cold-start a stream when its saved state is an empty object or an unrecognized legacy format instead of skipping it.
5.1.1 2026-10-12 88500 Replace the sync-oriented retry wording of unreachable-host errors with guidance on the host, port, firewall, and Airbyte Cloud IP allow list; note that Unencrypted needs an SSH tunnel on Airbyte Cloud. Explain the Discovered zero tables. connection-test failure: say that no table is visible to the login and how to grant SELECT. On Airbyte Cloud, default the Encryption option to Encrypted (trust server certificate). On Airbyte Cloud, reject localhost without an SSH tunnel and a URL or host:port pasted into Host with an explicit message before the connection is attempted.
5.1.0 2026-10-12 86318 Add Debezium's max_iteration_transactions (max.iteration.transactions) as an advanced CDC option, defaulting to 500.
5.0.2 2026-10-09 88429 Upgrade to Bulk CDK 1.1.14.
5.0.1 2026-09-09 84356 Change Debezium data.query.mode from direct to function to prevent syncs from hanging on large tables.
5.0.0 2026-06-23 77672 Map DECIMAL/NUMERIC columns with scale 0 to Airbyte integer instead of number so destinations preserve integral semantics.
4.4.12 2026-06-17 80156 Log a message when a DECIMAL/NUMERIC column with scale 0 is discovered, ahead of an upcoming number -> integer remapping. No functional change.
4.4.11 2026-06-11 79128 Fix incremental sync failure when the saved state has a null cursor (table was empty on prior CDK version).
4.4.10 2026-06-10 79149 Update cursor-based incremental query to prevent missing rows with high-precision datetime cursors.
4.4.9 2026-06-02 77998 Validate CDC access per configured stream to prevent zero-LSN errors caused by missing permissions.
4.4.8 2026-05-26 78415 Classify Azure read-replica error 3947 as transient so affected syncs retry instead of failing.
4.4.7 2026-05-12 78033 Re-release the Java connector base image revert after the 4.4.6 publish failure.
4.4.6 2026-05-07 77856 Revert the Java connector base image to resolve connection issues and remove registry rollback overrides.
4.4.5 2026-05-07 77843 Roll back source mssql to 4.4.3 to investigate a potential connection issue.
4.4.4 2026-05-07 77665 Fix sampling sync failure on empty tables in Full Refresh mode (NULL upper bound).
4.4.3 2026-05-05 77786 Make the hidden additional properties fields in spec optional. No functional change.
4.4.2 2026-04-29 77036 Fix TABLESAMPLE failure on views and tables without an ordered column in cursor-incremental syncs.
4.4.1 2026-04-23 76857 Fix Invalid column name error when sampling system-versioned temporal tables that have HIDDEN period columns.
4.4.0 2026-04-23 76143 Add Microsoft Entra ID service principal authentication for both JDBC and CDC paths.
4.3.6 2026-04-20 74729 Fix snapshot partitions restarting from the beginning of the table instead of resuming from the last checkpoint.
4.3.5 2026-02-23 73606 Fix CDC cursor overflow.
4.3.4 2026-02-17 72935 Update LSN validation to correctly detect when saved offset has been truncated.
4.3.3 2026-02-03 71821 Require a manual refresh when schema history is missing, bump CDK version.
4.3.2 2025-12-11 70836 Add Azure SQL Database compatibility for SQL Server Agent check
4.3.1 2025-12-09 70823 Bump up connector version number to release the connector
4.3.0-rc.10 2025-12-05 70306 Update CDK version to include fix for Debezium closing record race condition
4.3.0-rc.9 2025-12-01 70280 Fix connector concurrency default value
4.3.0-rc.8 2025-11-14 69754 Update to latest CDK version
4.3.0-rc.7 2025-11-14 69327 Change default schema to discover all schema instead of "dbo"
4.3.0-rc.6 2025-11-12 69311 Fix NPE in state migration when incremental_state is null; Reduce resourceAcquisitionHeartbeat to 100ms; Upgrade CDK version
4.3.0-rc.5 2025-11-08 69248 Add SQL Server identifier quoting to fix reserved keyword issues
4.3.0-rc.4 2025-11-05 69194 Fix composite primary key discovery to show all PK columns; separate PK discovery from sync strategy
4.3.0-rc.3 2025-10-31 69097 Fix connector state value type from string to json object
4.3.0-rc.2 2025-10-29 69093 Fix state parsing error for integer and number
4.3.0-rc.1 2025-10-28 63731 Migrate source mssql from old CDK to new CDK
4.2.6 2025-10-14 68091 Fix CDC table filtering to prevent unintended tables from being synced when using schema-level CDC
4.2.5 2025-08-13 64905 bumping up java cdk version for mssql
4.2.4 2025-07-03 62491 Improve Debezium performance by configuring the poll interval parameter.
4.2.3 2025-07-01 62052 Revert change to CDK interface signature.
4.2.2 2025-06-25 61729 Support the use of logical primary keys for CDC.
4.2.1 2025-06-23 62015 Fix previous merge. Improve cutoff date handling
4.2.0 2025-06-19 61685 Add an option to exclude today's data from cursor based incremental syncs when using temporal cursor (datetime).
4.1.29 2025-06-03 61320 Add error handling for connection issues and adopt the latest CDK version.
4.1.28 2025-05-15 60311 Migrate to new gradle flow.
4.1.27 2025-04-28 59124 Fix _ab_cdc_event_serial_no datatype in addMetaDataToRowsFetchedOutsideDebezium method
4.1.26 2025-03-27 56401 Fix non-unique value clustered index issue
4.1.25 2025-03-20 55878 Integrate with the latest mssql jdbc driver to solve a pre-gregorian date offset bug
4.1.24 2025-03-11 55709 Filter unwanted tables in discover to prevent null table issues
4.1.23 2025-03-06 55234 Update base image version for certified DB source connectors
4.1.22 2025-02-10 53217 Default to PK when the clustered index is composite.
4.1.21 2025-02-21 54189 Print state data only in the debugging log.
4.1.20 2025-01-26 52556 Improve tables discovery during read.
4.1.19 2025-01-16 51596 Bump driver versions to latest (jdbc, debezium, cdk)
4.1.18 2025-01-06 50943 Use airbyte/java-connector-base:2.0.0. This makes the image rootless. The connector will be incompatible with Airbyte < 0.64.
4.1.17 2024-12-17 49840 Use a base image: airbyte/java-connector-base:1.0.0
4.1.16 2024-11-13 48484 Enhanced error handling for MSSQL to improve system error detection and response.
4.1.15 2024-10-05 46515 Improving discovery of large SQL server database.
4.1.14 2024-09-17 45639 Adopt latest CDK to use the latest apache sshd mina to handle tcpkeepalive requests.
4.1.13 2024-09-05 45181 Fix incorrect categorizing resumable/nonresumable full refresh streams.
4.1.12 2024-09-10 45368 Remove excessive debezium logging.
4.1.11 2024-09-04 45142 Fix incorrect datetimeoffset format in cursor state.
4.1.10 2024-08-27 44759 Improve null safety in parsing debezium change events.
4.1.9 2024-08-27 44841 Adopt latest CDK.
4.1.8 2024-08-08 43410 Adopt latest CDK.
4.1.7 2024-08-06 42869 Adopt latest CDK.
4.1.6 2024-07-30 42550 Correctly report stream states.
4.1.5 2024-07-29 42852 Bump CDK version to latest to use new bug fixes on error translation.
4.1.4 2024-07-23 42421 Remove final transient error emitter iterators.
4.1.3 2024-07-22 42411 Hide the "initial load timeout in hours" field by default in UI
4.1.2 2024-07-22 42024 Fix a NPE bug on resuming from a failed attempt.
4.1.1 2024-07-19 42122 Improve wass error message + logging.
4.1.0 2024-07-17 42078 WASS analytics + bug fixes.
4.0.36 2024-07-17 41648 Implement WASS.
4.0.35 2024-07-05 40570 Bump debezium-connector-sqlserver from 2.6.1.Final to 2.6.2.Final.
4.0.34 2024-07-01 40516 Remove dbz hearbeat.
4.0.33 2024-06-30 40638 Fix a bug that could slow down an initial load of a large table using a different clustered index from the primary key.
4.0.32 2024-06-26 40558 Handle DatetimeOffset correctly.
4.0.31 2024-06-14 39419 Handle DatetimeOffset correctly.
4.0.30 2024-06-14 39349 Full refresh stream sending internal count metadata.
4.0.29 2024-06-14 39506 Adopt latest CDK.
4.0.28 2024-06-08 39342 Fix custom conversion in CDC for datetimeoffset type.
4.0.27 2024-05-29 38584 Set is_resumable flag in discover.
4.0.26 2024-05-16 38292 Improve cursor value query to return only one row
4.0.25 2024-05-29 38775 Publish CDK
4.0.24 2024-05-23 38640 Sync sending trace status messages indicating progress.
4.0.23 2024-05-15 38208 disable counts in full refresh stream in state message.
4.0.22 2024-05-14 38196 Bump jdbc driver version to 12.6.1.jre11
4.0.21 2024-05-07 38054 Resumeable refresh should run only if there is source defined pk.
4.0.20 2024-05-07 38042 Bump debezium version to latest.
4.0.19 2024-05-07 38029 Fix previous release.
4.0.18 2024-04-30 37451 Resumable full refresh read of tables.
4.0.17 2024-05-02 37781 Adopt latest CDK.
4.0.16 2024-05-01 37742 Adopt latest CDK. Remove Debezium retries.
4.0.15 2024-04-22 37541 Adopt latest CDK. reduce excessive logs.
4.0.14 2024-04-22 37476 Adopt latest CDK.
4.0.13 2024-04-16 37111 Populate null values in record message.
4.0.12 2024-04-15 37326 Allow up to 60 minutes of wait for the an initial CDC record.
4.0.11 2024-04-15 37325 Populate airbyte_meta.changes + error handling.
4.0.10 2024-04-15 37110 Internal cleanup.
4.0.9 2024-04-10 36919 Fix a bug in conversion of null values.
4.0.8 2024-04-05 36872 Update to connector's metadat definition.
4.0.7 2024-04-03 36772 Adopt latest CDK.
4.0.6 2024-03-25 36333 Deprecate Dbz state iterator.
4.0.5 2024-03-21 36364 Allow up to 40 minutes of wait for the an initial CDC record.
4.0.4 2024-03-20 36325 [Refactor] : Remove mssql initial source operations .
4.0.3 2024-03-19 36263 Fix a failure seen in CDC with tables containing default values.
4.0.2 2024-03-06 35792 Initial sync will now send record count in state message.
4.0.1 2024-03-12 36011 Read correctly null values of columns with default value in CDC.
4.0.0 2024-03-06 35873 Terabyte-sized tables support, reliability improvements, bug fixes.
3.7.7 2024-03-06 35816 Fix query that was failing on a case sensitive server.
3.7.6 2024-03-04 35721 Fix tests
3.7.5 2024-02-29 35739 Allow configuring the queue size used for cdc events.
3.7.4 2024-02-26 35566 Add config to throw an error on invalid CDC position.
3.7.3 2024-02-23 35596 Fix a logger issue
3.7.2 2024-02-21 35368 Change query syntax to make it compatible with Azure SQL Managed Instance.
3.7.1 2024-02-20 35405 Change query syntax to make it compatible with Azure Synapse.
3.7.0 2024-01-30 33311 Source mssql with checkpointing initial sync.
3.6.1 2024-01-26 34573 Adopt CDK v0.16.0.
3.6.0 2024-01-10 33700 Remove CDC config options for data_to_sync and snapshot isolation.
3.5.1 2024-01-05 33510 Test-only changes.
3.5.0 2023-12-19 33071 Fix SSL configuration parameters
3.4.1 2024-01-02 33755 Encode binary to base64 format
3.4.0 2023-12-19 33481 Remove LEGACY state flag
3.3.2 2023-12-14 33505 Using the released CDK.
3.3.1 2023-12-12 33225 extracting MsSql specific files out of the CDK.
3.3.0 2023-12-12 33018 Migrate to Per-stream/Global states and away from Legacy states
3.2.1 2023-12-11 33330 Parse DatetimeOffset fields with the correct format when used as cursor
3.2.0 2023-12-07 33225 CDC : Enable compression of schema history blob in state.
3.1.0 2023-11-28 32882 Enforce SSL on Airbyte Cloud.
3.0.2 2023-11-27 32573 Format Datetime and Datetime2 datatypes to 6-digit microsecond precision
3.0.1 2023-11-22 32656 Adopt java CDK version 0.5.0.
3.0.0 2023-11-07 31531 Remapped date, smalldatetime, datetime2, time, and datetimeoffset datatype to their correct Airbyte types
2.0.4 2023-11-06 #32193 Adopt java CDK version 0.4.1.
2.0.3 2023-10-31 32024 Upgrade to Debezium version 2.4.0.
2.0.2 2023-10-30 31960 Adopt java CDK version 0.2.0.
2.0.1 2023-08-24 29821 Set replication_method display_type to radio, update titles and descriptions, and make CDC the default choice
2.0.0 2023-08-22 29493 Set a default cursor for Cdc mode
1.1.1 2023-07-24 28545 Support Read Committed snapshot isolation level
1.1.0 2023-06-26 27737 License Update: Elv2
1.0.19 2023-06-20 27212 Fix silent exception swallowing in StreamingJdbcDatabase
1.0.18 2023-06-14 27335 Remove noisy debug logs
1.0.17 2023-05-25 26473 CDC : Limit queue size
1.0.16 2023-05-01 25740 Disable index logging
1.0.15 2023-04-26 25401 CDC : Upgrade Debezium to version 2.2.0
1.0.14 2023-04-19 25345 Logging : Log database indexes per stream
1.0.13 2023-04-19 24582 CDC : refactor for performance improvement
1.0.12 2023-04-17 25220 Logging changes : Log additional metadata & clean up noisy logs
1.0.11 2023-04-11 24656 CDC minor refactor
1.0.10 2023-04-06 24820 Fix data loss bug during an initial failed non-CDC incremental sync
1.0.9 2023-04-04 24833 Fix Debezium retry policy configuration
1.0.8 2023-03-28 24166 Fix InterruptedException bug during Debezium shutdown
1.0.7 2023-03-27 24373 Preparing the connector for CDC checkpointing
1.0.6 2023-03-22 20760 Removed redundant date-time datatypes formatting
1.0.5 2023-03-21 24207 Fix incorrect schema change warning in CDC mode
1.0.4 2023-03-21 24275 Fix error with CDC checkpointing
1.0.3 2023-03-15 24082 Fixed NPE during cursor values validation
1.0.2 2023-03-14 23908 Log warning on null cursor values
1.0.1 2023-03-10 23939 For network isolation, source connector accepts a list of hosts it is allowed to connect
1.0.0 2023-03-06 23112 Upgrade Debezium version to 2.1.2
0.4.29 2023-02-24 16798 Add event_serial_no to cdc metadata
0.4.28 2023-01-18 21348 Fix error introduced in 18959 in which option initial_waiting_seconds was removed
0.4.27 2022-12-14 20346 Consolidate date/time values mapping for JDBC sources
0.4.26 2022-12-12 18959 CDC : Don't timeout if snapshot is not complete.
0.4.25 2022-11-04 18732 Upgrade debezium version to 1.9.6
0.4.24 2022-10-25 18383 Better SSH error handling + messages
0.4.23 2022-10-21 18263 Fixes bug introduced in 15833 and adds better error messaging for SSH tunnel in Destinations
0.4.22 2022-10-19 18087 Better error messaging for configuration errors (SSH configs, choosing an invalid cursor)
0.4.21 2022-10-17 18041 Fixes bug introduced 2022-09-12 with SshTunnel, handles iterator exception properly
2022-10-13 15535 Update incremental query to avoid data missing when new data is inserted at the same time as a sync starts under non-CDC incremental mode
0.4.20 2022-09-14 15668 Wrap logs in AirbyteLogMessage
0.4.19 2022-09-05 16002 Added ability to specify schemas for discovery during setting connector up
0.4.18 2022-09-03 14910 Standardize spec for CDC replication. Replace the replication_method enum with a config object with a method enum field.
0.4.17 2022-09-01 16261 Emit state messages more frequently
0.4.16 2022-08-18 14356 DB Sources: only show a table can sync incrementally if at least one column can be used as a cursor field
0.4.15 2022-08-11 15538 Allow additional properties in db stream state
0.4.14 2022-08-10 15430 fixed a bug on handling special character on database name
0.4.13 2022-08-04 15268 Added [] enclosing to escape special character in the database name
0.4.12 2022-08-02 14801 Fix multiple log bindings
0.4.11 2022-07-22 14714 Clarified error message when invalid cursor column selected
0.4.10 2022-07-14 14574 Removed additionalProperties:false from JDBC source connectors
0.4.9 2022-07-05 14379 Aligned Normal and CDC migration + added some fixes for datatypes processing
0.4.8 2022-06-24 14121 Omit using 'USE' keyword on Azure SQL with CDC
0.4.5 2022-06-23 14077 Use the new state management
0.4.3 2022-06-17 13887 Increase version to include changes from 13854
0.4.2 2022-06-06 13435 Adjust JDBC fetch size based on max memory and max row size
0.4.1 2022-05-25 13419 Correct enum for Standard method.
0.4.0 2022-05-25 12759 13168 For CDC, Add option to ignore existing data and only sync new changes from the database.
0.3.22 2022-04-29 12480 Query tables with adaptive fetch size to optimize JDBC memory consumption
0.3.21 2022-04-11 11729 Bump mina-sshd from 2.7.0 to 2.8.0
0.3.19 2022-03-31 11495 Adds Support to Chinese MSSQL Server Agent
0.3.18 2022-03-29 11010 Adds JDBC Params
0.3.17 2022-02-21 10242 Fixed cursor for old connectors that use non-microsecond format. Now connectors work with both formats
0.3.16 2022-02-18 10242 Updated timestamp transformation with microseconds
0.3.15 2022-02-14 10256 Add -XX:+ExitOnOutOfMemoryError JVM option
0.3.14 2022-01-24 9554 Allow handling of java sql date in CDC
0.3.13 2022-01-07 9094 Added support for missed data types
0.3.12 2021-12-30 9206 Update connector fields title/description
0.3.11 2021-12-24 8958 Add support for JdbcType.ARRAY
0.3.10 2021-12-01 8371 Fixed incorrect handling "\n" in ssh key
0.3.9 2021-11-09 7748 Improve support for binary and varbinary data types
0.3.8 2021-10-26 7386 Fixed data type (smalldatetime, smallmoney) conversion from mssql source
0.3.7 2021-09-30 6585 Improved SSH Tunnel key generation steps
0.3.6 2021-09-17 6318 Added option to connect to DB via SSH
0.3.4 2021-08-13 4699 Added json config validator
0.3.3 2021-07-05 4689 Add CDC support
0.3.2 2021-06-09 3973 Add AIRBYTE_ENTRYPOINT for Kubernetes support
0.3.1 2021-06-08 3893 Enable SSL connection
0.3.0 2021-04-21 2990 Support namespaces
0.2.3 2021-03-28 2600 Add NCHAR and NVCHAR support to DB and cursor type casting
0.2.2 2021-03-26 2460 Destination supports destination sync mode
0.2.1 2021-03-18 2488 Sources support primary keys
0.2.0 2021-03-09 2238 Protocol allows future/unknown properties
0.1.11 2021-02-02 1887 Migrate AbstractJdbcSource to use iterators
0.1.10 2021-01-25 1746 Fix NPE in State Decorator
0.1.9 2021-01-19 1724 Fix JdbcSource handling of tables with same names in different schemas
0.1.9 2021-01-14 1655 Fix JdbcSource OOM
0.1.8 2021-01-13 1588 Handle invalid numeric values in JDBC source
0.1.6 2020-12-09 1172 Support incremental sync
0.1.5 2020-11-30 1038 Change JDBC sources to discover more than standard schemas
0.1.4 2020-11-30 1046 Add connectors using an index YAML file