From d732eaf80e7e5e07411422ed40bbcdd4c5c91ab9 Mon Sep 17 00:00:00 2001 From: Abhi Vaidyanatha Date: Thu, 16 Sep 2021 16:40:12 -0700 Subject: [PATCH] Add information on which destinations support Incremental - Deduped History in their docs (#6031) Co-authored-by: Abhi Vaidyanatha --- docs/integrations/destinations/azureblobstorage.md | 2 +- docs/integrations/destinations/bigquery.md | 1 + docs/integrations/destinations/dynamodb.md | 1 + docs/integrations/destinations/gcs.md | 1 + docs/integrations/destinations/kafka.md | 2 ++ docs/integrations/destinations/keen.md | 1 + docs/integrations/destinations/local-csv.md | 1 + docs/integrations/destinations/local-json.md | 1 + docs/integrations/destinations/meilisearch.md | 1 + docs/integrations/destinations/mongodb.md | 1 + docs/integrations/destinations/mssql.md | 1 + docs/integrations/destinations/mysql.md | 1 + docs/integrations/destinations/oracle.md | 1 + docs/integrations/destinations/postgres.md | 1 + docs/integrations/destinations/pubsub.md | 1 + docs/integrations/destinations/redshift.md | 1 + docs/integrations/destinations/s3.md | 1 + docs/integrations/destinations/snowflake.md | 1 + 18 files changed, 19 insertions(+), 1 deletion(-) diff --git a/docs/integrations/destinations/azureblobstorage.md b/docs/integrations/destinations/azureblobstorage.md index ae934399d3be5..91f3f36a60220 100644 --- a/docs/integrations/destinations/azureblobstorage.md +++ b/docs/integrations/destinations/azureblobstorage.md @@ -12,7 +12,7 @@ The Airbyte Azure Blob Storage destination allows you to sync data to Azure Blob | :--- | :---: | :--- | | Full Refresh Sync | ✅ | Warning: this mode deletes all previously synced data in the configured blob. | | Incremental - Append Sync | ✅ | The append mode would only work for "Append blobs" blobs as per Azure limitations, more details https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction#blobs | - +| Incremental - Deduped History | ❌ | As this connector does not support dbt, we don't support this sync mode on this destination. | ## Configuration diff --git a/docs/integrations/destinations/bigquery.md b/docs/integrations/destinations/bigquery.md index 788ac49a983b7..d1300c53d6384 100644 --- a/docs/integrations/destinations/bigquery.md +++ b/docs/integrations/destinations/bigquery.md @@ -39,6 +39,7 @@ Each stream will be output into its own table in BigQuery. Each table will conta | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | Yes | | | Namespaces | Yes | | ## Getting started diff --git a/docs/integrations/destinations/dynamodb.md b/docs/integrations/destinations/dynamodb.md index dfa230be31f16..ebaf106ccd28b 100644 --- a/docs/integrations/destinations/dynamodb.md +++ b/docs/integrations/destinations/dynamodb.md @@ -21,6 +21,7 @@ Each stream will be output into its own DynamoDB table. Each table will a collec | :--- | :---: | :--- | | Full Refresh Sync | ✅ | Warning: this mode deletes all previously synced data in the configured DynamoDB table. | | Incremental - Append Sync | ✅ | | +| Incremental - Deduped History | ❌ | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | ✅ | Namespace will be used as part of the table name. | ### Performance considerations diff --git a/docs/integrations/destinations/gcs.md b/docs/integrations/destinations/gcs.md index e667abf3bd31d..5ace4e98af353 100644 --- a/docs/integrations/destinations/gcs.md +++ b/docs/integrations/destinations/gcs.md @@ -12,6 +12,7 @@ The Airbyte GCS destination allows you to sync data to cloud storage buckets. Ea | :--- | :---: | :--- | | Full Refresh Sync | ✅ | Warning: this mode deletes all previously synced data in the configured bucket path. | | Incremental - Append Sync | ✅ | | +| Incremental - Deduped History | ❌ | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | ❌ | Setting a specific bucket path is equivalent to having separate namespaces. | ## Configuration diff --git a/docs/integrations/destinations/kafka.md b/docs/integrations/destinations/kafka.md index 658e884be51c9..206f6272f9f75 100644 --- a/docs/integrations/destinations/kafka.md +++ b/docs/integrations/destinations/kafka.md @@ -26,8 +26,10 @@ Each record will contain in its key the uuid assigned by Airbyte, and in the val | :--- | :--- | :--- | | Full Refresh Sync | No | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | Yes | | + ## Getting started ### Requirements diff --git a/docs/integrations/destinations/keen.md b/docs/integrations/destinations/keen.md index 05d6695340b35..4589117022d41 100644 --- a/docs/integrations/destinations/keen.md +++ b/docs/integrations/destinations/keen.md @@ -21,6 +21,7 @@ Each stream will output an event in Keen. Each collection will inherit the name | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | No | | ## Getting started diff --git a/docs/integrations/destinations/local-csv.md b/docs/integrations/destinations/local-csv.md index 8d3647e67d58e..5849081fe0822 100644 --- a/docs/integrations/destinations/local-csv.md +++ b/docs/integrations/destinations/local-csv.md @@ -24,6 +24,7 @@ Each stream will be output into its own file. Each file will contain 3 columns: | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | No | | #### Performance considerations diff --git a/docs/integrations/destinations/local-json.md b/docs/integrations/destinations/local-json.md index 9a3764200ef90..dbb05d5e50e5f 100644 --- a/docs/integrations/destinations/local-json.md +++ b/docs/integrations/destinations/local-json.md @@ -24,6 +24,7 @@ Each stream will be output into its own file. Each file will a collections of `j | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | No | | #### Performance considerations diff --git a/docs/integrations/destinations/meilisearch.md b/docs/integrations/destinations/meilisearch.md index 5906281b0b822..6095a730ada32 100644 --- a/docs/integrations/destinations/meilisearch.md +++ b/docs/integrations/destinations/meilisearch.md @@ -16,6 +16,7 @@ Each stream will be output into its own index in MeiliSearch. Each table will be | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | No | | ## Getting started diff --git a/docs/integrations/destinations/mongodb.md b/docs/integrations/destinations/mongodb.md index e4be56ef05991..52e673ba56ce4 100644 --- a/docs/integrations/destinations/mongodb.md +++ b/docs/integrations/destinations/mongodb.md @@ -20,6 +20,7 @@ Each stream will be output into its own collection in MongoDB. Each collection w | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | Yes | | ## Getting started diff --git a/docs/integrations/destinations/mssql.md b/docs/integrations/destinations/mssql.md index 38a5d6ad6d305..cb299c2c47644 100644 --- a/docs/integrations/destinations/mssql.md +++ b/docs/integrations/destinations/mssql.md @@ -28,6 +28,7 @@ Each stream will be output into its own table in SQL Server. Each table will con | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | Yes | | ## Getting started diff --git a/docs/integrations/destinations/mysql.md b/docs/integrations/destinations/mysql.md index b12b5b1f79f7a..2796c8f7d7388 100644 --- a/docs/integrations/destinations/mysql.md +++ b/docs/integrations/destinations/mysql.md @@ -20,6 +20,7 @@ Each stream will be output into its own table in MySQL. Each table will contain | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | Yes | | | Namespaces | Yes | | ## Getting started diff --git a/docs/integrations/destinations/oracle.md b/docs/integrations/destinations/oracle.md index 432d3b6f45ada..d1bd9393a9ae9 100644 --- a/docs/integrations/destinations/oracle.md +++ b/docs/integrations/destinations/oracle.md @@ -20,6 +20,7 @@ Each stream will be output into its own table in Oracle. Each table will contain | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | Yes | | | Namespaces | Yes | | | Basic Normalization | Yes | Only for raw tables, doesn't support for nested json yet | diff --git a/docs/integrations/destinations/postgres.md b/docs/integrations/destinations/postgres.md index c046952323f4f..aba0955e1ad9e 100644 --- a/docs/integrations/destinations/postgres.md +++ b/docs/integrations/destinations/postgres.md @@ -22,6 +22,7 @@ Each stream will be output into its own table in Postgres. Each table will conta | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | Yes | | | Namespaces | Yes | | ## Getting started diff --git a/docs/integrations/destinations/pubsub.md b/docs/integrations/destinations/pubsub.md index 4c37110de6efd..c27801533f215 100644 --- a/docs/integrations/destinations/pubsub.md +++ b/docs/integrations/destinations/pubsub.md @@ -28,6 +28,7 @@ The data will be a serialized JSON, containing the following fields | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | No | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | Yes | | ## Getting started diff --git a/docs/integrations/destinations/redshift.md b/docs/integrations/destinations/redshift.md index d259dfa2e0028..5d4e5beb4ed0f 100644 --- a/docs/integrations/destinations/redshift.md +++ b/docs/integrations/destinations/redshift.md @@ -26,6 +26,7 @@ Each stream will be output into its own raw table in Redshift. Each table will c | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | Yes | | | Namespaces | Yes | | #### Target Database diff --git a/docs/integrations/destinations/s3.md b/docs/integrations/destinations/s3.md index 57a8f348ded28..4d7d9177873f6 100644 --- a/docs/integrations/destinations/s3.md +++ b/docs/integrations/destinations/s3.md @@ -12,6 +12,7 @@ The Airbyte S3 destination allows you to sync data to AWS S3/ Minio S3. Each str | :--- | :---: | :--- | | Full Refresh Sync | ✅ | Warning: this mode deletes all previously synced data in the configured bucket path. | | Incremental - Append Sync | ✅ | | +| Incremental - Deduped History | ❌ | As this connector does not support dbt, we don't support this sync mode on this destination. | | Namespaces | ❌ | Setting a specific bucket path is equivalent to having separate namespaces. | ## Configuration diff --git a/docs/integrations/destinations/snowflake.md b/docs/integrations/destinations/snowflake.md index af38eeaa9fce7..95e89e40c5047 100644 --- a/docs/integrations/destinations/snowflake.md +++ b/docs/integrations/destinations/snowflake.md @@ -20,6 +20,7 @@ Each stream will be output into its own table in Snowflake. Each table will cont | :--- | :--- | :--- | | Full Refresh Sync | Yes | | | Incremental - Append Sync | Yes | | +| Incremental - Deduped History | Yes | | | Namespaces | Yes | | ## Getting started