diff --git a/docs/integrations/data-ingestion/clickpipes/mongodb/lifecycle.md b/docs/integrations/data-ingestion/clickpipes/mongodb/lifecycle.md index 821acabba65..0241ae6722a 100644 --- a/docs/integrations/data-ingestion/clickpipes/mongodb/lifecycle.md +++ b/docs/integrations/data-ingestion/clickpipes/mongodb/lifecycle.md @@ -19,11 +19,11 @@ After a pipe is provisioned, it enters the `Setup` state. This state is where we ## Snapshot {#snapshot} -Once setup is complete, we enter the `Snapshot` state. `Snapshot`, `Initial Snapshot` and `Initial Load` (more common) are interchangeable terms. In this state, we take a snapshot of the source MongoDB collections and load them into ClickHouse. Retention setting for the oplog should account for initial load time. The pipe will also enter the `Snapshot` state when a resync is triggered or when new tables are added to an existing pipe. +Once setup is complete, we enter the `Snapshot` state (unless it's a CDC-only pipe, which would transition to `Running`). `Snapshot`, `Initial Snapshot` and `Initial Load` (more common) are interchangeable terms. In this state, we take a snapshot of the source MongoDB collections and load them into ClickHouse. Retention setting for the oplog should account for initial load time. The pipe will also enter the `Snapshot` state when a resync is triggered or when new tables are added to an existing pipe. ## Running {#running} -Once the initial load is complete, the pipe enters the `Running` state. This is where the pipe begins `Change-Data Capture`. In this state, we start streaming changes from the source MongoDB cluster to ClickHouse. For information on controlling CDC, see [the doc on controlling CDC](./sync_control). +Once the initial load is complete, the pipe enters the `Running` state (unless it's a snapshot-only pipe, which would transition to `Completed`). This is where the pipe begins `Change-Data Capture`. In this state, we start streaming changes from the source MongoDB cluster to ClickHouse. For information on controlling CDC, see [the doc on controlling CDC](./sync_control). ## Paused {#paused} @@ -48,6 +48,10 @@ This state is coming soon. If you're using our [OpenAPI](https://clickhouse.com/ ::: This state indicates the pipe is in the phase of resync where it is performing an atomic swap of the _resync tables with the original tables. More information on resync can be found in the [resync documentation](./resync). +## Completed {#completed} + +This state applies to snapshot-only pipes and indicates that the snapshot has been completed and there's no more work to do. + ## Failed {#failed} If there is an irrecoverable error in the pipe, it will enter the `Failed` state. You can reach out to support or [resync](./resync) your pipe to recover from this state. diff --git a/docs/integrations/data-ingestion/clickpipes/mysql/lifecycle.md b/docs/integrations/data-ingestion/clickpipes/mysql/lifecycle.md index 3e37af23889..7ecc74be422 100644 --- a/docs/integrations/data-ingestion/clickpipes/mysql/lifecycle.md +++ b/docs/integrations/data-ingestion/clickpipes/mysql/lifecycle.md @@ -19,11 +19,11 @@ After a pipe is provisioned, it enters the `Setup` state. This state is where we ## Snapshot {#snapshot} -Once setup is complete, we enter the `Snapshot` state. `Snapshot`, `Initial Snapshot` and `Initial Load` (more common) are interchangeable terms. In this state, we take a snapshot of the source MySQL tables and load them into ClickHouse. Retention setting for binary logs should account for initial load time. For more information on initial load, see the [parallel initial load documentation](./parallel_initial_load). The pipe will also enter the `Snapshot` state when a resync is triggered or when new tables are added to an existing pipe. +Once setup is complete, we enter the `Snapshot` state (unless it's a CDC-only pipe, which would transition to `Running`). `Snapshot`, `Initial Snapshot` and `Initial Load` (more common) are interchangeable terms. In this state, we take a snapshot of the source MySQL tables and load them into ClickHouse. Retention setting for binary logs should account for initial load time. For more information on initial load, see the [parallel initial load documentation](./parallel_initial_load). The pipe will also enter the `Snapshot` state when a resync is triggered or when new tables are added to an existing pipe. ## Running {#running} -Once the initial load is complete, the pipe enters the `Running` state. This is where the pipe begins `Change-Data Capture`. In this state, we start reading binary logs from the source database and sync the data to ClickHouse in batches. For information on controlling CDC, see [the doc on controlling CDC](./sync_control). +Once the initial load is complete, the pipe enters the `Running` state (unless it's a snapshot-only pipe, which would transition to `Completed`). This is where the pipe begins `Change-Data Capture`. In this state, we start reading binary logs from the source database and sync the data to ClickHouse in batches. For information on controlling CDC, see [the doc on controlling CDC](./sync_control). ## Paused {#paused} @@ -48,6 +48,10 @@ This state is coming soon. If you're using our [OpenAPI](https://clickhouse.com/ ::: This state indicates the pipe is in the phase of resync where it is performing an atomic swap of the _resync tables with the original tables. More information on resync can be found in the [resync documentation](./resync). +## Completed {#completed} + +This state applies to snapshot-only pipes and indicates that the snapshot has been completed and there's no more work to do. + ## Failed {#failed} If there is an irrecoverable error in the pipe, it will enter the `Failed` state. You can reach out to support or [resync](./resync) your pipe to recover from this state. diff --git a/docs/integrations/data-ingestion/clickpipes/postgres/lifecycle.md b/docs/integrations/data-ingestion/clickpipes/postgres/lifecycle.md index 2ac02df64c5..3e7f30120b9 100644 --- a/docs/integrations/data-ingestion/clickpipes/postgres/lifecycle.md +++ b/docs/integrations/data-ingestion/clickpipes/postgres/lifecycle.md @@ -19,11 +19,11 @@ After a pipe is provisioned, it enters the `Setup` state. This state is where we ## Snapshot {#snapshot} -Once setup is complete, we enter the `Snapshot` state. `Snapshot`, `Initial Snapshot` and `Initial Load` (more common) are interchangeable terms. In this state, we take a snapshot of the source database tables and load them into ClickHouse. This does not use logical replication, but the replication slot is created at this step, therefore your `max_slot_wal_keep_size` and storage parameters should account for slot growth during initial load. For more information on initial load, see the [parallel initial load documentation](./parallel_initial_load). The pipe will also enter the `Snapshot` state when a resync is triggered or when new tables are added to an existing pipe. +Once setup is complete, we enter the `Snapshot` state (unless it's a CDC-only pipe, which would transition to `Running`). `Snapshot`, `Initial Snapshot` and `Initial Load` (more common) are interchangeable terms. In this state, we take a snapshot of the source database tables and load them into ClickHouse. This does not use logical replication, but the replication slot is created at this step, therefore your `max_slot_wal_keep_size` and storage parameters should account for slot growth during initial load. For more information on initial load, see the [parallel initial load documentation](./parallel_initial_load). The pipe will also enter the `Snapshot` state when a resync is triggered or when new tables are added to an existing pipe. ## Running {#running} -Once the initial load is complete, the pipe enters the `Running` state. This is where the pipe begins `Change-Data Capture`. In this state, we start logical replication from the source database to ClickHouse. For information on controlling CDC, see [the doc on controlling CDC](./sync_control). +Once the initial load is complete, the pipe enters the `Running` state (unless it's a snapshot-only pipe, which would transition to `Completed`). This is where the pipe begins `Change-Data Capture`. In this state, we start logical replication from the source database to ClickHouse. For information on controlling CDC, see [the doc on controlling CDC](./sync_control). ## Paused {#paused} @@ -48,6 +48,10 @@ This state is coming soon. If you're using our [OpenAPI](https://clickhouse.com/ ::: This state indicates the pipe is in the phase of resync where it is performing an atomic swap of the _resync tables with the original tables. More information on resync can be found in the [resync documentation](./resync). +## Completed {#completed} + +This state applies to snapshot-only pipes and indicates that the snapshot has been completed and there's no more work to do. + ## Failed {#failed} If there is an irrecoverable error in the pipe, it will enter the `Failed` state. You can reach out to support or [resync](./resync) your pipe to recover from this state.