Skip to content

Commit 167d08b

Browse files
committed
clarify & correct target schema requirements
1 parent 90d91b5 commit 167d08b

File tree

7 files changed

+68
-22
lines changed

7 files changed

+68
-22
lines changed

src/current/_includes/molt/molt-setup.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,12 @@ When you run `molt fetch`, you can configure the following options for data load
7474
- [Data load mode](#data-load-mode): Choose between `IMPORT INTO` and `COPY FROM`.
7575
- [Fetch metrics](#fetch-metrics): Configure metrics collection during initial data load.
7676

77+
<div class="filters filters-big clearfix">
78+
<button class="filter-button" data-scope="postgres">PostgreSQL</button>
79+
<button class="filter-button" data-scope="mysql">MySQL</button>
80+
<button class="filter-button" data-scope="oracle">Oracle</button>
81+
</div>
82+
7783
### Connection strings
7884

7985
{% include molt/molt-connection-strings.md %}

src/current/_includes/molt/replicator-flags-usage.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ The following [MOLT Replicator]({% link molt/molt-replicator.md %}) flags are **
55
| Flag | Description |
66
|-------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
77
| `--slotName` | **Required.** PostgreSQL replication slot name. Must match the slot name specified with `--pglogical-replication-slot-name` in the [MOLT Fetch command](#start-fetch). |
8-
| `--targetSchema` | **Required.** Target schema name on CockroachDB where tables will be replicated. |
8+
| `--targetSchema` | **Required.** Target schema name on CockroachDB where tables will be replicated. Schema name must be fully qualified in the format `database.schema`. |
99
| `--stagingSchema` | **Required.** Staging schema name for replication metadata and checkpoints. |
1010
| `--stagingCreateSchema` | **Required.** Automatically create the staging schema if it does not exist. Include this flag when starting replication for the first time. |
1111
| `--metricsAddr` | Enable Prometheus metrics at a specified `{host}:{port}`. Metrics are served at `http://{host}:{port}/_/varz`. |
@@ -14,7 +14,7 @@ The following [MOLT Replicator]({% link molt/molt-replicator.md %}) flags are **
1414
<section class="filter-content" markdown="1" data-scope="mysql">
1515
| Flag | Description |
1616
|-------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
17-
| `--targetSchema` | **Required.** Target schema name on CockroachDB where tables will be replicated. |
17+
| `--targetSchema` | **Required.** Target schema name on CockroachDB where tables will be replicated. Schema name must be fully qualified in the format `database.schema`. |
1818
| `--defaultGTIDSet` | **Required.** Default GTID set for changefeed. |
1919
| `--stagingSchema` | **Required.** Staging schema name for replication metadata and checkpoints. |
2020
| `--stagingCreateSchema` | **Required.** Automatically create the staging schema if it does not exist. Include this flag when starting replication for the first time. |
@@ -26,16 +26,16 @@ You can find the starting GTID in the `cdc_cursor` field of the `fetch complete`
2626
</section>
2727

2828
<section class="filter-content" markdown="1" data-scope="oracle">
29-
| Flag | Description |
30-
|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
31-
| `--sourceSchema` | **Required.** Source schema name on Oracle where tables will be replicated from. |
32-
| `--targetSchema` | **Required.** Target schema name on CockroachDB where tables will be replicated. |
33-
| `--scn` | **Required.** Snapshot System Change Number (SCN) for the initial changefeed starting point. |
34-
| `--backfillFromSCN` | **Required.** SCN of the earliest active transaction at the time of the snapshot. Ensures no transactions are skipped. |
35-
| `--stagingSchema` | **Required.** Staging schema name for replication metadata and checkpoints. |
36-
| `--stagingCreateSchema` | **Required.** Automatically create the staging schema if it does not exist. Include this flag when starting replication for the first time. |
37-
| `--metricsAddr` | Enable Prometheus metrics at a specified `{host}:{port}`. Metrics are served at `http://{host}:{port}/_/varz`. |
38-
| `--userscript` | Path to a userscript that enables table filtering from Oracle sources. Refer to [Table filter userscript](#table-filter-userscript). |
29+
| Flag | Description |
30+
|-------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|
31+
| `--sourceSchema` | **Required.** Source schema name on Oracle where tables will be replicated from. |
32+
| `--targetSchema` | **Required.** Target schema name on CockroachDB where tables will be replicated. Schema name must be fully qualified in the format `database.schema`. |
33+
| `--scn` | **Required.** Snapshot System Change Number (SCN) for the initial changefeed starting point. |
34+
| `--backfillFromSCN` | **Required.** SCN of the earliest active transaction at the time of the snapshot. Ensures no transactions are skipped. |
35+
| `--stagingSchema` | **Required.** Staging schema name for replication metadata and checkpoints. |
36+
| `--stagingCreateSchema` | **Required.** Automatically create the staging schema if it does not exist. Include this flag when starting replication for the first time. |
37+
| `--metricsAddr` | Enable Prometheus metrics at a specified `{host}:{port}`. Metrics are served at `http://{host}:{port}/_/varz`. |
38+
| `--userscript` | Path to a userscript that enables table filtering from Oracle sources. Refer to [Table filter userscript](#table-filter-userscript). |
3939

4040
You can find the SCN values in the message `replication-only mode should include the following replicator flags` after the [initial data load](#start-fetch) completes.
4141
</section>

src/current/_includes/molt/replicator-flags.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@
3939
| `--targetJitterTime` | `DURATION` | The time over which to jitter database pool disconnections.<br><br>**Default:** `15s` |
4040
| `--targetMaxLifetime` | `DURATION` | The maximum lifetime of a database connection.<br><br>**Default:** `5m0s` |
4141
| `--targetMaxPoolSize` | `INT` | The maximum number of target database connections.<br><br>**Default:** `128` |
42-
| `--targetSchema` | `STRING` | The SQL database schema in the target cluster to update. |
42+
| `--targetSchema` | `STRING` | The SQL database schema in the target cluster to update. CockroachDB schema names must be fully qualified in the format `database.schema`. |
4343
| `--targetStatementCacheSize` | `INT` | The maximum number of prepared statements to retain.<br><br>**Default:** `128` |
4444
| `--taskGracePeriod` | `DURATION` | How long to allow for task cleanup when recovering from errors.<br><br>**Default:** `1m0s` |
4545
| `--timestampLimit` | `INT` | The maximum number of source timestamps to coalesce into a target transaction.<br><br>**Default:** `1000` |

src/current/molt/migrate-failback.md

Lines changed: 22 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,12 @@ When you run `replicator`, you can configure the following options for replicati
101101
</section>
102102
- [Replicator metrics](#replicator-metrics): Monitor failback replication performance.
103103

104+
<div class="filters filters-big clearfix">
105+
<button class="filter-button" data-scope="postgres">PostgreSQL</button>
106+
<button class="filter-button" data-scope="mysql">MySQL</button>
107+
<button class="filter-button" data-scope="oracle">Oracle</button>
108+
</div>
109+
104110
### Connection strings
105111

106112
For failback, MOLT Replicator uses `--targetConn` to specify the destination database where you want to replicate CockroachDB changes, and `--stagingConn` for the CockroachDB staging database.
@@ -225,11 +231,19 @@ Create a CockroachDB changefeed to send changes to MOLT Replicator.
225231

226232
1. Create the CockroachDB changefeed pointing to the MOLT Replicator webhook endpoint. Use `cursor` to specify the logical timestamp from the preceding step.
227233

228-
{{site.data.alerts.callout_info}}
229-
Ensure that only **one** changefeed points to MOLT Replicator at a time to avoid mixing streams of incoming data.
230-
{{site.data.alerts.end}}
234+
<section class="filter-content" markdown="1" data-scope="postgres">
235+
The target schema is specified in the webhook URL path in the fully-qualified format `/database/schema`. For example, `/migration_schema/public` routes changes to the `public` schema in the `migration_schema` database.
236+
</section>
231237

232-
{{site.data.alerts.callout_success}}
238+
<section class="filter-content" markdown="1" data-scope="mysql">
239+
The target schema is specified in the webhook URL path. For example, `/migration_schema` routes changes to the `migration_schema` schema.
240+
</section>
241+
242+
<section class="filter-content" markdown="1" data-scope="oracle">
243+
The target schema is specified in the webhook URL path. For example, `/MIGRATION_SCHEMA` routes changes to the `MIGRATION_SCHEMA` schema.
244+
</section>
245+
246+
{{site.data.alerts.callout_info}}
233247
For details on the webhook sink URI, refer to [Webhook sink]({% link {{ site.current_cloud_version }}/changefeed-sinks.md %}#webhook-sink).
234248
{{site.data.alerts.end}}
235249

@@ -268,6 +282,10 @@ Create a CockroachDB changefeed to send changes to MOLT Replicator.
268282
1101234051444375553
269283
~~~
270284

285+
{{site.data.alerts.callout_success}}
286+
Ensure that only **one** changefeed points to MOLT Replicator at a time to avoid mixing streams of incoming data.
287+
{{site.data.alerts.end}}
288+
271289
1. Monitor the changefeed status, specifying the job ID:
272290

273291
~~~ sql

src/current/molt/migrate-load-replicate.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,12 @@ When you run `replicator`, you can configure the following options for replicati
8282
</section>
8383
- [Replicator metrics](#replicator-metrics): Monitor replication progress and performance.
8484

85+
<div class="filters filters-big clearfix">
86+
<button class="filter-button" data-scope="postgres">PostgreSQL</button>
87+
<button class="filter-button" data-scope="mysql">MySQL</button>
88+
<button class="filter-button" data-scope="oracle">Oracle</button>
89+
</div>
90+
8591
### Replication connection strings
8692

8793
MOLT Replicator uses `--sourceConn` and `--targetConn` to specify the source and target database connections.

src/current/molt/migrate-resume-replication.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,10 +70,12 @@ replicator oraclelogminer \
7070
--sourceConn $SOURCE \
7171
--sourcePDBConn $SOURCE_PDB \
7272
--sourceSchema migration_schema \
73+
--targetSchema defaultdb.public \
7374
--targetConn $TARGET \
7475
--stagingSchema _replicator \
7576
--metricsAddr :30005 \
76-
--userscript table_filter.ts
77+
--userscript table_filter.ts \
78+
-v
7779
~~~
7880

7981
{{site.data.alerts.callout_info}}

src/current/molt/molt-replicator.md

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -269,8 +269,8 @@ These server certificates must correspond to the client certificates specified i
269269

270270
Encode client certificates for changefeed webhook URLs:
271271

272-
- **Webhook URLs**: Use both URL encoding and base64 encoding: `base64 -i ./client.crt | jq -R -r '@uri'`
273-
- **Non-webhook contexts**: Use base64 encoding only: `base64 -w 0 ca.cert`
272+
- Webhook URLs: Use both URL encoding and base64 encoding: `base64 -i ./client.crt | jq -R -r '@uri'`
273+
- Non-webhook contexts: Use base64 encoding only: `base64 -w 0 ca.cert`
274274

275275
#### JWT authentication
276276

@@ -430,6 +430,13 @@ Specify the source Oracle schema to replicate from:
430430
~~~
431431
</section>
432432

433+
Specify the target schema on CockroachDB in fully-qualified `database.schema` format:
434+
435+
{% include_cached copy-clipboard.html %}
436+
~~~
437+
--targetSchema defaultdb.public
438+
~~~
439+
433440
To replicate from the correct position, specify the appropriate checkpoint value.
434441

435442
<section class="filter-content" markdown="1" data-scope="postgres">
@@ -476,6 +483,7 @@ At minimum, the `replicator` command should include the following flags:
476483
replicator pglogical \
477484
--sourceConn $SOURCE \
478485
--targetConn $TARGET \
486+
--targetSchema defaultdb.public \
479487
--slotName molt_slot \
480488
--stagingSchema _replicator \
481489
--stagingCreateSchema
@@ -490,6 +498,7 @@ For detailed steps, refer to [Load and replicate]({% link molt/migrate-load-repl
490498
replicator mylogical \
491499
--sourceConn $SOURCE \
492500
--targetConn $TARGET \
501+
--targetSchema defaultdb.public \
493502
--defaultGTIDSet '4c658ae6-e8ad-11ef-8449-0242ac140006:1-29' \
494503
--stagingSchema _replicator \
495504
--stagingCreateSchema
@@ -504,8 +513,9 @@ For detailed steps, refer to [Load and replicate]({% link molt/migrate-load-repl
504513
replicator oraclelogminer \
505514
--sourceConn $SOURCE \
506515
--sourcePDBConn $SOURCE_PDB \
507-
--sourceSchema migration_schema \
508516
--targetConn $TARGET \
517+
--sourceSchema migration_schema \
518+
--targetSchema defaultdb.public \
509519
--scn 26685786 \
510520
--backfillFromSCN 26685444 \
511521
--stagingSchema _replicator \
@@ -626,7 +636,11 @@ replicator start \
626636
--tlsPrivateKey ./certs/server.key
627637
~~~
628638

629-
For detailed steps, refer to [Migration failback]({% link molt/migrate-failback.md %}).
639+
After starting `replicator`, create a CockroachDB changefeed to send changes to MOLT Replicator. For detailed steps, refer to [Migration failback]({% link molt/migrate-failback.md %}).
640+
641+
{{site.data.alerts.callout_info}}
642+
When [creating the CockroachDB changefeed]({% link molt/migrate-failback.md %}#create-the-cockroachdb-changefeed) , you specify the target schema in the webhook URL path. For PostgreSQL targets, use the fully-qualified format `/database/schema` (for example, `/migration_schema/public`). For MySQL and Oracle targets, use just the schema name (for example, `/migration_schema`).
643+
{{site.data.alerts.end}}
630644

631645
## Monitoring
632646

0 commit comments

Comments
 (0)