Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
1205845
RavenDB-26046 - Add CDC Sink documentation
ayende Apr 2, 2026
9245674
RavenDB-26046 - Update CDC Sink docs for new Columns API and addition…
ayende Apr 3, 2026
173c224
Use explicit CdcColumnMapping() class name; note text columns in atta…
ayende Apr 3, 2026
8e4220b
attachment-handling: add SQL schema examples for root and embedded ta…
ayende Apr 3, 2026
2249ddb
attachment-handling: text columns use text/plain content type, binary…
ayende Apr 3, 2026
74224bb
CdcColumnType.Json: any JSON value, not just objects/arrays
ayende Apr 3, 2026
542657c
patching, troubleshooting: replace get() with load() for loading rela…
ayende Apr 3, 2026
b7694fd
Reformat long lines in code blocks; fix CdcColumnMapping Type= contin…
ayende Apr 3, 2026
13d4627
RavenDB-26046 - CDC Sink: add PostgreSQL type mapping page, fix $row/…
ayende Apr 3, 2026
4235bfe
RavenDB-26046 - Align CDC Sink docs with actual code API from PR #4698
ayende Apr 6, 2026
c48a94e
RavenDB-26046 - Add PostgreSQL, SQL Server, and MySQL overview pages
ayende Apr 6, 2026
4eeaacf
RavenDB-26046 - Add Source Schema Changes and Partial Export/Import t…
ayende Apr 6, 2026
d4753e2
RavenDB-26056 Fix broken links, address PR comments, document config …
ayende Apr 7, 2026
de13910
RavenDB-26056 Extract Source Schema Changes to dedicated page
ayende Apr 7, 2026
255254e
RavenDB-26056 MySQL: warn to apply schema changes one at a time
ayende Apr 7, 2026
cb100f5
RavenDB-26046 Address PR #2387 review comments
ayende Apr 15, 2026
e4a1285
RavenDB-26046 Fix config key names and add missing Postgres timeout s…
ayende Apr 15, 2026
f1e03d6
RavenDB-26046 Fix CDC Sink API reference: replace non-existent /verif…
Lwiel Jun 24, 2026
d49441c
RavenDB-26046 Fix deep-nesting JoinColumns to reference direct parent…
Lwiel Jun 24, 2026
9005fa9
RavenDB-26046 Fix document ID casing in examples (CollectionName is u…
Lwiel Jun 24, 2026
24a5192
RavenDB-26046 Correct REPLICA IDENTITY permission behavior (task fail…
Lwiel Jun 24, 2026
dd03158
RavenDB-26046 Remove unsupported 'set MaxFallbackTime to 0' claim; de…
Lwiel Jun 24, 2026
27cbc24
RavenDB-26046 Document OnDelete.IgnoreDeletes as REPLICA IDENTITY esc…
Lwiel Jun 24, 2026
72a08f4
RavenDB-26046 Fix slot/publication naming timing (at task creation, n…
Lwiel Jun 24, 2026
48603b6
RavenDB-26046 Normalize em dashes to spaced hyphens (match docs house…
Lwiel Jun 24, 2026
c254a25
RDoc-3836 Apply PR review comments and doc-wide style pass to CDC Sin…
Lwiel Jul 6, 2026
08e72d6
RDoc-3836 Restructure CDC Sink docs into Source Database Setup and Ma…
Lwiel Jul 6, 2026
c0da880
RDoc-3836 Split C#-only CDC Sink pages into csharp language partials
Lwiel Jul 6, 2026
68aceff
RDoc-3836 Apply second round of CDC Sink PR review comments
Lwiel Jul 6, 2026
f089ad0
RDoc-3836 Flatten nested clause in CDC Sink attachment-handling intro
Lwiel Jul 7, 2026
66c141d
RDoc-3836 Apply further review comments
Lwiel Jul 7, 2026
3a84460
RDoc-3836 Apply editorial updates to: overview & how-it-works
Danielle9897 Jul 7, 2026
45bf68b
RDoc-3836 Added article: "Create CDC Sink Task":
Danielle9897 Jul 8, 2026
c5dfa5a
RDoc-3836 Added article: "Update CDC Sink Task"
Danielle9897 Jul 8, 2026
bf30b88
RDoc-3836 Remove redundant files
Danielle9897 Jul 8, 2026
b1682d2
RDoc-3836 Added articles: "Enable/Disable CDC Sink Task" + "Update CD…
Danielle9897 Jul 8, 2026
16879bb
RDoc-3836 remove link to the removed studio-ui.mdx file
Danielle9897 Jul 8, 2026
c8d7950
RDoc-3836 Added article: "Get Task Info"
Danielle9897 Jul 8, 2026
8c2d826
RDoc-3836 Enhance section: "State persistence" + add images
Danielle9897 Jul 8, 2026
68b75dc
RDoc-3836 Add article: "Delete CDC Sink Task"
Danielle9897 Jul 8, 2026
80657c6
RDoc-3836 Add article: "Export/Import Task"
Danielle9897 Jul 9, 2026
352db91
RDoc-3836 Add article: "Backup/Restore Task"
Danielle9897 Jul 9, 2026
d18cd4d
RDoc-3836 Apply editorial updates to: configuration-syntax
Danielle9897 Jul 9, 2026
2c98d9c
RDoc-3836 Rename: configuration-syntax => configuration-reference
Danielle9897 Jul 9, 2026
f705fd9
RDoc-3836 Remove link to non-existing file: api-reference
Danielle9897 Jul 9, 2026
b5c3430
RDoc-3836 Apply editorial updates to: schema-design
Danielle9897 Jul 9, 2026
bd7551f
RDoc-3836 Apply editorial updates to: embedded-tables
Danielle9897 Jul 9, 2026
6e29e2b
RDoc-3836 Apply editorial updates to: linked-tables
Danielle9897 Jul 9, 2026
eead30a
RDoc-3836 Apply editorial updates to: column-mapping
Danielle9897 Jul 9, 2026
d1fb520
RDoc-3836 Apply editorial updates to: attachment-handling
Danielle9897 Jul 10, 2026
2fb4fde
RDoc-3836 Apply editorial updates to: property-retention
Danielle9897 Jul 10, 2026
c14d6d6
RDoc-3836 Add directory "Document Modeling" with 5 articles
Danielle9897 Jul 10, 2026
2f4f8e6
RDoc-3836 fix links
Danielle9897 Jul 10, 2026
da40678
RDoc-3836 fix "see also" links
Danielle9897 Jul 10, 2026
20aaec8
RDoc-3836 fix article: Patching
Danielle9897 Jul 12, 2026
7b29e22
RDoc-3836 fix article: Handling deletes
Danielle9897 Jul 12, 2026
da3ce09
RDoc-3836 fix links
Danielle9897 Jul 12, 2026
fb624c7
RDoc-3836 fix file: Source scheme changes
Danielle9897 Jul 12, 2026
bcec42c
RDoc-3836 fix article: Failover and consistency
Danielle9897 Jul 12, 2026
5190129
RDoc-3836 fix article: Server configuration
Danielle9897 Jul 12, 2026
91d221b
RDoc-3836 fix article: Monitoring
Danielle9897 Jul 12, 2026
2715f02
RDoc-3836 fix article: Troubleshooting
Danielle9897 Jul 12, 2026
421534a
RDoc-3836 fix article: PostgreSQL > Overview
Danielle9897 Jul 13, 2026
1407a9e
RDoc-3836 fix article: PostgreSQL > Prerequisites
Danielle9897 Jul 13, 2026
abe9ad7
RDoc-3836 fix article: PostgreSQL > WAL configuration
Danielle9897 Jul 13, 2026
0f4f6c4
RDoc-3836 fix article: PostgreSQL > Permissions and roles
Danielle9897 Jul 13, 2026
fea3075
RDoc-3836 fix article: PostgreSQL > Initial setup
Danielle9897 Jul 13, 2026
63d9ac9
RDoc-3836 fix article: PostgreSQL > Replica identity
Danielle9897 Jul 13, 2026
db898fa
RDoc-3836 fix article: PostgreSQL > Manual replica identity setup
Danielle9897 Jul 13, 2026
0d2f953
RDoc-3836 fix article: PostgreSQL > Cleanup and maintenance
Danielle9897 Jul 13, 2026
4bb1ba6
RDoc-3836 add content to: Failover, Backup/Restore, & Troubleshooting
Danielle9897 Jul 13, 2026
8eb60a5
RDoc-3836 fix article: PostgreSQL > Monitoring PostgreSQL
Danielle9897 Jul 13, 2026
ce7b877
RDoc-3836 fix article: SQL Server > Overview
Danielle9897 Jul 13, 2026
3a3b46c
RDoc-3836 fix article: MySQL > Overview
Danielle9897 Jul 13, 2026
787b249
RDoc-3836 fix article: PostgreSQL > Example > Simple migration
Danielle9897 Jul 13, 2026
8069a04
RDoc-3836 fix article: PostgreSQL > Example > Running balance
Danielle9897 Jul 14, 2026
324bbd8
RDoc-3836 fix article: PostgreSQL > Example > Denormalization
Danielle9897 Jul 14, 2026
7773a68
RDoc-3836 fix article: PostgreSQL > Example > Complex nesting
Danielle9897 Jul 14, 2026
7ddb686
RDoc-3836 rebase + fix links
Danielle9897 Jul 15, 2026
a1e4cb6
RDoc-3836 add operations to list + fix link
Danielle9897 Jul 15, 2026
a7a9a19
RDoc-3836 added links to cdc
Danielle9897 Jul 15, 2026
85831e3
RDoc-3836 add info regarding attachments + remove the internal operat…
Danielle9897 Jul 15, 2026
36d6f11
RDoc-3836 fix tables layout in configuration reference
Danielle9897 Jul 15, 2026
68208d1
RDoc-3977 Move examples to the root folder
Danielle9897 Jul 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -303,6 +303,10 @@ Task<Operation> SendAsync(IMaintenanceOperation<OperationIdResult> operation,
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; UpdateEtlOperation
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [ResetEtlOperation](../../../client-api/operations/maintenance/etl/reset-etl.mdx)

* **Sink tasks**:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [AddCdcSinkOperation](../../../server/ongoing-tasks/cdc-sink/manage-cdc-sink-tasks/create-task.mdx#create-a-cdc-sink-task-via-client-api)
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [UpdateCdcSinkOperation](../../../server/ongoing-tasks/cdc-sink/manage-cdc-sink-tasks/update-task.mdx#update-a-cdc-sink-task-via-client-api)

* **AI tasks**:
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; [AddEmbeddingsGenerationOperation](../../../ai-integration/generating-embeddings/embeddings-generation-task.mdx#configuring-an-embeddings-generation-task---from-the-client-api)

Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/connection-strings/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ RavenDB supports the following connection string types:
| Type | Connects to | Used by |
| - | - | - |
| **RavenDB** | Another RavenDB database | [RavenDB ETL](../../server/ongoing-tasks/etl/raven.mdx), [External Replication](../../server/ongoing-tasks/external-replication.mdx),<br/>and [Hub/Sink Replication](../../server/ongoing-tasks/hub-sink-replication.mdx) |
| **SQL** | A relational database | [SQL ETL](../../server/ongoing-tasks/etl/sql.mdx) and CDC Sink tasks |
| **SQL** | A relational database | [SQL ETL](../../server/ongoing-tasks/etl/sql.mdx) and [CDC Sink](../../server/ongoing-tasks/cdc-sink/overview.mdx) |
| **Snowflake** | A Snowflake warehouse | [Snowflake ETL](../../server/ongoing-tasks/etl/snowflake.mdx) |
| **OLAP** | Local or cloud storage | [OLAP ETL](../../server/ongoing-tasks/etl/olap.mdx) |
| **Elasticsearch** | An Elasticsearch cluster | [Elasticsearch ETL](../../server/ongoing-tasks/etl/elasticsearch.mdx) |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,8 @@ The specific fields and complete examples are documented in the relevant task or
and [Hub/Sink Replication](../../../../server/ongoing-tasks/hub-sink-replication.mdx).
See [Add a RavenDB connection string](../../../../server/ongoing-tasks/etl/raven.mdx#add-a-ravendb-connection-string).

* **SQL** - used by [SQL ETL](../../../../server/ongoing-tasks/etl/sql.mdx) and CDC Sink tasks.
* **SQL** - used by [SQL ETL](../../../../server/ongoing-tasks/etl/sql.mdx)
and [CDC Sink](../../../../server/ongoing-tasks/cdc-sink/overview.mdx).
See [Add an SQL connection string](../../../../server/ongoing-tasks/etl/sql.mdx#add-an-sql-connection-string).

* **Snowflake** - used by [Snowflake ETL](../../../../server/ongoing-tasks/etl/snowflake.mdx).
Expand Down
2 changes: 1 addition & 1 deletion docs/querying/rql/what-is-rql.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ The following keywords and methods are available in RQL:
- [long](../../querying/sorting-query-results/sort-query-results.mdx)
- [double](../../querying/sorting-query-results/sort-query-results.mdx)
- [alphaNumeric](../../querying/sorting-query-results/sort-query-results.mdx)
- [NULLS FIRST | NULLS LAST](../../querying/sorting-query-results/sort-query-results.mdx) // TODO...
- [NULLS FIRST | NULLS LAST](../../querying/sorting-query-results/sort-query-results.mdx#control-position-of-null-values)
- [random()](../../querying/sorting-query-results/sort-query-results.mdx#order-by-random)
- [score()](../../querying/sorting-query-results/sort-query-results.mdx#order-by-score)
- [spatial.distance()](../../client-api/session/querying/how-to-make-a-spatial-query.mdx#spatial-sorting)
Expand Down
4 changes: 4 additions & 0 deletions docs/server/ongoing-tasks/cdc-sink/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"position": 2,
"label": "CDC Sink"
}
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,290 @@
import Admonition from '@theme/Admonition';
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Panel from '@site/src/components/Panel';
import ContentFrame from '@site/src/components/ContentFrame';

<Admonition type="note" title="">

* CDC Sink lets you configure how DELETE events are handled for root tables and embedded tables
by setting the `OnDelete` property (`CdcSinkOnDeleteConfig`) on the table configuration.

* By default, a DELETE event deletes the corresponding RavenDB document.
For embedded tables, it removes the item from an array or map, or sets a `Value`-type property to `null`.
The `OnDelete` configuration changes this behavior.

* In this article:
* [Default DELETE behavior](#default-delete-behavior)
* [OnDelete configuration](#ondelete-configuration)
* [OnDelete behavior summary](#ondelete-behavior-summary)
* [Common patterns](#common-patterns)
* [Archive](#archive)
* [Audit trail (root document)](#audit-trail-root-document)
* [Silent ignore](#silent-ignore)
* [OnDelete for embedded tables](#ondelete-for-embedded-tables)
* [DELETE routing and REPLICA IDENTITY](#delete-routing-and-replica-identity)

</Admonition>

<Panel heading="Default DELETE behavior">

When `OnDelete` is not configured, CDC Sink applies the delete automatically:

* **Root table DELETE** - Deletes the corresponding RavenDB document.
* **Embedded table DELETE (Array/Map)** - Removes the item from the parent document's array or map.
* **Embedded table DELETE (Value)** - Sets the `Value`-type property on the parent document to `null`.

</Panel>

<Panel heading="OnDelete configuration">

`CdcSinkOnDeleteConfig` has two properties:

<Tabs>
<TabItem value="csharp" label="csharp">
```csharp
public class CdcSinkOnDeleteConfig
{
// JavaScript patch that runs when a DELETE event arrives
// Available variables: this, $row, $old
public string Patch { get; set; }

// When true, CDC Sink does not apply its automatic delete/removal
public bool IgnoreDeletes { get; set; }
}
```
</TabItem>
</Tabs>

* If `Patch` is set, it runs before CDC Sink applies or skips its automatic delete handling.
* If `IgnoreDeletes = true`, CDC Sink skips the automatic delete/removal after the patch runs.
* If `IgnoreDeletes = false` (default), CDC Sink applies the automatic delete/removal after the patch runs.

A DELETE event is discarded outright only when `IgnoreDeletes = true` **and** `Patch = null`
(that is, `OnDelete.IgnoreDeletes = true` with no `OnDelete.Patch`).
If a delete patch is configured, the event is still routed so the patch can run - even when `IgnoreDeletes = true`.

<Admonition type="note" title="">
When `IgnoreDeletes = true`, CDC Sink skips its automatic delete/removal but still saves changes made to `this` by the patch.
For root tables, `this` is the document being handled. For embedded tables, `this` is the parent document.
</Admonition>

</Panel>

<Panel heading="OnDelete behavior summary">

| IgnoreDeletes | Patch | Behavior |
|---------------|-------|----------|
| `false` | null | CDC Sink applies the normal delete/removal (default). |
| `false` | set | Patch runs, and CDC Sink still applies the normal delete/removal. |
| `true` | null | DELETE event is discarded silently; no patch runs. |
| `true` | set | Patch runs, and CDC Sink skips its automatic delete/removal. |

When CDC Sink applies its automatic delete/removal (that is, when `IgnoreDeletes = false`):

* For root tables, the delete patch runs before the document is deleted.
* For embedded tables, CDC Sink removes the item (or sets a `Value`-type property to `null`) before running the patch,
and `$old` contains the removed item.

When `IgnoreDeletes = true`, CDC Sink skips its automatic delete/removal.
If a patch is configured, its changes to `this` are saved unless the patch explicitly calls `del()` or `put()`.

</Panel>

<Panel heading="Common patterns">

<ContentFrame>

### Archive

Keep the document in RavenDB but mark it as archived.
The patch sets archive fields, and `IgnoreDeletes = true` prevents CDC Sink's automatic deletion:

<Tabs>
<TabItem value="csharp" label="csharp">
```csharp
OnDelete = new CdcSinkOnDeleteConfig
{
IgnoreDeletes = true,
Patch = @"
this.Archived = true;
this.ArchivedAt = new Date().toISOString();
"
}
```
</TabItem>
</Tabs>

The document remains in RavenDB with `Archived = true`.
Queries can filter on `Archived` to exclude archived records.

</ContentFrame>

<ContentFrame>

### Audit trail (root document)

When a root document is deleted, use `OnDelete.Patch` to write an audit document with `put()` before CDC Sink applies the delete.

Because `IgnoreDeletes` is `false` by default, CDC Sink deletes `this` after the patch runs.
Changes made only to `this` do not remain; use `put()` to create a separate document that survives the delete:

<Tabs>
<TabItem value="csharp" label="csharp">
```csharp
OnDelete = new CdcSinkOnDeleteConfig
{
// IgnoreDeletes defaults to false - CDC Sink deletes the RavenDB document after the patch
Patch = @"
put('DeletedOrders/' + id(this), {
OriginalId: id(this),
Customer: this.Customer,
Total: this.Total,
DeletedAt: new Date().toISOString(),
'@metadata': { '@collection': 'DeletedOrders' }
});
"
}
```
</TabItem>
</Tabs>

The patch creates a document in the `DeletedOrders` collection before `this` is deleted.
That audit document is not removed by this delete operation.

</ContentFrame>

<ContentFrame>

### Silent ignore

Set `IgnoreDeletes = true` and leave `Patch` unset to discard DELETE events without running a patch.
Use this for append-only data where deletes should not remove the RavenDB document or embedded item:

<Tabs>
<TabItem value="csharp" label="csharp">
```csharp
OnDelete = new CdcSinkOnDeleteConfig
{
// No patch - DELETE events are silently discarded
IgnoreDeletes = true
}
```
</TabItem>
</Tabs>

</ContentFrame>

</Panel>

<Panel heading="OnDelete for embedded tables">

The same `OnDelete` configuration works on embedded tables.
For embedded tables:

* `Patch`
runs on the parent document, not on the embedded item.
* `$old`
contains the embedded item's last known state before deletion.
* `IgnoreDeletes = true`
prevents CDC Sink from automatically removing the array/map item or setting a `Value`-type property to `null`.

---

**Example: Keep deleted items in an audit array**

Rather than removing a deleted line item from the array, move it to a separate `DeletedLines` property:

<Tabs>
<TabItem value="csharp" label="csharp">
```csharp
new CdcSinkEmbeddedTableConfig
{
SourceTableName = "order_lines",
PropertyName = "Lines",
// ...
OnDelete = new CdcSinkOnDeleteConfig
{
IgnoreDeletes = true,
Patch = @"
// Remove from active Lines
this.Lines = (this.Lines || [])
.filter(l => l.LineId !== $old?.LineId);

// Add to DeletedLines audit array
this.DeletedLines = this.DeletedLines || [];
this.DeletedLines.push({
LineId: $old?.LineId,
Product: $old?.Product,
Quantity: $old?.Quantity,
DeletedAt: new Date().toISOString()
});
"
}
}
```
</TabItem>
</Tabs>

With `IgnoreDeletes = true`, CDC Sink does not automatically remove the item -
the patch takes full control of both the `Lines` array and the `DeletedLines` audit trail.

---

**Example: Run a patch but still remove the item**

To run side-effect logic on DELETE while still removing the item from the array,
leave `IgnoreDeletes` as `false` (the default). CDC Sink removes the item, and the patch can use `$old` to inspect the removed item:

<Tabs>
<TabItem value="csharp" label="csharp">
```csharp
new CdcSinkEmbeddedTableConfig
{
SourceTableName = "order_lines",
PropertyName = "Lines",
// ...
OnDelete = new CdcSinkOnDeleteConfig
{
// IgnoreDeletes = false (default) - CDC Sink removes the item
Patch = @"
// Log total for the line being removed
this.RemovedTotal = (this.RemovedTotal || 0)
+ ($old?.UnitPrice || 0) * ($old?.Quantity || 0);
"
}
}
```
</TabItem>
</Tabs>

The `Lines` item is removed by CDC Sink. The patch only handles the side-effect logic.

</Panel>

<Panel heading="DELETE routing and REPLICA IDENTITY">

For embedded-table DELETE events, CDC Sink must know which parent document and embedded item to update.
The DELETE event must include the embedded table's primary key columns and the join columns that identify the parent.

For PostgreSQL, the default `REPLICA IDENTITY` includes only primary key columns in DELETE events.
If an embedded table's join columns are not part of its primary key, configure `REPLICA IDENTITY` so DELETE events include those join columns.

<Admonition type="note" title="">
This requirement is PostgreSQL-specific.
Other databases may include the needed DELETE-event columns by default or use a different mechanism.
See [REPLICA IDENTITY](../../../../server/ongoing-tasks/cdc-sink/source-database-setup/postgres/replica-identity.mdx) (PostgreSQL).
</Admonition>

---

**Skipping DELETE routing for embedded tables**:

* To discard embedded-table DELETE events entirely, set `OnDelete.IgnoreDeletes = true` and leave `OnDelete.Patch` unset.
In that configuration, CDC Sink does not need to route the DELETE event to a parent document, so PostgreSQL `REPLICA IDENTITY` changes are not needed for that delete behavior.

* This skip applies only when no patch is set.
If `OnDelete.Patch` is configured, the DELETE event is still routed and processed so the patch can run -
so the join columns (and `REPLICA IDENTITY`, if the join columns aren't in the primary key) are still required.

</Panel>
Loading
Loading