Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/source.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ The following document represents all possible fields that a change stream respo

### Source Connector Configuration Properties


| Name | Description | Type | Default | Valid Values | Importance |
|-----------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|-----------------------------------------------------------|------------------------------------------------|------------|
| connection.uri | The connection URI as supported by the official drivers. eg: ``mongodb://user@pass@locahost/``. | string | mongodb://localhost:27017,localhost:27018,localhost:27019 | A valid connection string | high |
Expand All @@ -66,7 +67,7 @@ The following document represents all possible fields that a change stream respo
| pipeline | An inline JSON array with objects describing the pipeline operations to run. Example: `[{"$match": {"operationType": "insert"}}, {"$addFields": {"Kafka": "Rules!"}}]` | string | [] | A valid JSON array | medium |
| collation | The json representation of the Collation options to use for the change stream. Use the `Collation.asDocument().toJson()` to create the specific json representation. | string | "" | A valid JSON document representing a collation | high |
| batch.size | The cursor batch size. | int | 0 | [0,...] | medium |
| change.stream.full.document | Determines what to return for update operations when using a Change Stream. When set to 'updateLookup', the change stream for partial updates will include both a delta describing the changes to the document as well as a copy of the entire document that was changed from *some time* after the change occurred. | string | "" | An empty string OR [default, updatelookup] | high |
| change.stream.full.document | Determines what to return for update operations when using a Change Stream. When set to 'updateLookup', the change stream for partial updates will include both a delta describing the changes to the document as well as a copy of the entire document that was changed from *some time* after the change occurred. | string | "" | An empty string OR [default, updateLookup] | high |
| poll.await.time.ms | The amount of time to wait before checking for new results on the change stream | long | 5000 | [1,...] | low |
| poll.max.batch.size | Maximum number of change stream documents to include in a single batch when polling for new data. This setting can be used to limit the amount of data buffered internally in the connector. | int | 1000 | [1,...] | low |
| topic.prefix | Prefix to prepend to database & collection names to generate the name of the Kafka topic to publish data to. | string | "" | | low |
Expand All @@ -75,6 +76,7 @@ The following document represents all possible fields that a change stream respo
| copy.existing.queue.size | The max size of the queue to use when copying data. | int | 16000 | [1,...] | medium |



### Custom pipelines

The following example can be used to only observe inserted files:
Expand Down