Source MongoDb: added support via TLS/SSL #6364
Conversation
fixed replica connection; fixed incremental read;
…it by default for other types
github-actions
bot
added
area/connectors
irynakruk
changed the title
irynakruk
changed the title
|
/test connector=connectors/source-mongodb-v2
|
|
/test connector=connectors/source-mongodb-v2
|
| case TIMESTAMP -> new BsonTimestamp(Long.parseLong(value)); | ||
| case DATE_TIME -> new BsonDateTime(new DateTime(value).getValue()); | ||
| case OBJECT_ID -> new ObjectId(value); | ||
| default -> null; |
There was a problem hiding this comment.
instead of default null why don't we return string?
Also I don't see a mapping from string or object, how come?
There was a problem hiding this comment.
I've changed to String, thanks! Also added String and Symbol.
This mapping needed only for cursor field, since it couldn't be object or array, it isn't present here.
| @@ -26,7 +26,29 @@ the Dockerfile. | |||
| We use `JUnit` for Java tests. | |||
|
|
|||
| ### Test Configuration | |||
| No specific configuration needed for testing, MongoDb Test Container is used. | |||
|
|
|||
| In order to test the MongoDb source, you need to have a standalone instance, a replica set or Atlas cluster. | |||
There was a problem hiding this comment.
it doesn't work with a docker image anymore?
There was a problem hiding this comment.
It works, but I've also added test for Atlas Cluster with ssl, I've corrected doc.
| @@ -98,13 +105,6 @@ | |||
| "default": "admin", | |||
| "examples": ["admin"], | |||
| "order": 4 | |||
| }, | |||
There was a problem hiding this comment.
you will need to make additionalProperties=true for this to be backwards compatible, and potentially handle it manually in the connector code
There was a problem hiding this comment.
Thanks a lot, I've added.
| ### TLS/SSL on a Connection | ||
|
|
||
| It is recommended to use encrypted connection. | ||
| Connection with TLS/SSL security protocol for MongoDb Atlas Cluster and Replica Set instances is enabled by default. Encryption is based on the underlying TLS/SSL support in the JDK. |
There was a problem hiding this comment.
suggest removing since it's an implementation detail
| Connection with TLS/SSL security protocol for MongoDb Atlas Cluster and Replica Set instances is enabled by default. Encryption is based on the underlying TLS/SSL support in the JDK. | |
| Connection with TLS/SSL security protocol for MongoDb Atlas Cluster and Replica Set instances is enabled by default. |
|
/test connector=connectors/source-mongodb-v2
|
|
/publish connector=connectors/source-mongodb-v2
|
What
Added support of TLS encryption by default when connecting to the MongoDB source: MongoDb Atlas Cluster and Replica Set.
For Standalone MongoDb instance added possibility to select whether to encrypt, as it may require additional configuration from customer.
Provided small fix for incremental sync.
How
For MongoDb Atlas Cluster and Replica Set added
ssl=trueto connection param.For Standalone MongoDb instance it is configurable.
Recommended reading order
MongoDbSource.javaspec.jsonPre-merge Checklist
Community member or Airbyter
airbyte_secret./gradlew :airbyte-integrations:connectors:<name>:integrationTest.README.mdbootstrap.md. See description and examplesdocs/integrations/<source or destination>/<name>.mdincluding changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>command is passing./publishcommand described here