java CDK: clean up dependencies, refactor modules #34745
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
octavia-squidington-iii
added
area/connectors
postamar
force-pushed
the
postamar/cdk-cleanup-base-pr
branch
2 times, most recently
from
c9f0502
to
5cb441e
Compare
| @@ -180,6 +180,7 @@ subprojects { subproj -> | |||
| } | |||
|
|
|||
| apply plugin: 'java' | |||
| apply plugin: 'java-test-fixtures' | |||
There was a problem hiding this comment.
This costs nothing and allows us to blanket-add lombok, junit, etc. dependencies to the fixtures sources.
| @@ -319,45 +324,27 @@ subprojects { subproj -> | |||
| finalizedBy jacocoTestReportTask | |||
| } | |||
|
|
|||
| // TODO: These should be added to the CDK or to the individual projects that need them: | |||
There was a problem hiding this comment.
Done. The dependencies have been hardcoded in the CDK, which no longer relies on deps.toml. We want to avoid significant version changes sneaking into the CDK.
| testFixturesImplementation jarName(it) | ||
| } | ||
| PRE_V017_TEST_IMPLEMENTATION.each {testImplementation jarName(it) } | ||
| PRE_V017_INTEGRATION_TEST_IMPLEMENTATION.each {integrationTestJavaImplementation jarName(it) } |
There was a problem hiding this comment.
This is required to maintain backward-compatibility with old CDK versions. We can remove these once we update all connectors to recent CDK versions.
There was a problem hiding this comment.
which is already happening for java sources : #34453
| useLocalCdk = false | ||
| cdkVersionRequired = '0.17.0' | ||
| features = ['db-destinations', 's3-destinations', 'typing-deduping', 'datastore-bigquery'] | ||
| useLocalCdk = true |
There was a problem hiding this comment.
destination-bigquery needs to be in this PR because it used to depend on destination-gcs, but no longer does.
| implementation ('com.github.airbytehq:json-avro-converter:1.1.0') { exclude group: 'ch.qos.logback', module: 'logback-classic'} | ||
|
|
||
| testImplementation 'org.apache.commons:commons-lang3:3.11' | ||
| testImplementation 'org.xerial.snappy:snappy-java:1.1.8.4' |
There was a problem hiding this comment.
Most of the code in destination-gcs has been hoisted into the CDK's s3-destinations module, which is a much better home for it because the deps are the same.
|
|
||
|
|
||
| implementation 'org.mongodb:mongodb-driver-sync:4.3.0' | ||
| } |
There was a problem hiding this comment.
This connector lacks a metadata.yaml file and is therefore dead. Removing.
| @@ -1,13 +1,22 @@ | |||
| import org.openapitools.generator.gradle.plugin.tasks.GenerateTask | |||
There was a problem hiding this comment.
This dependencies module was built from all the airbyte-* and *-oss modules, which are left-overs from the times of the monorepo. I ruled out importing things like airbyte-commons from the platform because of the considerable amount of dependencies they pull there. To be fair, dependencies are a lot less of a concern in that repo than they are here. Still, this code changes little enough that IMO it's safe to maintain two distinct copies of it.
postamar
changed the base branch from
master
to
postamar/enable-connector-tests-for-draft-prs
postamar
force-pushed
the
postamar/cdk-cleanup-base-pr
branch
from
9efadca
to
894c72a
Compare
octavia-squidington-iii
added
the
area/documentation
|
/publish-java-cdk
|
|
/approve-and-merge reason="CI all green except for destination-mongodb" |
|
Check that box |
This PR aims to change very little in terms of functionality and just moves stuff around for the most part. The CDK will be adopted (and thereby validated) in other PRs based on this one.