Skip to content

Split GCP Plugins into multiple modules#1106

Merged
fernst merged 1 commit intodevelopfrom
gcp-modules-2
Aug 15, 2022
Merged

Split GCP Plugins into multiple modules#1106
fernst merged 1 commit intodevelopfrom
gcp-modules-2

Conversation

@fernst
Copy link
Copy Markdown
Contributor

@fernst fernst commented Aug 12, 2022

Applied latest repository changes and cleanup

Moved E2E tests to separate submodule

Applied latest repository changes

@fernst fernst added the build Trigger unit test build label Aug 12, 2022
Comment thread google-cloud-bigquery/pom.xml Outdated
<name>Google Cloud BigQuery Plugins</name>
<description>BigQuery GCP Plugins</description>

<distributionManagement>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these change for submodules? I think it would keep the submodule pom files simpler if the common parts like distributionManagement, repository, checkstyle, build and even the common dependencies like cdap-api is moved to the top level pom. wdyt?

Copy link
Copy Markdown
Contributor Author

@fernst fernst Aug 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great point. I will consolidate some of the common dependencies in the parent pom (CDAP API modules that are in the provided scope and test dependencies)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Move the section and section also to parent pom?

@fernst fernst force-pushed the gcp-modules-2 branch 2 times, most recently from 9701dd3 to a67d218 Compare August 14, 2022 23:51
@yaojiefeng
Copy link
Copy Markdown
Contributor

How will we handle the import for pipeline json prior to 6.8.0? With this change, the pipeline json won't be able to find the correct artifact in the new instance.

@fernst
Copy link
Copy Markdown
Contributor Author

fernst commented Aug 15, 2022

One of the follow up items to this change is to add logic on the pipeline upgrader to handle pre-6.8 GCP artifacts

Applied latest repository changes and cleanup

Moved E2E tests to separate submodule

Applied latest repository changes

Simplied pom

Renamed storage module
@fernst
Copy link
Copy Markdown
Contributor Author

fernst commented Aug 15, 2022

TODOs after this PR is merged:

  1. Update E2E tests to install all required jars to run E2E tests. Tests are currently disabled.

  2. Update CDAP Upgrade logic to map old artifact name google-cloud to new artifact names google-cloud-*

@fernst fernst merged commit 8c64373 into develop Aug 15, 2022
@fernst fernst deleted the gcp-modules-2 branch August 17, 2022 16:58
@fernst fernst restored the gcp-modules-2 branch August 17, 2022 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Trigger unit test build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants