Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MINIFICPP-2027 Upgrade Google Cloud library to version 2.10.1 #1572

Closed
wants to merge 3 commits into from

Conversation

lordgamez
Copy link
Contributor

There were some major breaking changes in the GCP library since our previously used 1.37.0 which failed our upload test with the GCS server version 1.45.1. Upgrading the library should solve this issue.

https://issues.apache.org/jira/browse/MINIFICPP-2027


Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.

@lordgamez lordgamez marked this pull request as ready for review May 11, 2023 16:32
@lordgamez lordgamez requested a review from martinzink May 11, 2023 16:32
@lordgamez lordgamez marked this pull request as draft May 11, 2023 19:02
@szaszm
Copy link
Member

szaszm commented May 12, 2023

The windows failure looks related:


The following tests FAILED:
	 98 - DeleteGCSObjectTests.ServerGivesPermaError (Failed)
	 99 - DeleteGCSObjectTests.ServerGivesTransientErrors (Failed)
	100 - DeleteGCSObjectTests.HandlingSuccessfullDeletion (Failed)
	101 - DeleteGCSObjectTests.EmptyGeneration (Failed)
	104 - FetchGCSObjectTests.ServerError (Failed)
	105 - FetchGCSObjectTests.HappyPath (Failed)
	106 - FetchGCSObjectTests.EmptyGeneration (Failed)
	109 - GCPCredentialsTests.DefaultGCPCredentialsWithEnv (Failed)
	117 - ListGCSBucketTests.ServerGivesPermaError (Failed)
	118 - ListGCSBucketTests.ServerGivesTransientErrors (Failed)
	119 - ListGCSBucketTests.WithoutVersions (Failed)
	120 - ListGCSBucketTests.WithVersions (Failed)
	122 - PutGCSObjectTests.BucketFromAttribute (Failed)
	123 - PutGCSObjectTests.ServerGivesTransientErrors (Failed)
	124 - PutGCSObjectTests.ServerGivesPermaError (Failed)
Errors while running CTest
	125 - PutGCSObjectTests.NonRequiredPropertiesAreMissing (Failed)
	126 - PutGCSObjectTests.Crc32cMD5LocationTest (Failed)
	127 - PutGCSObjectTests.DontOverwriteTest (Failed)
	128 - PutGCSObjectTests.ValidServerSideEncryptionTest (Failed)
	130 - PutGCSObjectTests.NoContentType (Failed)
	131 - PutGCSObjectTests.ContentTypeFromAttribute (Failed)
	132 - PutGCSObjectTests.ObjectACLTest (Failed)
Error: Process completed with exit code 8.

@lordgamez
Copy link
Contributor Author

The windows failure looks related:


The following tests FAILED:
	 98 - DeleteGCSObjectTests.ServerGivesPermaError (Failed)
	 99 - DeleteGCSObjectTests.ServerGivesTransientErrors (Failed)
	100 - DeleteGCSObjectTests.HandlingSuccessfullDeletion (Failed)
	101 - DeleteGCSObjectTests.EmptyGeneration (Failed)
	104 - FetchGCSObjectTests.ServerError (Failed)
	105 - FetchGCSObjectTests.HappyPath (Failed)
	106 - FetchGCSObjectTests.EmptyGeneration (Failed)
	109 - GCPCredentialsTests.DefaultGCPCredentialsWithEnv (Failed)
	117 - ListGCSBucketTests.ServerGivesPermaError (Failed)
	118 - ListGCSBucketTests.ServerGivesTransientErrors (Failed)
	119 - ListGCSBucketTests.WithoutVersions (Failed)
	120 - ListGCSBucketTests.WithVersions (Failed)
	122 - PutGCSObjectTests.BucketFromAttribute (Failed)
	123 - PutGCSObjectTests.ServerGivesTransientErrors (Failed)
	124 - PutGCSObjectTests.ServerGivesPermaError (Failed)
Errors while running CTest
	125 - PutGCSObjectTests.NonRequiredPropertiesAreMissing (Failed)
	126 - PutGCSObjectTests.Crc32cMD5LocationTest (Failed)
	127 - PutGCSObjectTests.DontOverwriteTest (Failed)
	128 - PutGCSObjectTests.ValidServerSideEncryptionTest (Failed)
	130 - PutGCSObjectTests.NoContentType (Failed)
	131 - PutGCSObjectTests.ContentTypeFromAttribute (Failed)
	132 - PutGCSObjectTests.ObjectACLTest (Failed)
Error: Process completed with exit code 8.

Yes I moved the PR back to draft state due to this, I'm still investigating the issue

@lordgamez
Copy link
Contributor Author

Included the fix suggested in the following discussion: googleapis/google-cloud-cpp#11613 This solves the test issues on Windows

@lordgamez lordgamez marked this pull request as ready for review May 23, 2023 16:30
Copy link
Member

@martinzink martinzink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one small nitpick I think we should use our utils::Enviroment utilities to set the GOOGLE_APPLICATION_CREDENTIALS (so we dont need the google internal utility or the platform specific functions here) (I should have done this originally)

@szaszm szaszm closed this in 91594f5 Jun 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants