Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages.google_auth: 1.20.1 -> 1.22.1 #101029

Merged
merged 20 commits into from Nov 6, 2020

Conversation

@austinbutler
Copy link
Contributor

@austinbutler austinbutler commented Oct 18, 2020

Motivation for this change

Upgrades google_auth, fixes many Google Cloud packages. Will backport to 20.09 for ZHF.

https://hydra.nixos.org/build/127647323

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Oct 18, 2020

As I figured it might, this package requiring an updated google_auth opens a whole can of worms. That said, it's almost entirely various google-cloud Python packages so doesn't seem insurmountable. I'll see if I can tackle it, but if there's a particular way such a process needs to go please let me know.

2 packages marked as broken and skipped:
kmymoney python37Packages.rl-coach

40 packages failed to build:
dvc-with-remotes patroni python37Packages.google_cloud_automl python37Packages.google_cloud_bigquery python37Packages.google_cloud_bigtable python37Packages.google_cloud_container python37Packages.google_cloud_core python37Packages.google_cloud_datastore python37Packages.google_cloud_dns python37Packages.google_cloud_error_reporting python37Packages.google_cloud_firestore python37Packages.google_cloud_logging python37Packages.google_cloud_resource_manager python37Packages.google_cloud_runtimeconfig python37Packages.google_cloud_spanner python37Packages.google_cloud_storage python37Packages.google_cloud_texttospeech python37Packages.google_cloud_trace python37Packages.google_cloud_translate python37Packages.google_cloud_websecurityscanner python37Packages.tensorflow-bin_2 python38Packages.cirq python38Packages.google_cloud_automl python38Packages.google_cloud_bigquery python38Packages.google_cloud_bigtable python38Packages.google_cloud_container python38Packages.google_cloud_core python38Packages.google_cloud_datastore python38Packages.google_cloud_dns python38Packages.google_cloud_error_reporting python38Packages.google_cloud_firestore python38Packages.google_cloud_logging python38Packages.google_cloud_resource_manager python38Packages.google_cloud_runtimeconfig python38Packages.google_cloud_spanner python38Packages.google_cloud_storage python38Packages.google_cloud_texttospeech python38Packages.google_cloud_trace python38Packages.google_cloud_translate python38Packages.google_cloud_websecurityscanner

@austinbutler austinbutler marked this pull request as draft Oct 19, 2020
@ofborg ofborg bot requested a review from limeytexan Oct 19, 2020
@austinbutler austinbutler force-pushed the fix-google-cloud-kms branch from d24f006 to e3d40e5 Oct 20, 2020
@austinbutler austinbutler changed the title python3Packages.google_cloud_kms: 2.0.1 -> 2.2.0 python3Packages.google_auth: 1.20.1 -> 1.22.1 Oct 20, 2020
@austinbutler austinbutler force-pushed the fix-google-cloud-kms branch from 8bd28ed to 8098d2a Oct 20, 2020
@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Oct 20, 2020

All right, worked through all the issues except a Tensorflow package that doesn't build on master anyways.

@austinbutler austinbutler marked this pull request as ready for review Oct 20, 2020
@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Oct 28, 2020

@jonringer @costrouc mind taking a look? Also now that 20.09 is released would I still backport this?

@jonringer
Copy link
Contributor

@jonringer jonringer commented Oct 28, 2020

@jonringer @costrouc mind taking a look? Also now that 20.09 is released would I still backport this?

you may have to remove some "broken = true;", but yea. I'm fine with backporting non-major bumps. And if it doesn't cause regressions.

@jonringer
Copy link
Contributor

@jonringer jonringer commented Oct 28, 2020

only one I'm concerned about backporting:

python3Packages.google_cloud_automl: 1.0.1 -> 2.0.0

pyarrow
];

# prevent google directory from shadowing google imports
# call_api_applying_custom_retry_on_timeout requires credentials
# test_magics requires modifying sys.path
checkPhase = ''
Copy link
Contributor

@jonringer jonringer Oct 28, 2020

maybe disable this test?

builder for '/nix/store/2nm6nj24sswksj2p5gw7myyddpbywg4l-python3.8-google-cloud-bigquery-2.1.0.drv' failed with exit code 1; last 10 log lines:
  E       datetime.datetime(4567, 1, 1, 0, 0)
  E
  E       + [datetime.datetime(4567, 1, 1, 0, 0), datetime.datetime(9999, 12, 31, 0, 0)]
  E       - [datetime.datetime(4567, 1, 1, 0, 0, tzinfo=<UTC>),
  E       -  datetime.datetime(9999, 12, 31, 0, 0, tzinfo=<UTC>)]

  tests/unit/test_table.py:2294: AssertionError
  =========================== short test summary info ============================
  FAILED tests/unit/test_table.py::TestRowIterator::test_to_dataframe_timestamp_out_of_pyarrow_bounds
  ========== 1 failed, 1372 passed, 54 skipped, 1 deselected in 14.60s ===========

Copy link
Contributor

@jonringer jonringer Oct 28, 2020

pytestCheckHook would make this easy

Copy link
Contributor Author

@austinbutler austinbutler Oct 28, 2020

Strange, all tests pass on my machine. Just checked again with nix-build -A python38Packages.google_cloud_bigquery. I can disable the test, though.

Copy link
Contributor Author

@austinbutler austinbutler Oct 30, 2020

Disabled.

@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Oct 28, 2020

@jonringer @costrouc mind taking a look? Also now that 20.09 is released would I still backport this?

you may have to remove some "broken = true;", but yea. I'm fine with backporting non-major bumps. And if it doesn't cause regressions.

Not sure what you mean. I didn't explicitly add any broken = true at least. This is probably my own lack of knowledge.

only one I'm concerned about backporting:

python3Packages.google_cloud_automl: 1.0.1 -> 2.0.0

Are you still evaluating or do you want me to look into if it's possible to keep this on 1.x?

@jonringer
Copy link
Contributor

@jonringer jonringer commented Oct 29, 2020

Not sure what you mean. I didn't explicitly add any broken = true at least. This is probably my own lack of knowledge.

I did on release-20.09, as part of the release #101736

@jonringer
Copy link
Contributor

@jonringer jonringer commented Oct 29, 2020

Are you still evaluating or do you want me to look into if it's possible to keep this on 1.x?

For backporting, I would prefer not to backport this, unless it's already broken

@austinbutler austinbutler force-pushed the fix-google-cloud-kms branch from 8098d2a to b924871 Oct 30, 2020
@austinbutler austinbutler requested a review from jonringer Oct 30, 2020
@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Oct 30, 2020

Are you still evaluating or do you want me to look into if it's possible to keep this on 1.x?

For backporting, I would prefer not to backport this, unless it's already broken

Just checked and this one was not broken on 20.09. So I will keep that in mind when backporting.

@austinbutler austinbutler force-pushed the fix-google-cloud-kms branch from 87dce67 to 1706241 Nov 3, 2020
@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Nov 3, 2020

@jonringer would you mind taking another look? I think this is good to go.

@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Nov 6, 2020

@jonringer pretty please? 🙇

@jonringer
Copy link
Contributor

@jonringer jonringer commented Nov 6, 2020

oh yea, sorry, last time it was just taking longer than expected to review these packages

Copy link
Contributor

@jonringer jonringer left a comment

LGTM

failures are broken on target branch

https://github.com/NixOS/nixpkgs/pull/101029
2 packages marked as broken and skipped:
kmymoney python37Packages.rl-coach

1 package failed to build:
python37Packages.tensorflow-bin_2

118 packages built:
beancount deja-dup duplicity duply dvc-with-remotes fava gcalcli ledger2beancount lieer patroni python37Packages.beancount python37Packages.beancount_docverif python37Packages.cirq python37Packages.goobook python37Packages.google-auth-httplib2 python37Packages.google-auth-oauthlib python37Packages.google-cloud-access-context-manager python37Packages.google-cloud-iam python37Packages.google-cloud-org-policy python37Packages.google_api_core python37Packages.google_api_python_client python37Packages.google_auth python37Packages.google_cloud_asset python37Packages.google_cloud_automl python37Packages.google_cloud_bigquery python37Packages.google_cloud_bigquery_datatransfer python37Packages.google_cloud_bigtable python37Packages.google_cloud_container python37Packages.google_cloud_core python37Packages.google_cloud_dataproc python37Packages.google_cloud_datastore python37Packages.google_cloud_dlp python37Packages.google_cloud_dns python37Packages.google_cloud_error_reporting python37Packages.google_cloud_firestore python37Packages.google_cloud_iot python37Packages.google_cloud_kms python37Packages.google_cloud_language python37Packages.google_cloud_logging python37Packages.google_cloud_monitoring python37Packages.google_cloud_pubsub python37Packages.google_cloud_redis python37Packages.google_cloud_resource_manager python37Packages.google_cloud_runtimeconfig python37Packages.google_cloud_secret_manager python37Packages.google_cloud_securitycenter python37Packages.google_cloud_spanner python37Packages.google_cloud_speech python37Packages.google_cloud_storage python37Packages.google_cloud_tasks python37Packages.google_cloud_testutils python37Packages.google_cloud_texttospeech python37Packages.google_cloud_trace python37Packages.google_cloud_translate python37Packages.google_cloud_videointelligence python37Packages.google_cloud_vision python37Packages.google_cloud_websecurityscanner python37Packages.gspread python37Packages.kubernetes python37Packages.proto-plus python37Packages.pydrive python37Packages.tensorflow-tensorboard_2 python37Packages.weboob python38Packages.beancount_docverif python38Packages.cirq python38Packages.goobook python38Packages.google-auth-httplib2 python38Packages.google-auth-oauthlib python38Packages.google-cloud-access-context-manager python38Packages.google-cloud-iam python38Packages.google-cloud-org-policy python38Packages.google_api_core python38Packages.google_api_python_client python38Packages.google_auth python38Packages.google_cloud_asset python38Packages.google_cloud_automl python38Packages.google_cloud_bigquery python38Packages.google_cloud_bigquery_datatransfer python38Packages.google_cloud_bigtable python38Packages.google_cloud_container python38Packages.google_cloud_core python38Packages.google_cloud_dataproc python38Packages.google_cloud_datastore python38Packages.google_cloud_dlp python38Packages.google_cloud_dns python38Packages.google_cloud_error_reporting python38Packages.google_cloud_firestore python38Packages.google_cloud_iot python38Packages.google_cloud_kms python38Packages.google_cloud_language python38Packages.google_cloud_logging python38Packages.google_cloud_monitoring python38Packages.google_cloud_pubsub python38Packages.google_cloud_redis python38Packages.google_cloud_resource_manager python38Packages.google_cloud_runtimeconfig python38Packages.google_cloud_secret_manager python38Packages.google_cloud_securitycenter python38Packages.google_cloud_spanner python38Packages.google_cloud_speech python38Packages.google_cloud_storage python38Packages.google_cloud_tasks python38Packages.google_cloud_testutils python38Packages.google_cloud_texttospeech python38Packages.google_cloud_trace python38Packages.google_cloud_translate python38Packages.google_cloud_videointelligence python38Packages.google_cloud_vision python38Packages.google_cloud_websecurityscanner python38Packages.gspread python38Packages.kubernetes python38Packages.proto-plus python38Packages.pydrive python38Packages.tensorflow-tensorboard_2 python38Packages.weboob s3ql tuijam wal_e

@jonringer jonringer merged commit a361626 into NixOS:master Nov 6, 2020
19 checks passed
@austinbutler austinbutler deleted the fix-google-cloud-kms branch Nov 6, 2020
@austinbutler
Copy link
Contributor Author

@austinbutler austinbutler commented Nov 6, 2020

oh yea, sorry, last time it was just taking longer than expected to review these packages

All good, was just making me anxious since it was a large amount of packages and already had to merge in new changes once or twice. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants