Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure sdk update #60435

Merged
merged 82 commits into from Jul 16, 2019

Conversation

@mwilsoninsight
Copy link
Contributor

commented Apr 29, 2019

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

All modules build without error in python 3.7

@mwilsoninsight mwilsoninsight requested a review from FRidh as a code owner Apr 29, 2019

@ofborg ofborg bot added the 6.topic: python label Apr 29, 2019

@kalbasit

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

The commit messages do not follow CONTRIBUTING.md.

@mwilsoninsight mwilsoninsight force-pushed the mwilsoninsight:azure-sdk-update branch from b098d27 to f629523 Apr 29, 2019

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 29, 2019

The commit messages do not follow CONTRIBUTING.md.

Yep- sorry about that. Originally I wanted to save on the number of commits, but that doesn't quite work for following the contributing guide.

Thanks!

@kalbasit
Copy link
Member

left a comment

The commits look good now, thank you! Eval is failing though due to a syntax error. Please fix it and I'll proceed to have it tested on ofborg.

syntax error, unexpected '{', at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/eval-1-lassulus.ewr1.nix.ci/pkgs/development/python-modules/azure-servicemanagement-legacy/default.nix:37:15
@kalbasit

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

Alright, let's do first Python2 🤞

@GrahamcOfBorg build python2Packages.azure-applicationinsights python2Packages.azure-batch python2Packages.azure-common python2Packages.azure-cosmosdb-nspkg python2Packages.azure-cosmosdb-table python2Packages.azure-datalake-store python2Packages.azure-eventgrid python2Packages.azure-graphrbac python2Packages.azure-keyvault python2Packages.azure-loganalytics python2Packages.azure-mgmt-advisor python2Packages.azure-mgmt-applicationinsights python2Packages.azure-mgmt-authorization python2Packages.azure-mgmt-batch python2Packages.azure-mgmt-batchai python2Packages.azure-mgmt-billing python2Packages.azure-mgmt-cdn python2Packages.azure-mgmt-cognitiveservices python2Packages.azure-mgmt-commerce python2Packages.azure-mgmt-compute python2Packages.azure-mgmt-consumption python2Packages.azure-mgmt-containerinstance python2Packages.azure-mgmt-containerservice python2Packages.azure-mgmt-cosmosdb python2Packages.azure-mgmt-datafactory python2Packages.azure-mgmt-datalake-analytics python2Packages.azure-mgmt-datalake-nspkg python2Packages.azure-mgmt-datalake-store python2Packages.azure-mgmt-datamigration python2Packages.azure-mgmt-devspaces python2Packages.azure-mgmt-devtestlabs python2Packages.azure-mgmt-dns python2Packages.azure-mgmt-eventgrid python2Packages.azure-mgmt-eventhub python2Packages.azure-mgmt-hanaonazure python2Packages.azure-mgmt-iotcentral python2Packages.azure-mgmt-iothub python2Packages.azure-mgmt-iothubprovisioningservices python2Packages.azure-mgmt-keyvault python2Packages.azure-mgmt-loganalytics python2Packages.azure-mgmt-logic python2Packages.azure-mgmt-machinelearningcompute python2Packages.azure-mgmt-managementgroups python2Packages.azure-mgmt-managementpartner python2Packages.azure-mgmt-maps python2Packages.azure-mgmt-marketplaceordering python2Packages.azure-mgmt-media python2Packages.azure-mgmt-monitor python2Packages.azure-mgmt-msi python2Packages.azure-mgmt-network python2Packages.azure-mgmt-notificationhubs python2Packages.azure-mgmt-policyinsights python2Packages.azure-mgmt-powerbiembedded python2Packages.azure-mgmt-rdbms python2Packages.azure-mgmt-recoveryservices python2Packages.azure-mgmt-recoveryservicesbackup python2Packages.azure-mgmt-redis python2Packages.azure-mgmt-relay python2Packages.azure-mgmt-reservations python2Packages.azure-mgmt-resource python2Packages.azure-mgmt-scheduler python2Packages.azure-mgmt-search python2Packages.azure-mgmt-servicebus python2Packages.azure-mgmt-servicefabric python2Packages.azure-mgmt-signalr python2Packages.azure-mgmt-sql python2Packages.azure-mgmt-storage python2Packages.azure-mgmt-subscription python2Packages.azure-mgmt-trafficmanager python2Packages.azure-mgmt-web python2Packages.azure-servicebus python2Packages.azure-servicefabric python2Packages.azure-servicemanagament-legacy python2Packages.azure-storage-blob python2Packages.uamqp

EDIT: Eval still failing, @mwilsoninsight can you PTAL? Thanks!

@mwilsoninsight mwilsoninsight force-pushed the mwilsoninsight:azure-sdk-update branch from 9be969e to d8ea2bf Apr 30, 2019

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

I had a typo in keyvault =/

Just force pushed the fix

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

I see the same typo in service{bus,fabric} and uamqp- pushing those fixes shortly

@mwilsoninsight mwilsoninsight force-pushed the mwilsoninsight:azure-sdk-update branch from d8ea2bf to 6dc0690 Apr 30, 2019

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

@mwilsoninsight can you PTAL? Thanks!

@kalbasit, yep- had a typo in the metadata that found its way into a few other files due to copy/pasta. It just passed all the checks, so I think it should be all set 🤞

@nixos-discourse

This comment has been minimized.

Copy link

commented Apr 30, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/nixops-travis-ci-test-failure-on-azure-private-ip-pr/2441/12

@kalbasit

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

@GrahamcOfBorg build python2Packages.azure-applicationinsights python2Packages.azure-batch python2Packages.azure-common python2Packages.azure-cosmosdb-nspkg python2Packages.azure-cosmosdb-table python2Packages.azure-datalake-store python2Packages.azure-eventgrid python2Packages.azure-graphrbac python2Packages.azure-keyvault python2Packages.azure-loganalytics python2Packages.azure-mgmt-advisor python2Packages.azure-mgmt-applicationinsights python2Packages.azure-mgmt-authorization python2Packages.azure-mgmt-batch python2Packages.azure-mgmt-batchai python2Packages.azure-mgmt-billing python2Packages.azure-mgmt-cdn python2Packages.azure-mgmt-cognitiveservices python2Packages.azure-mgmt-commerce python2Packages.azure-mgmt-compute python2Packages.azure-mgmt-consumption python2Packages.azure-mgmt-containerinstance python2Packages.azure-mgmt-containerservice python2Packages.azure-mgmt-cosmosdb python2Packages.azure-mgmt-datafactory python2Packages.azure-mgmt-datalake-analytics python2Packages.azure-mgmt-datalake-nspkg python2Packages.azure-mgmt-datalake-store python2Packages.azure-mgmt-datamigration python2Packages.azure-mgmt-devspaces python2Packages.azure-mgmt-devtestlabs python2Packages.azure-mgmt-dns python2Packages.azure-mgmt-eventgrid python2Packages.azure-mgmt-eventhub python2Packages.azure-mgmt-hanaonazure python2Packages.azure-mgmt-iotcentral python2Packages.azure-mgmt-iothub python2Packages.azure-mgmt-iothubprovisioningservices python2Packages.azure-mgmt-keyvault python2Packages.azure-mgmt-loganalytics python2Packages.azure-mgmt-logic python2Packages.azure-mgmt-machinelearningcompute python2Packages.azure-mgmt-managementgroups python2Packages.azure-mgmt-managementpartner python2Packages.azure-mgmt-maps python2Packages.azure-mgmt-marketplaceordering python2Packages.azure-mgmt-media python2Packages.azure-mgmt-monitor python2Packages.azure-mgmt-msi python2Packages.azure-mgmt-network python2Packages.azure-mgmt-notificationhubs python2Packages.azure-mgmt-policyinsights python2Packages.azure-mgmt-powerbiembedded python2Packages.azure-mgmt-rdbms python2Packages.azure-mgmt-recoveryservices python2Packages.azure-mgmt-recoveryservicesbackup python2Packages.azure-mgmt-redis python2Packages.azure-mgmt-relay python2Packages.azure-mgmt-reservations python2Packages.azure-mgmt-resource python2Packages.azure-mgmt-scheduler python2Packages.azure-mgmt-search python2Packages.azure-mgmt-servicebus python2Packages.azure-mgmt-servicefabric python2Packages.azure-mgmt-signalr python2Packages.azure-mgmt-sql python2Packages.azure-mgmt-storage python2Packages.azure-mgmt-subscription python2Packages.azure-mgmt-trafficmanager python2Packages.azure-mgmt-web python2Packages.azure-servicebus python2Packages.azure-servicefabric python2Packages.azure-servicemanagament-legacy python2Packages.azure-storage-blob python2Packages.uamqp

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

Blerg- I see a failure with python2. I've only built with 3.7 so far, so tomorrow I'll start building for 3.6, 3.5, and 2.7 and sort out any failures :salute:

@kalbasit

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

@mwilsoninsight thank you for working on this! Please ping me when it's ready for another review.

@mwilsoninsight mwilsoninsight force-pushed the mwilsoninsight:azure-sdk-update branch from 6dc0690 to b9adf50 Apr 30, 2019

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

@mwilsoninsight thank you for working on this! Please ping me when it's ready for another review.

@kalbasit with today's work, it builds successfully against python2.7.

python3.6 threw a deprecation warning while building pylint =/

If context helps, I use a build.nix file with each version of python made using withPackages (ps: [<each of these packages>]) for testing builds/installs

3.5 threw up with the error that it didn't support contextvars-2.4 (which I don't think any of these modules reference...)

@FRidh FRidh self-assigned this Apr 30, 2019

@FRidh

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

Thanks for the PR. Please have a look also at #49657 (comment). How are we going to maintain this large set of packages?

@FRidh

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

What's up with removing bytecode? That should not be necessary as we create it during build time. Or was this bytecode part of the sdist?

@mwilsoninsight

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

@FRidh the rm statements were necessary to get the modules to build without complaining about colliding files.

These colliding files were of the same path but respective to azure-common and <colliding package>. The files in question were .py and .pyc files that existed in both locations.

The error I encountered primarily regarded namespacing concerns, so I figured that because azure-common was a propogatedBuildInput for many packages, that it may be creating the same file in those various locations throughout the nix store, but that the existence of the same files in those libs causes python confusion during namespace declaration.

To be totally honest, I'm not a Pythonista by trade, so I'm doing some major guess-work here.

@FRidh

This comment has been minimized.

Copy link
Member

commented Apr 30, 2019

When did you encounter the collisions, when you used python.withPackages?

mwilsoninsight and others added some commits Apr 29, 2019

pythonPackages.azure-mgmt-nspkg: refactor
metadata points to GitHub
pythonPackages.azure-nspkg: refactor
updated license and homepage

@kalbasit kalbasit force-pushed the mwilsoninsight:azure-sdk-update branch from 7e056a0 to 736f645 Jul 16, 2019

@kalbasit

This comment has been minimized.

Copy link
Member

commented Jul 16, 2019

Darwin issue fixed, retrying borg.

ofborg command

@GrahamcOfBorg build python2Packages.azure-applicationinsights python2Packages.azure-batch python2Packages.azure-common python2Packages.azure-cosmosdb-nspkg python2Packages.azure-cosmosdb-table python2Packages.azure-datalake-store python2Packages.azure-eventgrid python2Packages.azure-graphrbac python2Packages.azure-keyvault python2Packages.azure-loganalytics python2Packages.azure-mgmt-advisor python2Packages.azure-mgmt-applicationinsights python2Packages.azure-mgmt-authorization python2Packages.azure-mgmt-batch python2Packages.azure-mgmt-batchai python2Packages.azure-mgmt-billing python2Packages.azure-mgmt-cdn python2Packages.azure-mgmt-cognitiveservices python2Packages.azure-mgmt-commerce python2Packages.azure-mgmt-common python2Packages.azure-mgmt-compute python2Packages.azure-mgmt-consumption python2Packages.azure-mgmt-containerinstance python2Packages.azure-mgmt-containerservice python2Packages.azure-mgmt-cosmosdb python2Packages.azure-mgmt-datafactory python2Packages.azure-mgmt-datalake-analytics python2Packages.azure-mgmt-datalake-nspkg python2Packages.azure-mgmt-datalake-store python2Packages.azure-mgmt-datamigration python2Packages.azure-mgmt-devspaces python2Packages.azure-mgmt-devtestlabs python2Packages.azure-mgmt-dns python2Packages.azure-mgmt-eventgrid python2Packages.azure-mgmt-eventhub python2Packages.azure-mgmt-hanaonazure python2Packages.azure-mgmt-iotcentral python2Packages.azure-mgmt-iothub python2Packages.azure-mgmt-iothubprovisioningservices python2Packages.azure-mgmt-keyvault python2Packages.azure-mgmt-loganalytics python2Packages.azure-mgmt-logic python2Packages.azure-mgmt-machinelearningcompute python2Packages.azure-mgmt-managementgroups python2Packages.azure-mgmt-managementpartner python2Packages.azure-mgmt-maps python2Packages.azure-mgmt-marketplaceordering python2Packages.azure-mgmt-media python2Packages.azure-mgmt-monitor python2Packages.azure-mgmt-msi python2Packages.azure-mgmt-network python2Packages.azure-mgmt-notificationhubs python2Packages.azure-mgmt-nspkg python2Packages.azure-mgmt-policyinsights python2Packages.azure-mgmt-powerbiembedded python2Packages.azure-mgmt-rdbms python2Packages.azure-mgmt-recoveryservices python2Packages.azure-mgmt-recoveryservicesbackup python2Packages.azure-mgmt-redis python2Packages.azure-mgmt-relay python2Packages.azure-mgmt-reservations python2Packages.azure-mgmt-resource python2Packages.azure-mgmt-scheduler python2Packages.azure-mgmt-search python2Packages.azure-mgmt-servicebus python2Packages.azure-mgmt-servicefabric python2Packages.azure-mgmt-signalr python2Packages.azure-mgmt-sql python2Packages.azure-mgmt-storage python2Packages.azure-mgmt-subscription python2Packages.azure-mgmt-trafficmanager python2Packages.azure-mgmt-web python2Packages.azure-nspkg python2Packages.azure-servicebus python2Packages.azure-servicefabric python2Packages.azure-servicemanagement-legacy python2Packages.azure-storage-blob python2Packages.msrest python2Packages.uamqp

@GrahamcOfBorg build python3Packages.azure-applicationinsights python3Packages.azure-batch python3Packages.azure-common python3Packages.azure-cosmosdb-nspkg python3Packages.azure-cosmosdb-table python3Packages.azure-datalake-store python3Packages.azure-eventgrid python3Packages.azure-graphrbac python3Packages.azure-keyvault python3Packages.azure-loganalytics python3Packages.azure-mgmt-advisor python3Packages.azure-mgmt-applicationinsights python3Packages.azure-mgmt-authorization python3Packages.azure-mgmt-batch python3Packages.azure-mgmt-batchai python3Packages.azure-mgmt-billing python3Packages.azure-mgmt-cdn python3Packages.azure-mgmt-cognitiveservices python3Packages.azure-mgmt-commerce python3Packages.azure-mgmt-common python3Packages.azure-mgmt-compute python3Packages.azure-mgmt-consumption python3Packages.azure-mgmt-containerinstance python3Packages.azure-mgmt-containerservice python3Packages.azure-mgmt-cosmosdb python3Packages.azure-mgmt-datafactory python3Packages.azure-mgmt-datalake-analytics python3Packages.azure-mgmt-datalake-nspkg python3Packages.azure-mgmt-datalake-store python3Packages.azure-mgmt-datamigration python3Packages.azure-mgmt-devspaces python3Packages.azure-mgmt-devtestlabs python3Packages.azure-mgmt-dns python3Packages.azure-mgmt-eventgrid python3Packages.azure-mgmt-eventhub python3Packages.azure-mgmt-hanaonazure python3Packages.azure-mgmt-iotcentral python3Packages.azure-mgmt-iothub python3Packages.azure-mgmt-iothubprovisioningservices python3Packages.azure-mgmt-keyvault python3Packages.azure-mgmt-loganalytics python3Packages.azure-mgmt-logic python3Packages.azure-mgmt-machinelearningcompute python3Packages.azure-mgmt-managementgroups python3Packages.azure-mgmt-managementpartner python3Packages.azure-mgmt-maps python3Packages.azure-mgmt-marketplaceordering python3Packages.azure-mgmt-media python3Packages.azure-mgmt-monitor python3Packages.azure-mgmt-msi python3Packages.azure-mgmt-network python3Packages.azure-mgmt-notificationhubs python3Packages.azure-mgmt-nspkg python3Packages.azure-mgmt-policyinsights python3Packages.azure-mgmt-powerbiembedded python3Packages.azure-mgmt-rdbms python3Packages.azure-mgmt-recoveryservices python3Packages.azure-mgmt-recoveryservicesbackup python3Packages.azure-mgmt-redis python3Packages.azure-mgmt-relay python3Packages.azure-mgmt-reservations python3Packages.azure-mgmt-resource python3Packages.azure-mgmt-scheduler python3Packages.azure-mgmt-search python3Packages.azure-mgmt-servicebus python3Packages.azure-mgmt-servicefabric python3Packages.azure-mgmt-signalr python3Packages.azure-mgmt-sql python3Packages.azure-mgmt-storage python3Packages.azure-mgmt-subscription python3Packages.azure-mgmt-trafficmanager python3Packages.azure-mgmt-web python3Packages.azure-nspkg python3Packages.azure-servicebus python3Packages.azure-servicefabric python3Packages.azure-servicemanagement-legacy python3Packages.azure-storage-blob python3Packages.msrest python3Packages.uamqp

@kalbasit

This comment has been minimized.

Copy link
Member

commented Jul 16, 2019

All my testing are good, merging this PR.

@kalbasit kalbasit merged commit 6c2b929 into NixOS:master Jul 16, 2019

15 checks passed

Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
python2Packages.azure-applicationinsights, python2Packages.azure-batch, python2Packages.azure-common, python2Packages.azure-cosmosdb-nspkg, python2Packages.azure-cosmosdb-table, python2Packages.azure-datalake-store, python2Packages.azure-eventgrid, python2Packages.azure-graphrbac, python2Packages.azure-keyvault, python2Packages.azure-loganalytics, python2Packages.azure-mgmt-advisor, python2Packages.azure-mgmt-applicationinsights, python2Packages.azure-mgmt-authorization, python2Packages.azure-mgmt-batch, python2Packages.azure-mgmt-batchai, python2Packages.azure-mgmt-billing, python2Packages.azure-mgmt-cdn, python2Packages.azure-mgmt-cognitiveservices, python2Packages.azure-mgmt-commerce, python2Packages.azure-mgmt-common, python2Packages.azure-mgmt-compute, python2Packages.azure-mgmt-consumption, python2Packages.azure-mgmt-containerinstance, python2Packages.azure-mgmt-containerservice, python2Packages.azure-mgmt-cosmosdb, python2Packages.azure-mgmt-datafactory, python2Packages.azure-mgmt-datalake-analytics, p Success
Details
python3Packages.azure-applicationinsights, python3Packages.azure-batch, python3Packages.azure-common, python3Packages.azure-cosmosdb-nspkg, python3Packages.azure-cosmosdb-table, python3Packages.azure-datalake-store, python3Packages.azure-eventgrid, python3Packages.azure-graphrbac, python3Packages.azure-keyvault, python3Packages.azure-loganalytics, python3Packages.azure-mgmt-advisor, python3Packages.azure-mgmt-applicationinsights, python3Packages.azure-mgmt-authorization, python3Packages.azure-mgmt-batch, python3Packages.azure-mgmt-batchai, python3Packages.azure-mgmt-billing, python3Packages.azure-mgmt-cdn, python3Packages.azure-mgmt-cognitiveservices, python3Packages.azure-mgmt-commerce, python3Packages.azure-mgmt-common, python3Packages.azure-mgmt-compute, python3Packages.azure-mgmt-consumption, python3Packages.azure-mgmt-containerinstance, python3Packages.azure-mgmt-containerservice, python3Packages.azure-mgmt-cosmosdb, python3Packages.azure-mgmt-datafactory, python3Packages.azure-mgmt-datalake-analytics, p Success
Details
@jonringer jonringer referenced this pull request Aug 8, 2019
5 of 7 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.