Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python37Packages.py4j: 0.10.9 -> 0.10.9.1 #100623

Closed
wants to merge 1 commit into from

Conversation

@r-ryantm
Copy link
Contributor

@r-ryantm r-ryantm commented Oct 15, 2020

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.

meta.description for python37Packages.py4j is: "Py4J enables Python programs running in a Python interpreter to dynamically access Java objects in a Java Virtual Machine. Methods are called as if the Java objects resided in the Python interpreter and Java collections can be accessed through standard Python collection methods. Py4J also enables Java programs to call back Python objects."

meta.homepage for python37Packages.py4j is: "https://www.py4j.org/"

meta.changelog for python37Packages.py4j is: ""

Updates performed
  • Ran passthru.UpdateScript
To inspect upstream changes
Impact
Checks done (click to expand)

Rebuild report (if merged into master) (click to expand)
48 total rebuild path(s)

12 package rebuild(s)

12 x86_64-linux rebuild(s)
12 i686-linux rebuild(s)
12 x86_64-darwin rebuild(s)
12 aarch64-linux rebuild(s)


First fifty rebuilds by attrpath
python27Packages.databricks-connect
python27Packages.py4j
python27Packages.pyspark
python27Packages.python-ldap-test
python37Packages.databricks-connect
python37Packages.py4j
python37Packages.pyspark
python37Packages.python-ldap-test
python38Packages.databricks-connect
python38Packages.py4j
python38Packages.pyspark
python38Packages.python-ldap-test
Instructions to test this update (click to expand)

Either download from Cachix:

nix-store -r /nix/store/rhp1yg22rgdm2ffr73il971fhbprizdx-python3.7-py4j-0.10.9.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.

Or, build yourself:

nix-build -A python37Packages.py4j https://github.com/r-ryantm/nixpkgs/archive/0ebfe6addbd5025de12792de37cb4d943e4fc3cd.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:

ls -la /nix/store/rhp1yg22rgdm2ffr73il971fhbprizdx-python3.7-py4j-0.10.9.1
ls -la /nix/store/rhp1yg22rgdm2ffr73il971fhbprizdx-python3.7-py4j-0.10.9.1/bin


Pre-merge build results

We have automatically built all packages that will get rebuilt due to
this change.

This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.

Result of nixpkgs-review 1

6 packages failed to build:
  • python27Packages.databricks-connect
  • python27Packages.pyspark
  • python37Packages.databricks-connect
  • python37Packages.pyspark
  • python38Packages.databricks-connect
  • python38Packages.pyspark
6 packages built:
  • python27Packages.py4j
  • python27Packages.python-ldap-test
  • python37Packages.py4j
  • python37Packages.python-ldap-test
  • python38Packages.py4j
  • python38Packages.python-ldap-test

Maintainer pings

cc @shlevy for testing.

Copy link
Contributor

@jonringer jonringer left a comment

pyspark needs to be update:

  ERROR: Could not find a version that satisfies the requirement py4j==0.10.9 (from pyspark==3.0.0) (from versions: none)
  ERROR: No matching distribution found for py4j==0.10.9 (from pyspark==3.0.0)
https://github.com/NixOS/nixpkgs/pull/100623
3 packages marked as broken and skipped:
python27Packages.databricks-connect python37Packages.databricks-connect python38Packages.databricks-connect

3 packages failed to build:
python27Packages.pyspark python37Packages.pyspark python38Packages.pyspark

6 packages built:
python27Packages.py4j python27Packages.python-ldap-test python37Packages.py4j python37Packages.python-ldap-test python38Packages.py4j python38Packages.python-ldap-test

jskrzypek added a commit to jskrzypek/nixpkgs that referenced this issue Nov 23, 2020
Pyspark switched to pinning py4j==0.10.9 with v3.0.0 - see this commit:
https://github.com/apache/spark/\
   commit/fc4e56a54c15e20baf085e6061d3d83f5ce1185d

This meant that since the bump to pyspark v3.0.0 - in this commit:
https://github.com/NixOS/nixpkgs/\
   commit/5181547ae6624b462919a806c4d0888e6e4630f4 -
the patch was no longer matching on the 'py4j==0.10.7' string that was
working previously.

The failing patch went unnoticed previously because the version of py4j
pinned by pyspark>=3.0.0 was the same as the py4j provided by nixpkgs.

However, a recent PR (NixOS#101636) bumped the version of py4j to 0.10.9.1 in
this commit:
https://github.com/NixOS/nixpkgs/\
   commit/43a91282d66223c5cb978d53fbe1033f56dd7f2b
which caused the version pinned by pyspark to no longer match the
version provided by nixpkgs. FWIW, @jonringer flagged this issue on
another PR that tried to bump py4j: NixOS#100623.

My solution here was to upgrade the patch's target string to match the
version found in pyspark's current setup.py.
jonringer added a commit that referenced this issue Nov 24, 2020
Pyspark switched to pinning py4j==0.10.9 with v3.0.0 - see this commit:
https://github.com/apache/spark/\
   commit/fc4e56a54c15e20baf085e6061d3d83f5ce1185d

This meant that since the bump to pyspark v3.0.0 - in this commit:
https://github.com/NixOS/nixpkgs/\
   commit/5181547ae6624b462919a806c4d0888e6e4630f4 -
the patch was no longer matching on the 'py4j==0.10.7' string that was
working previously.

The failing patch went unnoticed previously because the version of py4j
pinned by pyspark>=3.0.0 was the same as the py4j provided by nixpkgs.

However, a recent PR (#101636) bumped the version of py4j to 0.10.9.1 in
this commit:
https://github.com/NixOS/nixpkgs/\
   commit/43a91282d66223c5cb978d53fbe1033f56dd7f2b
which caused the version pinned by pyspark to no longer match the
version provided by nixpkgs. FWIW, @jonringer flagged this issue on
another PR that tried to bump py4j: #100623.

My solution here was to upgrade the patch's target string to match the
version found in pyspark's current setup.py.
@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Nov 27, 2020

Result of nixpkgs-review pr 100623 run on x86_64-linux 1

3 packages failed to build:
  • python27Packages.databricks-connect
  • python37Packages.databricks-connect
  • python38Packages.databricks-connect
9 packages built:
  • python27Packages.py4j
  • python27Packages.pyspark
  • python27Packages.python-ldap-test
  • python37Packages.py4j
  • python37Packages.pyspark
  • python37Packages.python-ldap-test
  • python38Packages.py4j
  • python38Packages.pyspark
  • python38Packages.python-ldap-test

@SuperSandro2000 SuperSandro2000 marked this pull request as draft Jan 20, 2021
@dotlambda
Copy link
Member

@dotlambda dotlambda commented Feb 5, 2021

py4j is on version 0.10.9.1 on master

@dotlambda dotlambda closed this Feb 5, 2021
@r-ryantm r-ryantm deleted the auto-update/python3.7-py4j branch Feb 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants