Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pyopencl: enable package build for macOS #99921

Merged
merged 1 commit into from May 8, 2021

Conversation

@ryanorendorff
Copy link
Contributor

@ryanorendorff ryanorendorff commented Oct 7, 2020

Motivation for this change

pyopencl can work on macOS (specifically Catalina), but did not build in nixpkgs due to a linux only optional dependency and missing header files. This PR enables pyopencl to be used on macOS.

Tested using the following command from the root of the nixpkgs repo

nix-shell -E "with (import ./default.nix {}); (pkgs.python3.withPackages (p: with p; [numpy pyopencl setuptools])).env"

and then running the example on the front page of the pyopencl documentation: https://documen.tician.de/pyopencl/

wget https://documen.tician.de/pyopencl/_downloads/d058e815d3a1f246da550c694cce0c92/demo.py 
python demo.py

Tested on macOS Catalina 10.15.5 and Nixos 20.03.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@ryanorendorff
Copy link
Contributor Author

@ryanorendorff ryanorendorff commented Oct 7, 2020

I can add my handle to maintainers if you want me to keep track of the mac build.

@ryanorendorff
Copy link
Contributor Author

@ryanorendorff ryanorendorff commented Oct 14, 2020

Hmm I am not quite sure why the build fails with the following error on the CI test.

anonymous function at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/packet-spot-eval-3/pkgs/development/python-modules/pyopencl/default.nix:1:1 called without required argument 'libGL_driver', at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/packet-spot-eval-3/lib/customisation.nix:69:16

I double checked that the shell test still passes.

Any thoughts?

Copy link
Member

@SuperSandro2000 SuperSandro2000 left a comment

Please squash the commits together.

@ryanorendorff
Copy link
Contributor Author

@ryanorendorff ryanorendorff commented Nov 27, 2020

Please squash the commits together.

Done!

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Dec 27, 2020

@ryanorendorff please fix the ofborg eval error.

@ryanorendorff
Copy link
Contributor Author

@ryanorendorff ryanorendorff commented Apr 29, 2021

@SuperSandro2000 Fixed the ofborg eval error, which was due to the use of the alias libGL_driver = mesa.drivers (see here). There was a slight complication in just adding mesa driver as the function definition directly; there is a python package named mesa, so the driver mesa library was inaccessible due to name shadowing. I had to pass the mesa driver in through callPackage in the python modules top-level.

Tested on macOS 11.3 with the test mentioned in the PR description. The demo runs successfully with the expected output.

Thanks @rb2k for the suggestion on removing the alias reference.

@rb2k
Copy link
Contributor

@rb2k rb2k commented May 7, 2021

@jonringer / @FRidh any chance you could take a peek at this :) ?

@rb2k
Copy link
Contributor

@rb2k rb2k commented May 7, 2021

Result of nixpkgs-review pr 99921 run on x86_64-darwin 1

2 packages marked as broken and skipped:
  • python38Packages.loo-py
  • python39Packages.loo-py
6 packages built:
  • python38Packages.pyopencl
  • python38Packages.reikna
  • python38Packages.sasmodels
  • python39Packages.pyopencl
  • python39Packages.reikna
  • python39Packages.sasmodels

Copy link
Contributor

@jonringer jonringer left a comment

otherwise LGTM

pkgs/top-level/python-packages.nix Outdated Show resolved Hide resolved
Copy link
Contributor

@jonringer jonringer left a comment

LGTM

Result of nixpkgs-review pr 99921 run on x86_64-linux 1

8 packages built:
  • python38Packages.loo-py
  • python38Packages.pyopencl
  • python38Packages.reikna
  • python38Packages.sasmodels
  • python39Packages.loo-py
  • python39Packages.pyopencl
  • python39Packages.reikna
  • python39Packages.sasmodels

@jonringer jonringer merged commit 03aa75f into NixOS:master May 8, 2021
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants