Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildPython*: add updateScript to passthru #50983

Merged
merged 1 commit into from Dec 7, 2018
Merged

Conversation

@FRidh
Copy link
Member

FRidh commented Nov 24, 2018

All Python packages now have an updateScript. The script calls
update-python-libraries and passes it the position of the derivation
expression obtained using meta.position. This works fine in case a Nix
expression represents only a single derivation. If there are more in it,
update-python-libraries will fail.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

pkgs/development/interpreters/python/mk-python-derivation.nix Outdated
updateScript = let
filename = builtins.head (lib.splitString ":" self.meta.position);
in writeScript "update-python" ''
${update-python-libraries} ${filename} --commit

This comment has been minimized.

Copy link
@FRidh

FRidh Nov 24, 2018

Author Member

will have to disable committing in case we run concurrently with #50977

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 24, 2018

Contributor

I think it would be nice if update.py handled the committing. Though in my experience git’s locking is pretty bad for things like this.

This comment has been minimized.

Copy link
@FRidh

FRidh Nov 24, 2018

Author Member

Yes, I agree.

passthru = {
updateScript = let
filename = builtins.head (lib.splitString ":" self.meta.position);
in writeScript "update-python" ''

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 24, 2018

Contributor
Suggested change
in writeScript "update-python" ''
in writeScript "update-python" ''
#!${python.stdenv.shell}
passthru = {
updateScript = let
filename = builtins.head (lib.splitString ":" self.meta.position);
in writeScript "update-python" ''

This comment has been minimized.

Copy link
@jtojnar

jtojnar Nov 25, 2018

Contributor

With #50977 (comment), you can do

passthru = {
  updateScript = let
    filename = builtins.head (lib.splitString ":" self.meta.position);
  in [ update-python-libraries filename "--commit" ];
}; 
# Commits are created sequentially.
if args.commit:
logging.info("Committing updates...")
list(map(lambda x: _commit(**x), filter(bool, results)))

This comment has been minimized.

Copy link
@Mic92

Mic92 Nov 26, 2018

Contributor

I find this easier to read:

Suggested change
list(map(lambda x: _commit(**x), filter(bool, results)))
for result in results:
if results:
_commit(**result)

or if your really want code golfing:

Suggested change
list(map(lambda x: _commit(**x), filter(bool, results)))
[_commit(**result) for result in results if result]
All Python packages now have an updateScript. The script calls
`update-python-libraries` and passes it the position of the derivation
expression obtained using `meta.position`. This works fine in case a Nix
expression represents only a single derivation. If there are more in it,
`update-python-libraries` will fail.
@FRidh FRidh force-pushed the FRidh:update-script branch to 56a0ed1 Dec 3, 2018
@FRidh FRidh merged commit 74d0b82 into NixOS:master Dec 7, 2018
9 checks passed
9 checks passed
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@FRidh FRidh deleted the FRidh:update-script branch Dec 7, 2018
@jtojnar

This comment has been minimized.

Copy link
Contributor

jtojnar commented Dec 22, 2018

This seems to break nix-shell maintainers/scripts/update.nix --argstr path python3.pkgs.pygobject3

No error is printed but if I modify the script, I can see that the updateScript from the expression is overridden by the one from this PR:

INFO:root:Updating packages...
WARNING:root:Path /home/jtojnar/projects/nixpkgs/pkgs/development/python-modules/pygobject/3.nix: url does not point to PyPI.
INFO:root:Finished updating packages.
INFO:root:0 package(s) updated
@FRidh

This comment has been minimized.

Copy link
Member Author

FRidh commented Dec 23, 2018

@jtojnar pushed 3d5025b to master

@nixos-discourse

This comment has been minimized.

Copy link

nixos-discourse commented May 21, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/automating-updates-of-ruby-packages/2996/2

@jonringer jonringer mentioned this pull request May 23, 2019
4 of 10 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.