pythonPackages.pybase64: init at 0.2.1 #34589

Ma27 · 2018-02-04T13:12:52Z

Motivation for this change

Adds the pybase64 package, a simple python-based libbase64 wrapper.
Please have a look at the commit messages for further reference.

Things done

Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)
- NixOS
- macOS
- other Linux distributions
Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/)
Fits CONTRIBUTING.md.

Ma27 · 2018-02-04T13:19:32Z

currently running nox-review pr 34589...

Ma27 · 2018-02-04T13:27:06Z

ok, the package flexget fails for me, however this seems unrelated (I also tested this on the latest nixpkgs master without this change), I'd submit a separate PR to fix this.

FRidh · 2018-02-04T18:51:39Z

pkgs/top-level/python-packages.nix

@@ -3839,7 +3839,8 @@ in {
    };
  });

-  nose-parameterized = callPackage ../development/python-modules/nose-parameterized {};
+  nose-parameterized = builtins.trace "Warning: `nose-parameterized` is deprecated! Use `parameterized` instead."


Ma27 · 2018-02-04T21:20:31Z

@FRidh done. The flexget package is still broken, however this is an issue on master and I already started investigating, however I need some after-FOSDEM sleep first :-)

FRidh · 2018-02-05T09:08:56Z

pkgs/development/python-modules/parameterized/default.nix

+  # Tests require some python3-isms but code works without.
+  doCheck = isPy3k;
+
+  buildInputs = [ nose glibcLocales ];


checkInputs

`pythonPackages.parameterized` is the successor of `nose-parameterized` as the authors of the module decided to support more testing frameworks and stopped focusing on `noes` only. `nose-parameterized` is still available in `pypi` with version `0.6.0`, but is officially deprecated. However the renaming happened quite recently so it is possible that there are still folks relying on `nose-parameterized`. Therefore I moved the expression to provide a `pythonPackages.parameterized` derivation and added a package override which builds `nose-parameterized` after yielding a deprecation warning.

@ironpinguin

`pybase64` is a tiny wrapper for `libbase64` written in python. /cc @ironpinguin

Ma27 · 2018-02-05T12:58:04Z

done.

dotlambda · 2018-02-05T23:16:03Z

This causes a Hydra build to fail: https://hydra.nixos.org/build/68773666

Ma27 · 2018-02-06T06:08:40Z

@dotlambda so, yielding warnings (in this case because of a deprecation by the package author) causes our builds to fail? oO

For me it seems weird to me to break a build because of warnings (in this case because of a deprecation warning due to a deprecated package which is something that happens from time to time), but @FRidh might know more about this.

However if there's no simple workaround to keep the deprecation warning without breaking the build I'd rather drop the warning for now as I think that patching the parameterized package in order to look like nose-parameterized is IMO not a better solution since we just patch around to satisfy some minor warnings.

Ma27 · 2018-02-07T08:48:20Z

@FRidh FYI I currently don't know how to fix the flexget package as the tests during nix-build are fialing, however everything is perfectly fine when using virtualenv. As I don't know to much about Python I'll stop working on this for now...

dotlambda · 2018-02-07T10:15:17Z

I think we might be able to fix the flexget package by pinning pytest to a certain version (probably 3.2.3 or 3.2.4). I'll try this and report whether it succeeds.

Ma27 · 2018-02-07T11:39:17Z

👍 I think we could also create a package set using ‘pypi2nix’ as this would solve the dependency version problem

dotlambda · 2018-02-08T08:36:28Z

I think the best solution would be to just disable tests completely. Even when up-/downgrading pytest, I have to disable about half of the tests.

Ma27 · 2018-02-08T09:04:41Z

In this case we’d have to test the resulting binary manually right? As I mentioned we could try if pypi2nix actually fixes these issues, however I can’t have a look at this before the next weekend %)

…

On 8. Feb 2018, at 9:36 AM, Robert Schütz ***@***.***> wrote: I think the best solution would be to just disable tests completely. Even when up-/downgrading pytest, I have to disable about half of the tests. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

dotlambda · 2018-02-08T09:06:40Z

I don't think derivations generated by pypi2nix are fit for inclusion in Nixpkgs yet.

Ma27 · 2018-02-08T09:11:42Z

can you elaborate, please? We use tools like hackage2nix and node2nix in nixpkgs to generate derivations. The thing is: such applicatiojs pin their dependencies explicitly, so we’d either have to patch and test such applications whenever we bump a python library or we work around this issue by generating package sets for bigger applications (which actually happens with several nodejs and ruby applications)

…

On 8. Feb 2018, at 9:36 AM, Robert Schütz ***@***.***> wrote: I think the best solution would be to just disable tests completely. Even when up-/downgrading pytest, I have to disable about half of the tests. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

FRidh · 2018-02-08T10:49:09Z

can you elaborate, please?

Security and testing. Give it some thought how these are affected by the use of pypi2nix.

such applicatiojs pin their dependencies explicitly

Which is bad practice. They should not pin on patch level, but unfortunately it is often done.

have to patch and test such applications whenever we bump a python library

That's why most of our packages have tests, so we can see this happen, and we typically see it happen! It just requires maintainers of these packages to actively maintain their package.

Ma27 · 2018-02-08T12:32:47Z

Security and testing. Give it some thought how these are affected by the use of pypi2nix.

ACK, you're right.

That's why most of our packages have tests, so we can see this happen, and we typically see it happen! It just requires maintainers of these packages to actively maintain their package.

Okay then, I already started chasing and fixing some build/test issues of flexget, however I lack the inside knowledge in how Python packaging works in Nix, so I had some broken tests where I didn't know how to fix them.
Will push the branch, then we can see what we can do next ok? :-)

Ma27 requested a review from FRidh as a code owner February 4, 2018 13:12

GrahamcOfBorg added the 6.topic: python label Feb 4, 2018

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Feb 4, 2018

FRidh reviewed Feb 4, 2018

View reviewed changes

Ma27 force-pushed the package-pybase64 branch from 300d27d to f518ad6 Compare February 4, 2018 21:19

FRidh reviewed Feb 5, 2018

View reviewed changes

Ma27 force-pushed the package-pybase64 branch from f518ad6 to cb8343f Compare February 5, 2018 12:55

Ma27 added 2 commits February 5, 2018 13:55

pythonPackages.pybase64: init at 0.2.1

14da2e2

`pybase64` is a tiny wrapper for `libbase64` written in python. /cc @ironpinguin

Ma27 force-pushed the package-pybase64 branch from cb8343f to 14da2e2 Compare February 5, 2018 12:56

FRidh merged commit 422f99b into NixOS:master Feb 5, 2018

Ma27 deleted the package-pybase64 branch February 5, 2018 13:25

dotlambda mentioned this pull request Feb 5, 2018

pythonPackages.dateparser: switch to parameterized and thereby fix Hydra test #34643

Merged

8 tasks

dotlambda mentioned this pull request Feb 9, 2018

flexget: mark as broken #34771

Merged

8 tasks

Ma27 mentioned this pull request Mar 3, 2018

flexget: 2.10.82 -> 2.12.8 #35557

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pythonPackages.pybase64: init at 0.2.1 #34589

pythonPackages.pybase64: init at 0.2.1 #34589

Ma27 commented Feb 4, 2018 •

edited

Ma27 commented Feb 4, 2018

Ma27 commented Feb 4, 2018

FRidh Feb 4, 2018

Ma27 commented Feb 4, 2018

FRidh Feb 5, 2018

Ma27 commented Feb 5, 2018

dotlambda commented Feb 5, 2018

Ma27 commented Feb 6, 2018

Ma27 commented Feb 7, 2018

dotlambda commented Feb 7, 2018

Ma27 commented Feb 7, 2018

dotlambda commented Feb 8, 2018

Ma27 commented Feb 8, 2018 via email

dotlambda commented Feb 8, 2018

Ma27 commented Feb 8, 2018 via email

FRidh commented Feb 8, 2018

Ma27 commented Feb 8, 2018

pythonPackages.pybase64: init at 0.2.1 #34589

pythonPackages.pybase64: init at 0.2.1 #34589

Conversation

Ma27 commented Feb 4, 2018 • edited

Motivation for this change

Things done

Ma27 commented Feb 4, 2018

Ma27 commented Feb 4, 2018

FRidh Feb 4, 2018

Choose a reason for hiding this comment

Ma27 commented Feb 4, 2018

FRidh Feb 5, 2018

Choose a reason for hiding this comment

Ma27 commented Feb 5, 2018

dotlambda commented Feb 5, 2018

Ma27 commented Feb 6, 2018

Ma27 commented Feb 7, 2018

dotlambda commented Feb 7, 2018

Ma27 commented Feb 7, 2018

dotlambda commented Feb 8, 2018

Ma27 commented Feb 8, 2018 via email

dotlambda commented Feb 8, 2018

Ma27 commented Feb 8, 2018 via email

FRidh commented Feb 8, 2018

Ma27 commented Feb 8, 2018

Ma27 commented Feb 4, 2018 •

edited