Refactor ~200 package from python-packages.nix -> python-modules

[costrouc]

Things done

Strictly moving package to python-modules. Did not attempt to upgrade package unless it added python 3 support or was broken. Also converted fetch... to fetchPypi to make future upgrades easier. All packages have been tested in sandbox mode to build properly.

As of the last edit there are ~456 remaining packages.

grep "buildPythonPackage" python-packages.nix | wc
    456    2322   18504

General steps taken:

• added any missing meta information (license, description, homepage)
• fetch... -> fetchPypi
• name = "something-1.0.1"; ->version="1.0.1"; pname="something";
• kept pkgs reference (would have taken a lot of time to refactor out)
• !isPy27 -> isPy3k
• moved disabled next to version, pname.
• remove platforms.all let buildPythonPackage determine

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Fits CONTRIBUTING.md.

---

[costrouc]

I was creating these commits starting from master and it looks like it is too far ahead. Not sure what to do. It was 63 commits but grew larger when putting on staging. Sorry for everyone getting pinged.

[Mic92]

You also want to rebase this.

[FRidh]

Thanks a lot! I've rebased and pushed this to staging.

[FRidh]

Ohh, the merge conflicts after this change 😢

[costrouc]

@FRidh yes I was very worried about that. Could I do something that would make it easier to merge? I still have about 90 commits to add as well (I am testing them now). I was hoping to check all the meta attributes too which I have not done yet. https://github.com/nixos/ofborg#running-meta-checks-locally

[FRidh]

Done!

I don't think there is really anything that can be done. It seems like it is more an issue of timing; there are already quite some package changes in master and staging-next so I suppose the best moment would be when they've all just been merged together.

Don't let it stop you, go ahead.

[costrouc]

Awesome! I will rebase the next 90 commits on the new staging in a new PR when I am ready. I am going to have to wait awhile because some of the packages are taking quite some time to build. Thanks for making this easy! Can't wait for the moment when python-packages.nix can be truly alphabetical

[Mic92]

Regarding alphabetical order some tooling to maintain the order would be nice. Humans are not good at that for large files and then we have merges that can mess up the order.

[costrouc]

@Mic92 I've thought about this a bit and I'm thinking that in order to do proper alphabetical sorting we would need the AST representation of the file. This could also be helpful in enforcing a style to files and many other things. I really liked your project pythoninx and have played with it. Does nix/libexpr expose the AST that could somehow be exposed in python?

Also in regards to sorting I think that normalized sorting would be helpful. For example Example1_a-sDF would be sorted as example1-a-sdf.

[edolstra]

Once upon a time we had a Stratego program for sorting all-packages.nix, using the old SDF grammar to get a syntax tree that includes whitespace/comments. https://github.com/NixOS/nixpkgs/blob/7ee6cf21ea5c4a07499ba4e7cb7834653c77af27/maintainers/scripts/sort-attrs.str