Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parallel-full: init SQL and CSV support #77619

Merged
merged 1 commit into from Apr 3, 2020
Merged

Conversation

@tomberek
Copy link
Contributor

@tomberek tomberek commented Jan 13, 2020

Motivation for this change

Allow using parallel's SQL features. --sqlmaster and --sqlworker

Things done

Created parallel-extra to prevent adding dependencies to existing users.

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after): increased by 14.7M to add SQL support (93.2M -> 107.9M)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@erikarvstedt
Copy link
Member

@erikarvstedt erikarvstedt commented Jan 14, 2020

Does parallel-extra enable all extra features? Then parallel-full is a better name.

@tomberek tomberek force-pushed the tomberek:tom/parallel_sql branch from 822746f to aba7a2a Jan 29, 2020
@tomberek
Copy link
Contributor Author

@tomberek tomberek commented Jan 29, 2020

Added CSV support and changed name to parallel-full. I suppose we could add every database this could support, but that's a bit too much.

@tomberek tomberek changed the title parallel: init SQL support parallel: init SQL and CSV support Jan 29, 2020
@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Apr 3, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/132

wrapProgram $out/bin/parallel \
--prefix PATH : "${stdenv.lib.makeBinPath [ procps perl ]}"
--prefix PATH : "${stdenv.lib.makeBinPath [ procps perl ]}" \
--set PERL5LIB "${perlPackages.makeFullPerlPath extraPerlPackages}"

This comment has been minimized.

@Mic92

Mic92 Apr 3, 2020
Contributor

Since it is just an environment variable added I would make parallel-full a wrapper around parallel that adds these packages.

This comment has been minimized.

@tomberek

tomberek Apr 3, 2020
Author Contributor

Not 100% sure if the wrapper approach i used is correct, but it works. Open to a better approach.

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved
@tomberek tomberek force-pushed the tomberek:tom/parallel_sql branch from bdb48ab to de25fed Apr 3, 2020
parallel: add pacakges for CSV function
parallel: use wrapper to include more libraries
parallel: remove unneeded with
parallel: remove args

Co-Authored-By: Jörg Thalheim <Mic92@users.noreply.github.com>
parallel: move default pacakges for full into wrapper
@tomberek tomberek force-pushed the tomberek:tom/parallel_sql branch from de25fed to 52f46fe Apr 3, 2020
@Mic92 Mic92 changed the title parallel: init SQL and CSV support parallel-full: init SQL and CSV support Apr 3, 2020
@Mic92 Mic92 merged commit 939976a into NixOS:master Apr 3, 2020
16 checks passed
16 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="52f46fe"; rev="52f46fe1e16e2dd5a80747e2b26c6ce1de7be1e3"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
parallel, parallel.passthru.tests on aarch64-linux Success
Details
parallel, parallel.passthru.tests on x86_64-linux Success
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.