Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

youtube-dl: build the lazy_extractors module to improve startup time #61115

Merged
merged 1 commit into from May 8, 2019

Conversation

@ivan
Copy link
Contributor

commented May 8, 2019

Motivation for this change

youtube-dl imports every extractor it has unless the youtube_dl/extractor/lazy_extractors.py module is built.

This updates nixpkgs' youtube-dl to build the module, improving startup time.

Testing on an Intel 4790K / NVMe drive:

before:

# time youtube-dl --help > /dev/null
youtube-dl --help > /dev/null  0.26s user 0.06s system 100% cpu 0.322 total

# strace youtube-dl 'https://www.youtube.com/watch?v=nJ9IdzcQMoc' 2>&1 | grep extractor/ | grep \.pyc | wc -l
789

after:

# time youtube-dl --help > /dev/null
youtube-dl --help > /dev/null  0.13s user 0.05s system 100% cpu 0.182 total

# strace youtube-dl 'https://www.youtube.com/watch?v=nJ9IdzcQMoc' 2>&1 | grep extractor/ | grep \.pyc | wc -l
6
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@ivan ivan force-pushed the ivan:youtube-dl-lazy-extractors branch from 03e3dde to 840f488 May 8, 2019

@worldofpeace worldofpeace merged commit a80ee5b into NixOS:master May 8, 2019

13 checks passed

Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
@worldofpeace

This comment has been minimized.

Copy link
Member

commented May 8, 2019

hmm, backport or to not backport...

@ivan

This comment has been minimized.

Copy link
Contributor Author

commented May 8, 2019

I would leave it in master for a bit first to see if anything breaks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.