Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x265: build a single shared library for all bit-depths #92052

Merged
merged 1 commit into from Aug 15, 2020

Conversation

@dominikh
Copy link
Contributor

dominikh commented Jul 2, 2020

This builds the 10-bit and 12-bit versions of x265 as static
libraries, which then get linked into the 8-bit dynamic library and
executable. This causes x265 to default to 8-bit, but make 10- and
12-bit available to callers that use introspection, such as ffmpeg.

$ x265 -V
x265 [info]: HEVC encoder version 3.2
x265 [info]: build info [Linux][GCC 9.3.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

$ ffmpeg -h encoder=libx265
[...]
    Supported pixel formats: yuv420p yuvj420p yuv422p yuvj422p yuv444p yuvj444p gbrp yuv420p10le yuv422p10le yuv444p10le gbrp10le yuv420p12le yuv422p12le yuv444p12le gbrp12le gray gray10le gray12le

Inspired by @codyopel's comment on #80405.

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
This builds the 10-bit and 12-bit versions of x265 as static
libraries, which then get linked into the 8-bit dynamic library and
executable. This causes x265 to default to 8-bit, but make 10- and
12-bit available to callers that use introspection, such as ffmpeg.

    $ x265 -V
    x265 [info]: HEVC encoder version 3.2
    x265 [info]: build info [Linux][GCC 9.3.0][64 bit] 8bit+10bit+12bit
    x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

    $ ffmpeg -h encoder=libx265
    [...]
        Supported pixel formats: yuv420p yuvj420p yuv422p yuvj422p yuv444p yuvj444p gbrp yuv420p10le yuv422p10le yuv444p10le gbrp10le yuv420p12le yuv422p12le yuv444p12le gbrp12le gray gray10le gray12le

Inspired by @codyopel's comment on #80405.
@mdlayher
Copy link
Member

mdlayher commented Jul 2, 2020

/marvin opt-in

@mdlayher
Copy link
Member

mdlayher commented Jul 2, 2020

For anyone reading, I'm running nixpkgs-review pr 92052 locally but it's taking a very long time to pull all the reverse dependencies and such. I'll share another update when it's done.

@mdlayher
Copy link
Member

mdlayher commented Jul 2, 2020

I let it run for about 3 hours and it noted 28 package failures before starting compilation on really big packages like Blender. I don't think my desktop is beefy enough to run this whole workload: https://gist.github.com/mdlayher/7e4cc832f5cc0f2ec7e6934bd74a8c0f

@mdlayher
Copy link
Member

mdlayher commented Jul 2, 2020

I can confirm that this also works locally on my machine, but I'm out of my depth as far as diagnosing any possible reverse dependency issues:

✔ ~/src/github.com/nixos/nixpkgs [:bdd65f3f69d|✔] 
14:21 $ nix run -f default.nix x265
[3 built, 5 copied (34.5 MiB), 7.9 MiB DL]
✔ ~/src/github.com/nixos/nixpkgs [:bdd65f3f69d|✔] 
14:25 $ nix-build -A x265
/nix/store/16lp8nyar1rwx7iz0flvk0kybcnnwwp0-x265-3.2
✘-1 ~/src/github.com/nixos/nixpkgs [:bdd65f3f69d|✔] 
14:25 $ ./result/bin/x265 -V
x265 [info]: HEVC encoder version 3.2
x265 [info]: build info [Linux][GCC 9.3.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
@FRidh FRidh changed the base branch from master to staging Aug 15, 2020
@FRidh FRidh merged commit 90cd27e into NixOS:staging Aug 15, 2020
16 checks passed
16 checks passed
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./pkgs/t
Details
grahamcofborg-eval-lib-tests nix-build --arg pkgs import ./. {} ./lib/tests/release.nix
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./nixos/
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./nixos/
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./nixos/
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./pkgs/t
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="bdd65f3"; rev="bdd65f3f69dfcd0c3cfa9a00085d732a8f9715e4"; } ./pkgs/t
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
x265, x265.passthru.tests on aarch64-linux Success
Details
x265, x265.passthru.tests on x86_64-linux Success
Details
@B4dM4n B4dM4n mentioned this pull request Aug 25, 2020
6 of 10 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.