Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gstreamer-1.16.0 #54398

Merged
merged 4 commits into from Jun 20, 2019

Conversation

@nh2
Copy link
Contributor

commented Jan 21, 2019

  • Based upon the (tiny) #54380; let's merge that first. (done)
  • Wait for 1.16 to be released (#54398 (comment))
  • Fix any fallout when 1.16 is released

Motivation for this change

The latest gstreamer version The current gstreamer development version, leading up to the next to-be-released 1.16 stable version. Already doing some work here so we have less work to do when 1.16 comes out.

Edit: This is now, and the PR has been updated.

Copying an important part of the commit message:


During the 1.14 -> 1.15 upgrade, lots of stuff stopped working because gstreamer changed what features are enabled by default and which ones are automatically turned on/off via pkgconfig dependency detection.

This resulted in the gstreamer ("core" attribute in nixpkgs) package to have only 15 of its previous 163 build targets enabled, and downstream packages breaking correspondingly.

To ease maintainability and to ensure users will find the expected features available (and when not, will see in the nix file why not), we now pass the -Dauto_features=enabled Meson build flag to all gstreamer builds, which sets all auto dependencies to enabled, and we explicitly disable those that we can't build .

This means in particular that gst-plugins-bad now has vastly more integrations (namely all for which nixpkgs has libraries available).

@nh2's notes while working on this (for posterity)

  • various meson patches: Removed: Seem no longer necessary, but I haven't checked all the details and whether the patches were fully merged or only partially.
    See e.g. https://lists.freedesktop.org/archives/gstreamer-bugs/2018-August/223857.html where they say

    Committed without the gl part which was installed into libdir on purpose. You will have to take account of that in your packaging.

  • libice: gstreamer-check-1.0.pc not found (and indeed it's not there)
  • core configure output:
    -Found pkg-config: /nix/store/4iij6zf2z7gx68rx3kg28xi5rqvnfja6-pkg-config-0.29.2/bin/pkg-config (0.29.2)
    -Dependency libunwind found: YES 1.3.1
    -Found CMake: NO
    -Dependency libdw found: NO (tried pkgconfig and cmake)
    -Message: Support for backtraces is partial only.
    +Dependency libunwind skipped: feature libunwind disabled
    +Dependency libdw skipped: feature libdw disabled
    +Checking for function "backtrace" : YES
    • likely the same as before (partial support), but I improved it by adding an elfutils dep which provides libdw
  • core configure output:
    -Build targets in project: 163
    +Build targets in project: 15
  • gst-validate build complains that GstPbutils-1.0.gir couldn't be found / GEN'd even though it says checking for GST_PBUTILS... yes before
    • probably due to core configure output: Program g-ir-scanner skipped: feature introspection disabled

Things done

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Tested via:

NIX_PATH=nixpkgs=. nix-shell -p gst_all_1.gstreamer -p gst_all_1.gst-plugins-good -p gst_all_1.gst-plugins-base -p gst_all_1.gst-plugins-bad -p gst_all_1.gst-plugins-ugly -p gst_all_1.gst-libav -p gst_all_1.gst-rtsp-server -p gst_all_1.gst-validate -p gst_all_1.gst-vaapi

CC @matthewbauer

TODO

  • Merge #54380
  • Somebody should double-check whether indeed all patches removed in this PR have all their contents merged upstream, or if some were merged only partially
  • Check that the vastly-increased gst-plugins-bad integrations work on Darwin, make those conditional that don't
  • This needs some real testing from some real NixOS users, as I haven't checked whether any of the multimedia functionality actually works (I only built it)

@nh2 nh2 changed the title The latest gstreamer version. gstreamer-1.15.1 Jan 21, 2019

@nh2 nh2 requested a review from matthewbauer Jan 21, 2019

@nh2 nh2 referenced this pull request Jan 21, 2019

Merged

SRT support for gstreamer #54380

5 of 10 tasks complete
@jtojnar

This comment has been minimized.

Copy link
Contributor

commented Jan 21, 2019

I did not enable gtksink by default due to its closure size (150 MB vs 300 MB).

Also please note that 1.15 is an unstable development branch for 1.16 stable release.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jan 21, 2019

I did not enable gtksink by default due to its closure size (150 MB vs 300 MB).

@jtojnar That makes sense; the PR doesn't change this, it is still optional.

Also please note that 1.15 is an unstable development branch for 1.16 stable release.

Oops OK. Let's wait until 1.16 before continuing with this then.

@nh2 nh2 changed the title gstreamer-1.15.1 WIP: gstreamer-1.15.1 (to eventually become the gstreamer-1.16 PR) Jan 21, 2019

@jtojnar

This comment has been minimized.

Copy link
Contributor

commented Jan 21, 2019

The removed patches come from https://bugzilla.gnome.org/show_bug.cgi?id=794856 and have been committed.

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jan 21, 2019

The removed patches come from https://bugzilla.gnome.org/show_bug.cgi?id=794856 and have been committed.

@jtojnar Thanks! I ticked off the bullet point in the issue description based on this feedback.

closure size

We will also want to do a closure size check on the added -bad dependencies eventually.

I think it makes sense that it'll increase and will likely be worth it, as for end users nothing is more annoying than a gstreamer plugin but it not working on your machine because the distro hasn't enabled it, and -bad is supposed to contain as many integrations as possible, but we should still know the number.

Also, I wouldn't be opposed to introducing feature flags as arguments to the callPackage for each of the external dependencies so that users can easily build a smaller -bad package for their use cases, but I think the default should be ? true for most features so that in the default case multimedia "just works".

@nh2 nh2 force-pushed the nh2:gstreamer-1.15.1 branch from 39eb078 to 802bc29 Jan 22, 2019

@jtojnar

This comment has been minimized.

Copy link
Contributor

commented Mar 6, 2019

115.2 was released, 1.16 in few weeks: https://gstreamer.freedesktop.org/news/#2019-02-27T10:00:00Z

@nh2 nh2 force-pushed the nh2:gstreamer-1.15.1 branch from 512cee1 to 4876f0b May 1, 2019

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 1, 2019

1.16 was released.

I've updated the PR with another commit to update to 1.16 and fix a few build errors.

I've also made an upstream PR to fix a script not being executable, which broke the build:

https://gitlab.freedesktop.org/gstreamer/gstreamer/merge_requests/141

@nh2 nh2 changed the title WIP: gstreamer-1.15.1 (to eventually become the gstreamer-1.16 PR) gstreamer-1.16.0 May 1, 2019

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 1, 2019

Let's see if this works:

@GrahamcOfBorg build gst_all_1

@adisbladis

This comment has been minimized.

Copy link
Member

commented May 1, 2019

@GrahamcOfBorg build gst_all_1

@nh2 nh2 force-pushed the nh2:gstreamer-1.15.1 branch 2 times, most recently from 7750c09 to d563757 May 1, 2019

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 3, 2019

@GrahamcOfBorg build gst_all_1

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 3, 2019

I've backed out the evaluation fixes Darwin out of this PR, as that isn't really related to the upgrade, and will handle those with a separate branch.

If ofborg goes green, this is good to merge from my side.

@nh2 nh2 referenced this pull request May 3, 2019

Open

libvpx: 1.7.0 -> 1.8.0 #60826

3 of 12 tasks complete
@jtojnar

This comment has been minimized.

Copy link
Contributor

commented May 3, 2019

Dependency changes described in release notes do not affect us.

What is up with ffmpeg? And do we care about the rust bindings?

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 4, 2019

Dependency changes described in release notes do not affect us.

Thanks for checking!

What is up with ffmpeg?

I've updated gst-libav (which is the ffmpeg binding, right?) in this PR as one of the many gst packages, and it builds successfully.

And do we care about the rust bindings?

To my knowledge they aren't packaged in nixpkgs yet.

I'll leave that to somebody else, as I have no experience with packaging Rust for nixpkgs, and care mainly about the update to 1.16 and the improved meson build system I've added in this PR.

@nh2 nh2 referenced this pull request May 4, 2019

Open

Fix many gstreamer packages on Darwin #60922

4 of 11 tasks complete

@nh2 nh2 force-pushed the nh2:gstreamer-1.15.1 branch from d563757 to b84a8ad May 4, 2019

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented May 5, 2019

@jtojnar Good to go?

fdo-mirror pushed a commit to freedesktop/gstreamer-gst-plugins-bad that referenced this pull request May 13, 2019

meson: Fix typo in gsm header file name
This was wrong since commit c360cee.

Also fix incorrect indentation (tab instead of spaces).

Found using hermetic builds with Nix:

    NixOS/nixpkgs#54398 (comment)
@nixos-discourse

This comment has been minimized.

Copy link

commented May 14, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/what-are-your-goals-for-19-09/2875/24

jtojnar added a commit to jtojnar/nixpkgs that referenced this pull request Jun 19, 2019

meson: enable auto_features by default
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491

nh2 added some commits Jan 21, 2019

gstreamer: 1.14.4 -> 1.15.1, and extended gst-plugins-bad.
During the 1.14 -> 1.15 upgrade, lots of stuff stopped working because
gstreamer changed what features are enabled by default and which ones are
automatically turned on/off via pkgconfig dependency detection.

This resulted in the `gstreamer` ("core" attribute in nixpkgs) package
to have only 15 of its previous 163 build targets enabled, and downstream
packages breaking correspondingly.

To ease maintainability and to ensure users will find the expected features
available (and when not, will see in the nix file why not), we now pass
the `-Dauto_features=enabled` Meson build flag to all gstreamer builds,
which sets all `auto` dependencies to `enabled`, and we explicitly disable
those that we can't build.

This means in particular that `gst-plugins-bad` now has vastly more integrations
(namely all for which nixpkgs has libraries available).
gstreamer: 1.15.1 -> 1.16.0
Changes needed:

* core: dbghelp disabled as it's not needed
* core: Fixed script not being executable and skipped by patchShebangs
* base: gtk_doc disabled because of wayland symbol error
* ges: Patch removed that was merged upstream:
  https://bugzilla.gnome.org/show_bug.cgi?id=794856#c16

@nh2 nh2 force-pushed the nh2:gstreamer-1.15.1 branch from b84a8ad to 3be4c04 Jun 19, 2019

@nh2

This comment has been minimized.

Copy link
Contributor Author

commented Jun 19, 2019

@GrahamcOfBorg build gst_all_1

jtojnar added a commit to jtojnar/nixpkgs that referenced this pull request Jun 19, 2019

meson: enable auto_features by default
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491

@nh2 nh2 merged commit a4c5d37 into NixOS:master Jun 20, 2019

16 of 17 checks passed

gst_all_1 on x86_64-linux Timed out, unknown build status
Details
Evaluation Performance Report Evaluator Performance Report
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-darwin nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A darwin-tested
Details
grahamcofborg-eval-nixos nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release-combined.nix -A tested
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
gst_all_1 on aarch64-linux Success
Details
gst_all_1.gst-plugins-bad, gstreamer on aarch64-linux Success
Details
gst_all_1.gst-plugins-bad, gstreamer on x86_64-linux Success
Details

jtojnar added a commit to jtojnar/nixpkgs that referenced this pull request Jun 25, 2019

meson: enable auto_features by default
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491

jtojnar added a commit to jtojnar/nixpkgs that referenced this pull request Jun 25, 2019

meson: enable auto_features by default
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491

jtojnar added a commit to jtojnar/nixpkgs that referenced this pull request Jun 25, 2019

meson: enable auto_features by default
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491
@nixos-discourse

This comment has been minimized.

Copy link

commented Jul 3, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/what-are-your-goals-for-19-09/2875/29

@nixos-discourse

This comment has been minimized.

Copy link

commented Jul 31, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/nixos-weekly-11-nixery-nixfmt-and-cachix-releases-nixcon-2019-tickets-a-job-and-first-impressions-post/3591/1

fdo-mirror pushed a commit to freedesktop/gstreamer-gst-plugins-bad that referenced this pull request Aug 8, 2019

meson: Fix typo in gsm header file name
This was wrong since commit c360cee.

Also fix incorrect indentation (tab instead of spaces).

Found using hermetic builds with Nix:

    NixOS/nixpkgs#54398 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.