Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gstreamer-1.16.0 #54398

Merged
merged 4 commits into from Jun 20, 2019
Merged

gstreamer-1.16.0 #54398

merged 4 commits into from Jun 20, 2019

Conversation

@nh2
Copy link
Contributor

@nh2 nh2 commented Jan 21, 2019

  • Based upon the (tiny) #54380; let's merge that first. (done)
  • Wait for 1.16 to be released (#54398 (comment))
  • Fix any fallout when 1.16 is released

Motivation for this change

The latest gstreamer version The current gstreamer development version, leading up to the next to-be-released 1.16 stable version. Already doing some work here so we have less work to do when 1.16 comes out.

Edit: This is now, and the PR has been updated.

Copying an important part of the commit message:


During the 1.14 -> 1.15 upgrade, lots of stuff stopped working because gstreamer changed what features are enabled by default and which ones are automatically turned on/off via pkgconfig dependency detection.

This resulted in the gstreamer ("core" attribute in nixpkgs) package to have only 15 of its previous 163 build targets enabled, and downstream packages breaking correspondingly.

To ease maintainability and to ensure users will find the expected features available (and when not, will see in the nix file why not), we now pass the -Dauto_features=enabled Meson build flag to all gstreamer builds, which sets all auto dependencies to enabled, and we explicitly disable those that we can't build .

This means in particular that gst-plugins-bad now has vastly more integrations (namely all for which nixpkgs has libraries available).

@nh2's notes while working on this (for posterity)

  • various meson patches: Removed: Seem no longer necessary, but I haven't checked all the details and whether the patches were fully merged or only partially.
    See e.g. https://lists.freedesktop.org/archives/gstreamer-bugs/2018-August/223857.html where they say

    Committed without the gl part which was installed into libdir on purpose. You will have to take account of that in your packaging.

  • libice: gstreamer-check-1.0.pc not found (and indeed it's not there)
  • core configure output:
    -Found pkg-config: /nix/store/4iij6zf2z7gx68rx3kg28xi5rqvnfja6-pkg-config-0.29.2/bin/pkg-config (0.29.2)
    -Dependency libunwind found: YES 1.3.1
    -Found CMake: NO
    -Dependency libdw found: NO (tried pkgconfig and cmake)
    -Message: Support for backtraces is partial only.
    +Dependency libunwind skipped: feature libunwind disabled
    +Dependency libdw skipped: feature libdw disabled
    +Checking for function "backtrace" : YES
    • likely the same as before (partial support), but I improved it by adding an elfutils dep which provides libdw
  • core configure output:
    -Build targets in project: 163
    +Build targets in project: 15
  • gst-validate build complains that GstPbutils-1.0.gir couldn't be found / GEN'd even though it says checking for GST_PBUTILS... yes before
    • probably due to core configure output: Program g-ir-scanner skipped: feature introspection disabled

Things done

  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

Tested via:

NIX_PATH=nixpkgs=. nix-shell -p gst_all_1.gstreamer -p gst_all_1.gst-plugins-good -p gst_all_1.gst-plugins-base -p gst_all_1.gst-plugins-bad -p gst_all_1.gst-plugins-ugly -p gst_all_1.gst-libav -p gst_all_1.gst-rtsp-server -p gst_all_1.gst-validate -p gst_all_1.gst-vaapi

CC @matthewbauer

TODO

  • Merge #54380
  • Somebody should double-check whether indeed all patches removed in this PR have all their contents merged upstream, or if some were merged only partially
  • Check that the vastly-increased gst-plugins-bad integrations work on Darwin, make those conditional that don't
  • This needs some real testing from some real NixOS users, as I haven't checked whether any of the multimedia functionality actually works (I only built it)
@nh2 nh2 changed the title The latest gstreamer version. gstreamer-1.15.1 Jan 21, 2019
@nh2 nh2 requested a review from matthewbauer Jan 21, 2019
@nh2 nh2 mentioned this pull request Jan 21, 2019
10 tasks
@jtojnar
Copy link
Contributor

@jtojnar jtojnar commented Jan 21, 2019

I did not enable gtksink by default due to its closure size (150 MB vs 300 MB).

Also please note that 1.15 is an unstable development branch for 1.16 stable release.

Loading

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jan 21, 2019

I did not enable gtksink by default due to its closure size (150 MB vs 300 MB).

@jtojnar That makes sense; the PR doesn't change this, it is still optional.

Also please note that 1.15 is an unstable development branch for 1.16 stable release.

Oops OK. Let's wait until 1.16 before continuing with this then.

Loading

@nh2 nh2 changed the title gstreamer-1.15.1 WIP: gstreamer-1.15.1 (to eventually become the gstreamer-1.16 PR) Jan 21, 2019
@jtojnar
Copy link
Contributor

@jtojnar jtojnar commented Jan 21, 2019

The removed patches come from https://bugzilla.gnome.org/show_bug.cgi?id=794856 and have been committed.

Loading

@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jan 21, 2019

The removed patches come from https://bugzilla.gnome.org/show_bug.cgi?id=794856 and have been committed.

@jtojnar Thanks! I ticked off the bullet point in the issue description based on this feedback.

closure size

We will also want to do a closure size check on the added -bad dependencies eventually.

I think it makes sense that it'll increase and will likely be worth it, as for end users nothing is more annoying than a gstreamer plugin but it not working on your machine because the distro hasn't enabled it, and -bad is supposed to contain as many integrations as possible, but we should still know the number.

Also, I wouldn't be opposed to introducing feature flags as arguments to the callPackage for each of the external dependencies so that users can easily build a smaller -bad package for their use cases, but I think the default should be ? true for most features so that in the default case multimedia "just works".

Loading

@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from 39eb078 to 802bc29 Jan 22, 2019
@jtojnar
Copy link
Contributor

@jtojnar jtojnar commented Mar 6, 2019

115.2 was released, 1.16 in few weeks: https://gstreamer.freedesktop.org/news/#2019-02-27T10:00:00Z

Loading

@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from 512cee1 to 4876f0b May 1, 2019
@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 1, 2019

1.16 was released.

I've updated the PR with another commit to update to 1.16 and fix a few build errors.

I've also made an upstream PR to fix a script not being executable, which broke the build:

https://gitlab.freedesktop.org/gstreamer/gstreamer/merge_requests/141

Loading

@nh2 nh2 changed the title WIP: gstreamer-1.15.1 (to eventually become the gstreamer-1.16 PR) gstreamer-1.16.0 May 1, 2019
@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 1, 2019

Let's see if this works:

@GrahamcOfBorg build gst_all_1

Loading

@adisbladis
Copy link
Member

@adisbladis adisbladis commented May 1, 2019

@GrahamcOfBorg build gst_all_1

Loading

pkgs/development/libraries/gstreamer/bad/default.nix Outdated Show resolved Hide resolved
Loading
pkgs/development/libraries/gstreamer/bad/default.nix Outdated Show resolved Hide resolved
Loading
pkgs/development/libraries/gstreamer/bad/default.nix Outdated Show resolved Hide resolved
Loading
@nh2 nh2 force-pushed the gstreamer-1.15.1 branch 2 times, most recently from 7750c09 to d563757 May 3, 2019
@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 3, 2019

@GrahamcOfBorg build gst_all_1

Loading

@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 3, 2019

I've backed out the evaluation fixes Darwin out of this PR, as that isn't really related to the upgrade, and will handle those with a separate branch.

If ofborg goes green, this is good to merge from my side.

Loading

@nh2 nh2 mentioned this pull request May 3, 2019
12 tasks
@jtojnar
Copy link
Contributor

@jtojnar jtojnar commented May 3, 2019

Dependency changes described in release notes do not affect us.

What is up with ffmpeg? And do we care about the rust bindings?

Loading

@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 4, 2019

Dependency changes described in release notes do not affect us.

Thanks for checking!

What is up with ffmpeg?

I've updated gst-libav (which is the ffmpeg binding, right?) in this PR as one of the many gst packages, and it builds successfully.

And do we care about the rust bindings?

To my knowledge they aren't packaged in nixpkgs yet.

I'll leave that to somebody else, as I have no experience with packaging Rust for nixpkgs, and care mainly about the update to 1.16 and the improved meson build system I've added in this PR.

Loading

@nh2 nh2 mentioned this pull request May 4, 2019
11 tasks
@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from d563757 to b84a8ad May 4, 2019
@nh2
Copy link
Contributor Author

@nh2 nh2 commented May 5, 2019

@jtojnar Good to go?

Loading

fdo-mirror pushed a commit to freedesktop/gstreamer-gst-plugins-bad that referenced this issue May 13, 2019
This was wrong since commit c360cee.

Also fix incorrect indentation (tab instead of spaces).

Found using hermetic builds with Nix:

    NixOS/nixpkgs#54398 (comment)
@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented May 14, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/what-are-your-goals-for-19-09/2875/24

Loading

nh2 added 4 commits Jun 19, 2019
During the 1.14 -> 1.15 upgrade, lots of stuff stopped working because
gstreamer changed what features are enabled by default and which ones are
automatically turned on/off via pkgconfig dependency detection.

This resulted in the `gstreamer` ("core" attribute in nixpkgs) package
to have only 15 of its previous 163 build targets enabled, and downstream
packages breaking correspondingly.

To ease maintainability and to ensure users will find the expected features
available (and when not, will see in the nix file why not), we now pass
the `-Dauto_features=enabled` Meson build flag to all gstreamer builds,
which sets all `auto` dependencies to `enabled`, and we explicitly disable
those that we can't build.

This means in particular that `gst-plugins-bad` now has vastly more integrations
(namely all for which nixpkgs has libraries available).
Changes needed:

* core: dbghelp disabled as it's not needed
* core: Fixed script not being executable and skipped by patchShebangs
* base: gtk_doc disabled because of wayland symbol error
* ges: Patch removed that was merged upstream:
  https://bugzilla.gnome.org/show_bug.cgi?id=794856#c16
@nh2 nh2 force-pushed the gstreamer-1.15.1 branch from b84a8ad to 3be4c04 Jun 19, 2019
@nh2
Copy link
Contributor Author

@nh2 nh2 commented Jun 19, 2019

@GrahamcOfBorg build gst_all_1

Loading

@nh2 nh2 merged commit a4c5d37 into NixOS:master Jun 20, 2019
17 checks passed
Loading
jtojnar added a commit to jtojnar/nixpkgs that referenced this issue Jun 25, 2019
We disabled them by default when they were introduced in

NixOS#42994

but enabling them probably makes more sense:

NixOS#54398
NixOS#63491
@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Jul 3, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/what-are-your-goals-for-19-09/2875/29

Loading

@nixos-discourse
Copy link

@nixos-discourse nixos-discourse commented Jul 31, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/nixos-weekly-11-nixery-nixfmt-and-cachix-releases-nixcon-2019-tickets-a-job-and-first-impressions-post/3591/1

Loading

fdo-mirror pushed a commit to freedesktop/gstreamer-gst-plugins-bad that referenced this issue Aug 8, 2019
This was wrong since commit c360cee.

Also fix incorrect indentation (tab instead of spaces).

Found using hermetic builds with Nix:

    NixOS/nixpkgs#54398 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants