nixos/wireplumber: add extraConfig / extraScripts options for WirePlumber 0.5

[hcsch]

Description of changes

Follow up to #282377 and #291946. #282377 broke the use of environment.etc."wireplumber<...>" and did not yet add an extraConfig option to make up for that, so currently WirePlumber can only be configured via services.pipewire.wireplumber.configPackages. This is hardly ergonmomic, so this PR adds extraConfig / extraScripts options to fix that.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[hcsch]

@K900 I've created this PR now to try and fix the lack of an extraConfig option for WirePlumber. I'm not quite sure about some details yet though:

Do you know if WirePlumber .conf files (pre 0.5) can be split into multiple files in a conf.d of the same name? I couldn't find anything about that and in some very basic tests that didn't seem to be the case. If you can't split up .conf files in 0.4 I'll have to patch together a single config file from strings to link into the configFiles env. Not sure how duplicate definitions in .conf files work for 0.4 either though and I haven't gotten around to testing that. If duplicate definitions don't work or conf files don't support merging of objects, then users would have to copy the whole (or parts of the) default .conf file into their config, not just what they want to change.

All in all, this is not looking quite that trivial if extraConfig should work well and allow configuration of every aspect of WirePlumber. Perhaps reverting would be better until 0.5 lands after all...
I'm quite honestly a bit stressed out by how much breakage my changes have caused and want to get further changes right

[hcsch]

From a bit of testing I've done and from what I can tell based on https://gitlab.freedesktop.org/pipewire/wireplumber/-/blob/0.4/src/config, the top-level, non-lua .conf files for single-instance / multi-instance mode can't have fragments merged into them from the respective .conf.d in 0.4. Just copying the default contents and append some attributes (even if they are then duplicated in the file) works though. From how wireplumber behaved when I tested this (with the context.properties > log.level attribute) it seems that the latest definition of an attribute is what is used. Since you have to redefine top-level attributes though, you'll inadvertently unset some defaults if you don't copy those defaults over, even if you only want to specify one sub-attribute. So while people will be able to add config lines to .conf files, they will likely have to copy over some defaults as far as I can tell right now.

From a cursory search it also seems that nixpkgs doesn't currently (explicitly) support the multi-instance mode of wireplumber, right? I've added options for the config files relevant for that as well, just in case linking to the appropriate documentation. If you think it'd be less confusing to just not have those options, please do tell.

[hcsch]

A caveat with the extraConfig option is that due to how it will append to the default .conf files provided by the wireplumber package the options will take lib.types.lines as opposed to pkgs.formats.json.type. Otherwise the module would have to strip outer curly braces and the resulting .conf file would be a mix of the lax SPA JSON and strict JSON. That should probably work, but I'm not sure that is that good an approach. With the extraConfig options just being strings, users can also just copy parts of the contents of the default .conf files as opposed to having to rewrite the default they want to change a sub-attribute of in nix.

[hcsch]

51e50d7 should probably go into a separate PR. I'll strip that out into a separate one when cleaning up this PR.

[hcsch]

I've tested the current state of this PR on my NixOS setup with nixpkgs on nixos-unstable. Appending to /etc/wireplumber/wireplumber.conf via extraConfig.wireplumber works, adding a lua config to /etc/wireplumber/main.lua.d/ via extraLuaConfig.main works, and so does adding a lua script to /etc/wireplumber/scripts/ via extraScripts.
I've also tested that wireplumber picks up log.level = 4 in wireplumber.conf configured with the extraConfig option (see example for the option).

[K900]

This is a very cool example.

[hcsch]

This one came easily and I'm rather happy with it as well. I'll try and see if I can find similarly good examples for the others as well

[hcsch]

I've added an example to the bluetooth and policy Lua config each. I gotta note that I've not tested these examples. I might try to today or in the next few days, but especially the policy config seems a bit hard to test since I know of nothing that actually uses policy defined endpoints at the moment (in the usual desktop setup of wireplumber that I have). Then again, maybe I just need to look harder.

[hcsch]

I've created #293248 to document the breaking change as it is now. Once that lands I'll add a commit to this PR to update the release notes again with the options added in this PR as a migration option (currently #293248 adds a note stating that configPackages is the only way to configure WirePlumber until this PR lands).

[K900]

Wireplumber 0.5 is RC1 now, and moved back to old config structure: #278760

Maybe we should update this PR to use 0.5 config already?

[hcsch]

Probably a good idea since it can be one option less with 0.5. extraLuaConfig can just be rolled into extraConfig then after all.
Do you think 0.5 will become stable soon enough?

[hcsch]

Sorry for the lot of force-pushes, perhaps I should be making more draft commits and squashing them again later.
I've successfully tested the bluetooth headphone config example and the log-level example. The virtual-item/policy one I don't think I'll be able to test and the stream rules for setting a default volume for Celluloid I sadly haven't been able to get to work yet.
Do you perhaps have some ideas for that @K900, or do you think I should just scrap that example for now, as nice as it might be?

Over all the current changes should work for configuring WP 0.5 with extraConfig and the example should be okay-ish (minus the ones that I haven't gotten to work)

[hcsch]

Oh and should I roll #293248 into this PR, or should I keep that separate?
(I'll also update the wiki again once this gets merged)

[K900]

No strong preference either way. I'll take a closer look at this tomorrow.

[K900]

Overall very cool, can you promise the examples still work? :)

[sersorrel]

There's now some proper migration documentation from wireplumber's side: https://pipewire.pages.freedesktop.org/wireplumber/daemon/configuration/migration.html

It seems from some brief experimentation that scripts in /etc/wireplumber/scripts aren't loaded any more, they have to be somewhere in $XDG_DATA_DIRS (I guess that's probably going to be /run/current-system/sw/share).

[camelpunch]

This PR would definitely have saved me some time. I'm tempted to go further in a future PR by providing a nix-native configuration for the JSON config, like many other modules do.

[hcsch]

Sorry for the lack of activity last week. I've rebased to the current master and have also adopted the style of inheriting functions from lib in the let ... in at the top according to #299299. I think I'll just remove the examples I can't get to work / verify (Celluloid default volume and virtual-items). They can be added again later if/when somebody gets them to work. I'll add a short note about the dotted attribute names and how to write them in nix expressions to the description and then this should be good to go.

[hcsch]

How do you think that extraScripts should be best added? I'm currently thinking of adding them to XDG_DATA_DIRS in the environment for the wireplumber service. That way they should be able to override existing script files provided by wireplumber. So something like /nix/store/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-extra-scripts/share:/run/current-system/sw/share as XDG_DATA_DIRS for the service.

[hcsch]

Then again. It'd be nicer for users if you could just take a peek at /run/current-system/sw/share/wireplumber/scirpts and everything relevant was in there (except for user local changes), I imagine. And /run/current-system/sw/share should be in XDG_DATA_DIRS by default anyways. Not quite sure how share/wireplumber currently ends up in /run/current-system/sw or how to add files to the that are preferred over those in the wireplumber package. Do I just chuck a package with them in environment.systemPackages with lib.mkAfter or lib.mkOrder 1200 or something like that?

[hcsch]

I've changed the implementation of extraConfig, extraScripts, and configPackages to now use the generated config env package in the XDG_DATA_DIRS variable of the WirePlumber service. This way environment.etc isn't used at all. Putting distro provided configuration / script files into XDG_DATA_DIRS also seems to be the currently recommended approach. I'm not sure if this would have been possible with WirePlumber 0.4, but this way we should cause almost no breakage for people upgrading from 23.11 to 24.05 (beyond that caused by the configuration format change).

I've tested that configuration is picked up by WirePlumber with the LDAC quality example form extraConfig. Scripts are picked up as well, which I tested with the following component and script definitions:

{
  services.pipewire.wireplumber = {
    extraConfig = {
      "test-script-component" = {
        "wireplumber.components" = [{
          name = "test-script.lua";
          type = "script/lua";
          provides = "custom.test-script";
        }];
        "wireplumber.profiles".main."custom.test-script" = "required";
      };
    };
    extraScripts = {
      "test-script" = ''
        print("Hello, world!")
      '';
    };
  };
}

This successfully prints "Hello, world!" in the WirePlumber log.

I still have to tweak the implementation a bit so that scripts can not only be directly in scripts/, but also in subdirectories, and then this should ready for real this time. I'll squash the commits at that point.

[K900]

Upstream has a big migration guide now: https://pipewire.pages.freedesktop.org/wireplumber/daemon/configuration/migration.html

[hcsch]

Finally got around to finishing this. extraScripts now supports putting scripts in any relative path one wishes below share/wireplumber/scripts. I've tested both extraConfig and extraScripts with the example given in extraScripts. WirePlumber successfully prints "Hello, world!". I sadly don't really have a better example script, but I hope this is enough to at least show people the syntax for how to add scripts.
extraConfig and extraScripts now get grouped into their own packages built with pkgs.buildEnv. This way you can easily tell where config/script files are coming from with a tool like nix-tree (see image below showing the deps of unit-wireplumber.service in user-units < etc).

This is ready for review again now, and I'll squash the commits after that for merging.

Thanks for your help and patience so far :)

[K900]

SGTM. Squash?

[hcsch]

I've squashed the commits. If you have any more suggestions for what to test before getting this merged, feel free to tell me and I'll try my best to give those a try.