Kdenlive/mlt: enable glaxnimate to add vectorial animations

[tobiasBora]

Description of changes

Fix #209923

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

@turion @goibhniu

[tobiasBora]

This PR adds also a new functionality in KDEnlive to edit vectorial animations. See tutorial here https://www.youtube.com/watch?v=em-km2xzVnw (french, but should be fairly easy to follow)

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/kdenlive-missing-mlt-module-glaxnimate/24486/6

[SuperSandro2000]

Please follow the contributing guide when naming your commits.

[tobiasBora]

Ok so I investigated a bit further: Mlt creates its own library installed in $out/lib/mlt-7/libmltglaxnimate.so. The source (a single file + some yaml) of this library requires a few things from glaxnimate notably src/core/application_info_generated.in.hpp (just containing #define … for the version of the project, the description…) and io/io_registry.hpp (see here, the cmake rule being here). The problem is that glaxnimate is not exporting any libraries, so instead of copy/pasting code from glaxnimate, MLT just uses some code using git submodule. To compile these files, a CMake file has been written here. While it's surprisingly long, as far as I understand it just adds glaxnimate's code in the include directory, add the source, and installs the library.

Note that as I understand, because all .cpp files of glaxnimate are included in the source of the library, there is no need anymore to link any potential (non-existing) glaxnimate library since it's packed into MLT, which is confirmed by a quick look at the final library:

$ ldd libmltglaxnimate.so | rg glax

So to fix it properly, we would need to patch glaxnimate to install the libraries, and patch MLT to use glaxnimate instead of a submodule… This might be doable, but quite a lot of work compared to simply adding the submodule as I do right now… Note that I filled a bug here to ask glaxnimate to install the library files (they might not want in case their library is not stable enough). If the library is not stable, then I guess it's a good idea to keep the submodule as upgrading glaxnimate might change the version used by MLT and break other stuff.

@SuperSandro2000 thanks, I'll follow them better (I didn't know that case was important and used by bots), but since my commit touches two packages, how should I write that? Do I need to write one commit per program?

[tobiasBora]

So I got an answer by glaxnimate's team: https://gitlab.com/mattbas/glaxnimate/-/issues/545. They say they plan to modify their software to cut it in smaller, reusable libraries, but they had no time to do it so far. So I guess for now it's better to use submodules rather than spending time on writing patches that would eventually be unnecessary.

If you are good with that, I can remove glaxnimate from the list of dependencies, squash & modify the commit name once @SuperSandro2000 gives me the appropriate name to give to the commit, and we can merge if everything is good for you.

[SuperSandro2000]

Do I need to write one commit per program?

Yes

So I guess for now it's better to use submodules rather than spending time on writing patches that would eventually be unnecessary.

If the required work is just installing the so, even if it is done via a cp command and linking against that, then that is preferred. Package collections, like nixpkgs, want to avoid vendoring as much as possible because it makes having a single source for a package with all patches impossible.

[tobiasBora]

If the required work is just installing the so, even if it is done via a cp command and linking against that, then that is preferred.

@SuperSandro2000 I understand that Nix tries to limit third party sources, but the software needs the source since glaxnimate provides no library, so I don't think a .so file is even produced (but I did it a while ago so I might remember poorly). I have no idea how to change the whole compilation process in order to avoid this dependency on the source (and I don't have time to spend one week on this), so if you know how to do it feel free to… until then people won't be able to use glaxnimate in KDEnlive.

Overall, I think maintaining a clean repo is nice, but IMHO it should not come at the price of not implementing a functionality because a perfect clean solution is not possible to implement.

[tobiasBora]

Where are we stuck currently?

The code works (or at least used to work, I've not tested recently), I am just waiting for @SuperSandro2000 to accept the fact that I rely on a git submodule following what is done upstream. Since glaxnimate provides no API/.so library for now (it might come later if they find people with good enough cmake expertise), I see no other solution.

Then, I'll properly rebase the commits on the latest nixpkgs version.

[turion]

@tobiasBora see https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#commit-conventions for naming conventions for the commits. I think ideally you would split the commit into two, with these names:

kdenlive: enable glaxnimate
mlt: enable glaxnimate

[tobiasBora]

Should be good now. Note that I handle the path to glaxnimate in a better way now, directly modifying the default path. This way, upgrades will also upgrade this path (unless you changed it yourself), but if you tried before an earlier version that saved the path to glaxnimate, you may need to restore the path to its default value:

If you don't, you might get a prompt asking for the path to glaxnimate when you start to use the plugin (cf tutorial in second message).

[turion]

@ofborg build kdenlive

[turion]

There is something wrong with the build:

building '/nix/store/qw3y9318g1v0i0s8x90dav38ijg4qva6-mlt-7.22.0.drv'...
Running phase: qtPreHook
Error: wrapQtAppsHook is not used, and dontWrapQtApps is not set.
error: builder for '/nix/store/qw3y9318g1v0i0s8x90dav38ijg4qva6-mlt-7.22.0.drv' failed with exit code 1;
       last 2 log lines:
       > Running phase: qtPreHook
       > Error: wrapQtAppsHook is not used, and dontWrapQtApps is not set.

https://github.com/NixOS/nixpkgs/pull/209941/checks?check_run_id=20524628144

I can reproduce the build error locally.

[turion]

It must somehow be related to this part:

  ] ++ lib.optionals (qt != null) [
    qt.wrapQtAppsHook
];

But it builds on master. I don't understand what change creates the error.

[turion]

I think this doesn't quite work as intended anymore. Maybe the issue is that this unconditionally adds qtbase and other qt packages, but mlt can only deal with them when qt != null in the inputs. Maybe throw an error if enableGlaxnimate = true && qt == null?

[tobiasBora]

Oh interesting… I was also getting this error when compiling mlt alone, but I thought that it was expected since qt was not set, thinking that mlt was only compilable with a non-empty qt. Since kdenlive was able to compile mlt I thought that mlt was getting sent qt somehow. But now I'm very confused: why is mlt failing to build while ffmpeg = mlt.override {ffmpeg-full} is not?

[tobiasBora]

Oh, ok, now I get why mlt in kdenlive builds fine: in pkgs/top-level/qt5-packages.nix, there is

  mlt = pkgs.mlt.override {
    qt = qt5;
  };

so KDEnlive is using the one with Qt, explaining why it is not failing to build.

When I have a bit more time, I will try to see if qt is really needed, and if it is needed I will maybe enable glaxnimate only if qt is not null (throwing an error would break the current version, so I'm thinking it might not be the best idea).

[ShalokShalom]

Just as a heads up, this is still the case in the unstable build with Qt6 (kdePackages.kdenlive)

[tobiasBora]

Ok thanks. I need to take a bit of time to fix this issue. The next month will be rather busy for me, I'll see if I can find some time before but it's definitely in my todo list.

[tobiasBora]

This version should work now (let's wait maybe for the tests to finish first). By just specifying the only actually needed library, we get rid of the mlt error (actually, mlt seems to automatically disable glaxnimate support if qt is disabled, without providing any error).

[tobiasBora]

Seems like all tests passes, are we good now?

[SuperSandro2000]

Doesn't this need to be done with substituteAll on the patch file?

[tobiasBora]

Sure, but substituteAllInPlace is already part of Kdenlive's derivation as this file is already patched similarly for other binaries.

[SuperSandro2000]

Can we make this explicit? Using substituteAllInPlace with ambient variables is kinda surpising.

[tobiasBora]

What do you mean? substituteAllInPlace just expects these variables to be in the environment (whether this is good or not is discussed here, but so far no agreement has been reached), and my goal was just to implement this fix following what kdenlive's package is already doing, as I don't want to risk adding bugs doing some sort of refactoring. Or do you mean to put them in env = {} to make it somewhat clearer?

[chayleaf]

I agree, it would be cleaner to use the substituteAll builder on the patch itself. That said, I don't mind either way.

[chayleaf]

Suggested change

, glaxnimate