Build system incomplete? How to build?

[hartwork]

Hi!

From a quick look my impression is that the current build system at src/CMakeLists.txt is incomplete, e.g. because:

• there is no call to project(..)
• there is no call to cmake_minimum_required(..)
• there is no call finding libvisual::libvisual to make the target available
• the if(NOT ENABLE_LIBVISUAL) bypasses most of the build system by default

How do you currently build projectM_libvisual?

I see partial CMake removal commits b79b98a and 000b437 but they don#t explain themselves to me. What was the idea?

I'm happy to provide pull request but I first need to understand status quo better. Thanks!

Best, Sebastian

[hartwork]

@projectM-visualizer any thoughts?

[revmischa]

I've no idea if this plugin has worked in the last decade or not, probably the work was not complete, if you want to update it you are more than welcome! Feel free to open PRs or chat with us on Discord

[kblaschke]

Having the build system work again is great, will have a look at the PR!

Besides that, the plug-in will require some refactoring to catch up with changes in libprojectM to make it build against the upcoming 4.0 release, specifically:

• Use the new C API instead of the C++ one.
• Replace the "config.inp" file support with something else, as it was removed from libprojectM

If you've got the time to implement these changes as well, that'd be gorgeous!

[hartwork]

Hello @kblaschke,

I was already wondering about how API- and ABI-compatible projectM >=4 and <4 are, so that is good to know. Given that distros have projectM <4 and a broken libvisual-projectm 2.1.0 packaged today, I believe there should be at least one more release of libvisual-projectm as 2.1.x on the projectm <4 track, e.g. to give distros an easy way to a fixed package, before moving on to supporting projectM >=4. Let's decouple things that can be decoupled and go for migration to projectM >=4 after the next release rather than before, please. What do you think?

Best, Sebastian

[kblaschke]

Basically zero compatibility, version 4 has a completely new API and doesn't export any of the previous symbols anymore. V3 only had a C++ API, the new version is just pure C with basic types to ensure maximum compatibility.

But we've incremented the .so version, which means Linux distros can install version 3 and 4 libraries at the same time (but not the -dev package).

Should also be relatively easy to implement both APIs in the libvisual plug-in using #ifdefs and select the appropriate one based on the development headers.

[hartwork]

Basically zero compatibility, version 4 has a completely new API and doesn't export any of the previous symbols anymore. V3 only had a C++ API, the new version is just pure C with basic types to ensure maximum compatibility.

Hi @kblaschke, I understand.

But we've incremented the .so version, which means Linux distros can install version 3 and 4 libraries at the same time (but not the -dev package).

If people are linking against -lprojectM then just a bump of soversion is likely not enough, same with pkg-config where pkg-config --libs libprojectM would need to not start pulling in 4.x to not break existing build systems. I believe it needs something like libprojectM-4.so.* and libprojectM-4 for a pkg-config name or something. Libvisual has the same problem and they (before I got involved upstream) solved it like this:

# pkg-config --libs libvisual-0.4
-lm -ldl -lvisual-0.4

What do you think?

Should also be relatively easy to implement both APIs in the libvisual plug-in using #ifdefs and select the appropriate one based on the development headers.

If we support both versions at the same time, the CI will also have to and then we need to compile two different versions of projectM from sources and all. Can be done, but given the complexity and effort — just to understand better — why keep supporting the old API after release libvisual-projectm 2.1.1?

[kblaschke]

What do you think?

Might work, yet I don't really see a point in having multiple versions of the pkg-config file or library installed while the include dir will only allow a single version at any time anyway, no matter how we call the .pc files. When building against a third-party library, a build system always needs to perform some pre-build checks. Sadly, pkg-config doesn't provide a versioning mechanism on its own, leaving the build scripts (e.g. configure) to checks headers etc. and figure it out.

Can be done, but given the complexity and effort — just to understand better — why keep supporting the old API after release libvisual-projectm 2.1.1?

Sure, I don't see any reason to support the old API anymore. The CI shouldn't have to figure it out - ideally, the build script should be good enough to check against which projectM version it should build (e.g. checking for projectm.hpp -> 3.x, projectm.h -> 4.x) and configure the build accordingly. Not installing the development headers and libraries system-wide really helps with that. CMake easily supports it via CMAKE_PREFIX_PATH, other build systems may or may not have such options. At least for pkg-config, there are environment variables to point it to additional or even completely separate directories with the .pc files to use. I frequently do that when building with crossdev toolchains, e.g. when using an older glibc.

[hartwork]

@kblaschke I did reach out to you via e-mail in the mean time. If it did end up in spam, let me repeat my offer to take this to a call. We both love visualization, have best intentions and share the same native language, yet I see unexpectedly little common ground in our exchange here, not 100% sure why. This community is small and we're in the very same boat and I'd like to understand your side and motivations better.

With that said, what you'll read below is the best I can do without voice chat:

What do you think?

Might work, yet I don't really see a point in having multiple versions of the pkg-config file or library installed while the include dir will only allow a single version at any time anyway, no matter how we call the .pc files. When building against a third-party library, a build system always needs to perform some pre-build checks. Sadly, pkg-config doesn't provide a versioning mechanism on its own, leaving the build scripts (e.g. configure) to checks headers etc. and figure it out.

You said earlier that "which means Linux distros can install version 3 and 4 libraries at the same time" but if both 3.x and 4.x have to provide symlink libprojectM.so as in current…

# ls -l /usr/lib64/libprojectM.so
lrwxrwxrwx 1 root root 20 27. Nov 04:29 /usr/lib64/libprojectM.so -> libprojectM.so.3.1.1

…then it will be impossible to even install both runtime libraries at the same time, because only one package can install a symlink and then you'll need meta packages and means to flip symlinks and that kind of unpleasant complexity, that could be avoided easily.
Let's make it simple, I would like to argue that two things that have "basically zero compatibility" should have zero file overlap. How much cost does it cause to rename the related files for 4.x while you still can? Maybe I can help reduce that pain if it turns out to be significant. I'm sort of worried to get this right before 4.0 final by now.

Can be done, but given the complexity and effort — just to understand better — why keep supporting the old API after release libvisual-projectm 2.1.1?

Sure, I don't see any reason to support the old API anymore.

The reason is temporary and is giving people a quick way out of their broken libvisual-projectm 2.1.0. I got one 👍 on my comment #1 (comment) above, so I seem to not be alone with that. Baby steps and decoupling, first get out a cheap fix, then update on the real thing when users know what their in for and that it's alive.

The CI shouldn't have to figure it out - ideally, the build script should be good enough to check against which projectM version it should build (e.g. checking for projectm.hpp -> 3.x, projectm.h -> 4.x) and configure the build accordingly.

Two different include directories are the classic way to avoid forcing build systems to even do magic detection like that, e.g. SDL 1 and SDL 2 use different folders. I have 148 include subfolders on my machine of projects that all follow that approach:

# ls -1d /usr/include/*-[0-9]*/ | wc -l
148

Not installing the development headers and libraries system-wide really helps with that.

That or resolving naming conflicts at the source, yes.

CMake easily supports it via CMAKE_PREFIX_PATH, other build systems may or may not have such options. At least for pkg-config, there are environment variables to point it to additional or even completely separate directories with the .pc files to use. I frequently do that when building with crossdev toolchains, e.g. when using an older glibc.

All possible yes, but not very convenient. I thought we want to have something that users will have fun using.

[kblaschke]

I did reach out to you via e-mail in the mean time. If it did end up in spam, let me repeat my offer to take this to a call. We both love visualization, have best intentions and share the same native language, yet I see unexpectedly little common ground in our exchange here, not 100% sure why.

Sorry, didn't properly check my mail in the last days, this slipped in-between all the other stuff. I know we share a common language, but since GitHub is public, I'll keep it to English here. We can surely talk on Discord, you can find me on the projectM server or add me directly (my Discord username is CodAv#0175).

You said earlier that "which means Linux distros can install version 3 and 4 libraries at the same time" but if both 3.x and 4.x have to provide symlink libprojectM.so as in current…

Sure, the .so symlink can only exist once, and will regularly point to the latest version. That's also the reason why the actual linked library in the executable will be stored with the SOVERSION suffixed to the extension, e.g. .so.3. Using ldd on any executable will show this. On my system, libmenu.so is an example:

/usr/lib64/libmenu.so -> libmenu.so.6
/usr/lib64/libmenu.so.5
/usr/lib64/libmenu.so.6 -> libmenu.so.6.3
/usr/lib64/libmenu.so.6.3

The .so symlink is more of a compatibility and convenience thing. Libraries on Linux generally follow this structure, linking in a chain like this: .so -> .so.SOVERSION -> .so.LIBVERSION. The linker itself will always use the SOVERSION-suffixed symlink if SOVERSION is set.

This detail aside, we're talking about the pkg-config files, which are really a pain. Renaming them is totally fine for me. Feel free to add it to this macro here, which will take care of all four pkg-config files currently being installed, and create a pull request on my RC2 fixes branch:

https://github.com/projectM-visualizer/projectm/blob/master/cmake/GeneratePkgConfigFiles.cmake

You can retrieve the SOVERSION directly from the target and append it if set:

get_target_property(_target_soversion ${target} SOVERSION)

If you want to add a version number to the include dir as well, e.g. -4.0, I'm also fine with it, as it's probably best to do that before the 4.0 release. I'd also propose to rename it to projectM-4.0 from the current name, libprojectm.
If you do so, please also change the includes of any frontends already using the new 4.0 API, e.g. projectM-visualizer/frontend-sdl2 and projectM-visualizer/frontend-music-plugin.

If there's any other questions or topics, my offer stands to talk on Discord if I've got the time!

[hartwork]

Hi @kblaschke, thanks for your reply!

I know we share a common language, but since GitHub is public, I'll keep it to English here.

Yes, I have and had no intentions on speaking German on GitHub 😃 It was only about 1:1 voice chat.

We can surely talk on Discord, you can find me on the projectM server or add me directly (my Discord username is CodAv#0175).

When trying to join the Discord I get this error…

…asking for phone verification like last time I tried. Discord has no business with my phone number, so I won't be able to join Discord because of how they operate.

Sure, the .so symlink can only exist once, and will regularly point to the latest version. That's also the reason why the actual linked library in the executable will be stored with the SOVERSION suffixed to the extension, e.g. .so.3. Using ldd on any executable will show this. On my system, libmenu.so is an example:

/usr/lib64/libmenu.so -> libmenu.so.6
/usr/lib64/libmenu.so.5
/usr/lib64/libmenu.so.6 -> libmenu.so.6.3
/usr/lib64/libmenu.so.6.3

The .so symlink is more of a compatibility and convenience thing. Libraries on Linux generally follow this structure, linking in a chain like this: .so -> .so.SOVERSION -> .so.LIBVERSION. The linker itself will always use the SOVERSION-suffixed symlink if SOVERSION is set.

My point was that there can only be a single libprojectM.so (symlink) file so the user won't be able to compile and link software against <4 and >=4.0 using -l:projectM in parallel in practice. That's why the name conflict is worth resolving.

This detail aside, we're talking about the pkg-config files, which are really a pain.

I saw the comment in code about them being "a mess" and why. I'm personally very happy with pkg-config and what the project did for the ecosystem.

Renaming them is totally fine for me.

Cool!

Feel free to add it to this macro here, which will take care of all four pkg-config files currently being installed,

Done.

and create a pull request on my RC2 fixes branch:

Given that branch rc2-fixes is in your personal repo rather than https://github.com/projectM-visualizer/projectm, it will not be seen by anyone but you really and it's also rather uncommon to do, and maybe by mistake? I have there therefore created a pull request against the regular main repository where my fork is from and where v4.0.0-rc1 was born, and we can go from there. The pull request is projectM-visualizer/projectm#699.

You can retrieve the SOVERSION directly from the target and append it if set:

get_target_property(_target_soversion ${target} SOVERSION)

I have not made use of that yet, if it's important please let me know during review where you'd want me to use it, if itÄ's important to you, Thanks!

If you want to add a version number to the include dir as well, e.g. -4.0, I'm also fine with it, as it's probably best to do that before the 4.0 release.

Agreed, yes.

I'd also propose to rename it to projectM-4.0 from the current name, libprojectm.

Okay, done.

If you do so, please also change the includes of any frontends already using the new 4.0 API, e.g. projectM-visualizer/frontend-sdl2 and projectM-visualizer/frontend-music-plugin.

I can do that, but let's first get the main pull request merged and #2 on libvisual-projectm reviewed at least, so I don't have even more things hanging in the air, and we can take it from there.

If there's any other questions or topics, my offer stands to talk on Discord if I've got the time!

It will have to be Jitsi, BigBlueButton, Signal, Telegram, (..) or plain phone with me — not Discord — but thanks for the offer!

Best, Sebastian

[kblaschke]

Given that branch rc2-fixes is in your personal repo rather than https://github.com/projectM-visualizer/projectm, it will not be seen by anyone but you really and it's also rather uncommon to do, and maybe by mistake?

No, it's not a mistake and also not uncommon, it's rather a "best practice".

Let me explain: I always work in my fork, and then create a PR once I've got my changes sorted and ready. This both keeps the main repository clean. In addition to that, people tend to use branches in the main repo and even base their own work on them, which is fine. But my personal working branch will be rebased, squashed and edited frequently, which causes a lot of problems for the others.

For more important changes, I create a draft PR like the one you can see in the main repo. This makes the branch visible, but still makes it clear not to base any work on this unless there's a good reason.

If you personally don't like this practice, and only want to merge directly into the master branch, then you're free to do so.

That said, please keep any further discussion on the topic of the actual changes. If you want to discuss the other things in more detail, please feel free to send me an email.