Compatibility with non-Linux systems (libdbus support) #92
Conversation
These provide a small set of wrapper functions around the low-level libdbus API that act like a shim for sd-bus.
This commit adds necessary modifications to the original Mako code, and puts them behind preprocessor conditions defined by the build system.
This commit adds the dbus-1 library as a dependency if neither systemd, nor elogind is available. It also adds the sd-bus wrappers to the build system in such case.
This commit applies fixes for the following: pool-buffer.c: mkstemp is in POSIX.1-2008, thus requires _POSIX_C_SOURCE to be 200809L or higher, and _X_OPEN_SOURCE 500 does not provide that. event-loop.c: TFD_CLOEXEC is defined as O_CLOEXEC, which also appeared in POSIX.1-2008, thus _POSIX_C_SOURCE 199309L is not enough.
busctl is part of sd-bus, so we don't have it when we are using subd.
A call to dbus_watch_handle fails if there is not enough memory. In such case, the "application may spin in a busy loop on the file descriptor until memory becomes available, but nothing more catastrophic should happen" (from libdbus documentation). I think this is better, than potentially blocking the handling of other fd events with a sleep.
This function was too big, and also didn't do all the necessary resource deallocation in case of errors.
strdup requires _POSIX_C_SOURCE 200112L open_memstream requires _POSIX_C_SOURCE 200809L
Actions were not working, because I misunderstood the return values of the sd_bus_message_read function. The makoctl.libdbus script also needed a fix because dbus-send's syntax for message arguments differs from busctl's.
There's no need to dynamically allocate the list that represents the message's iterator stack in sd_bus_message. +bonus: Fixed two typos.
The return value of successful realloc calls was ignored. This commit also replaces malloc with calloc for the DBusWatch * and pollfd arrays, and improves deallocation in case of errors.
I use plurals in variable names to imply an array, or some kind of list, and this is a struct (that contains two arrays, but still). Also things like watches->watches could be confusing.
For some reason GLibc hides strdup with 200112L despite it being part of IEEE Std. 1003.1-2001.
|
Wow, this is quite impressive work. I wouldn't have bet this would work.
Indeed, I'm not a fan of merging all of this. Maybe if the shim lived in a separate repo that could be a better solution. For those who don't have systemd installed, I had in mind to extract sd-bus from systemd and provide a standalone library for it. It would work well on Linux, but I'm not sure about FreeBSD (epoll-shim can probably make our life easier). Would you be interested in such a project? |
|
I agree, this is awesome but I don't think it belongs in a single dbus client. Extracting this into a library that could be an optional dependency would be ideal. |
Thank you, and as I've said, I'm not surprised :) A little "history", if you don't mind: One of my first attempts at this was to create a separate library that provides the I decided against this design mainly because it would have required significant changes to mako's source, and this is why the sd-bus shim was born. It is not a separate lib mainly because of three reasons:
I know my last point is from an idealist viewpoint, and not practical at all, so I might reconsider extracting this as a library. Maybe as an extension, or rather an additional thin layer above subd.
I have also played with this idea, but got discouraged during my initial attempts (too much Linuxism), and also by the fact that systemd developers said they do not want sd-bus to be platform-independent, and they intentionally use Linux specific features for performance reasons. I'm sure those parts could be replaced, but it feels that it would be a huge undertaking, even for two person. BTW, there is someone on the Gentoo forums, who extracted sd-bus from systemd: https://forums.gentoo.org/viewtopic-t-1077936-start-0.html |
|
It seems like there should be room in the ecosystem for a dbus library that isn't sd-bus, since dbus itself is not systemd and there are alternative implementations of the server side (like dbus-broker) as well. You're right that this isn't a small undertaking, but I could see adding support for an alternative and non-linux-specific library. As you've noted, mako uses a fairly small portion of the dbus api, so having two backends would be relatively low-maintenance assuming the libraries are stable. |
|
Instead of creating a shim for sd-bus (and playing catch up all the time), I think a one time port to gdbus would be better, as glib is cross platform and would work on FreeBSD. This however also introduces a dependency on glib for mako (but also frees it of a dependency on libsystemd). As for the usecase the author has, it would be interesting if mako provides its own interface for notifications (just makoctl talking to the daemon over a socket) instead of the D-Bus name, so that simple usecases can be made possible (like calling from irc clients and scripts) without using D-Bus at all. You could just disable entirely and continue using mako for notifications. |
That's fair. I decided to use sd-bus because all other libraries are pretty bad, but it's true that being tied to Linux is Bad™. Sorry about that, but I don't have an immediate perfect solution for this issue.
Have you tried building only what's necessary for sd-bus?
I'd rather switch completely to the new lib if it's as good as sd-bus. But yeah, it's a lot of work. Completely forking sd-bus might be an option.
-1, glib is godawful.
Not sure we want to support both a D-Bus API and another IPC API. |
|
Sorry for my absence, end of year is always super busy.
Yep, it is definitely something there's room for. There are a few DBus implementations out there, but all of them seem to be tied to Linux. I thought about this a lot, but I don't really have the time to start something this big.
I didn't want to use such a big dependency, and my main goal was to introduce as few modifications to Mako's code as possible. Plus I wanted to learn the libdbus API :)
No need to apologize, I know sd-bus is very appealing, and I know there is no good platform independent alternative. That said, I think the libdbus library is not that awful, and with the automatic
I didn't even get to compile sd-bus sources. They have a lot of utility stuff on which sd-bus depends, and I was not able to build them. Maybe I was just impatient, and maybe I'll give another try during the Christmas holidays. Right now my plan is to extract the shim as a library. |
|
I would suggest libnih-dbus, but it is GPL-2.0-only, and probably not ready to go outside of Linux (although I imagine it could be ported easily). Linking to it would make the resulting GPL-2.0-only. Something tells me *BSD people would not like that. In any case, congrats on writing all that C code! I had a mind to just RIIR haha.
Indeed, extracted logind and sd-bus are not easy to port. Being a moving target makes the effort that much more difficult. |
|
Can we close this out? I'd love to see this find a home, but this PR is not an ideal one. :) |
Hi,
Currently Mako is Linux-only, because sd-bus is Linux-only, and I think there is very little chance it will ever be ported to anything else. I wanted to use Mako though, so started working on bringing libdbus support in. I've had several concepts for how to do it, and finally settled on creating a sort of shim for sd-bus, that would require only a few modifications in Mako's original code. That said, the shim itself is quite large (~800 LOC), and I honestly wouldn't be surprised if you were unwilling to merge that amount of code just to satisfy the needs of - probably - only a handful of people. In case you are, here is a little summary of what's happening inside the shim.
First and foremost, only features currently required by Mako are implemented. This is not just true at the API level, but also at the function level, meaning that there are functions that satisfy Mako's needs, but are not necessarily a 1:1 copy of the original sd-bus functions semantically. Such limitations are commented in the code if known.
The largest difference comes from the fact that libdbus needs you to listen on an arbitrary number of file descriptors instead of just one, like sd-bus. These file descriptors (and their associated DBusWatch watches) are dynamically added/removed/enabled/disabled whenever libdbus needs them. Because of this, libdbus support needs a completely different
init_event_loopfunction, andrun_event_loopalso needs to be modified. The part that automatically handles watches/fds is also one of the three large parts of the shim.Another big difference, and another large part is something that aims to mimic the way sd-bus handles vtables. While sd-bus auto-implements the
org.freedesktop.DBus.Peer,org.freedesktop.DBus.Properties, andorg.freedesktop.DBus.Introspectableinterfaces based on the vtable data, my code only auto-implements the latter for now (it is enough for Mako to work properly). The vtable syntax is also different. It's worth noting that unlike the one vs. many fd difference, these ones could be made more similar to sd-bus relatively easily.The third large part consists of the sd-bus function implementations. The role of
sd_busandsd_bus_errorare roughly the same as their libdbus counterpartsDBusConnectionandDBusError, so I found the easiest way to preserve function signatures was totypedefthem. All of them are opaque types, so this should not be a problem. I've alsotypedef-d my object member implementation (subd_member) tosd_bus_vtable, since they also do roughly the same.DBusMessagehowever is a bit "less" than its sd-bus counterpart, sincesd_bus_messageknows whatsd_busit is associated with, and also keeps track of its current read/write position. Sosd_bus_messagehere is not a simpletypedeftoDBusMessage, but a structure that holds aDBusMessage, a stack of iterators (DBusMessageIter) to track position, and thesd_bus(DBusConnection) pointerb.I think this is enough for a short intro. I've tried to comment explanations of what and why is happening everywhere, maybe it's even a bit too excessive at certain places.
I've been using variations of this code for a few weeks now on my HardenedBSD system, where irssi, and a Python script called from procmail generates notifications. I've also tried to call all methods using dbus-send and/or d-feet, and did the same on a Debian Stretch machine.
Oh, and one more thing: if you want to try this without Xwayland, you will need to start a dbus session, because libdbus can't autolaunch a session without an X session. You can do this by starting the compositor via dbus-run-session, e.g.: