add support for multiple backend (epoll, libevent...)

[aconchillo]

No description provided.

[civodul]

Bon dia @aconchillo, awesome work! 👍

FWIW, I'm planning to use Fibers in the Shepherd, the init used on Guix System. It has to work on GNU/Hurd, which currently lacks epoll but has good'ol poll that libevent must be using there. In short: I'm very interested in seeing this merged. :-)

[emixa-d]

    printf("Max events fired. Ignoring\n");

I don't think that applications expect fibers to write to stdout. (current-error-port) would probably be fine though.

[emixa-d]

Also, can this ignoring cause some events to be missed?

[aconchillo]

Bon dia @aconchillo, awesome work! 👍

Hola! Thanks! @Habush was the one to do all the initial work, I just took the torch 😄

FWIW, I'm planning to use Fibers in the Shepherd, the init used on Guix System. It has to work on GNU/Hurd, which currently lacks epoll but has good'ol poll that libevent must be using there. In short: I'm very interested in seeing this merged. :-)

Hopefully some time before summer (I hope you can wait a bit longer). I've been very busy lately (actually in the middle of job switching right now) but I started working on the autotools changes a couple of months ago even though I couldn't do much.

Bear with me! 😅

[emixa-d]

If we add a fallback for clock_nanosleep, shouldn't it be static (maybe not possible with the FFI) or at least namespaces (e.g., _fibers_clock_nanosleep or such)? Otherwise, there's a potential risk with interfering with other libraries' or the program's replacement of clock_nanosleep.

Edit: alternatively, IIUC, there are a number of flags you could set when loading a library or maybe when compiling to keep the symbol both visible and that-library-only (not visible to other libraries or the program unless specifically requested from that library).

[emixa-d]

I don't think unconditionally adding a replacement for clock_nanosleep is good -- what if Darwin improves later and adds a clock_nanosleep? I think some AC_CHECK_FUN would be appropriate.

[emixa-d]

Looking at my local Guile REPL, we already have 'getaffinity' / 'setaffinity'.

[aconchillo]

Looking at my local Guile REPL, we already have 'getaffinity' / 'setaffinity'.

Not on macOS REPL.

[emixa-d]

On epoll platforms, the epoll library is not loadeded when cross-compiling

(eval-when (eval load compile)
  ;; When cross-compiling, the cross-compiled 'epoll.so' cannot be loaded by
  ;; the 'guild compile' process; skip it.
  (unless (getenv "FIBERS_CROSS_COMPILING")
    (dynamic-call "init_fibers_epoll"
                  (dynamic-link (extension-library "fibers-epoll")))))

However, AFAICT, the libevent and darwin code doesn't have anything equivalent, is this somehow unnecessary when cross-compiling to non-Linux?

[aconchillo]

@emixa-d hi! you made some comments to a particular commit. it might be too much to ask, but would you mind adding the comments in this PR? otherwise it's very hard to keep track of those comments. thank you!

[aconchillo]

@emixa-d hi! you made some comments to a particular commit. it might be too much to ask, but would you mind adding the comments in this PR? otherwise it's very hard to keep track of those comments. thank you!

also, just realized your comment are for a very old commit and some of them don't apply anymore (e.g. posix-clock ones).

[emixa-d]

I put the comments on the PR this time, and added a few more.

[aconchillo]

@Habush @wingo @civodul an initial pass of this PR is done. we now have all unit tests passing with epoll and libevent on Linux and libevent on macOS. There are a couple of tests I had to disable on macOS because of the lack of SOCK_NONBLOCK (which btw, we should probably support?).

Also, the macOS implementation is considerably slower (because the nanosleep clock is not implemented properly yet).

I still have to go through @emixa-d's comments, but right now you can do:

./configure

which will use epoll on Linux, and:

./configure --with-libevent=yes

which will enable the libevent bakend both on Linux or macOS.

[aconchillo]

On epoll platforms, the epoll library is not loadeded when cross-compiling

(eval-when (eval load compile)
  ;; When cross-compiling, the cross-compiled 'epoll.so' cannot be loaded by
  ;; the 'guild compile' process; skip it.
  (unless (getenv "FIBERS_CROSS_COMPILING")
    (dynamic-call "init_fibers_epoll"
                  (dynamic-link (extension-library "fibers-epoll")))))

However, AFAICT, the libevent and darwin code doesn't have anything equivalent, is this somehow unnecessary when cross-compiling to non-Linux?

I assume we would need to do the same. Otherwise the library will be loaded and will of course fail, I assume that was the intention here.

[aconchillo]

Also, can this ignoring cause some events to be missed?

I believe that only one callback will be triggered for a registered event. So, in theory, if we add 3 events and all of them are active we will get 3 callbacks only.

However, one problem would be adding more FDs to monitor. In that case, we would have more events than what we previously anticipated. So, I think the fix would be to increase the events vector when we add new events. Since libevent doesn't have the notion of maxevents like epoll_wait has.

[aconchillo]

@emixa-d All comments have been addressed, I just added a question for this one: #53 (comment)

[aconchillo]

@emixa-d I'm waiting for the author of that article to mention something about th elicense or give us permission to add a copyright line with his name on it and with LGPL3+. If I don't hear back, what would be the options? Looking at that code I don't see many other ways to implement that.

Aside from that, is there anything else missing that would be a blocker? (Other improvements can be done in future PRs). Not sure if you had time to review again. No rush at all! Thank you! 🙌

[emixa-d]

On options: there exist a bunch of exceptions in copyright.

For example, https://fsfe.org/freesoftware/legal/faq.en.html mentions

Which files are copyrightable?

All files that are original works of authorship are copyrightable. In essence. If someone sat down typing their own original thoughts on a keyboard, then that person holds copyright over the output. Common examples are source code, documentation, audio, and video.

However, there are some edge cases. For example, the program

print("Hello, user!")

probably does not meet the threshold of originality because it is too trivial. Similarly, data files and configuration files may not meet that threshold either.

That's a bit of an extreme case, but you could look into whether this 'threshold of originality' or some other limitation applies in our case.

If all fails, we could also ignore the setaffinity stuff on Darwin -- while from what I've read, setting the affinity appropriately can be improve performance a little, to my understanding it's not essential.

[emixa-d]

Aside from that, is there anything else missing that would be a blocker? (Other improvements can be done in future PRs). Not sure if you had time to review again. No rush at all! Thank you! 🙌

One thing I forgot: you are making a pipe, but aren't setting O_CLOEXEC, which makes spawning new processes in a clean environment more complicated. Preferably O_CLOEXEC would be set atomically (instead of with fcntl), using the optional argument of 'pipe':

From the Guile NEWS (changes in 3.0.9)

** `pipe' now takes flags as an optional argument

This lets you pass flags such as O_CLOEXEC and O_NONBLOCK, as with the
pipe2(2) system call found on GNU/Linux and GNU/Hurd, instead of having
to call `fnctl' afterwards.  See "Ports and File Descriptors" in the
manual for details.

Don't know if Darwin has that, though. However, according to the documentation, 'pipe' will throw a 'system-error' with ENOSYS when the second argument is unavailable so you could implement a fcntl fallback with that.

[gdt]

if setaffinity is an optimization, then building on a platform without support should issue a configure or build time warning and/or a run-time notice, and just use a null implementation that does nothing. It's far better to function slightly slower than to not function.

I previously had the impression from how this was handled, despite what I would guess it does, that it was somehow mandatory.

[gdt]

FWIW:

• POSIX says pipe(2) does not accept flags
• NetBSD has O_CLOEXEC in pipe2
• macOS 10.13 (there` isn't really darwin these days IMHO) does not have pipe2
• Other people say macOS lacks pipe2

[aconchillo]

Aside from that, is there anything else missing that would be a blocker? (Other improvements can be done in future PRs). Not sure if you had time to review again. No rush at all! Thank you! 🙌

One thing I forgot: you are making a pipe, but aren't setting O_CLOEXEC, which makes spawning new processes in a clean environment more complicated. Preferably O_CLOEXEC would be set atomically (instead of with fcntl), using the optional argument of 'pipe':

From the Guile NEWS (changes in 3.0.9)

** `pipe' now takes flags as an optional argument

This lets you pass flags such as O_CLOEXEC and O_NONBLOCK, as with the
pipe2(2) system call found on GNU/Linux and GNU/Hurd, instead of having
to call `fnctl' afterwards.  See "Ports and File Descriptors" in the
manual for details.

Don't know if Darwin has that, though. However, according to the documentation, 'pipe' will throw a 'system-error' with ENOSYS when the second argument is unavailable so you could implement a fcntl fallback with that.

Ah, good catch! Unfortunately, macOS doesn't have pipe2 so for now I just set O_CLOEXEC using fcntl. Once guile 3.0.9 is released we can switch to the pipe optional argument.

[aconchillo]

if setaffinity is an optimization, then building on a platform without support should issue a configure or build time warning and/or a run-time notice, and just use a null implementation that does nothing. It's far better to function slightly slower than to not function.

I previously had the impression from how this was handled, despite what I would guess it does, that it was somehow mandatory.

I decided to get rid of macOS getaffinity/setaffinity implementation. The unit tests timing look mostly the same as before, so no real impact. Also, it seems it's not actually possible to implement setaffinity/getaffinity properly, see for example:

https://developer.apple.com/forums/thread/44002

So, with this we also solve the copyright issues. Please, take a final look and let me know how everything looks.

Once you guys give the thumbs up I will merge this PR and cut a 1.2.0 release soon. Again, thank you so much for all the feedback! This was started 1 and a half years ago by @Habush and it's taken a while, mainly due to lack of time, but we almost there!

[aconchillo]

Unless there are more comments, I'm planning to merge this PR next Thursday Dec 22 and cut Fibers 1.2.0 before Dec 25 as a holidays gift.

[aconchillo]

The time has come. If there are any issues or we fnd better ways on how to do this we can fix later but I would say we got to a nice help. Thank you @Habush for starting this and @emixa-d and @gdt for providing such valuable feedback and reviews.

Off we go! Merging!

[Habush]

Hi @aconchillo. I'm glad to see this work develop into this and be finally merged. Thank you for all your work. Happy Holidays!

[aconchillo]

Just realized I said I would release Dec 25... oh well 😅