Question: Why would heim depend on a runtime in the future?

[cjbassi]

In the readme it states:

At the moment heim is not dependant on any async runtime, but it will in a future and most probably it will be a runtime crate.

And I'm curious for what reason a library would need to depend on an async runtime?

Btw, really appreciate this library and it's looking good so far!

[svartalf]

Ah, yeah, this is a huge headache for me. I would love not to depend on any async runtime (which would be an obvious choice), but there are two things I need: an async files I/O and a Netlink sockets.

Both are very important for Linux implementation: sync FS is used under the hood right now (which is kinda okay, since I'm working with procfs and sysfs only, but it is still might be a blocking operation) and netlink is the only reasonable choice to fetch the network connections (parsing /proc/net/* files is knowingly slow and provides literally nothing).

I had a discussion at #wg-async-ecosystem about the netlink issue, and it seems that the only realistic way for now would be to implement low-level netlink sockets for all reactors, hide them behind some kind of abstraction and let the library users choose what reactor they are using (probably with crate features like reactor-tokio?).

Same goes to async FS, but I hope that this issue will be solved by community, since FS is far more popular thing than Netlink :)

So, this is a hard problem and I'm still trying to find the most reasonable solution to it ¯\(ツ)/¯

[cjbassi]

Interesting, thanks for the info. A quick glance at async FS in the rust ecosystem reveals that the main (and only) implementation is tokio::fs which explicitly requires the tokio runtime. Hopefully that gets worked on like you said, since we still are quite early in rust async developments 🚀

[little-dude]

I had a discussion at #wg-async-ecosystem about the netlink issue, and it seems that the only realistic way for now would be to implement low-level netlink sockets for all reactors, hide them behind some kind of abstraction and let the library users choose what reactor they are using (probably with crate features like reactor-tokio?).

Hello, I'm the author of https://github.com/little-dude/netlink. I'm wondering what issue exactly you're having with netlink and asynchronousity. These crates have tokio integration, although there are some unresolved issues (like this: little-dude/netlink#39). Fwiw I wouldn't recommend using my crates for now, it's more at the "proof of concept" stage, still.

[svartalf]

Hey, @little-dude! Yes, I saw your crate (tremendous piece of work, I should say), but there are few reasons why I probably will not integrate it:

1. It is based on tokio. While there is nothing bad with it, I want heim to be runtime-agnostic. There will be a tokio integration at Tokio 0.2 integration #82 (and with other runtimes too eventually), but it will be optional, without it library will fallback to a more simple implementation.
2. The only part of netlink protocol I need is the sock_diag, which is unfortunately still in progress

I had not investigated it fully yet, but I was thinking about some async-netlink-socket crate, which provides mio-based socket and high-level sockets around it (for tokio via PollEvented and for other runtimes with some glue and sticks also). With it I'll be able to implement a very small implementation of sock_diag protocol, which could be integrated into the heim; I hope with async_await it will be much easier, because first prototype I wrote was a diabolic hell of combinators.

So, as I said already, I'm still not sure what to do exactly with it and would love to hear your thoughts about it.

[little-dude]

I had not investigated it fully yet, but I was thinking about some async-netlink-socket crate, which provides mio-based socket and high-level sockets around it (for tokio via PollEvented and for other runtimes with some glue and sticks also).

I think netlink-sys does what you describe. It has a basic socket implementation, an optional Evented socket (with --feature mio_support), and an optional TokioSocket (with --feature tokio_support)

The only part of netlink protocol I need is the sock_diag, which is unfortunately still in progress

Yeah, and that's because a fundamental flaws in my netlink_packet crate unfortunately. I have ideas about fixing that, but I'm not sure when I'll be able to pick it up.

I hope with async_await it will be much easier, because first prototype I wrote was a diabolic hell of combinators.

Ha, I know what you're talking about!

[little-dude]

After working for a bit on my netlink crates here is more info:

• I think I'll finally be able to add sock_diag messages. This required a big refactoring that I just finished. I'll see if the author of the PR is interested in continuing their work, but if they are not I'll probably pick it up
• After taking a closer look, there are two crates that depend on some parts of the tokio ecosystem: netlink-sys, uses PollEvented when compiled with --features=tokio_support, and netlink-proto uses tokio_io::codec. So as far as I understand, there is not dependency on the tokio runtime, and things should work with other runtimes

(fwiw, if you look at the netlink crates, don't look at doc.rs, I haven't published the updated version there).