Async support in Mirage/Xen

[avsm]

Here's a thread from the mailing list that lays out some of the steps to support Async in Mirage/Xen mode.

We've been making steady progress on porting Async to the Xen backend, but it's still going to > take some time to complete. Here's the steps:

• Lwt splits out the logical threading core (the Lwt module) and the Unix-specific execution portions (Lwt_unix). Async needs to be restructured along similar lines, which has already begun: there is a Core_kernel that is Unix-independent and is used by our Irmin db. There is also an Async_kernel.
• An Async_xen needs to be written which implements the subset of Async_unix functionality necessary to drive traffic. Async_unix is much bigger than Lwt_unix, so finding a viable subset is important (for instance, Async_unix.Reader and Async_xen.Reader need to have a similar module type, but cannot be exactly the same).
• Frenetic needs to use Async_kernel in the OS-independent portions, instead of using Async (which packs in both Async_kernel and Async_unix).

[philtomson]

Ah, I suspect I need this Async support to get my mqtt_client running in Xen and I was about to open an issue here about this.

The other conclusion I was reaching was that there needed to be a V[1|2]_ASYNC.mli in mirage/types... not sure if that should be a separate issue or part of this one.

I was thinking that where V2_LWT.mli has module type declarations like:

module type IPV4 = IPV4
  with type 'a io = 'a Lwt.t
   and type buffer = Cstruct.t
   and type ipv4addr = Ipaddr.V4.t

... a V2_ASYNC.mli would need to have:

module type IPV4 = IPV4
  with type 'a io = 'a Deferred.t
  and type buffer = Async_cstruct.t
  and type ipv4addr = Ipaddr.V4.t

(though I'm not sure about that last one for Ippaddr)
I noticed that an Async_cstruct already exists here:
https://github.com/mirage/ocaml-cstruct/blob/master/async/async_cstruct.ml

[avsm]

That's exactly right @philtomson. The issue is what lies behind the module types -- we need to build equivalent implementations of (e.g.) the network stack to support Async. There's one light on the horizon: https://github.com/janestreet/lwt-async which builds a type 'a Lwt.t = 'a Deferred.t, so we could embed all the Lwt code in an Async runtime with this.

[philtomson]

@avsm: is it as easy as:

type 'a Lwt.t = 'a Deferred.t  

?
Seems the semantics have subtle differences.

[avsm]

Seems the semantics have subtle differences.

Indeed. I've never tried it. Async is more conservative than Lwt in general, so it might just work. (note that it's Lwt embedded in Async, not the other way around, which would be harder)

[rgrinberg]

@avsm I think that lwt-async keeps up with Async but it doesn't keep with the latest Lwt. Luckily Lwt is fairly static.

[hannesm]

I suspect eio will happen before we manage to get async supported.