Wrap lwt and Async?

[bobpoekert]

Most native reason code is going to be wrapping opam libraries for doing things like http requests and database queries, and those native libraries are going to be using either lwt or Async. It would be great if you could, for example, have an http client library that used either bs-fetch or cohttp under the hood, depending on the platform, and exposed a common api for both. Is this something that it would make sense for repromise to do, or would it make more sense for that to be a separate library?

[aantron]

I think this would definitely make sense :)

My original plan was to implement the "core" of the wrapping in Repromise and/or Lwt/Async, and leave more specific things, like isomorphic HTTP clients, to other libraries that would then build over that core (for example, lwt-node, I believe, started in that direction, cc @kennetpostigo).

I think having strong native/node/(partially browser) ecosystem compatibility would make Reason a uniquely powerful language. It should save an enormous amount of learning and development time for users.

[kennetpostigo]

Hey @bobpoekert @aantron !

I'm working on these at the moment all built with reason

• Http Module
• Cross-Platform fetch to make network requests.

At the moment these use lwt under the hood, but if all goes well, @aantron and I can converge on using Repromise promises. Or if repromise becomes lwt 5.0/6.0 i'd just keep depending on lwt and update w/e needs to be updated.

[bobpoekert]

@kennetpostigo How are you implementing your cross-platform fetch? Do you have converters/wrappers from bucklescript promises to lwt promises and a stripped-down version of lwt that runs on bucklescript? Or do you have some other promise type that wraps both?

[kennetpostigo]

@bobpoekert js side uses what bucklescript exposes, but native side uses unix pipes

[bobpoekert]

@kennetpostigo but what's the common abstraction that cross-platform consumer code uses?

[kennetpostigo]

@bobpoekert bsb-native lets you compile to native or js. Theres a few libraries that do this like rex-json, even repromise uses bsb-native last I checked, among other libraries out there.

[aantron]

@kennetpostigo @bobpoekert is asking, I believe, what is the underlying synchronization primitive (e.g. promise implementation) that is being compiled to native and js, that lwt-node would build its isomorphic API over.

The answer to that is, I think, that lwt-node is not there yet.

Repromise has isomorphic native/js promises, but it is not yet compatible with Lwt (or Async).

lwt-node uses Lwt right now, and so presumably is native-only for the time being.

[kennetpostigo]

oh right, like I said before, right now for js we're using what bucklescript offers and for native we are using unix pipes. Theres no code sharing tho for the http client(fetch)

[aantron]

There is now a super basic POC of Repromise/Lwt interop here.

Basically, Lwt_unix (the I/O part of Lwt) drives the process I/O loop, and calls into Repromise when I/O resolves a promise and triggers a callback.

There are also two functions for converting Lwt and Repromise promises between each other.

The test case has a simple wrapper around Lwt_unix.sleep, that turns it into a Repromise-friendly sleep.

[lessp]

Hey @aantron, this looks like a super-promising library, are there still plans for this?

Perhaps this would also be interesting from the previous discussion's perspective? https://github.com/anuragsoni/fetch/

[aantron]

Yep, still planning to look into this in great detail :)

From the other end, there is also https://github.com/aantron/luv, for rewriting Lwt over libuv (what's used by Node internally for I/O). That's what I've spent the most time on recently.

Most recently, though, I went on a small hiatus, but planning to come back to all this soon.

[lessp]

Oh, very interesting, there's lots of potential for making the community more approachable here!

Thanks for all your hard work, excited to follow the progress on these! 🙏

[aantron]

I'm going to close this issue to keep the repo tidy, because progress needs to be tracked in other repos that would drive Repromise:

• repromise_lwt is a Repromise/Lwt interop demo, in which the native version of Repromise is being driven by Lwt (Lwt_engine inside Lwt_unix). That is, Lwt plays the role of Node.js, or the browser JS runtime.
• Luv is a binding to libuv, the same C async I/O library that drives Node.js. It has an example where native Repromise, driven by Luv, is doing an HTTP GET (low-level version, not fetch). It has another example of native Repromise and Lwt coexisting in one process, because both are being driven by Luv.

Even though this issue is closed, further discussion is welcome!