This repository has been archived by the owner. It is now read-only.

unix: remove libev #485

Closed
bnoordhuis opened this Issue Jul 2, 2012 · 26 comments

Comments

Projects
None yet
@bnoordhuis
Contributor

bnoordhuis commented Jul 2, 2012

This issue tracks the removal of libev in v0.9 / master.

@ghost ghost assigned bnoordhuis Jul 2, 2012

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Jul 2, 2012

Contributor

In case any project watchers are wondering, libev served us well but:

  1. It only supports level-triggered I/O. On Linux, we want to use edge-triggered mode - it cuts down the number of syscalls by a substantial margin.
  2. libev's inner loop does a lot of things we don't really need. Gutting the inner loop like we did in 649ad50 gave a 40% performance increase on some benchmarks.
Contributor

bnoordhuis commented Jul 2, 2012

In case any project watchers are wondering, libev served us well but:

  1. It only supports level-triggered I/O. On Linux, we want to use edge-triggered mode - it cuts down the number of syscalls by a substantial margin.
  2. libev's inner loop does a lot of things we don't really need. Gutting the inner loop like we did in 649ad50 gave a 40% performance increase on some benchmarks.
@nathansobo

This comment has been minimized.

Show comment
Hide comment
@nathansobo

nathansobo Aug 2, 2012

@bnoordhuis: Will this result in a single event loop shared between Unix and Windows versions of the library? Will the interface to it be sufficiently abstract that it might be possible to replace it with a different event loop? The Chromium Embedded Framework maintainer and I are interested in getting libuv working on top of Chromium's render process event loop as a means of making the Node.js API available from JavaScript in a Chromium window. We're curious about the roadmap for removing libev, since we're thinking this attempt will be easier once libuv has its own event loop.

nathansobo commented Aug 2, 2012

@bnoordhuis: Will this result in a single event loop shared between Unix and Windows versions of the library? Will the interface to it be sufficiently abstract that it might be possible to replace it with a different event loop? The Chromium Embedded Framework maintainer and I are interested in getting libuv working on top of Chromium's render process event loop as a means of making the Node.js API available from JavaScript in a Chromium window. We're curious about the roadmap for removing libev, since we're thinking this attempt will be easier once libuv has its own event loop.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Aug 2, 2012

Contributor

Will the interface to it be sufficiently abstract that it might be possible to replace it with a different event loop?

If you mean 'will it be possible to embed libuv in another event loop?' then the answer is yes. We'll give you a timeout and something (file descriptor, handle) to poll on in your own event loop.

Contributor

bnoordhuis commented Aug 2, 2012

Will the interface to it be sufficiently abstract that it might be possible to replace it with a different event loop?

If you mean 'will it be possible to embed libuv in another event loop?' then the answer is yes. We'll give you a timeout and something (file descriptor, handle) to poll on in your own event loop.

@nathansobo

This comment has been minimized.

Show comment
Hide comment
@nathansobo

nathansobo Aug 3, 2012

That sounds much easier than what I had in mind. So presumably you'll expose a function as part of the public interface that we can call when the file descriptor changes to invoke the appropriate callbacks? Very excited about this. Godspeed.

nathansobo commented Aug 3, 2012

That sounds much easier than what I had in mind. So presumably you'll expose a function as part of the public interface that we can call when the file descriptor changes to invoke the appropriate callbacks? Very excited about this. Godspeed.

@milani

This comment has been minimized.

Show comment
Hide comment
@milani

milani Aug 6, 2012

@nathansobo We could bring nodejs APIs to CEF using proxies in AppJS. But it's better to merge the event loops. Someone did it and published under the name node-webkit you maybe heard of.

milani commented Aug 6, 2012

@nathansobo We could bring nodejs APIs to CEF using proxies in AppJS. But it's better to merge the event loops. Someone did it and published under the name node-webkit you maybe heard of.

@nathansobo

This comment has been minimized.

Show comment
Hide comment
@nathansobo

nathansobo Aug 7, 2012

@milani Yes, but Marshall Greenblatt (the author of CEF) has concerns about the way it was done. He says that the event loops are highly optimized to maximize performance on the OS, so he's concerned that moving to an event loop based on libuv directly could negatively impact performance. He prefers an approach where the default Chromium event loop remains intact, driving libuv with the Chromium event loop rather than the other way around. I think only way to do that efficiently is with an API like @bnoordhuis describes, where you can have the main event loop wait on a file handle / timeout. Thoughts?

nathansobo commented Aug 7, 2012

@milani Yes, but Marshall Greenblatt (the author of CEF) has concerns about the way it was done. He says that the event loops are highly optimized to maximize performance on the OS, so he's concerned that moving to an event loop based on libuv directly could negatively impact performance. He prefers an approach where the default Chromium event loop remains intact, driving libuv with the Chromium event loop rather than the other way around. I think only way to do that efficiently is with an API like @bnoordhuis describes, where you can have the main event loop wait on a file handle / timeout. Thoughts?

@jbergstroem

This comment has been minimized.

Show comment
Hide comment
@jbergstroem

jbergstroem commented Aug 9, 2012

Close?

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Aug 9, 2012

Contributor

Close?

Close but not quite there yet, if that is what you mean. :-)

Contributor

bnoordhuis commented Aug 9, 2012

Close?

Close but not quite there yet, if that is what you mean. :-)

@jbergstroem

This comment has been minimized.

Show comment
Hide comment
@jbergstroem

jbergstroem commented Aug 9, 2012

@bnoordhuis indeed ^_^

@denik

This comment has been minimized.

Show comment
Hide comment
@denik

denik Oct 6, 2012

Once edge-triggered epoll is used, will Poll watchers become edge-triggered too? or will you implement level-triggered Poll on top of edge-triggered epoll?

denik commented Oct 6, 2012

Once edge-triggered epoll is used, will Poll watchers become edge-triggered too? or will you implement level-triggered Poll on top of edge-triggered epoll?

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Oct 6, 2012

Contributor

@denik Poll watchers will default to level-triggered mode. I may add a flag for edge-triggered mode.

FWIW, I'm reconsidering edge-triggered mode. It reduces the number of system calls substantially but it slows down some benchmarks because a lot more time is spent inside kernel-side spinlocks. :-(

It's probably because everyone and his dog uses level-triggered mode and as a result that code path has seen a lot more optimization. I may send some patches to the LKML but that doesn't help us in the near future.

Contributor

bnoordhuis commented Oct 6, 2012

@denik Poll watchers will default to level-triggered mode. I may add a flag for edge-triggered mode.

FWIW, I'm reconsidering edge-triggered mode. It reduces the number of system calls substantially but it slows down some benchmarks because a lot more time is spent inside kernel-side spinlocks. :-(

It's probably because everyone and his dog uses level-triggered mode and as a result that code path has seen a lot more optimization. I may send some patches to the LKML but that doesn't help us in the near future.

@zcbenz

This comment has been minimized.

Show comment
Hide comment
@zcbenz

zcbenz Oct 10, 2012

@bnoordhuis When will libev be completely removed? I've waited this for a long time :)

And will libuv provide an API like uv_backend_fd() to help integrating libuv to other event loops after libev is removed?

zcbenz commented Oct 10, 2012

@bnoordhuis When will libev be completely removed? I've waited this for a long time :)

And will libuv provide an API like uv_backend_fd() to help integrating libuv to other event loops after libev is removed?

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Oct 10, 2012

Contributor

When will libev be completely removed?

It's mostly done, I just need to clean it up a little. Time permitting, I'll land it this or next week.

And will libuv provide an API like uv_backend_fd() to help integrating libuv to other event loops after libev is removed?

Yes. And mea culpa, that's long overdue.

Contributor

bnoordhuis commented Oct 10, 2012

When will libev be completely removed?

It's mostly done, I just need to clean it up a little. Time permitting, I'll land it this or next week.

And will libuv provide an API like uv_backend_fd() to help integrating libuv to other event loops after libev is removed?

Yes. And mea culpa, that's long overdue.

@TooTallNate

This comment has been minimized.

Show comment
Hide comment
@TooTallNate

TooTallNate Oct 10, 2012

Contributor

And will libuv provide an API like uv_backend_fd() to help integrating libuv to other event loops after libev is removed?

I believe this thread on the libuv mailing list is related: https://groups.google.com/d/topic/libuv/qSSaBRBwFoY/discussion

Contributor

TooTallNate commented Oct 10, 2012

And will libuv provide an API like uv_backend_fd() to help integrating libuv to other event loops after libev is removed?

I believe this thread on the libuv mailing list is related: https://groups.google.com/d/topic/libuv/qSSaBRBwFoY/discussion

@nathansobo

This comment has been minimized.

Show comment
Hide comment
@nathansobo

nathansobo Oct 23, 2012

@bnoordhuis is the code for removing libev visible publicly on a branch somewhere? I'd love to get a look at progress.

nathansobo commented Oct 23, 2012

@bnoordhuis is the code for removing libev visible publicly on a branch somewhere? I'd love to get a look at progress.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Oct 23, 2012

Contributor

@nathansobo Sure, it's in my rm-ev branch.

Contributor

bnoordhuis commented Oct 23, 2012

@nathansobo Sure, it's in my rm-ev branch.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Nov 16, 2012

Contributor

Fixed in 1282d64 and 665a316.

Contributor

bnoordhuis commented Nov 16, 2012

Fixed in 1282d64 and 665a316.

@bnoordhuis bnoordhuis closed this Nov 16, 2012

@nathansobo

This comment has been minimized.

Show comment
Hide comment
@nathansobo

nathansobo Dec 7, 2012

@bnoordhuis Any updates on uv_backend_fd()?

nathansobo commented Dec 7, 2012

@bnoordhuis Any updates on uv_backend_fd()?

@indutny

This comment has been minimized.

Show comment
Hide comment
@indutny
Contributor

indutny commented Dec 7, 2012

@WabidWombat

This comment has been minimized.

Show comment
Hide comment
@WabidWombat

WabidWombat Apr 12, 2016

I am tasked with porting some server code from *nix to Windows. The code was written using libev and libeio, each of which I am at best marginally familiar with. At this point the choice is to either (1) make the libeio and libev code Windows-happy, or (2) port it to libuv.

Can anyone give me a rough idea of the relative difficulty/merits/demerits? I am just looking for order-of-magnitude here.

By the way, the select() barrier of 64 connections and the related performance issues loom large in our application.

Thanks

WabidWombat commented Apr 12, 2016

I am tasked with porting some server code from *nix to Windows. The code was written using libev and libeio, each of which I am at best marginally familiar with. At this point the choice is to either (1) make the libeio and libev code Windows-happy, or (2) port it to libuv.

Can anyone give me a rough idea of the relative difficulty/merits/demerits? I am just looking for order-of-magnitude here.

By the way, the select() barrier of 64 connections and the related performance issues loom large in our application.

Thanks

@indutny

This comment has been minimized.

Show comment
Hide comment
@indutny

indutny Apr 12, 2016

Contributor

@WabidWombat I would suggest using porting to libuv. There are lots of edge cases to deal with, and lots of similarities between libev's API and libuv. Should be much easier than writing everything from the scratch.

Contributor

indutny commented Apr 12, 2016

@WabidWombat I would suggest using porting to libuv. There are lots of edge cases to deal with, and lots of similarities between libev's API and libuv. Should be much easier than writing everything from the scratch.

@WabidWombat

This comment has been minimized.

Show comment
Hide comment
@WabidWombat

WabidWombat Apr 12, 2016

To make sure I understand you - you are recommending porting the existing code onto windows, rather than using libuv?

WabidWombat commented Apr 12, 2016

To make sure I understand you - you are recommending porting the existing code onto windows, rather than using libuv?

@indutny

This comment has been minimized.

Show comment
Hide comment
@indutny

indutny Apr 12, 2016

Contributor

My english is far from my own expectations, sorry! I suggest using libuv.

Contributor

indutny commented Apr 12, 2016

My english is far from my own expectations, sorry! I suggest using libuv.

@WabidWombat

This comment has been minimized.

Show comment
Hide comment
@WabidWombat

WabidWombat Apr 12, 2016

спасибо!

WabidWombat commented Apr 12, 2016

спасибо!

@dawud-tan

This comment has been minimized.

Show comment
Hide comment
@dawud-tan

dawud-tan Oct 9, 2016

@bnoordhuis so, currently, libuv doesn't use epollet yet?

dawud-tan commented Oct 9, 2016

@bnoordhuis so, currently, libuv doesn't use epollet yet?

@saghul

This comment has been minimized.

Show comment
Hide comment
@saghul

saghul Oct 10, 2016

Contributor

No it does not.

On Oct 9, 2016 20:20, "陳大衛" notifications@github.com wrote:

@bnoordhuis https://github.com/bnoordhuis so, currently, libuv doesn't
use epollet yet?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#485 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATYGJYKmrrD3ZPVQsGsORXM6zbgACuOks5qyT5ugaJpZM4ADX8z
.

Contributor

saghul commented Oct 10, 2016

No it does not.

On Oct 9, 2016 20:20, "陳大衛" notifications@github.com wrote:

@bnoordhuis https://github.com/bnoordhuis so, currently, libuv doesn't
use epollet yet?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#485 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATYGJYKmrrD3ZPVQsGsORXM6zbgACuOks5qyT5ugaJpZM4ADX8z
.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.