-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is it possible to use libc functions for networking and IO with Photon coroutines? #148
Comments
Network: Photon has similar API to libc socket, for instance, send/recv/sendmsg/recvmsg/bind/listen/accept, but the socket itself is a C++ class encapsulation. Could you show me what the current V code of the networking looks like? Then we could discuss how to hide the C++ class. IO: There are several io engines in BTW, If we decide to use io_uring event engine, things would be even more simpler. We just need to integrate the code from |
We could make photon IO's API exactly the same to libc API. |
libc compatibility would've been great, but yeah, still means that actual libc functions can't be used. Here's the code that does the http request (no ssl):
dial tcp code: https://github.com/vlang/v/blob/2904c399b5ca124549b13c165c8864c05a9bd80c/vlib/net/tcp.v#L25 and finally TcpSocket.connect(): https://github.com/vlang/v/blob/2904c399b5ca124549b13c165c8864c05a9bd80c/vlib/net/tcp.v#L498 In the end, it's a simple libc |
Glad to hear the good news!
Yes. Photon socket classes are also implemented with libc functions. You can mimic our code to realize yours. Please checkout |
Turned out libc functions work great with Photon, it was the Boehm GC (libgc) that was causing segfaults. Making Photon work with GC is the only thing left to be done. Stackful coroutines have their own stacks with dynamic sizes, such behavior makes it hard for the GC to detect alive objects. Does photon have a way to register coroutine stacks and let the GC know about context switching? As I understand, Boehm GC has a way to let know about a new stack during context switching, but I didn't find this feature in Photon... |
I found a solution for boost coroutines. Should be similar for Photon: |
That's an interesting issue. I'll see what I can do. |
@medvednikov Let's start another issue for GC |
Thanks. Closing this, since libc io functions work fine with Photon: |
An update on this: libc networking does work, even with the GC, but looks like it's sequential and never context switching it's just as slow as normal threading similar to what you did in curl.cpp? https://github.com/alibaba/PhotonLibOS/blob/main/net/curl.cpp shouldn't this curl.cpp logic still be working for all code besides code which uses some system threading stuff / context switchign on its own? |
The default libc socket fd is blocking. Photon socket uses non-blocking fd. See the docs and example Once you have initialized the Env in a vCPU, you can send/recv packets in an non-blocking way, by using Photon socket. As to the HTTP component, libcurl is of course integrated. But another alternative is the Photon's own HTTP client/server link. It's simpler, faster, and has less dependencies. If V already has its own HTTP implementation, you can just change the underlying libc socket to Photon socket. Just like most other modules in Photon, curl.cpp is not able to work in a system threading environment. It has to be Photon coroutine Env. |
We also have plain function wrappers around libc's networking functions. Please see https://github.com/alibaba/PhotonLibOS/blob/main/net/basic_socket.h for details. |
Thanks @lihuiba, is this made to integrate photon easily with existing libc code? v is currently already using non blocking fd by setting O_NONBLOCK, but Im guessing this still going to function properly with photon. I'm trying to work out the best way to integrate it with our net code. I guess the only options are to to rewrite it using your client/socket. Or use the plain functions you provide in |
Those APIs have none special requirements. You only need to pass a non-blocking fd. |
Thanks @beef9999 |
Yes, it's made to integrate photon easily with any C/C++ code that uses plain fd. Actually the C++ socket classes are only wrappers to these functions. |
Thanks @lihuiba, I thought this was the case 😄 but I thought better to double check. |
@joe-conigliaro You are welcome. |
Hello,
I've successfully integrated Photon into the V programming language, the coroutines work with simple examples.
However when I try to use existing networking and IO libraries which use standard libc functions, I get undebuggable segfaults.
It's easy to use Photon's custom sleep functions, but re-writing the entire standard library to use Photon's net/IO is next to impossible.
Is there a way to use standard libc functions together with Photon coroutines?
The text was updated successfully, but these errors were encountered: