Is it possible to use libc functions for networking and IO with Photon coroutines?

[medvednikov]

Hello,

I've successfully integrated Photon into the V programming language, the coroutines work with simple examples.

However when I try to use existing networking and IO libraries which use standard libc functions, I get undebuggable segfaults.

// Build with
// v -use-coroutines examples/simple_coroutines.v
//

import coroutines
import time
import net.http

fn foo(a int) {
    for {
        println('hello from foo2() a=${a}')
        resp := http.get('https://vlang.io/utc_now') or { panic(err) } // segfaults here
        println('resp=${resp.body}')
        coroutines.sleep(2 * time.second)
    }
}

fn main() {
    go foo(10)
    for {
        println('hello from MAIN')
        coroutines.sleep(1 * time.second)
    }
    println('done')
}

It's easy to use Photon's custom sleep functions, but re-writing the entire standard library to use Photon's net/IO is next to impossible.

Is there a way to use standard libc functions together with Photon coroutines?

[beef9999]

Network: Photon has similar API to libc socket, for instance, send/recv/sendmsg/recvmsg/bind/listen/accept, but the socket itself is a C++ class encapsulation. Could you show me what the current V code of the networking looks like? Then we could discuss how to hide the C++ class.

IO: There are several io engines in fs/localfs.h The psync engine will block the current vCPU (but generic), the aio engine won't block (but not generic), the io_uring engine won't block, and has the best performance, but requires high version kernel.

BTW, If we decide to use io_uring event engine, things would be even more simpler. We just need to integrate the code from io/iouring-wrapper.h. There are no other dependencies in fs or net. However, if the event engine is epoll, you will still to add some functional code, for example, set non-blocking fd ...

[beef9999]

We could make photon IO's API exactly the same to libc API.

[medvednikov]

libc compatibility would've been great, but yeah, still means that actual libc functions can't be used.

Here's the code that does the http request (no ssl):

    host_name, _ := net.split_address(host)!
    s := req.build_request_headers(method, host_name, path)
    mut client := net.dial_tcp(host)!
    client.set_read_timeout(req.read_timeout)
    client.set_write_timeout(req.write_timeout)
    // TODO this really needs to be exposed somehow
    client.write(s.bytes())!
    $if trace_http_request ? {
        eprintln('> ${s}')
    }
    mut bytes := io.read_all(reader: client)!
    client.close()!
    response_text := bytes.bytestr()
    $if trace_http_response ? {
        eprintln('< ${response_text}')
    }
    return parse_response(response_text)

dial tcp code: https://github.com/vlang/v/blob/2904c399b5ca124549b13c165c8864c05a9bd80c/vlib/net/tcp.v#L25

and finally TcpSocket.connect(): https://github.com/vlang/v/blob/2904c399b5ca124549b13c165c8864c05a9bd80c/vlib/net/tcp.v#L498

In the end, it's a simple libc connect() call: C.connect(s.handle, voidptr(&a), a.len()).

[lihuiba]

I've successfully integrated Photon into the V programming language, the coroutines work with simple examples.

Glad to hear the good news!

Is there a way to use standard libc functions together with Photon coroutines?

Yes. Photon socket classes are also implemented with libc functions. You can mimic our code to realize yours.

Please checkout photon/net/basic_socket.cpp to see the details.

[medvednikov]

Turned out libc functions work great with Photon, it was the Boehm GC (libgc) that was causing segfaults. Making Photon work with GC is the only thing left to be done.

Stackful coroutines have their own stacks with dynamic sizes, such behavior makes it hard for the GC to detect alive objects. Does photon have a way to register coroutine stacks and let the GC know about context switching?

As I understand, Boehm GC has a way to let know about a new stack during context switching, but I didn't find this feature in Photon...

[medvednikov]

I found a solution for boost coroutines. Should be similar for Photon:

https://stackoverflow.com/questions/28241646/how-to-use-boost-coroutine-with-boehm-gc/64623242#64623242

[lihuiba]

Making Photon work with GC is the only thing left to be done.

That's an interesting issue. I'll see what I can do.

[lihuiba]

@medvednikov Let's start another issue for GC
#150

[medvednikov]

Thanks.

Closing this, since libc io functions work fine with Photon:

https://github.com/vlang/v/blob/f73e99cddd49ffab68be8a7fe3374e609d228d18/examples/coroutines/simple_coroutines.v

[medvednikov]

An update on this:

libc networking does work, even with the GC, but looks like it's sequential and never context switching

it's just as slow as normal threading
we need to talk to photon APIs

similar to what you did in curl.cpp?

https://github.com/alibaba/PhotonLibOS/blob/main/net/curl.cpp

shouldn't this curl.cpp logic still be working for all code besides code which uses some system threading stuff / context switchign on its own?

[beef9999]

The default libc socket fd is blocking. Photon socket uses non-blocking fd. See the docs and example
https://photonlibos.github.io/docs/api/network
https://photonlibos.github.io/docs/introduction/write-first-example#5-socket

Once you have initialized the Env in a vCPU, you can send/recv packets in an non-blocking way, by using Photon socket.

---

As to the HTTP component, libcurl is of course integrated. But another alternative is the Photon's own HTTP client/server link. It's simpler, faster, and has less dependencies.

If V already has its own HTTP implementation, you can just change the underlying libc socket to Photon socket.

---

Just like most other modules in Photon, curl.cpp is not able to work in a system threading environment. It has to be Photon coroutine Env.

[lihuiba]

We also have plain function wrappers around libc's networking functions. Please see https://github.com/alibaba/PhotonLibOS/blob/main/net/basic_socket.h for details.

[joe-conigliaro]

We also have plain function wrappers around libc's networking functions. Please see https://github.com/alibaba/PhotonLibOS/blob/main/net/basic_socket.h for details.

Thanks @lihuiba, is this made to integrate photon easily with existing libc code?

v is currently already using non blocking fd by setting O_NONBLOCK, but Im guessing this still going to function properly with photon.

I'm trying to work out the best way to integrate it with our net code. I guess the only options are to to rewrite it using your client/socket. Or use the plain functions you provide in basic_socket.h (if that would work, it would be the simplest).

[beef9999]

https://github.com/alibaba/PhotonLibOS/blob/main/net/basic_socket.h

Those APIs have none special requirements. You only need to pass a non-blocking fd.

[joe-conigliaro]

Thanks @beef9999

[lihuiba]

is this made to integrate photon easily with existing libc code?

Yes, it's made to integrate photon easily with any C/C++ code that uses plain fd.

Actually the C++ socket classes are only wrappers to these functions.

[joe-conigliaro]

Thanks @lihuiba, I thought this was the case 😄 but I thought better to double check.

[lihuiba]

@joe-conigliaro You are welcome.