corosio produces much bigger binaries than asio

[pfeatherstone]

I'm comparing a simple echo server written in corosio and asio. I'm using the example echo server on the corosio webpage. I've attached the code. On my machine, compiling with gcc 14.2.0 i get the following binary sizes:

• echo_corosio: 683280 bytes
• echo_asio: 108952 bytes

CMakeLists.txt

main_asio.cpp

main_corosio.cpp

[pfeatherstone]

Could you open the "discussions" tab on the corosio project?

[pfeatherstone]

when i compile with clang18 i get:

• echo_corosio: 692496 bytes
• echo_asio: 101664 bytes

[pfeatherstone]

so i'm not convinced of the promise of "smaller binaries" due to less template bloat.

[InternalHigh]

What could be the reason for this?

[pfeatherstone]

I was surprised by how little difference LTO made for corosio.

[pfeatherstone]

In that simple corosio echo server, it's building ALL of corosio, including UDP, DNS resolution, thread pools, absolutely everything, and the linker can't get rid of it. ChatGPT is saying that corosio's io_context on Linux is constructing the epoll backend like so:

auto& sched = ctx.make_service<detail::epoll_scheduler>(
    static_cast<int>(concurrency_hint));

ctx.make_service<detail::epoll_tcp_service>();
ctx.make_service<detail::epoll_tcp_acceptor_service>();
ctx.make_service<detail::epoll_udp_service>();
ctx.make_service<detail::epoll_local_stream_service>();
ctx.make_service<detail::epoll_local_stream_acceptor_service>();
ctx.make_service<detail::epoll_local_datagram_service>();

So it's bringing in a lot of logic. Asio's io_context doesn't do this. So it's no wonder LTO won't achieve much here.

[pfeatherstone]

@sgerbino Have you had a chance to look at this? is there a way to modularize the build? So only link to what you need ? Is it possible there is too much cross-talk inside corosio and capy and the libraries aren't linker friendly.

[sgerbino]

Yes, there is. And you've found half of the bloat I'm looking at. The services should be lazily created and therefore not linked in the echo example. The other half is that every backend is compiled into a single translation unit, so the linker links select and io_uring even though the default io_context on Linux only ever constructs epoll. That may be fixed by restructuring the backends into separate TUs so a program only links the backend it actually uses.

I'm also looking at the claim you're referencing, I assume it's WhyNotAsio in the Capy documentation, which is about the template instantiation cascades that the type-erased streams avoid. This is not exercised in the echo server example. If you're referencing some other claim about binary size, please let me know so I can make sure any claim made is accurate.

[pfeatherstone]

Yes i was referencing:

Write any_stream& and accept any stream. Your function compiles once. It links anywhere. Your build times drop. Your binaries shrink. Your error messages become readable. And because coroutines are ordinary functions (not templates), you get natural ABI stability. Link against a new stream implementation without recompiling your code.

The numbers contradicted this statement. However, as we've just found out, the binary bloat has nothing to do with type-erasure, but rather corosio not properly separating functionality into separate TUs and the linker not able to strip away unwanted symbols.

[pfeatherstone]

Is this easily fixable ?

[sgerbino]

This should alleviate the eager registration problem. It shows considerable improvement on binary size on my machine.
#373

[sgerbino]

Yes i was referencing:

Write any_stream& and accept any stream. Your function compiles once. It links anywhere. Your build times drop. Your binaries shrink. Your error messages become readable. And because coroutines are ordinary functions (not templates), you get natural ABI stability. Link against a new stream implementation without recompiling your code.

The numbers contradicted this statement. However, as we've just found out, the binary bloat has nothing to do with type-erasure, but rather corosio not properly separating functionality into separate TUs and the linker not able to strip away unwanted symbols.

I think there is a misunderstanding on what the claim is, I will show a gist demonstration once develop is updated.

[sgerbino]

I had AI work up an example in both asio and corosio and take measurements. As a project grows with more stream-generic functions and more stream types, the template version's binary grows with the product of the two (every function is recompiled for every stream type). With type-erased any_stream, each function is only compiled once.

The echo demo is a poor demonstration of the advantage stated in WhyNotAsio. It has one stream type, which is the case where templates are smaller. When you get to protocol-level libraries with several stream types, dozens of stream-generic functions, you hit the crossover point where smaller binaries can be the result. Here's a gist of what I mean: https://gist.github.com/sgerbino/076b2112026d44918a8f5f946a9c1b62

Shrinking the binary (like the change we just made) will help to make the crossover point sooner rather than later. I'm sure there is more work we can do to get it even smaller. The lazy service registration was an obvious win (and what it should've done to begin with).

[sgerbino]

Just adding this note, if you think the wording is misleading in the Capy documentation feel free to suggest a way it should be rephrased to make this more clear.

[pfeatherstone]

I think it's fine but you have to build a fairly large protocol it seems to get to the point where capy+corosio wins. Beating Asio on the small echo example should be a necessary target in my view otherwise someone can easily dismiss the binary size claim (as I did).

[pfeatherstone]

What would be really good is if the echo example achieved similar binary size to a hand rolled event loop in C with BSD sockets when using LTO, stripping and potentially getting HALO to kick in