Wingfoil — a Rust stream-processing graph framework with iceoryx2 connectivity

[0-jake-0]

Hi all,

Wanted to share something we recently shipped that builds on iceoryx2, in case it's useful to others here and to get any feedback you have on the integration choices.

Context

Wingfoil is a Rust stream-processing library — you wire up a graph of nodes where each node ticks when its upstreams produce a value, and the graph runs either in real time or replayed from historical data, designed for high-frequency trading and real time AI pipelines.

What the adapter looks like

Two graph nodes:

use wingfoil::adapters::iceoryx2::{iceoryx2_pub, iceoryx2_sub};
use wingfoil::*;

#[repr(C)]
#[derive(Clone, Copy, Debug, Default, ZeroCopySend)]
struct Quote { price: u64, qty: u32 }

// Subscriber side — pulls from shared memory, emits into the graph
let quotes = iceoryx2_sub::<Quote>("market_quotes")
    .collapse()
    .for_each(|q, _| risk.update(q));

// Publisher side — graph values written into shared memory
mid_stream.iceoryx2_pub(opts);

Three polling modes

The trickiest design call was how the subscriber pulls from the service, given wingfoil's graph engine runs synchronously per cycle. We landed on three modes, picked at construction time:

• Spin — direct try_receive inside cycle(). ~1–5 µs end-to-end, pins a core.
• Threaded — background thread with a 10 µs yield, channel into the graph. Lower CPU, ~10–100 µs.
• Signaled — event-driven WaitSet on a matching Event service; publisher signals after each send.

These cover the real situations we've hit: spin for the latency-critical path, signaled for the wake-up-when-something-happens case, threaded for the middle ground where the consumer is doing other work too.

Links

• Adapter source: https://github.com/wingfoil-io/wingfoil/tree/main/wingfoil/src/adapters/iceoryx2
• Pub example: https://github.com/wingfoil-io/wingfoil/blob/main/wingfoil/examples/iceoryx2/pub.rs
• Sub example: https://github.com/wingfoil-io/wingfoil/blob/main/wingfoil/examples/iceoryx2/sub.rs
• Wingfoil: https://github.com/wingfoil-io/wingfoil

Thanks for building iceoryx2 — happy to answer questions or dig into anything above.

[elfenpiff]

Great work @0-jake-0

[orecham]

Hey @0-jake-0 ,

We're getting ready to launch a community forum for iceoryx2 over at community.iceoryx.io.

If you're up for it, we'd love for you to repost this in our Showcase category over there! It's a really clean solution to a design crossroads that many users are likely to hit, and exactly the kind of post we hope to accumulate.

Thanks again for sharing this.