Skip to content

Horrible eBPF performance of TC program when reading from HashMap #1144

@bugrazoid

Description

@bugrazoid

Hello. I'm currently mastering eBPF using Rust + Aya. The following problem has arisen. I'm making a port forwarding program using TC. I record incoming connections in a HashMap and indicate which port it was initially directed to. Accordingly, the outgoing connection from this map retrieves the source port. The problem is that productivity drops very quickly, by about 10 times. Moreover, if you redirect the port separately and read the map separately, then performance is maintained.

The problematic code with low productivity (~5-10k rps):

let key = RedirectLocalPortKey::new(packet.remote_ip(), packet.remote_port(), packet.local_ip());
if let Some(original_port) = unsafe { REDIRECT_EGRESS.get(&key) }.copied() {
    packet.set_local_port(original_port);
}

If you use static data, the productivity is good (~50k rps):

let key = RedirectLocalPortKey::new(packet.remote_ip(), packet.remote_port(), packet.local_ip());
if packet.local_port() == 8081 {}
    packet.set_local_port(8080);
}

If you also read data from HashMap, then performance is maintained (~50k rps):

let key = RedirectLocalPortKey::new(packet.remote_ip(), packet.remote_port(), packet.local_ip());
if packet.local_port() == 8081 {}
    packet.set_local_port(8080);
}
if let Some(port) = unsafe { REDIRECT_EGRESS.get(&key) }.copied() {
    info!(ctx, "egress: {}", port);
}

And another variant (~20k rps):

if packet.local_port().inner() == 8081 {
    packet.set_local_port(Port::from(1080));
}

if let Some(port) = unsafe { REDIRECT_EGRESS.get(&key) }.copied() {
    packet.set_local_port(port.original_port);
}

Full code here

Help me understand what I'm doing wrong?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions