Ringbuf Support for Python API

[willfindlay]

This pull request contains an implementation for ringbuf support in bcc's Python API.

Fixes #2985.

Summary of Changes

• Added ringbuf helpers from libbpf API to libbcc
• Added a new RingBuf class to represent the ringbuf map
• Added a new EventArrayBase class to reduce code duplication between PerfEventArray and RingBuf
• Added BPF_RINGBUF_OUTPUT macro for BPF programs
• Added documentation

Examples

The following example programs describe how the API works.

On the BPF side, we have two choices for submitting events. map.ringbuf_output works like map.perf_submit for perf buffers,
while map.ringbuf_reserve/map.ringbuf_discard/map.ringbuf_submit present an alternative approach that allows for finer control.

map.ringbuf_output works like:

BPF_RINGBUF_OUTPUT(events);

struct data_t data {
    u32 pid;
};

TRACEPOINT_PROBE(raw_syscalls, sys_enter) {
    u32 pid = bpf_get_current_pid_tgid();

    struct data_t data = { .pid = pid };

    events.ringbuf_output(&data, sizeof(data), 0 /* flags */);
    
    return 0;
}

map.ringbuf_reserve/map.ringbuf_discard/map.ringbuf_submit work like:

BPF_RINGBUF_OUTPUT(events);

struct data_t data {
    u32 pid;
};

TRACEPOINT_PROBE(raw_syscalls, sys_enter) {
    u32 pid = bpf_get_current_pid_tgid();

    struct data_t *data = events.ringbuf_reserve(sizeof(struct data_t));
    if (!data)
        return 1;

    data->pid = pid;

    events.ringbuf_submit(data, 0 /* flags */);
    // or, to discard the data...
    events.ringbuf_discard(data, 0 /* flags */);

    return 0;
}

On the Python side, the user calls open_ring_buffer to associate a callback with the buffer. There are two options for reading data from kernelspace: ring_buffer_poll which works like perf_buffer_poll and ring_buffer_consume which reads data without polling first.

import time

from bcc import BPF

b = BPF(src_file=b'source.c')

def callback(ctx, data, size):
    event = b['events'].event(data)
    print(event.pid)

b['events'].open_ring_buffer(callback)

while True:
    # Option 1...
    b.ring_buffer_poll(30)
    # Option 2...
    b.ring_buffer_consume()
    time.sleep(0.5)

Unlike perf buffer callbacks, ringbuf supports return values from the callback, which can be used to indicate error conditions to stop polling early. If a callback does not return an integer, we fall back to returning 0, which approximates perf buffer behavior.

[yonghong-song]

[buildbot, test this please]

[willfindlay]

Is there an easy way to see what tests specifically are failing? Seems to be something in test_tools_smoke.py from what I can tell.

[yonghong-song]

Is there an easy way to see what tests specifically are failing? Seems to be something in test_tools_smoke.py from what I can tell.

Probably a flaky test. Let me try again.

[yonghong-song]

[buildbot, test this please]

[willfindlay]

Ah seems like it was indeed a flaky test.

[willfindlay]

Thank you for your comments @anakryiko. I left a few followup questions / concerns as replies. I should have time to update this PR later today or tomorrow.

[willfindlay]

@anakryiko I have now addressed all of your feedback (with the exception of pages vs bytes in the ringbuf macro, as discussed). Things seem to be in a good place now, but if you have any other concerns let me know.

[willfindlay]

@yonghong-song Should be good to test again. I've added unit tests under tests/python/test_ringbuf.py, as well as examples and documentation.

[anakryiko]

It looks good from API perspective, would be good to get a review from someone more familiar with BCC internals, though. See also the point of being able to size ringbuf from user-space.

[anakryiko]

is there a way to set ringbuf size from Python API in user-space? There are probably cases, where this size is determined dynamically by user space logic

[willfindlay]

I don't think this would be easy to implement given the way bcc treats maps. We could potentially change this in the future, but for now, I think the current API is fine. For now, if the user desires different map sizes according to userspace logic, they can just programmatically generate the correct C code. This is already a common pattern in bcc tools.

[anakryiko]

ring_buffer_consume is very specialized use-case, I'd do all the examples with ring_buffer_poll() and just mention ring_buffer_consume as an alternative, as you did in the example above.

[anakryiko]

@yonghong-song, do you squash all those 44 commits into one when you land? I wonder if it makes sense to ask people to re-organize and squash into fewer logic commits before landing. E.g., implementation, testing and documentation commits. 44 commits are pretty unwieldy.

[yonghong-song]

@anakryiko Thanks for review. I won't push these 44 commits. I indeed frequently squash them into one commit. llvm sometimes also has giant commits. I think it is okay not to put burden on the developer for already done work.
I will take a look shortly.

[yonghong-song]

[buildbot, test this please]

[yonghong-song]

[buildbot, test this please]

[yonghong-song]

Merged. Thanks @willfindlay and @anakryiko!

[raoufkh]

Hello

First, thank you @willfindlay for this work.

I have one question, in TC and XDP programs, how can we send the network packet to the user space program. Using perf buffers, this was possible by using the perf_submit_skb method while perf_submit allows only a data struct we define.

I'm trying to include an unsigned char * pointer in my struct (which I will send to the user space program using ringbuf_output but I don't manage to copy the network packet inside the struct. Doing so, the struct memory space will contain the value of the pointer (memory address of netwokr packet) but not the network packet itself. Regarding to the linux documentation, ring buffers should allow sending entire network packet to the user space.

In the BCC reference guide and by browsing the libbpf library, I saw that when opening a ring buffer, there is a ctx argument to pass but there is no information on how to use it. Does this argument allow to solve my problem?

Could you provide help please?

Regards
Raouf