Skip to content

Commit a64af0e

Browse files
borkmannAlexei Starovoitov
authored andcommitted
bpf, libbpf: use correct barriers in perf ring buffer walk
Given libbpf is a generic library and not restricted to x86-64 only, the compiler barrier in bpf_perf_event_read_simple() after fetching the head needs to be replaced with smp_rmb() at minimum. Also, writing out the tail we should use WRITE_ONCE() to avoid store tearing. Now that we have the logic in place in ring_buffer_read_head() and ring_buffer_write_tail() helper also used by perf tool which would select the correct and best variant for a given architecture (e.g. x86-64 can avoid CPU barriers entirely), make use of these in order to fix bpf_perf_event_read_simple(). Fixes: d0cabbb ("tools: bpf: move the event reading loop to libbpf") Fixes: 3911169 ("samples: bpf: add bpf_perf_event_output example") Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Cc: Peter Zijlstra <peterz@infradead.org> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
1 parent 09d6215 commit a64af0e

File tree

1 file changed

+4
-6
lines changed

1 file changed

+4
-6
lines changed

tools/lib/bpf/libbpf.c

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@
2727
#include <linux/list.h>
2828
#include <linux/limits.h>
2929
#include <linux/perf_event.h>
30+
#include <linux/ring_buffer.h>
3031
#include <sys/stat.h>
3132
#include <sys/types.h>
3233
#include <sys/vfs.h>
@@ -2418,13 +2419,12 @@ bpf_perf_event_read_simple(void *mem, unsigned long size,
24182419
unsigned long page_size, void **buf, size_t *buf_len,
24192420
bpf_perf_event_print_t fn, void *priv)
24202421
{
2421-
volatile struct perf_event_mmap_page *header = mem;
2422+
struct perf_event_mmap_page *header = mem;
2423+
__u64 data_head = ring_buffer_read_head(header);
24222424
__u64 data_tail = header->data_tail;
2423-
__u64 data_head = header->data_head;
24242425
int ret = LIBBPF_PERF_EVENT_ERROR;
24252426
void *base, *begin, *end;
24262427

2427-
asm volatile("" ::: "memory"); /* in real code it should be smp_rmb() */
24282428
if (data_head == data_tail)
24292429
return LIBBPF_PERF_EVENT_CONT;
24302430

@@ -2467,8 +2467,6 @@ bpf_perf_event_read_simple(void *mem, unsigned long size,
24672467
data_tail += ehdr->size;
24682468
}
24692469

2470-
__sync_synchronize(); /* smp_mb() */
2471-
header->data_tail = data_tail;
2472-
2470+
ring_buffer_write_tail(header, data_tail);
24732471
return ret;
24742472
}

0 commit comments

Comments
 (0)