-
Notifications
You must be signed in to change notification settings - Fork 4k
generate perf event data structure in Python automatically #2198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
[buildbot, test this please] |
src/python/bcc/table.py
Outdated
'long long': ct.c_longlong, | ||
'unsigned long long': ct.c_ulonglong, | ||
'u64': ct.c_ulonglong, | ||
'void *': ct.c_void_p } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add s8, s16, s32, s64 as well?
How do we handle __int128
and unsigned __int128
? I guess in such cases, the old way is needed where __int128
is coded as s64[2]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. The old way works great for __int128 here too:
'__int128' : (ct.c_longlong * 2),
s8/16/32/64 also added. Any other type that is not supported will stop
the script from running with message
"Type: '%s' not recognized. Please define the data with ctypes manually."
src/python/bcc/table.py
Outdated
i = 0 | ||
while i < num_fields: | ||
field_name = lib.bpf_perf_event_field_name(self.bpf.module, i) | ||
field_type = lib.bpf_perf_event_field_type(self.bpf.module, i) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to have just one libbcc call like lib.bpf_perf_event_field
to get field name and type in one shot?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. We can concatenate type and name in libbcc and then split them in Python.
@boat0 buildbot is okay. I see the error with llvm 3.8.0 and llvm 3.8.1 on ubuntu.
Could you check whether you can find an alternative implementation? It would be good to find an implementation working for all llvm's. But if you cannot, using Also, could you add some tests to test some of your supported types including array types? |
Now we use getPointeeType().getTypePtr() instead of getPointeeOrArrayElementType() |
OK. I will do it in a subsequent PR. tcpconnect.py has two events defined and features __int128 and array types:
|
d8dbbd5
to
b3868c5
Compare
[buildbot, test this please] |
When ring buffers are opened to receive custom perf event, we have to define the event data structure twice: once in BPF C and once in Python. It is tedious and error-prone. This patch implements the automatic generation of Python data structure from the C declaration, thus making the redundant definition in Python unnecessary. Example: // define output data structure in C struct data_t { u32 pid; u64 ts; char comm[TASK_COMM_LEN]; }; BPF_PERF_OUTPUT(events); ... Old way: # define output data structure in Python TASK_COMM_LEN = 16 # linux/sched.h class Data(ct.Structure): _fields_ = [("pid", ct.c_ulonglong), ("ts", ct.c_ulonglong), ("comm", ct.c_char * TASK_COMM_LEN)] def print_event(cpu, data, size): event = ct.cast(data, ct.POINTER(Data)).contents ... New way: def print_event(cpu, data, size): event = b["events"].event(data) ...
Take tcpconnect.py as an example to show the new, simpler way of outputing perf event. Other tools/examples can be simplified in a similar way.
Last build failed at test_usdt2.py in which event type is defined as a simple int type.
|
[buildbot, test this please] |
The following
If users try to map __int128 differently in python, they should try to explicit ct_type definitions in python instead. |
Simplify code following iovisor#2198 (iovisor#2198). Some tools are not touched: mountsnoop.py, trace.py, lib/*.py, old/*.py.
…e in Python Follow-up to iovisor#2198.
…e in Python Follow-up to iovisor#2198.
This is great! Thanks a lot @boat0! The developer tutorial would also need to be updated to take this pull request into account. |
pr submitted #2226. Thanks! |
) * generate perf event data structure in Python automatically When ring buffers are opened to receive custom perf event, we have to define the event data structure twice: once in BPF C and once in Python. It is tedious and error-prone. This patch implements the automatic generation of Python data structure from the C declaration, thus making the redundant definition in Python unnecessary. Example: // define output data structure in C struct data_t { u32 pid; u64 ts; char comm[TASK_COMM_LEN]; }; BPF_PERF_OUTPUT(events); ... Old way: # define output data structure in Python TASK_COMM_LEN = 16 # linux/sched.h class Data(ct.Structure): _fields_ = [("pid", ct.c_ulonglong), ("ts", ct.c_ulonglong), ("comm", ct.c_char * TASK_COMM_LEN)] def print_event(cpu, data, size): event = ct.cast(data, ct.POINTER(Data)).contents ... New way: def print_event(cpu, data, size): event = b["events"].event(data) ... * tools/tcpconnect.py: deduce perf event data structure automatically Take tcpconnect.py as an example to show the new, simpler way of outputing perf event. Other tools/examples can be simplified in a similar way.
…sor#2204) Simplify code following iovisor#2198 (iovisor#2198). Some tools are not touched: mountsnoop.py, trace.py, lib/*.py, old/*.py.
…e in Python (iovisor#2211) Follow-up to iovisor#2198.
) * generate perf event data structure in Python automatically When ring buffers are opened to receive custom perf event, we have to define the event data structure twice: once in BPF C and once in Python. It is tedious and error-prone. This patch implements the automatic generation of Python data structure from the C declaration, thus making the redundant definition in Python unnecessary. Example: // define output data structure in C struct data_t { u32 pid; u64 ts; char comm[TASK_COMM_LEN]; }; BPF_PERF_OUTPUT(events); ... Old way: # define output data structure in Python TASK_COMM_LEN = 16 # linux/sched.h class Data(ct.Structure): _fields_ = [("pid", ct.c_ulonglong), ("ts", ct.c_ulonglong), ("comm", ct.c_char * TASK_COMM_LEN)] def print_event(cpu, data, size): event = ct.cast(data, ct.POINTER(Data)).contents ... New way: def print_event(cpu, data, size): event = b["events"].event(data) ... * tools/tcpconnect.py: deduce perf event data structure automatically Take tcpconnect.py as an example to show the new, simpler way of outputing perf event. Other tools/examples can be simplified in a similar way.
…sor#2204) Simplify code following iovisor#2198 (iovisor#2198). Some tools are not touched: mountsnoop.py, trace.py, lib/*.py, old/*.py.
…e in Python (iovisor#2211) Follow-up to iovisor#2198.
When ring buffers are opened to receive custom perf event, we have
to define the event data structure twice: once in BPF C and once
in Python. It is redundant and error-prone.
This patch implements the automatic generation of Python data structure
from the C declaration, thus making the redundant definition in Python
unnecessary. Example:
Old way:
New way: