Sync master to 6.0/stage #30
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The main changes are: 1. Skipping the new attribute forms. 2. Handling DW_FORM_strx*, DW_FORM_line_strp, and DW_FORM_implicit_const for the attributes that we care about. 3. Parsing the new unit header format. 4. Parsing the new line number program header format. Note that Clang currently produces an incorrect DWARF 5 line number program header for the Linux kernel (https://reviews.llvm.org/D105662), so some types are not properly deduplicated in that case. Closes osandov#104. Signed-off-by: Omar Sandoval <osandov@osandov.com>
These were added in DWARF 5. They need to know the CU that they're being evaluated in, but the parameters for drgn_eval_dwarf_expression() were already getting unwieldy. Wrap the evaluation context in a new struct drgn_dwarf_expression_context, add the additional CU information, and implement the operations. Signed-off-by: Omar Sandoval <osandov@osandov.com>
The DWARF 5 format is a little more complicated than DWARF 2-4 but functionally very similar. Signed-off-by: Omar Sandoval <osandov@osandov.com>
The DWARF spec and dwarf.h list them in hexadecimal, so make it easier to cross reference. Signed-off-by: Omar Sandoval <osandov@osandov.com>
s/grather/greater/g Signed-off-by: Omar Sandoval <osandov@osandov.com>
If the kernel is compiled with CONFIG_CPUMASK_OFFSTACK, then the full struct cpumask::bits array may not be allocated. Use nr_cpu_ids as the limit instead of the length of the array. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a way to create an object from raw bytes. One example where I've wanted this is creating a struct pt_regs from a PRSTATUS note or other source. Signed-off-by: Omar Sandoval <osandov@osandov.com>
And the libdrgn implementation, drgn_object_read_bytes(). Signed-off-by: Omar Sandoval <osandov@osandov.com>
We're adding NUMA node mask helpers in osandov#107, so make sure we can run them. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Linux v5.14 renamed task_struct::state to task_struct::__state. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Sometimes we want to traverse numa nodes in the system, so add kernel nodemask helpers to support this. Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
Commit a97f6c4 ("Associate types with program") changed repr() for drgn.Type to include a "prog." prefix, but it didn't update the documentation to reflect that. It also forgot to update a global type constructor to the new Program methods. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add an example of stack traces and parameters/local variables and use some more interesting helpers. Signed-off-by: Omar Sandoval <osandov@osandov.com>
The API reference has all of the details, but add a short example to the user guide (and move it before symbols, as stack traces are probably more interesting/important). Signed-off-by: Omar Sandoval <osandov@osandov.com>
Extract for_each_set_bit() that was added internally for the cpumask and nodemask helpers, and add for_each_clear_bit() and test_bit() to go with it. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to find the corresponding "struct net_device *" object
given an interface index number. As an example:
>>> netdev = netdev_get_by_index(prog["init_net"], 1)
>>> netdev.name.string_().decode()
'lo'
Or pass a "Program" as the first argument, and let the helper find in
its initial network namespace (i.e. "init_net"):
>>> netdev = netdev_get_by_index(prog, 3)
>>> netdev.name.string_().decode()
'enp0s3'
Also add a test for this new helper to tests/helpers/linux/test_net.py.
For now, a user may combine this new helper with socket.if_nametoindex()
to look up by interface name:
>>> netdev = find_netdev_by_index(prog, socket.if_nametoindex("dummy0"))
>>> netdev.name.string_().decode()
'dummy0'
However, as mentioned by Cong, one should keep in mind that
socket.if_nametoindex() is based on system's current name-to-index
mapping, which may be different from that of e.g. a kdump.
Thus, as suggested by Omar, a better way to do name lookups would be
simply linear-searching the name hash table, which is slower, but less
erorr-prone.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
And document that it returns NULL if the UID is not found. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Mainly unused imports, unused variables, unnecessary f-strings, and regex literals missing the r prefix. I'm not adding it to the CI linter because it's too noisy, though. Signed-off-by: Omar Sandoval <osandov@osandov.com>
All of these take a type which can also be a drgn.Type, not just a str. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Add a helper to get the network device ("struct net_device *") given an
interface name. As an example:
>>> netdev = netdev_get_by_name(prog["init_net"], "lo")
>>> netdev.ifindex.value_()
1
Or pass a "Program" as the first argument, and let the helper find in
the initial network namespace (i.e. "init_net"):
>>> netdev = netdev_get_by_index(prog, "dummy0")
>>> netdev.ifindex.value_()
2
Also add a test for this new helper to tests/helpers/linux/test_net.py.
This helper simply does a linear search over the name hash table of the
network namespace, since implementing hashing in drgn is non-trivial.
It is obviously slower than net/core/dev.c:netdev_name_node_lookup() in
the kernel, but still useful.
Linux kernel commit ff92741270bf ("net: introduce name_node struct to be
used in hashlist") introduced struct netdev_name_node for name lookups.
Start by assuming that the kernel has this commit, and fall back to the
old path if that fails.
Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
With mypy 0.920, two warnings appear on current main: $ mypy --strict --no-warn-return-any drgn _drgn.pyi drgn/helpers/linux/__init__.py:36: error: Need type annotation for "__all__" (hint: "__all__: List[<type>] = ...") drgn/helpers/linux/__init__.py:38: error: unused "type: ignore" comment Found 2 errors in 1 file (checked 33 source files) The "unused" type:ignore directive was necessary for prior versions, so add --no-warn-unused-ignores, so that we pass on multiple versions. Apply a List[str] annotation to the __all__ variable to silence the other error. Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
This minor change is a quality of life improvement ensuring developers receive more warnings and diagnostics in their editors. Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
PR osandov#133 adds a test case using multiprocessing.Barrier(), which needs /dev/shm. Signed-off-by: Omar Sandoval <osandov@osandov.com>
In preparation for introducing an API to represent threads, the linux helper iterators, radix_tree_for_each, idr_for_each, for_each_pid, and for_each_task have been rewritten in C. This will allow them to be accessed from libdrgn, which will be necessary for the threads API. Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Currently, we create a section filled with zeroes to contain the symbols in our ELF symbol tests. We can just use a NOBITS section with no file data instead. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Specifically, we want logs from vmtest.download. Signed-off-by: Omar Sandoval <osandov@osandov.com>
The next change will add a C helper that needs per_cpu_ptr(). Signed-off-by: Omar Sandoval <osandov@osandov.com>
PR osandov#129 will need to get the idle thread for a CPU when the idle thread crashed. Add a helper for this. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Reserve "sock" for "struct socket *" objects, according to our kernel naming convention. Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
Add helpers to convert between sockets and inodes. As an example: >>> file = fget(task, fd) >>> sock = SOCKET_I(file.f_inode) >>> sock.type.value_() 2 >>> import socket >>> int(socket.SOCK_DGRAM) 2 >>> inode = SOCK_INODE(sock) Also add tests for the new helpers to tests/helpers/linux/test_net.py. Signed-off-by: Peilin Ye <peilin.ye@bytedance.com>
I missed that the kernel has an idle_task() function which uses cpu_rq()->idle instead of idle_threads; the latter is technically architecture-specific. So, replace idle_thread() with idle_task(), which is architecture-independent and more consistent with the kernel. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Signed-off-by: Omar Sandoval <osandov@osandov.com>
Kernels built without multiprocessing support don't have __per_cpu_offset; instead, per_cpu_ptr() is a no-op. Make the helper do the same and update the test case to work on !SMP as well. Signed-off-by: Omar Sandoval <osandov@osandov.com>
We would like to test drgn against kernel core dumps (e.g., for osandov#129). One option would be to include some vmcore files in the repository and test against those. But those can be huge, and we'd need a lot of them to test different kernel versions. Instead, we can run vmtest, enable kdump, and trigger a crash. To do that, we first need to enable a few kernel config options. Signed-off-by: Omar Sandoval <osandov@osandov.com>
We can avoid the need for the kexec tool if we load the kdump kernel ourselves, which is much easier with kexec_file_load(). Add the config options to enable it. Signed-off-by: Omar Sandoval <osandov@osandov.com>
prakashsurya
approved these changes
Jan 5, 2022
sebroy
approved these changes
Jan 5, 2022
This is another undocumented convention. Signed-off-by: Omar Sandoval <osandov@osandov.com>
This is one place where I broke the convention that I just documented. Signed-off-by: Omar Sandoval <osandov@osandov.com>
Now that the vmtest kernel supports kdump, add a script that can be used to crash and enter the kdump environment on demand. Use that to crash after running the normal test suite so that we can run tests against /proc/vmcore. vmcore tests live in their own directory; presently the only test is a simple sanity check that ensures we can can attach to /proc/vmcore. Signed-off-by: Omar Sandoval <osandov@osandov.com> Signed-off-by: Kevin Svetlitski <svetlitski@fb.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Needed for Ubuntu 20.04 transition and can be done ahead of deployment date