-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support remote symbolization #61
Labels
enhancement
New feature or request
Comments
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 17, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 17, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 17, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 17, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 17, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 17, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
First set of changes enabling address normalization: #114 |
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 18, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 18, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 18, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 18, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 19, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
d-e-s-o
added a commit
to d-e-s-o/blazesym
that referenced
this issue
Apr 19, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
d-e-s-o
added a commit
to d-e-s-o/blazesym
that referenced
this issue
Apr 20, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Apr 20, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (libbpf#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
that referenced
this issue
Apr 20, 2023
We want to support symbolization in scenarios where addresses are captured on one system and then symbolized elsewhere (#61). In a nutshell, we are going to perform address "normalization" on the "local" system (the one that captured the addresses), send them to the remote, and then symbolize these normalized addresses on the remote. This change introduces the logic for normalizing addresses. Basically, we use the corresponding proc maps entry, parse the contents, identify the ELF files to which those addresses map, and then perform normalization based on the ELF information. I introduced a new public module, normalize. We have to think some more about how best to expose this functionality, but this is a reasonable starting point. The normalization does not yet use advanced caching of sorts: each ELF entry handled is parsed as we found it. That is not particularly effective but not wrong. We can optimize further subsequently without adjusting the API. Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
to danielocfb/blazesym
that referenced
this issue
Oct 17, 2023
With the normalization API rework (mainly libbpf#359), we no longer intend to add specific remote symbolization APIs. Rather, the existing symbolization APIs were adjusted to be compatible for use with the outputs of normalization. As such, let's mark the remote symbolization TODO (and issue) as done. Closes: libbpf#61 Signed-off-by: Daniel Müller <deso@posteo.net>
danielocfb
pushed a commit
that referenced
this issue
Oct 17, 2023
With the normalization API rework (mainly #359), we no longer intend to add specific remote symbolization APIs. Rather, the existing symbolization APIs were adjusted to be compatible for use with the outputs of normalization. As such, let's mark the remote symbolization TODO (and issue) as done. Closes: #61 Signed-off-by: Daniel Müller <deso@posteo.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Symbolization is a potentially resource intensive process and it may not be feasible to perform it on the very system where addresses are recorded. Embedded devices, for example, with limited disk space and CPU capacity, cannot afford to perform symbolization on the device itself: debug information can be large and would be prohibitive to disk space usage and so it is unlikely to be stored on the device itself and the process of symbolization is likely to impact other running applications negatively, would be taking excessive amounts of time, or both.
For that and other reasons, we'd like to support remote (or off-device) symbolization. The below (preliminary) API proposal flushes out the idea somewhat.
The local side normalizes a list of addresses using the
normalize_addresses
function:The resulting normalized addresses together with information about their owners have to be conveyed to the remote for the actual symbolization to happen. The transfer of this information is outside of blazesym‘s purview and a responsibility of the user. For Rust users, we will provide serde derives for convenient serialization & deserialization.
On the remote system, blazesym‘s existing BlazeSymbolizer can be used to perform the symbolization using the newly added symbolize_normalized method:
The API should also allow us to enable
debuginfod
support, by having an implementor ofAddressMetaResolver
that speaks the corresponding protocol and fetches debug information from a service using it.The text was updated successfully, but these errors were encountered: