Add remote device support via bpfd #2298

This patch adds BPFd, a standalone executable designed to provide BCC the ability to work across system and architecture boundaries. This is done by loading the BPFd executable onto a remote target device to have it act as a proxy for whenever a BCC tool wishes to perform an operation on the system (e.g. load BPF programs, read /proc/kallsyms, attach kprobes, etc.). This arrangement allows developers to have kernel sources and the LLVM stack on a separate host machine (e.g. the development machine) instead of needing to set all these up on the target device, thereby allowing for the drastic reduction of space required on a target for BCC tools to run. The reduction of the space requirement, in particular, becomes a much more critical factor for devices that have more limited disk space (e.g. embedded devices). In addition, the above set-up also allows developers to run clang on a different architecture than the target's architecture, thus facilitating cross-compilation development. However, the natural disadvantage for cross-developers is that there is a need to have a copy of the target's kernel sources on the host for the above set-up to work. For more information, please check out the README in the original BPFd repository (https://github.com/joelagnel/bpfd) and this LWN article explaining the purpose of and how BPFd works in more detail (https://lwn.net/Articles/744522/) Signed-off-by: Jazel Canseco <jcanseco@google.com> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>

The bpf shared library (i.e. libbpf.so) is included since the bpfd executable dynamically links to it. Signed-off-by: Jazel Canseco <jcanseco@google.com> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>

This modifies BCC to query BPFd whenever BCC wishes to perform an operation on a remote target device (e.g. load BPF programs, read /proc/kallsyms, attach kprobes, etc.) If no remote target device has been configured, BCC defaults to performing the operations on the local system just like before. Signed-off-by: Jazel Canseco <jcanseco@google.com> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>

Signed-off-by: Jazel Canseco <jcanseco@google.com>

libbcc is big because it links to llvm/clang and, linking directly to it, bpfd has a much bigger footprint than necessary. Since we don't want to compile restricted-C to eBPF on embedded devices running just bpfd, we can achieve a much smaller disk footprint by linking only the minimum objects required by bpfd to run and communicate with the host bcc to load/unload the pre-compiled ebpf programs. Before this commit: $ ldd src/cc/bpfd/bpfd linux-vdso.so.1 (0x00007ffecc3df000) libz.so.1 => /usr/lib/libz.so.1 (0x00007f02b5606000) libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f02b5601000) libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x00007f02b5592000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f02b5571000) libelf.so.1 => /usr/lib/libelf.so.1 (0x00007f02b5557000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f02b53c8000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f02b5241000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f02b5227000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f02b5063000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f02b7a67000) $ ls -lsha src/cc/bpfd/bpfd 40M -rwxr-xr-x 1 adi adi 40M Mar 25 10:59 src/cc/bpfd/bpfd After: $ ldd ./src/cc/bpfd/bpfd linux-vdso.so.1 (0x00007ffffc5f1000) libelf.so.1 => /usr/lib/libelf.so.1 (0x00007f93b60d6000) libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f93b5f47000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f93b5f2d000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f93b5d69000) libz.so.1 => /usr/lib/libz.so.1 (0x00007f93b5b52000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f93b614f000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f93b59cd000) $ ls -lsha ./src/cc/bpfd/bpfd 204K -rwxr-xr-x 1 adi adi 203K Mar 23 19:18 ./src/cc/bpfd/bpfd Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>

Building bpfd should not require also building full BCC, because bpfd is much smaller (~180kb release binary), has fewer dependencies and is easier to cross compile and run on resource constrained embedded devices without LLVM/Python. This enables standalone bpfd builds, for example, to cross-compile bpfd for 32bit arm systems: $ mkdir build; cd build $ cmake -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchain-arm.cmake .. $ make When running the above cross-build instructions, the proper cross-compilation dependencies (libelf, libz, libstdc++) and toolchains are needed (can be added to CMAKE_FIND_ROOT_PATH). Bpfd is also built when compiling full BCC. Distribution packagers can package bpfd stand-alone like the BCC python bindings or together with the rest of bcc. Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>

This backend allows BCC to connect and run eBPF on remote targets with bpfd's stdin/stdout attached to ssh sockets. It deliberately avoids the complications of ssh authentication and authorization, so the ssh communication channel should be setup before BCC runs. The environment variables necessary to use this backend: BCC_REMOTE=ssh BCC_REMOTE_SSH_USER=<user> BCC_REMOTE_SSH_ADDR=<ip-addr-or-hostname> Optional variables: BCC_REMOTE_SSH_CMD=<cmd> (default runs 'bpfd' in $PATH) BCC_REMOTE_SSH_PORT=<port> (default 22) Special priviledges (usually root/sudo access) to run BCC need to be present only on the remote machine which runs bpfd, the local host running python/bcc/llvm can run as a normal user. This backend can also be used on machines with different architectures than x86 by enabling BCC cross-compilation via the ARCH environment variable (example ARCH=arm64). Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add remote device support via bpfd #2298

Add remote device support via bpfd #2298

Commits on Apr 23, 2019

Add remote device support via bpfd #2298

Are you sure you want to change the base?

Add remote device support via bpfd #2298

Commits on Apr 23, 2019