Support multiple languages / platforms

[danobi]

Creating this issue to decide directory structure and code layout.

Current structure:

.
├── libbpf
├── LICENSE
├── README.md
├── src
│   ├── bootstrap.bpf.c
│   ├── bootstrap.c
│   ├── bootstrap.h
│   ├── fentry.bpf.c
│   ├── fentry.c
│   ├── kprobe.bpf.c
│   ├── kprobe.c
│   ├── Makefile
│   ├── minimal.bpf.c
│   ├── minimal.c
│   ├── uprobe.bpf.c
│   ├── uprobe.c
│   ├── vmlinux_508.h
│   └── vmlinux.h -> vmlinux_508.h
└── tools
    ├── bpftool
    └── gen_vmlinux_h.sh

Proposed structure:

.
├── examples
│   ├── c
│   │   ├── bootstrap
│   │   │   ├── bootstrap.bpf.c
│   │   │   ├── bootstrap.c
│   │   │   ├── bootstrap.h
│   │   │   └── Makefile
│   │   ├── fentry
│   │   │   ├── fentry.bpf.c
│   │   │   ├── fentry.c
│   │   │   └── Makefile
│   │   ├── kprobe
│   │   │   ├── kprobe.bpf.c
│   │   │   ├── kprobe.c
│   │   │   └── Makefile
│   │   ├── Makefile
│   │   ├── minimal
│   │   │   ├── Makefile
│   │   │   ├── minimal.bpf.c
│   │   │   └── minimal.c
│   │   └── uprobe
│   │       ├── Makefile
│   │       ├── uprobe.bpf.c
│   │       └── uprobe.c
│   └── rust
│       ├── Cargo.toml
│       ├── Makefile
│       └── xdp
│           ├── Cargo.toml
│           ├── Makefile
│           └── src
│               ├── bpf
│               │   └── xdp.bpf.c
│               └── main.rs
├── libbpf
├── LICENSE
├── README.md
├── tools
│   ├── bpftool
│   └── gen_vmlinux_h.sh
└── vmlinux
    ├── Makefile
    ├── vmlinux_508.h
    └── vmlinux.h -> vmlinux_508.h

Basic idea is top level entries (tools/, vmlinux/, etc.) are shared between all examples. Anything language / platform specific must be completely contained to the examples/* directory. Each language/platform has a makefile that does the right thing.

Not sure if each individual example should have its own makefile. Maybe it'd be nice to be able to cd into any example and just type make.

[chenhengqi]

I have some small questions.

1. Do we need a symbolic link in src/bpf to vmlinux.h ?
2. How does rust example link against libbpf ?

[anakryiko]

I like it overall, but I'd leave libbpf submodule at the top level still. It is not used from Rust, but I feel like it's a good idea to keep it at top level, because some other language might come up that will build libbpf from sources (e.g., Go + Cgo). In in general, it's libbpf-bootstrap, so libbpf should be more apparent ;)

As for Makefile in each example subdir. It's trivial to have a simlar makefile that will delegate to upper level Makefile for build and clean up, so yeah, cd examples/c/bootstrap && make should just work, IMO.

[danobi]

@chenhengqi

Do we need a symbolic link in src/bpf to vmlinux.h ?

Yeah, we could. Or we could just have src/bpf/xdp.bpf.c do #include "../../../vmlinux/vmlinux.h".

How does rust example link against libbpf ?

libbpf-rs depends on libbpf-sys. libbpf-sys vendors (submodule) a specific libbpf version and builds/links to it at compile time: https://github.com/alexforster/libbpf-sys/blob/3c1c21e45c7ea74f6d0fe18ab99624d9ee14fc96/build.rs#L13-L23

[danobi]

I like it overall, but I'd leave libbpf submodule at the top level still. It is not used from Rust, but I feel like it's a good idea to keep it at top level, because some other language might come up that will build libbpf from sources (e.g., Go + Cgo). In in general, it's libbpf-bootstrap, so libbpf should be more apparent ;)

Sure, sounds good.

As for Makefile in each example subdir. It's trivial to have a simlar makefile that will delegate to upper level Makefile for build and clean up, so yeah, cd examples/c/bootstrap && make should just work, IMO.

Ok, I'll look into that

I'll update the issue summary

[waruqi]

The kprobe program depends on vmlinux.h. If I want to compile the android bpf program, do I need to generate the vmlinux.h file corresponding to the android system? How should I organize them?

└── vmlinux
    ├── vmlinux_508.h
    └── vmlinux.h -> vmlinux_508.h

Or this?

- vmlinux
  - linux/vmlinux.h
  - android/vmlinux.h

[danobi]

Maybe:

└── vmlinux
    ├── Makefile
    ├── xmake.lua
    ├── vmlinux_508.h
    └── vmlinux.h -> vmlinux_508.h

and then depending on if you use make or xmake vmlinux_508.h and vmlinux.h is overwritten. And if you retarget xmake to android, vmlinux* gets overwritten

[anakryiko]

No overwriting, please. See iovisor/bcc#3265 for how we dealt with it in libbpf-tools. Do you think that's feasible here as well?

[waruqi]

So only one vmlinux.h can support linux and android at same time? It seems great!

[anakryiko]

Yes, it's just a source of type definitions, nothing more. And you don't even need vmlinux.h, you can define all the types on your own and mark them with __attribute__((preserve_access_index)) to make them CO-RE-compatible.