Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images #102

astoycos · 2022-08-10T14:40:33Z

Starting work on #99

packaging/container-deployment/Containerfile.bytecode

astoycos · 2022-08-15T15:09:18Z

The first draft of this is functional, there are still some TODOs in the spec + implementation but I want to get some review before moving forward.

docs/admin/shipping-bytecode.md

donaldh · 2022-08-16T07:53:33Z

docs/admin/shipping-bytecode.md

+
+We provide two distinct spec variants here to ensure interoperatiblity with existing registries
+and packages which do no support the new custom media types defined here.
+


Can you add a bullet list with the names of the two spec variants here.

Done, this is a rough draft eventually this may turn into 2 separate documents like solo did for wasm modules here

docs/admin/shipping-bytecode.md

astoycos · 2022-08-24T21:33:20Z

@dave-tucker This should be ready to go (At least for the quick and dirty version) there's still some todo's and I'd feel better with a few unit tests but for the e2e K8s demo this is a must

LMK what you think specifically about 9f6e937 and 4b5fb08

dave-tucker

A few nits around error handling but overall this LGTM 👍
Needs a rebase also

dave-tucker · 2022-08-30T11:18:05Z

bpfd/src/server/pull_bytecode.rs

+use serde_json::Value;
+use tar::Archive;
+
+const CONTAINERIZED_BYTECODE_PATH: &str = "/etc/bpfd/bytecode/";


This should be in /var

So this might break @Billy99's work... If we're not running with sudo will we have the ability to create/ put files in /var/bpfd/bytecode?

I had code changes pending on my branch to change the user from bpfctl to bpfd-clients. I can change the directory with those changes and make sure it has the correct owner.

Ack thanks!

See #118
FYI: I change the name if needed.

dave-tucker · 2022-08-30T11:20:59Z

bpfd/src/server/pull_bytecode.rs

+        .expect("Not a valid bytecode image reference");
+
+    let protocol = client::ClientProtocol::Https;
+    // TODO(astoycos): Add option/flag to authenticate against private image repositories


That should live in bpfd.toml we can have something like:

[ registry ] [ registry.docker-io ] username = foo password = bar

We can then infer the registry from the image path and login if required.
Can be added later so lets add an issue to track it,

Sounds good to me I will make an issue to track this

dave-tucker · 2022-08-30T11:25:14Z

bpfd/src/server/pull_bytecode.rs

+    debug! {"Pulling bytecode from image path: {}", image_url}
+    let image: Reference = image_url
+        .parse()
+        .expect("Not a valid bytecode image reference");


We should explicitly handle this error (and other errors here) vs. using expect which will panic.
i.e. .map_err(ImageError::InvalidImageUrl)?.

+1 let me whip up some custom error types :)

Add some documentation around delivering bytecode to bpfd via container images. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

astoycos · 2022-08-30T14:20:54Z

Done this should be ready to go barring CI

Add a new flag to bpfctl titled `--from-image` which will signal that the provided <PATH> is actually a valid container image URL Update bpfd to be able to pull and extract EBPF bytecode from a container image if properly configured. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

This allows us to not have to worry about installing openSSL on different underlying distros However for use with fedora we still need to install perl :/ see -> openssl/openssl#13761 Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

Add option to only compile the bpf dispatcher written in C using clang. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

Update dockerfile to only build C ebpf dispacher program for now, which speeds up build time. Move from the nigthly rust image to rust:1.63.0-bullseye. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

dave-tucker · 2022-08-30T14:48:09Z

rustfmt me please 😆

astoycos · 2022-08-30T14:52:44Z

I always forget 🥇

dave-tucker reviewed Aug 10, 2022

View reviewed changes

packaging/container-deployment/Containerfile.bytecode Outdated Show resolved Hide resolved

astoycos force-pushed the bytecode-image branch 2 times, most recently from d28f3b0 to 50dc5aa Compare August 15, 2022 15:07

astoycos marked this pull request as ready for review August 15, 2022 15:08

astoycos force-pushed the bytecode-image branch 2 times, most recently from d4ab23c to fa41b45 Compare August 15, 2022 15:35

astoycos requested a review from dave-tucker August 15, 2022 17:07

donaldh reviewed Aug 16, 2022

View reviewed changes

docs/admin/shipping-bytecode.md Outdated Show resolved Hide resolved

donaldh reviewed Aug 16, 2022

View reviewed changes

docs/admin/shipping-bytecode.md Outdated Show resolved Hide resolved

astoycos force-pushed the bytecode-image branch from 92c8b5d to fabf0eb Compare August 24, 2022 20:36

astoycos changed the title ~~WIP Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images~~ Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images Aug 24, 2022

astoycos force-pushed the bytecode-image branch 2 times, most recently from 23f91fd to 4b5fb08 Compare August 24, 2022 21:05

dave-tucker mentioned this pull request Aug 30, 2022

Enable running bpfd as systemd service #110

Merged

dave-tucker reviewed Aug 30, 2022

View reviewed changes

Add example bytecode image spec

5e47c90

Add some documentation around delivering bytecode to bpfd via container images. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

astoycos mentioned this pull request Aug 30, 2022

Add config option to specify container registry credentials for pulling bytecode images #119

Closed

astoycos force-pushed the bytecode-image branch from 4b5fb08 to 79b77f0 Compare August 30, 2022 14:20

astoycos mentioned this pull request Aug 30, 2022

Add Unit tests to bytecode image pulling #120

Closed

Andrew Stoycos added 4 commits August 30, 2022 10:41

Use vendoed openSSL

ab2bc80

This allows us to not have to worry about installing openSSL on different underlying distros However for use with fedora we still need to install perl :/ see -> openssl/openssl#13761 Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

Add opt to only compile c code

e718590

Add option to only compile the bpf dispatcher written in C using clang. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

Update bpfd dockerfile

61609fa

Update dockerfile to only build C ebpf dispacher program for now, which speeds up build time. Move from the nigthly rust image to rust:1.63.0-bullseye. Signed-off-by: Andrew Stoycos <astoycos@redhat.com>

astoycos force-pushed the bytecode-image branch from 79b77f0 to 61609fa Compare August 30, 2022 14:48

astoycos merged commit 8d837c1 into bpfman:main Aug 30, 2022

dave-tucker mentioned this pull request Sep 21, 2022

Using CNI images to ship eBPF bytecode #99

Closed

astoycos deleted the bytecode-image branch April 27, 2023 20:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images #102

Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images #102

astoycos commented Aug 10, 2022

astoycos commented Aug 15, 2022

donaldh Aug 16, 2022

astoycos Aug 24, 2022

astoycos commented Aug 24, 2022 •

edited

Loading

dave-tucker left a comment

dave-tucker Aug 30, 2022

astoycos Aug 30, 2022

Billy99 Aug 30, 2022

astoycos Aug 30, 2022

Billy99 Aug 30, 2022

dave-tucker Aug 30, 2022

astoycos Aug 30, 2022

dave-tucker Aug 30, 2022

astoycos Aug 30, 2022

astoycos commented Aug 30, 2022

dave-tucker commented Aug 30, 2022

astoycos commented Aug 30, 2022


		We provide two distinct spec variants here to ensure interoperatiblity with existing registries
		and packages which do no support the new custom media types defined here.

Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images #102

Add example bytecode image spec + ability for bpfd to pull and extract bytecode from images #102

Conversation

astoycos commented Aug 10, 2022

astoycos commented Aug 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

astoycos commented Aug 24, 2022 • edited Loading

dave-tucker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

astoycos commented Aug 30, 2022

dave-tucker commented Aug 30, 2022

astoycos commented Aug 30, 2022

astoycos commented Aug 24, 2022 •

edited

Loading