Skip to content

BoB L1: new tdx-init, LH 8.0.0 #43

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+32 −24
Open

BoB L1: new tdx-init, LH 8.0.0 #43

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2f22972
bob-common: bump tdx-init version
ilyaluk Nov 13, 2025
14fcc2e
bob-l1: include trixie-backports for rust 1.89
ilyaluk Nov 13, 2025
b21e753
bob-l1: bump lighthouse to 8.0.0
ilyaluk Nov 13, 2025
a6dd9b5
bob-l1: move readme, clarify image measurements
ilyaluk Nov 13, 2025
ef5dd27
Drop empty fluentbit filter events
alexhulbert Nov 25, 2025
2cf8e72
update commit
ilyaluk Nov 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bob-common/mkosi.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ make_git_package \
# Build tdx-init
make_git_package \
"tdx-init" \
"v0.1.1" \
"v0.3.0" \
"https://github.com/flashbots/tdx-init" \
'go build -trimpath -ldflags "-s -w -buildid=" -o ./build/tdx-init' \
"build/tdx-init:/usr/bin/tdx-init"
Expand Down
4 changes: 2 additions & 2 deletions bob-common/mkosi.extra/etc/fluent-bit/delay.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ function log_delay(tag, ts_table, record)
-- 2) Check if we've already processed this second
if last_processed_second == now_floor then
-- Skip the flush; Return no output
return 2, ts_table, {}
return -1, ts_table, {}
end

-- 3) Otherwise, do the flush logic once for this second
Expand All @@ -49,7 +49,7 @@ function log_delay(tag, ts_table, record)

-- 4) Return any flushed logs
if #to_emit == 0 then
return 2, ts_table, {}
return -1, ts_table, {}
else
local new_ts = { sec = now_sec, nsec = 0 }
return 1, new_ts, to_emit
Expand Down
2 changes: 1 addition & 1 deletion bob-l1/mkosi.build
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ LIGHTHOUSE_BUILD_CMD="
"
make_git_package \
"lighthouse" \
"v7.1.0" \
"v8.0.0" \
Copy link
Member

@alexhulbert alexhulbert Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lighthouse build process can be simplified starting with v8 like so: https://github.com/flashbots/flashbots-images/pull/38/files#diff-3c7f2c5d727d89202366ab0b86107555d64c34fed340037e8204c3f8959d80e7R28-R38

I'll make the necessary changes to this PR to get it working

Copy link
Contributor Author

@ilyaluk ilyaluk Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

"https://github.com/sigp/lighthouse.git" \
"$LIGHTHOUSE_BUILD_CMD" \
"target/release/lighthouse:/usr/bin/lighthouse"
4 changes: 3 additions & 1 deletion bob-l1/mkosi.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
[Build]
SandboxTrees=bob-l1/mkosi.sandbox
Environment=LIGHTHOUSE_BINARY KERNEL_CONFIG_SNIPPETS=bob-l1/kernel.config KERNEL_VERSION=6.13.12
WithNetwork=true

Expand All @@ -12,7 +13,8 @@ Packages=openntpd
BuildPackages=build-essential
git
gcc
cargo
cargo/trixie-backports
Copy link
Member

@alexhulbert alexhulbert Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, i will look for other places we're using rustup in the repo to get newer version of rust and switch them out for this technique now that the backports version is new enough

rustc/trixie-backports
zlib1g-dev
libzstd-dev
libleveldb-dev
Expand Down
1 change: 0 additions & 1 deletion bob-l1/mkosi.extra/etc/systemd/system/lighthouse.service
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ ExecStart=/usr/bin/lighthouse bn \
--execution-endpoint http://localhost:8551 \
--execution-jwt /tmp/jwt.hex \
--checkpoint-sync-url https://mainnet.checkpoint.sigp.io \
--disable-deposit-contract-sync \
--datadir "/persistent/lighthouse" \
--disable-optimistic-finalized-sync \
--disable-quic \
Expand Down
6 changes: 6 additions & 0 deletions bob-l1/mkosi.sandbox/etc/apt/sources.list.d/debian-backports.sources
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Types: deb deb-src
URIs: https://snapshot.debian.org/archive/debian/20251113T083151Z
Copy link
Member

@alexhulbert alexhulbert Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm slightly leaning towards a mkosi.* script so that we don't have to hard code the timestamp twice. What do you think? We could also just make this part of the base module. Everything should have backports, especially since it can help us avoid using rustup throughout the repo since it nontrivially slows down builds, depends on network access, and downloads untrusted binaries.

Copy link
Contributor Author

@ilyaluk ilyaluk Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, adding backports repo via mkosi.conf is not supported at the moment (or I could not figure it out). I assume that's becuase backports is a different suite. There's upstream issue about this, afaiu: systemd/mkosi#1755

Also, we can't add it via scripts (e.g. systemd/mkosi#3439), because build packages installation happens in sandbox, not image itself.

Suites: trixie-backports
Components: main
Enabled: yes
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
37 changes: 19 additions & 18 deletions bob-common/readme.md → bob-l1/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ git clone https://github.com/flashbots/flashbots-images.git
cd flashbots-images

# build the BOB (TEE searcher sandbox) image
make build IMAGE=bob
make build IMAGE=bob-l1
```

### 2. audit the VM image
Expand Down Expand Up @@ -216,41 +216,38 @@ Flashbots has adapted Edgeless Constellation’s [measured-boot](https://github.
Only [PCR 4, 9, and 11](https://constellation-docs.netlify.app/constellation/2.2/architecture/attestation#runtime-measurements) are meaningful, since the other PCR’s in Azure’s vTPM are not reproducible due to their proprietary closed-source implementations. But, these 3 measurements are enough to ensure Flashbots does not have access to the searcher VM, as any change in the image will generate different PCR 4, 9, and 11 measurements! You can test and verify this claim yourself by changing a line of code, building the new image, and running the measurement software again.

```bash
# clone and build
git clone https://github.com/flashbots/measured-boot
cd measured-boot
go build
cd flashbots-images

# measure
./measured-boot /path/to/flashbots-images/build/tdx-debian-azure.efi output.json --direct-uki
# assuming you've run make build IMAGE=bob-l1
make measure
```

<details>
<summary>Expected Output</summary>

```
ubuntu@schmangelina-bob-mkosi-builder:~/measured-boot$ ./measured-boot /home/ubuntu/flashbots-images/build/tdx-debian.efi output.json --direct-uki
EFI Boot Stages:
Stage 1 - Unified Kernel Image (UKI): f04271b7b053dde1741e103c8d64aa0e2c5042cdfb7c08ea25bf64ae005b6381
ubuntu@builder:~/flashbots-images$ make measure
EFI Boot Stages:
Stage 1 - Unified Kernel Image (UKI): 320af1bf8257b6fd1a47b8fa865bdde7bdfdbf235894804b6b15b676296b1ba4
Stage 2 - Linux : eb1a69b12b47b6b3d4716bad94323d27173cba5f4285b918a2bf59ea5cb3c9ea
Linux LOAD_FILE2 protocol:
cmdline: "console=tty0 console=ttyS0,115200n8 mitigations=auto,nosmt spec_store_bypass_disable=on nospectre_v2\x00"
initrd (digest aebd8d9d0db231daf59ccc069b2a0cd82f825e849317344d417ff1730ec0779e)
initrd (digest 0cc531c70b473425e513310dfb4cbcfd5161444a07d318b4d5b816f557d589a6)
UKI sections:
Section 1 - .linux ( 5829632 bytes): 0da293e37ad5511c59be47993769aacb91b243f7d010288e118dc90e95aaef5a, 7439b377dbba898b0db23928be49fb906aa5551cfc01395bc37b8bd50d8f5530
Section 2 - .osrel ( 308 bytes): 3fb9e4e3cc810d4326b5c13cef18aee1f9df8c5f4f7f5b96665724fa3b846e08, 94e5e922dec19c3ab3e3c85b5d30dbb563098a430418a70c11a5b729721fae39
Section 3 - .cmdline ( 101 bytes): 461203a89f23e36c3a4dc817f905b00484d2cf7e7d9376f13df91c41d84abe46, 5b20d03fb990ccafdcfa1ddb37feff37141e728776ed89f335798f3c3899a135
Section 4 - .initrd ( 163161430 bytes): 15ee37e75f1e8d42080e91fdbbd2560780918c81fe3687ae6d15c472bbdaac75, aebd8d9d0db231daf59ccc069b2a0cd82f825e849317344d417ff1730ec0779e
Section 4 - .initrd ( 166037465 bytes): 15ee37e75f1e8d42080e91fdbbd2560780918c81fe3687ae6d15c472bbdaac75, 0cc531c70b473425e513310dfb4cbcfd5161444a07d318b4d5b816f557d589a6
Section 5 - .uname ( 7 bytes): da7a6d941caa9d28b8a3665c4865c143db8f99400ac88d883370ae3021636c30, 2200d673ad92228af377b9573ed86e7a4e36a87a2a9a08d8c1134aca3ddb021c
Section 6 - .sbat ( 309 bytes): ff552fd255be18a3d61c0da88976fc71559d13aad12d1dfe1708cf950cc4b74c, eae67f3a8f5614d71bd75143feeecbb3c12cd202192e2830f0fb1c6df0f4a139
Section 7 - .data : not measured
Section 8 - .reloc : not measured
Section 9 - .rodata : not measured
Section 10 - .sdmagic: not measured
Section 11 - .text : not measured
PCR[ 4]: 52f267b72dc8a06a2aa50281aa49539c3ea08e1fd1e037bc84e00f12abd38071
PCR[ 9]: a0b3cce18e7e3073ae6332bebb23d4438873f3e73f68f882627bee5c798e03c4
PCR[11]: 04b26f0af2bffab1d37442f5e73974660578b891a0ef2f3697bc3d06b0317978
PCR[ 4]: 176543f594059b26292565a3c07b5eaa34122cf2ce7f53b149b6fb85c3046d30
PCR[ 9]: 817c80c72f0a42bd72d4c7130f0d48c39a6d3ac6def92da085dec16feb822518
PCR[11]: 46a1b5dd625d967205699242a2de2815e539424c3132306b91f31bcda442693f
PCR[12]: 0000000000000000000000000000000000000000000000000000000000000000
PCR[13]: 0000000000000000000000000000000000000000000000000000000000000000
PCR[15]: 0000000000000000000000000000000000000000000000000000000000000000
Expand All @@ -267,19 +264,23 @@ Then, copy and paste PCR 4, 9, and 11 into the following format and save as `mea
"attestation_type": "azure-tdx",
"measurements": {
"4": {
"expected": "52f267b72dc8a06a2aa50281aa49539c3ea08e1fd1e037bc84e00f12abd38071"
"expected": "176543f594059b26292565a3c07b5eaa34122cf2ce7f53b149b6fb85c3046d30"
},
"9": {
"expected": "a0b3cce18e7e3073ae6332bebb23d4438873f3e73f68f882627bee5c798e03c4"
"expected": "817c80c72f0a42bd72d4c7130f0d48c39a6d3ac6def92da085dec16feb822518"
},
"11": {
"expected": "04b26f0af2bffab1d37442f5e73974660578b891a0ef2f3697bc3d06b0317978"
"expected": "46a1b5dd625d967205699242a2de2815e539424c3132306b91f31bcda442693f"
}
}
}
]
```

> Note: at the time of the writing, those measurements were acquired by building from commit ef5dd2727ba4569d530c67822dc96778f54a295a, if you're viewing this from main branch please ensure to build from the same commit to get the same measurements.

> Note: at the time of the writing, compiling bob-l1 image is not reproducible if building under ARM mac with Rosetta. Please use x86_64 Linux for now.

### 3. audit and run the remote attestation software which requests the measurement from Azure’s vTPM

Flashbots again leverages Edgeless Constellation’s [attested TLS](https://docs.edgeless.systems/constellation/architecture/attestation#attested-tls-atls) and other attestation primitives to interact with Azure’s attestation service. CVM-reverse-proxy fetches Azure's vTPM measurement and compares it with the locally supplied measurement.
Expand Down