Rust tools for working with Realtek firmware #261

marshallpierce · 2023-08-28T23:41:23Z

Further adventures in porting tools to Rust to flesh out the supported API.

These tools didn't feel like examples, so I made a top level bumble CLI tool that hosts them all as subcommands. I also moved the usb probe not-really-an-example into it as well. I'm open to suggestions on how best to organize the subcommands to make them intuitive to explore with --help, and how to leave room for other future tools.

I also adopted the per-OS project data dir for a default firmware location so that users can download once and then use those .bin files from anywhere without having to sprinkle .bin files in project directories or reaching inside the python package dir hierarchy.

marshallpierce · 2023-08-28T23:50:18Z

Here's the current subcommand hierarchy. Is this intuitive? Does it leave the necessary conceptual room for more tools?

% ... cg run --bin bumble
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/bumble`
Usage: bumble <COMMAND>

Commands:
  firmware  Manage device firmware
  usb       USB operations
  help      Print this message or the help of the given subcommand(s)
...

% bumble usb

USB operations

Usage: bumble usb <COMMAND>

Commands:
  probe  Probe the USB bus for Bluetooth devices
  help   Print this message or the help of the given subcommand(s)

% bumble usb probe --help
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/bumble usb probe --help`
Probe the USB bus for Bluetooth devices

Usage: bumble usb probe [OPTIONS]

Options:
      --verbose  Show additional info for each USB device

% bumble firmware

Manage device firmware

Usage: bumble firmware <COMMAND>

Commands:
  realtek  Manage Realtek chipset firmware

% bumble firmware realtek

Manage Realtek chipset firmware

Usage: bumble firmware realtek <COMMAND>

Commands:
  download  Download Realtek firmware
  drop      Drop firmware from a USB device
  info      Show driver info for a USB device
  load      Load firmware onto a USB device
  parse     Parse a firmware file

marshallpierce · 2023-08-28T23:43:18Z

tools/rtk_util.py

@@ -61,9 +61,8 @@ async def do_load(usb_transport, force):
        # Get the driver.
        driver = await rtk.Driver.for_host(host, force)
        if driver is None:
-            if not force:


I think it's erroneous to only return if not force -- otherwise, it would then try to driver.download_firmware(), which would surely fail if driver is None.

rust/Cargo.toml

marshallpierce · 2023-08-28T23:51:40Z

rust/src/wrapper/drivers/rtk.rs

+            }
+            if opcode == 0 && length == 1 {
+                project_id = offset
+                    .checked_sub(1)


The equivalent logic in python doesn't do this check before reading offset - 1, so I think on a suitably mangled file it could read from the header instead of stopping short.

rust/src/wrapper/drivers/rtk.rs

rust/Cargo.toml

rust/README.md

rust/src/cli/firmware/realtek.rs

rust/src/cli/usb/mod.rs

rust/src/wrapper/drivers/rtk.rs

whitevegagabriel · 2023-08-29T20:16:20Z

rust/src/wrapper/drivers/rtk.rs

+        // look for project id from the end
+        let mut offset = payload.len();
+        let mut project_id: Option<u8> = None;
+        while offset >= 2 {


Does this mean that a project ID can be found within a patch?

I'm assuming that there is some undocumented footer of opcodes after the patches, but I don't know for sure

If the opcodes are to be found after the patches (or not at all), then it might be more correct to only check the post-patch bytes (if present).

But I recognize that this is copying what the Python implementation does.

yep, we could chop the search down to only the space after patches, but unless we know that's actually valid, I don't think it makes sense to do anything other than blindly follow in the footsteps of the Python impl.

Any insights @barbibulle?

This code got moved; it's now at https://github.com/google/bumble/pull/261/files#diff-1a70751eee41dc84f6ba4d3247b476d244e914ad403c219c88d4c154f9b0c991R103.

The python implementation just mirrors the Linux driver implementation, where looking for the project ID is here: https://github.com/torvalds/linux/blob/99d99825fc075fd24b60cc9cf0fb1e20b9c16b0f/drivers/bluetooth/btrtl.c#L677
And you're right that the search for opcodes should stop at the end of the patch block. I think the C implementation takes the shortcut of assuming that there will always be an opcode list with at least an end opcode, so it cheats a bit by not calculating the size of the patch block. There is still a check to stop at the end of the header, because that's easy, fixed size, and that ensures the search doesn't get out of bounds of the firmware buffer at least. But a malformed file with no end or project_id opcode after the patch block could lead to incorrectly finding a lookalike inside the patch block.

rust/src/wrapper/drivers/rtk.rs

whitevegagabriel

Approval with the assumption that

whitevegagabriel · 2023-08-30T15:51:31Z

rust/src/wrapper/drivers/rtk.rs

+        // look for project id from the end
+        let mut offset = payload.len();
+        let mut project_id: Option<u8> = None;
+        while offset >= 2 {


Any insights @barbibulle?

whitevegagabriel

Approval with the assumption that the rtk firmware parsing in Python is also correct (aside from the unchecked sub when getting project_id)

barbibulle · 2023-08-30T18:28:36Z

bumble/drivers/__init__.py

@@ -66,3 +69,21 @@ async def get_driver_for_host(host):
        return driver

    return None
+
+
+def project_data_dir() -> pathlib.Path:


The JSOn KeyStore implementation currently uses appdirs for this prupose, but I see that the community is moving to platformdirs instead, as you are doing here, so I'll make the change in JsonKeyStore to switch to that as well.

bumble/drivers/__init__.py

rust/src/cli/firmware/realtek.rs

rust/src/main.rs

Further adventures in porting tools to Rust to flesh out the supported API. These tools didn't feel like `example`s, so I made a top level `bumble` CLI tool that hosts them all as subcommands. I also moved the usb probe not-really-an-`example` into it as well. I'm open to suggestions on how best to organize the subcommands to make them intuitive to explore with `--help`, and how to leave room for other future tools. I also adopted the per-OS project data dir for a default firmware location so that users can download once and then use those .bin files from anywhere without having to sprinkle .bin files in project directories or reaching inside the python package dir hierarchy.

marshallpierce commented Aug 28, 2023

View reviewed changes

marshallpierce force-pushed the mp/rust-realtek-tools branch from c891a57 to a403f79 Compare August 28, 2023 23:53

marshallpierce commented Aug 28, 2023

View reviewed changes

rust/src/wrapper/drivers/rtk.rs Outdated Show resolved Hide resolved

marshallpierce marked this pull request as ready for review August 28, 2023 23:59

marshallpierce mentioned this pull request Aug 29, 2023

Port l2cap_bridge sample to Rust #262

Merged

whitevegagabriel reviewed Aug 29, 2023

View reviewed changes

marshallpierce force-pushed the mp/rust-realtek-tools branch from a403f79 to 6e6b762 Compare August 29, 2023 22:10

whitevegagabriel reviewed Aug 30, 2023

View reviewed changes

marshallpierce force-pushed the mp/rust-realtek-tools branch 3 times, most recently from 611276d to 72712d0 Compare August 30, 2023 16:08

whitevegagabriel approved these changes Aug 30, 2023

View reviewed changes

barbibulle approved these changes Aug 30, 2023

View reviewed changes

marshallpierce force-pushed the mp/rust-realtek-tools branch from 72712d0 to 7e8d43f Compare August 30, 2023 21:35

marshallpierce force-pushed the mp/rust-realtek-tools branch from 7e8d43f to 0e2fc80 Compare August 30, 2023 21:37

barbibulle approved these changes Sep 5, 2023

View reviewed changes

barbibulle merged commit fd4d1bc into google:main Sep 5, 2023
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rust tools for working with Realtek firmware #261

Rust tools for working with Realtek firmware #261

marshallpierce commented Aug 28, 2023

marshallpierce commented Aug 28, 2023

marshallpierce Aug 28, 2023

marshallpierce Aug 28, 2023

whitevegagabriel Aug 29, 2023

marshallpierce Aug 29, 2023

whitevegagabriel Aug 30, 2023 •

edited

marshallpierce Aug 30, 2023

whitevegagabriel Aug 30, 2023

marshallpierce Aug 30, 2023

barbibulle Sep 1, 2023

whitevegagabriel left a comment

whitevegagabriel Aug 30, 2023

whitevegagabriel left a comment

barbibulle Aug 30, 2023

Rust tools for working with Realtek firmware #261

Rust tools for working with Realtek firmware #261

Conversation

marshallpierce commented Aug 28, 2023

marshallpierce commented Aug 28, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whitevegagabriel Aug 30, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whitevegagabriel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whitevegagabriel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

whitevegagabriel Aug 30, 2023 •

edited