feat(linux): own udev rules setup — deploy fails with 'Permission denied' but advises cables and BOOTSEL

[zackees]

Problem

On Linux, fbuild deploy can flash successfully and then be unable to reopen the port it just flashed, because serial device nodes are root:dialout 0660 and the invoking user is typically not in dialout. Nothing in the toolchain sets this up.

The failure is misleading, which is the expensive part. Real output from an unattended RP2350 bench run:

wrote /run/media/niteris/RP2350/NEW.UF2
deploy succeeded (full flash); FBUILD_DEPLOY_PORT=; firmware deployed to RP2040 via
BOOTSEL mass-storage (/run/media/niteris/RP2350); the firmware was flashed and accepted,
but no healthy, openable runtime CDC port reappeared within 30s (elapsed 30023ms;
prior port none; requested serial none; catalogue candidates: /dev/ttyACM1
(serial 2DCB876B587EA334; health healthy); last open error /dev/ttyACM1: Permission denied).
To recover manually: use a direct motherboard USB port and a known-good data cable,
then 1) hold BOOT/BOOTSEL, 2) press and release RESET while still holding BOOT, ...

fbuild correctly identifies the candidate, reports it as health healthy, and even prints Permission denied — then advises cables, BOOTSEL and RESET. The remedy has nothing to do with the cause. On an unattended bench this reads as a hardware fault and costs a long debugging detour.

Worse, the state is self-perpetuating: a one-shot chmod/chgrp on the node does not survive, because the deploy cycle itself re-enumerates the board (BOOTSEL → application), and udev recreates the node as root:dialout before fbuild reopens it. Only a udev rule keyed on vendor ID actually holds.

Nothing currently installs rules

• fbuild has no udev, setup, or doctor subcommand. fbuild device is list / status / lease / release / take. Nothing writes to /etc/udev/rules.d or checks group membership.
• FastLED only diagnoses, and only on the legacy path: ci/compiler/pio.py::check_usb_permissions() looks for /etc/udev/rules.d/99-platformio-udev.rules, membership in dialout/uucp/plugdev/tty, alternative rules files, and root. It reports and never installs — and it lives in the PlatformIO compiler path, which the fbuild deploy path never calls.
• FastLED's own tooling already assumes the group exists: ci/codec_cpu/device_profile.py shells out through sg dialout -c.

Historically PlatformIO shipped 99-platformio-udev.rules for users to install by hand. fbuild replaced pio for build and deploy but did not carry that setup step across, so a fresh Linux bench has no component responsible for serial access.

Why this belongs in fbuild

fbuild is the only component that knows the full device set. The vid-ingest series (#723–#739, plus "chore(usb): migrate all VID/PID catalogue data to FastLED/boards") means fbuild already ingests a catalogue of ~36 vendors / ~1056 products — Raspberry Pi 0x2E8A, Espressif 0x303A, FTDI 0x0403, WCH 0x1A86, SiLabs 0x10C4/0x2544, PJRC 0x16C0, NXP 0x1FC9, Nordic 0x1915, Adafruit 0x239A, Arduino 0x2341/0x2A03, and the rest.

So fbuild can generate the correct rules from its own registry rather than anyone hardcoding a vendor list that drifts. Every consumer repo re-deriving this is exactly the duplication the VID/PID migration was meant to end.

Proposal

1. Classify the error correctly (highest value, lowest risk — worth doing even if nothing else lands).

When the post-deploy reopen fails with EACCES, say so and point at the fix, instead of emitting cable/BOOTSEL/RESET advice. The information is already in hand — the message literally contains Permission denied.

2. fbuild doctor — read-only preflight: are the candidate nodes openable, is the user in a group that grants access, do any recognised rules files exist. Cheap to run before a deploy, and turns a 30 s timeout into an immediate diagnosis.

3. fbuild udev generate — emit rules to stdout, derived from the ingested VID catalogue:

SUBSYSTEM=="tty", ATTRS{idVendor}=="2e8a", GROUP="plugdev", MODE="0660"
SUBSYSTEM=="tty", ATTRS{idVendor}=="303a", GROUP="plugdev", MODE="0660"
...

Printing rather than writing matters: on NixOS /etc is generated from declarative config, so a hand-written file there is both out-of-band and liable to be clobbered. Users need the content to paste into services.udev.extraRules, not a mutation.

4. fbuild udev install (optional) — write /etc/udev/rules.d/99-fbuild.rules and reload. Should require root and print the sudo command rather than escalating on its own. A --transient variant targeting /run/udev/rules.d/ is useful for CI and ephemeral benches, since tmpfs clears on reboot and leaves no persistent change.

Workaround in the meantime

SUBSYSTEM=="tty", ATTRS{idVendor}=="2e8a", GROUP="users", MODE="0660"
SUBSYSTEM=="tty", ATTRS{idVendor}=="303a", GROUP="users", MODE="0660"

Written to /run/udev/rules.d/99-fastled-bench.rules + udevadm control --reload-rules && udevadm trigger. After that, deploy reported FBUILD_DEPLOY_PORT=/dev/ttyACM1 and the full RP2350 driver matrix ran (PIO0/PIO1/PIO2 all pass, RPC smoke pass). Adding the user to dialout is the clean permanent equivalent.

Environment

NixOS, Linux 6.18.48, x86_64, fbuild 2.5.22. Reproduced with an RP2350 (2e8a) and an ESP32-C6 (303a) on the same host.

Related: FastLED#3899 (RP2350 HIL), FastLED#3714 (unattended deploy & recovery), fbuild#1423 (SWD/probe-rs transport for non-enumerating RP boards).

Generated with Claude Code

https://claude.ai/code/session_01KkufoNxfnNRU9psT3R9F51