Opt-in HTTP tap for the raw /dev/diag byte stream

[lukejenkins]

Prerequisites

• I have read CONTRIBUTING.md.

What problem does this feature solve or what does it enhance?

The Linux diag character device only permits one consumer at a time in memory-device mode. As soon as rayhunter holds that handle, any second tool that wants to read /dev/diag (QCSuper, other DIAG analyzers, bespoke debugging/capture scripts) is refused by the kernel with EBUSY.

Users on a supported device therefore have to choose: run rayhunter's IMSI-catcher analysis or run their second tool — never both at the same time. That forces a workflow of "stop rayhunter → run the other tool → restart rayhunter and lose the passive-monitoring gap", which undercuts rayhunter's core value proposition as a persistent background monitor.

This is a known rough edge. It's also a tiny sliver of the much larger multiplexer/unification argument that @trevormjaymes made in #895 (comment) — where the proposed end-state is a full DiagServer that owns /dev/diag, with rayhunter and other clients as fan-out consumers. That is a substantial architectural shift; I'm not proposing to take it on here. I'm proposing the minimum change that lets a second DIAG tool coexist with rayhunter today, on one supported device, gated behind an opt-in flag — a stepping stone that could be replaced or absorbed by a fuller multiplexer later without changing the public wire contract much.

Proposed Solution

Add an opt-in, off-by-default HTTP endpoint on the existing axum server:

GET /api/diag/stream

• Controlled by a new diag_stream_enabled boolean in config.toml. Default false; behavior unchanged for every existing user.
• Also gated on !debug_mode, since there is no real DiagDevice to tap in debug mode.
• When active, rayhunter's existing single-consumer read of /dev/diag is tee'd into a bounded tokio::sync::broadcast channel before parsing. Each HTTP subscriber gets its own BroadcastStream view.
• Response is Content-Type: application/octet-stream, Transfer-Encoding: chunked, X-DTAP-Version: 1. Body is the raw bytes rayhunter reads from /dev/diag — Qualcomm DIAG memory-device-mode MessagesContainer records, byte-for-byte identical to what a direct /dev/diag consumer would see. (HDLC framing lives inside each inner message's data.)
• Slow subscribers observe RecvError::Lagged(n) and drop frames; a single warning is logged per lag. rayhunter's own parser is never gated on subscriber speed — that's the key safety property.
• When disabled, the endpoint returns 503 Service Unavailable with a descriptive message so clients get a clear signal instead of hanging.

The implementation is small (~150 lines of Rust, all behind diag_stream_enabled). I made a proof-of-concept and tested it end-to-end on hardware, but per the CONTRIBUTING.md guidance on substantial contributions I'm raising it here before opening a PR. The working branch — if it helps the discussion — is lukejenkins/rayhunter:feat/diag-stream-http.

Testing already run on an Orbic RC400L: wire-format smoke test, multi-subscriber byte-for-byte equality check, independent parser validation of captured output (unwrap MessagesContainer → HDLC CRC-CCITT verify → DLF decode → 7/7 LteRrcOtaMsg parsed cleanly), and a coexistence check confirming rayhunter's own QMDL recording runs uninterrupted while two subscribers are live.

Alternatives Considered

• Stop rayhunter, run the other tool, restart — the status quo. Defeats the point of passive monitoring; creates a gap during which IMSI-catcher activity would go undetected.
• Ship a separate diag-multiplexer daemon that owns /dev/diag and offers IPC to rayhunter + other clients — closer to the full architecture sketched in #895 (comment). Cleaner long-term, but a much larger change: new binary, new installer story, new recovery story, breaks the current single-binary model. Worth doing eventually; out of scope here.
• Export only rayhunter's post-decode PCAP stream — lossy. Rayhunter's PCAP conversion only covers a subset of log codes; a second tool running DIAG-level analysis needs the raw bytes. Also doesn't help tools that depend on DIAG request/response commands rather than passive log reads.
• Background-export each read to a file, let a second tool tail the file — works, but: disk I/O on small-flash devices, file-rotation complexity, and a tailer sees writes at rayhunter's read cadence with whatever buffering the filesystem adds. An HTTP stream is cheaper to reason about and doesn't touch storage.
• Add a compile-time feature flag instead of a runtime config flag — runtime is preferable: same binary works for everyone; users don't need to rebuild to enable it, and distributions don't have to pick a single default.

AI-assisted contribution disclosure

This issue body was written by me. The patch itself was drafted with AI assistance (Claude); design, tap insertion point, backpressure semantics, 503 gating, X-DTAP-Version, config-flag naming, and wording were reviewed line-by-line. Hardware validation on RC400L was run and reviewed by me.

[untitaker]

it's pretty obvious that large portions of this issue were also written with AI, despite what you're saying. this feature also doesn't solve anything, we don't have anybody asking for running multiple diag consumers on top of rayhunter. the problem is the preexisting consumers on the device itself who will open diag directly.

[lukejenkins]

Crap, I thought I'd removed that crud before posting. The intent wasn't to deceive, and I apologize for hitting post with that line included. I am a human with good intentions, just not the best judgment of how tired is too tired to post.

To your point actually /I/ hit the issue of wanting to do other work with diag logs, and I couldn't use my rayhunter running rc400l's because of the single device limitation of /dev/diag . I'll go forward with running custom builds if I need to, but I think that finding at least one other person who has talked about needing multiple endpoints for diag was enough of a possible user base for this small LOC change that it might be of interest to the rest of the project.

[untitaker]

okay, fair feature request. I'm not sure either how we can solve this, I'm not sure I trust the conclusion in that other issue. sorry for the fast closing, we get a lot of noise.

[lukejenkins]

I brought up the other issue as it was talking about multiple users of diag, other mentions are waaay back in the qcsuper dependency days. I don't want to hitch the horse of this issue to the wagon of #895. It looks like it would be quite the undertaking.

I tried to keep my change as small as possible in the rayhunter project and I'll keep the crud that I want to with some diag for wardriving in my own project. I went with the option of adding an http endpoint for getting the messages as that seemed like the smallest change, but if the preference is something else on the rayhunter modem, I'd be great with that too.

I know that file io based options grabbing the diag logs as they're written is possible today without no changes to rayhunter, but fighting with file io has been slow and problematic for me so far.

And no problem about the fast close, I probably would have done the same in your shoes.

[cooperq]

one optional workaround while we mull this over is to run rayhunter-check on the pcaps produced by qcsuper or whatever else you are using.

[lukejenkins]

Its the other way around actually, I want rayhunter to rayhunter while I simultaneously use the diag feed for my own purposes; most of which need a low latency diag feed.

[lukejenkins]

Sorry misclick on close, please re-open.

[cooperq]

I see, and that sounds like a fun edge case but its very much an edge case, you are the only user I know of that wants to do this. AFAICT this means adding a decent amount of code to support this one edge case. Unless you think there is a general use case for this I think its probably best left in a fork.

[lukejenkins]

Thinking about this more, I think that the better place to put the tap is upstream from rayhunter. To add support for devices that use /dev/ffs-diag rather than /dev/diag , diag_device.rs is going to need to change a lot. Once there are some knobs to configure here, I'll pick things back up. Thanks for talking it through with me. I'm good to close the issue if you folks don't have anything to add.