Feedback from Medical Users

[Kieirra]

Several of you have mentioned the idea of a custom rule import system, and I’m considering it. Nothing is decided yet.

With version 1.8.0 introducing regex support in custom rules, this could make advanced configurations more flexible. I’m exploring whether it would make sense to allow importing custom rules and possibly custom dictionaries as well, especially for specific professional fields such as healthcare.

If you work in the medical field and use custom rules or a custom dictionary (for medications, abbreviations, specific terminology, etc.), feel free to share the actual rules you are using. This will help me assess whether this feature would be useful and worth implementing.

[Kieirra]

I am tagging you @thiswillbeyourgithub as you were one of the first users to suggest it.

[Kieirra]

https://framagit.org/interhop/murmure/-/blob/main/README.md?ref_type=heads

[thiswillbeyourgithub]

Regarding the dictionnary (not rules):

• i can provide a list of all the medication used in france/europe. Either commercial names or INN.
• i can provide a list of the 2020 french medical dictionnary: https://www.academie-medecine.fr/le-dictionnaire/index.php

Can you clarify again how the dictionnary works at the code level? Is it scalable with 100k terms? Will it obliterate generating out of dictionnary words?

[thiswillbeyourgithub]

Regarding the rules, i've been so far curating a pretty good list to make dictation "natural". Things like:

• open/close parenthesis
• open/close quotation marks
• newlines
• dashes
• commas
• collons
• converting units like "nanogramme par millilitre"

But because it took about 3-7 rules per rule to make it work reliably without regex, I didn't try to create medically related rules until regexes are implemented.

[Kieirra]

While the dictionary can technically handle 1,000+ entries without dropping performance, adding massive datasets (like entire medical databases) will actually degrade the overall recognition accuracy due to phonetic collisions. It also negatively impacts the 'LLM Connect' feature, since these dictionary words are injected into the LLM context.

A 'good' candidate for the phonetic dictionary is a high-frequency, everyday word that Parakeet consistently misunderstands and replaces with a specific incorrect word.

But because it took about 3-7 rules per rule to make it work reliably without regex, I didn't try to create medically related rules until regexes are implemented.

I can compile a nightly build for you if you'd like. Just keep in mind it might contain some bugs, and you'll need to replace it once 1.8.0 is officially released. Let me know your OS

[Kieirra]

Actually there's a simpler alternative to the reverse proxy that satisfies the "no inbound traffic to workstations" rule: an outbound-only tunnel.

It's very close to Option 3 of the documentation, except the relay is self-hosted inside your own DMZ instead of Cloudflare's:

• A small agent runs on each workstation and opens a persistent outbound HTTPS connection to a relay placed in your DMZ.
• When a phone sends a request, it hits the relay, which forwards it back through the connection already opened by the workstation.
• Workstations don't listen on any port, they can't be scanned or reached from the LAN.
• Your firewall rule stays fully intact, traffic is always outbound-initiated.

With TLS passthrough on the relay, the relay never decrypts the content, it only forwards encrypted bytes. Same end-to-end encryption property as a TURN server with SRTP, with way less complexity on both sides.

This works with the current version of Murmure, no code change needed. Happy to document it as the recommended enterprise deployment pattern if your security team is ok with it.

[thiswillbeyourgithub]

Hi,

I'm quite possibly the least knwoledgeablz about network here but i'm getting the feeling that we have come full circle back to my original suggestion haven't we?

Currently my workstation's browser is able to use the port 443 to my self-hosting server to connect to my phone via TURNS("relay").

Now say we move the relay inside the hospital, but in the DMZ so it can be reached by the phone and by the workstation.

And make it so that only murmure can initiate a connection to that relay, and only after it passed some SSO token.

The phone can join to that relay connection only via a token in the QR code of murmure.

Possibly, the phone has to pass SSO too to even have the right to pass the QR code token. I'm actually not sure this authentication is actually needed and if it would not weaken the overall security model because it would make a personal device able to do a SSO connection (risk of keylogging for example to steal the first factor, and 'notification fatigue' increasing the chance of validating a second factor when it's wrong), without actually improving the security. I think we require SSO only for suspicious activity (many attempts, never before seen IP, IP outside of the region, etc).

Once the phone is connected: the data can transit through the relay to the murmure app, and we can configure the relay to only transit from the phone to the computer and not the other way around (or just the bare minimum signaling like some ACK).

Did I get that right? If not, what's different from what you're saying @Kieirra?

I don't really see what's the technical risk that is not already present on the workstations right now, as proven because parakeet_web basically does this already (minus SSO).

I hope this is useful and I'm not just slowing you guys down. Don't be afraid to tell me I'm just a psychiatrist :/

[Kieirra]

From my understanding, in your setup, the website is already exposed, so it can easily initialize the connection with TURN.

But in Murmure's case, even if I change the protocol to allow TURN, we still face a fundamental issue: the smartphone needs to download the Murmure web interface (the HTML/JS for the PWA) and perform the initial Signaling.

Even if a hospital hosts its own Signaling and TURN servers in their DMZ, they still need a way to serve the Murmure web app from the locked-down workstation to the external phone. To achieve this without opening inbound ports on the workstation, they must use an outbound tunnel.

And this leads to a paradox: if we have to set up a tunnel just to expose the web service the first time, we don't really need the complexity of TURN anymore. We can just use that tunnel for everything.

For me, a tunnel elegantly solves this problem. It requires no inbound traffic on the workstation, working exactly like the outbound architecture you described. The workstation sends its data (outbound) to the tunnel hosted in the DMZ or cloud, and the smartphone connects to that tunnel. Everything goes through it securely. Furthermore, on my side, it allows me to keep the exact same codebase for both enterprise and solo users.

[thiswillbeyourgithub]

Thank you for taking the time to get me up to speed :)

Even if a hospital hosts its own Signaling and TURN servers in their DMZ, they still need a way to serve the Murmure web app from the locked-down workstation to the external phone. To achieve this without opening inbound ports on the workstation, they must use an outbound tunnel.

Why would the mumure web app be served from murmure? It could very well be served by the "signaling" server, no? I believe that's what I do in websend (which is being heavily refactored as we speak). Sure it requires being a bit careful to make sure it remains backwards compatible but it seems doable and would incentive keeping things simple and robust instead of fancy.

[Kieirra]

The PWA is bundled inside the Murmure binary (built by Vite, packaged as a Tauri resource). If it had to be served from the signaling server, admins would need to extract it on every Murmure update, and since the PWA ships the WebRTC code paired with the workstation's Rust server, the two versions have to stay in sync.

Honestly, I'd rather avoid shipping a separate PWA artifact next to the binary on every release for a use case that still feels quite specific. Murmure should stay a simple general-purpose app that works for different users and different scenarios with a single release. If this specific scenario (where even an outbound tunnel isn't an option) becomes recurring, I'm open to revisiting this.

[aparrot89]

It’s simply incredible what you managed to achieve in such a short time. Huge congratulations!
I think, for now, we can focus on these features only.
As you know, I work in a hospital.

• Just a quick feedback.
  In a hospital, typically, a computer can be used by multiple healthcare professionals.
  In some cases, a healthcare professional always uses the same computer.
  In other cases, the healthcare professional changes computers every day.
  In yet other cases, the healthcare professional switches computers throughout the day (typically in the emergency department).
  It is also quite common for computers to be logged in with so-called generic accounts.
  We therefore need to find a way to validate or invalidate the connection duration between the mobile device and the computer.
  Perhaps every time the mobile application is closed, the connection should be renewed?
• As soon as the functionality is roughly ready, I think I can check with my IT department @jothoma1 to see if it’s possible to test the feature in preview—and of course, I’ll provide feedback.

[Kieirra]

Thank you for the message.

Noted !

[Kieirra]

Just a heads up, Smart Speech Mic has been merged on main. You can test using your phone as a wireless microphone by scanning a QR code in Murmure. One constraint though, the phone needs to be on the same local network as the computer, so in a hospital setting it would need to be connected to the hospital's network. Let me know if you need a Windows or Linux build for testing.

[Kieirra]

In a hospital, typically, a computer can be used by multiple healthcare professionals.

It shouldn't be a problem as long as only one user is connected at a time; the last user to connect will disconnect any others.

In some cases, a healthcare professional always uses the same computer.

In this case, the user can even add the website to their smartphone's home screen. The website is a PWA, acting as a native mobile app without requiring any installation. This way, they won’t need to scan the QR code each time.

In other cases, the healthcare professional changes computers every day.

They will need to scan the QR code every time. There’s no alternative.

It is also quite common for computers to be logged in with so-called generic accounts.

No account or login is required. The QR code acts as the login. However, anyone who scans the QR code can control the PC as if they were using a mouse with a microphone.

We therefore need to find a way to validate or invalidate the connection duration between the mobile device and the computer.
Perhaps every time the mobile application is closed, the connection should be renewed?

There is a way, but it’s not automatic. You can reset the QR code, which changes a UUID and invalidates all connections for users who have saved the connection on their phone’s home screen. Maybe I could add it for the CLI, if you want to make scripts that reset them every day, for example.

As soon as the functionality is roughly ready, I think I can check with my IT department @jothoma1 to see if it’s possible to test the feature in preview—and of course, I’ll provide feedback.

Let me know if you need help building the application. Normally, you just need to clone the repo on the main branch, follow the instructions in the contributing.md, and you can build the application. Otherwise, I will provide you directly with the binary of the main branch for testing.