Expert pubsub backend

[Steve-Mcl]

Description

Adds backend support for Expert requests/responses and inflight request/response messages over pubsub.

Key parts:

• New endpoint /api/v1/user/expert-creds (POST) in forge/routes/api/user.js for Expert User to get creds for MQTT client
• New admin endpoints /api/v1/admin/expert-agent-creds (DELETE and POST) in forge/routes/api/admin.js
• New controller function createClientForExpertClient in forge/db/controllers/BrokerClient.js for FE broker client
• New controller functions createClientForExpertAgent, removeClientForExpertAgent for generating BE agent broker client
• Adds to forge/comms/aclManager.js
  • new verify function checkExpertTopic
  • new ACLs for FE user expert-client:* as expertClient
  • new ACLs for BE agent expert-agent:* as expertAgent

Tests added:

test/unit/forge/db/controllers/BrokerClient_spec.js

▼ BrokerClient
  ▼ authenticateCredentials
    ✔ should return true if password matches
    ✔ should return false if password does not match
    ✔ should return false if user not found
    ✔ should return false if password is missing
    ✔ should delete broker client if username starts with frontend: and password is correct
    ✔ should delete broker client if username starts with expert-client: and password is correct
  ▼ createClientForExpertAgent
    ✔ should create broker client and return username and password
    ✔ should update password if client already exists
    ✔ should return null if app.comms is not available
  ▼ removeClientForExpertAgent
    ✔ should remove broker client
  ▼ createClientForExpertClient
    ✔ should create broker client for expert client
    ✔ should destroy existing client before creating new one
    ✔ should return null if app.comms is not available

test/unit/forge/comms/authRoutesV2_spec.js

      Expert Acls
        Expert Client
          ✔ denies subscription to project topics
          ✔ denies publish to project topics
          ✔ denies subscription to device topics
          ✔ denies publish to device topics
          ✔ denies publish/subscribe to platform topics
          ✔ allows subscription to chat response topics (instance)
          ✔ allows subscription to chat response topics (device)
          ✔ denies publish to chat response topics (instance)
          ✔ denies publish to chat response topics (device)
          ✔ denies subscription to another client's chat response topics
          ✔ denies publish to another client's chat response topics
          ✔ denies subscription with mismatching session id
          ✔ denies subscription with invalid entity type
          ✔ denies subscription with invalid entity id
          ✔ allows publish to chat request topics (instance)
          ✔ allows publish to chat request topics (device)
          ✔ denies subscription to chat request topics (instance)
          ✔ denies subscription to chat request topics (device)
          ✔ denies publish with invalid entity id
          ✔ allows subscription to inflight request topics (instance)
          ✔ allows subscription to inflight request topics (device)
          ✔ denies publish to inflight request topics (instance)
          ✔ denies publish to inflight request topics (device)
          ✔ denies subscription to another client's chat response topics
          ✔ denies publish to another client's chat response topics
          ✔ denies subscription to inflight request with mismatching session id
          ✔ allows publish to inflight response topics (instance)
          ✔ allows publish to inflight response topics (device)
          ✔ denies publish to inflight response with mismatching session id
        Expert Agent
          ✔ denies subscription to project topics
          ✔ denies publish to project topics
          ✔ denies subscription to device topics
          ✔ denies publish to device topics
          ✔ denies publish/subscribe to platform topics
          ✔ allows subscription to chat request topics
          ✔ denies subscription to wildcard request topics
          ✔ denies publish to chat request topics (instance)
          ✔ denies publish to chat request topics (device)
          ✔ allows publish to chat response topics (instance)
          ✔ allows publish to chat response topics (device)
          ✔ denies subscription to chat response topics
          ✔ denies publish with invalid entity type
          ✔ denies publish with invalid entity id
          ✔ allows subscription to inflight response topics
          ✔ denies publish to inflight response topics (instance)
          ✔ denies publish to inflight response topics (device)
          ✔ allows publish to inflight request topics (instance)
          ✔ allows publish to inflight request topics (device)
          ✔ denies publish to inflight request with bad entity type
          ✔ denies publish to inflight request with bad entity id

test/unit/forge/routes/api/user_spec.js

  User API
    User Expert Creds
      ✔ user can request expert credentials (111ms)
      ✔ user cannot request expert credentials without sessionId (49ms)
      ✔ user cannot request expert credentials with invalid sessionId (49ms)
      ✔ user cannot request expert credentials if not logged in

Related Issue(s)

Checklist

• I have read the contribution guidelines
• Suitable unit/system level tests have been added and they pass
• Documentation has been updated
  • Upgrade instructions
  • Configuration details
  • Concepts
• Changes flowforge.yml?
  • Issue/PR raised on FlowFuse/helm to update ConfigMap Template
  • Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production
• Link to Changelog Entry PR, or note why one is not needed.

Labels

• Includes a DB migration? -> add the area:migration label

[codecov]

Codecov Report

❌ Patch coverage is 76.59574% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.62%. Comparing base (a3d579c) to head (7cc82bd).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
forge/comms/aclManager.js	71.87%	27 Missing ⚠️
forge/routes/api/admin.js	25.00%	6 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7008   +/-   ##
=======================================
  Coverage   76.62%   76.62%           
=======================================
  Files         405      405           
  Lines       20566    20566           
  Branches     4972     4972           
=======================================
  Hits        15758    15758           
  Misses       4808     4808           

Flag	Coverage Δ
backend	76.62% <76.59%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hardillb]

OK, I've read it all apart from the aclManager.js

What I've read looks OK, but need to spend time to actually step through the acl matching.

[cstns]

are we ok to mark this as ready for review and review it?

[Steve-Mcl]

OK, I've read it all apart from the aclManager.js

What I've read looks OK, but need to spend time to actually step through the acl matching.

@hardillb this has been marked as ready for review.

To (hopefully) help with the understanding the acl matching, below may or may not be useful:

ACL matching cheat sheet for expertClient / expertAgent

The 2 clients

Username	Who
expert-client:<userid>:<sessionid>	A logged-in user's browser session	one per browser tab
expert-agent:<userid>:<apiversion>	The single backend "Expert" worker that services every user	one process

The topic shape

ff/v1/expert/<userid>/<sessionid>/<entity>/<entityId>/support/<channel>/...

^entity [a|p|d|t] == app|project|device|team

Two channels:

• chat - .../support/chat/request (client→agent) and .../support/chat/response (agent→client)
• inflight - .../support/inflight/<toolName>/request (agent→client) and .../inflight/<toolName>/response (client→agent)

The pub on one side mirrors the sub on the other - client pub chat/request pairs with agent sub chat/request, etc.

Two-stage match (in verify, aclManager.js:383-456)

1. Pick the bucket by username prefix + sub/pub → e.g. ACLS.expertClient.sub / ACLS.expertClient.pub.
2. First matching regex wins. If that ACL entry has a verify, run it with the regex captures, the username parts, and the ACL's config flags. No regex match → denied.

What gets wildcarded, and why

+ (single-level MQTT wildcard) is only honored where the ACL config opts in:

	userid	sessionid	entity/entityId
client sub	❌ (must match own user)	❌ (must match own session)	✅ (one sub covers all entities in this session)
client pub	❌	❌	❌ (publishes are always specific)
agent sub	✅	✅	✅ (one agent fans-in everything)
agent pub	❌	❌	❌ (replies are always specific)

Entity wildcarding requires both entityType and entityId to be + together - you can't wildcard one half (aclManager.js:185-189).

What checkExpertTopic enforces (aclManager.js:118-253)

In order, denying on first failure:

1. Client-type sanity: agent ACL must be hit by an agent username, client ACL by a client. Stops a client from ever falling into agent rules.
2. Topic part count : chat=5 segments after the prefix, inflight=6. Cheap shape check.
3. userid present and (for clients) matches the username's userid - a client cannot pub/sub on another user's topics.
4. userid exists in DB: for clients always, for agents only on pub. Agent subs can use +.
5. sessionid present; for clients must match the username's sessionid; for agents + allowed; otherwise must be ≥ 8 chars.
6. Entity resolves : p/d/a/t looked up in DB to get the owning teamId (and applicationHash where relevant). Unknown entityType or missing record → deny.
7. User is a team member of that owning team.
8. For inflight only: RBAC check on the tool name via expertRbacToolCheck - expert:status-message is free, automation:select-nodes / get-nodes / get-flows need project:flows:view, anything else falls back to project:flows:edit.

Wildcarded-entity subs skip steps 6–8 (no entity to look up, no team to check) - that's safe because the agent is trusted infrastructure and clients can't wildcard the user/session, so a client wildcard sub only ever sees its own session's traffic.

The mental model in one line

Username decides who you are; the regex bucket decides what shape of topic you're allowed to touch; checkExpertTopic decides whether the userid/session/entity in the topic is actually yours and you have permission to see it - with carefully scoped wildcards so the agent can fan-in and a client can fan-out across its own session without ever crossing a user or team boundary.

[hardillb]

Approved, but please just check my comments