Fix MCP feature enumeration with with cap, deadline and per instance fault containement

[Steve-Mcl]

Description

TL;DR

The POST /api/v1/expert/mcp/features endpoint checked each candidate instance/device for liveness sequentially, so one slow or unreachable instance held up all the others (worst case N x timeout). It also fetched features via a bare Promise.all with no per-item error handling - and since a device MQTT call rejects on timeout, a single unresponsive device could reject the whole batch and 500 the endpoint.

This PR replaces that with a single bounded, pipelined per-instance scan: each instance flows liveness -> token -> features independently under a concurrency limit, wrapped so one failure can never sink the batch. The scan is bounded by a total server cap and an overall deadline, and the response now carries a summary describing coverage. All limits are configurable via app.config.expert.insights.*.

Details

• Pipelined operations (instead of 2 barriers!). wall clock is now the slowest single instance chain, not the sum of scans.
• Fault containment (bug fix): each per-instance scan is wrapped in try..catch, so a rejecting device call is skipped rather than failing the whole request.
• Total cap MAX_SERVERS (default 20); the remainder is reported as unenumerated.
  • This is a change in behaviour agreed in AI meeting today (2026-07-16) - deemed necessary to keep things snappy AND avoid timeouts / db pool exhaustion. Agreement was A) we have very few insights users today (no SH have insights) and B) it is still marked as a BETA feature and as such, our BETA testing has revealed this need.
• Bounded concurrency ENUM_CONCURRENCY (default 5, aligned with the DB pool).
• Overall deadline ENUM_DEADLINE_MS (default 10s) returns partial results; in-flight MQTT calls run out their own timeouts (no abort for MQTT).
• Per device feature timeout reduced 10s -> 5s (DEVICE_FEATURES_TIMEOUT); liveness stays at 3s.
• All limits configurable via app.config.expert.insights.{ MAX_SERVERS, ENUM_CONCURRENCY, ENUM_DEADLINE_MS, DEVICE_LIVENESS_TIMEOUT, DEVICE_FEATURES_TIMEOUT }, each falling back to the defaults above.
  • NOTE: These are not advertised in docs for now. Only used as emergency internal tweaks (and for test overrides)
• New summary in the response (and schema): { instanceCount, mcpServerCount, enumeratedCount, unenumeratedCount, limit, limited, deadlineReached }.
  • Following discussion in AI meeting, it was stated a summary would be included so that we can debug and at some point inform users of limits.

Builds on the recently merged Device.status gate, which already skips offline devices before they reach the scan.

Tests

test/unit/forge/ee/routes/expert/index_spec.js:

• The existing endpoint tests passed against the refactor without modification - the behaviour is unchanged for the common cases. The only edit was extra assertions in the "return early with no registrations" test, to cover the always-present summary.
• Added a buildDeviceRegistrations(count) helper producing N distinct online device registrations for fan-out/cap coverage.
• Cap: 25 registrations against the cap of 20 - asserts only 20 are enumerated, the summary reflects the cap (enumeratedCount 20, unenumeratedCount 5, limited true), and instances beyond the cap are never contacted.
• Deadline + fault containment: four devices (rejects at liveness, rejects at features, exceeds a shortened deadline, responds in time). Asserts the request still returns 200, only the healthy device is enumerated, deadlineReached: true, and failed instances are skipped rather than reported incompatible.

Related Issue(s)

closes #7832

Checklist

• I have read the contribution guidelines
• Suitable unit/system level tests have been added and they pass
• Documentation has been updated
  • Upgrade instructions
  • Configuration details
  • Concepts
• Changes flowforge.yml?
  • Issue/PR raised on FlowFuse/helm to update ConfigMap Template
  • Issue/PR raised on FlowFuse/CloudProject to update values for Staging/Production
• Link to Changelog Entry PR, or note why one is not needed.

Labels

• Includes a DB migration? -> add the area:migration label

[codecov]

Codecov Report

❌ Patch coverage is 94.33962% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.17%. Comparing base (214e2c7) to head (9d781a7).

Files with missing lines	Patch %	Lines
forge/ee/routes/expert/index.js	94.33%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7841      +/-   ##
==========================================
+ Coverage   75.12%   75.17%   +0.04%     
==========================================
  Files         430      430              
  Lines       22863    22886      +23     
  Branches     6063     6069       +6     
==========================================
+ Hits        17176    17204      +28     
+ Misses       5687     5682       -5     

Flag	Coverage Δ
backend	75.17% <94.33%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cstns]

Changes look good to me. One thing I'm unclear about is that since you changed the signature of the /features endpoint, the type drift check didn't fail..

Not going to block it for that thogh