Re-running the gates against the full tree across 21 fleet repos (excluding openregister) produced 276 failing gates. Triaging the authorisation gates line by line — reading every call path — turned up five distinct false-positive modes. Together they account for ~130 of the 276, and one of them ships remediation advice that would introduce a vulnerability if followed.
Measurement: gate script ConductionNL/.github@ad00986 (main, includes #147/#148/#149), ajv installed under hydra-gates/scripts/lib, each repo at its current origin/development, no --scope-to-diff. Each run was executed in its own mount namespace with a private tmpfs /tmp, because the runner writes its detail logs to hardcoded /tmp/hydra-gate-<name>.log paths and reads the counts back out of them — two concurrent runs silently corrupt each other's numbers (see item 6).
1. gate-38 skip-link — fails in 21 of 21 repos
Uniformity across independent inputs points at the instrument, and it is.
The gate greps a root component for a literal <NcContent> / <NcAppContent>. The fleet has migrated to <CnAppRoot> (nc-vue), and CnAppRoot renders <NcContent> internally — src/components/CnAppRoot/CnAppRoot.vue:60. Every manifest-driven app root therefore fails a check it satisfies.
Second cause in the same gate: it scopes templates/settings/*.php. Those are Nextcloud settings-framework templates; the framework owns the page shell, so such a file can never contain a skip-link and can never pass.
Fix: accept <CnAppRoot> (and any component that transitively renders NcContent) as a shell; drop templates/settings/*.php from scope.
2. gate-9 semantic-auth, rule public-page-annotation-with-auth-body — 36 of 39 findings, and the advice is dangerous
The gate tells you: "remove #[PublicPage] or remove body auth check". Every one of the 36 is a webhook or public-portal endpoint that correctly bypasses NC session auth and authenticates the caller itself. Following either half of that advice breaks the endpoint or removes its only authentication.
Verified call paths:
openconnector PaymentsController::webhook — HMAC signature verify with constant-time compare + timestamp tolerance, 401 before any state change. Its docblock already says "the signature check IS the auth body for this route".
portaliq ContributionController::inbox (and 9 siblings) — resolves a portal subject(), 401 when absent; per-row subject + tenant + trust-boundary filtering.
- Same shape:
openconnector DSO / IwmoIjw / Notificaties / NotifyNl / OpenFormulieren / Peppol / StufZkn inbound, procest CaseFederationController ×3, hermiq EgressAuthorize + McpRun, launchpad PublicShareController ×2, doriath ApplicationController::create + ApplicationTokenController::exchange, scholiq PaymentTransactionController::callback, shillinq PortalPaymentInitiationController::initiate, softwarecatalog ×2, petstore ×1.
Fix: the rule must distinguish "public but self-authenticating" (signature verify, bearer/portal token, capability token → correct) from "#[PublicPage] while relying on an IUserSession session check" (→ the real mismatch). Only 3 of the 39 findings are the no-admin-required-annotation-with-admin-body shape the gate was actually built for, and those were real — fixed in openbuild#127.
3. gate-7 no-admin-idor — 14 of 15 findings false, even after #149
#149 taught the gate to follow delegation; two blind spots remain.
(a) A guard that returns bool/null instead of throwing or returning 401/403. hermiq ×7: AgentVersionController::index/diff → loadAccessibleAgent($id, $userId) → canUserAccessAgent() (private/owner/invited); AgentsController::index applies canUserAccessAgent() per row inside the result loop. _GUARD_HELPER_NAME_RE requires the name to end in Admin|Access|Permission|Permitted|Owner|Allowed|Authorised — canUserAccessAgent ends in Agent, loadAccessibleAgent starts with load; and _HELPER_GUARD_BODY_RE wants a throw or a 401/403/404, which a bool-returning predicate never has.
(b) Endpoints with no object identifier at all. nldesign CatalogController::tokenSets (public catalogue, no args), nldesign ContrastController::evaluate (pure colour math on caller-supplied input), procest AssistantController::availability (boolean feature probe), hermiq AgentsController::create/stats/tools. There is no object to be insecure-direct-referenced; the gate should not ask for a per-object guard where no object is addressed.
doriath CxpRelayController::put/get are a capability-URL relay (the 24-char ISecureRandom pairing id is the bearer credential, payloads are sealed) and SettingsController::getPolicy is a documented read-open-to-all-authenticated. Both false.
The one real finding was docudesk AnonymizationController::updateRelation — genuinely unguarded through four service hops to an unscoped primary-key write. Fixed in docudesk#382. So gate-7's true-positive rate here is 1 in 15 — high enough to be worth keeping, low enough that the noise will train people to skip it.
4. gate-14 route-reachability — 22 of 30 findings are AppHost delegation
openbuild (7), shillinq (7), doriath (5), planix (3) are reported as having unrouted dashboard#page, dashboard#catchAll, settings#index/create/load, preferences#getPreference/setPreference. All are supplied by OCA\OpenRegister\AppHost\Routes::standard(). The gate parses only the literal array in appinfo/routes.php, so it cannot see routes contributed by the shared builder.
Tell: docudesk passed this gate only because it happens to carry a literal $canonicalRoutes fallback array for the openregister-absent case. Passing should not depend on that.
Fix: when appinfo/routes.php calls Routes::standard(), fold in Routes::canonicalRoutes() before deciding.
The other 8 findings are real and worth keeping — see item 5.
5. gate-6 orphan-auth — 2 of 5 false
launchpad LiveTileController::validateSource — a routed controller method (liveTile#validateSource, POST, appinfo/routes.php:557). Its caller is the NC router, not PHP code.
pipelinq LogBerichtenboxAdapter::checkMailbox — implements BerichtenboxAdapterInterface::checkMailbox; callers go through the interface.
Fix: exempt routed controller methods and declared-interface implementations.
The other three are real decidesk#60-shaped orphans — implemented, unit-tested, never wired into any production path — and are being filed against their repos: procest BewijsstukService::assertMutable, pipelinq PortalTenantService::isSelfSignupAllowed, pipelinq ZgwCoexistenceValidator::validateWritePath.
6. The runner is not safe to run twice at once
run-hydra-gates.sh writes ~50 detail logs to hardcoded /tmp/hydra-gate-<name>.log and derives each verdict from wc -l on that shared file. Two concurrent runs — trivially easy across repos, or a stray backgrounded run nobody noticed — produce numbers that are plausible but wrong, in both directions.
This bit me during this exercise: a first measurement pass reported softwarecatalog with 7 gate-7 IDOR findings. Running check_no_admin_idor.py directly against the same tree returned zero — the 7 belonged to hermiq, running in parallel. procest moved 22 → 21, softwarecatalog 18 → 16, zaakafhandelapp 8 → 9 once the runs were isolated. Only gate-6 was immune, because it is the one gate already using mktemp.
Fix: derive a per-run log directory once (mktemp -d, or honour TMPDIR) and hang every gate's log off it — the pattern gate-6 already uses. Cheap, and it removes a whole class of "measured, but wrong" from every future fleet sweep.
Suggested priority
- item 6 — until it lands, no fleet-wide number from this runner is trustworthy.
- item 2 — the remediation text is actively harmful.
- items 1, 4 — pure noise, ~43 findings, and they are what makes people stop reading gate output.
- items 3, 5 — narrow the guard vocabulary and the scope.
Nothing was suppressed, baselined or excluded anywhere in this exercise.
Re-running the gates against the full tree across 21 fleet repos (excluding openregister) produced 276 failing gates. Triaging the authorisation gates line by line — reading every call path — turned up five distinct false-positive modes. Together they account for ~130 of the 276, and one of them ships remediation advice that would introduce a vulnerability if followed.
Measurement: gate script
ConductionNL/.github@ad00986(main, includes #147/#148/#149),ajvinstalled underhydra-gates/scripts/lib, each repo at its currentorigin/development, no--scope-to-diff. Each run was executed in its own mount namespace with a privatetmpfs /tmp, because the runner writes its detail logs to hardcoded/tmp/hydra-gate-<name>.logpaths and reads the counts back out of them — two concurrent runs silently corrupt each other's numbers (see item 6).1. gate-38
skip-link— fails in 21 of 21 reposUniformity across independent inputs points at the instrument, and it is.
The gate greps a root component for a literal
<NcContent>/<NcAppContent>. The fleet has migrated to<CnAppRoot>(nc-vue), andCnAppRootrenders<NcContent>internally —src/components/CnAppRoot/CnAppRoot.vue:60. Every manifest-driven app root therefore fails a check it satisfies.Second cause in the same gate: it scopes
templates/settings/*.php. Those are Nextcloud settings-framework templates; the framework owns the page shell, so such a file can never contain a skip-link and can never pass.Fix: accept
<CnAppRoot>(and any component that transitively rendersNcContent) as a shell; droptemplates/settings/*.phpfrom scope.2. gate-9
semantic-auth, rulepublic-page-annotation-with-auth-body— 36 of 39 findings, and the advice is dangerousThe gate tells you: "remove
#[PublicPage]or remove body auth check". Every one of the 36 is a webhook or public-portal endpoint that correctly bypasses NC session auth and authenticates the caller itself. Following either half of that advice breaks the endpoint or removes its only authentication.Verified call paths:
openconnector PaymentsController::webhook— HMAC signature verify with constant-time compare + timestamp tolerance, 401 before any state change. Its docblock already says "the signature check IS the auth body for this route".portaliq ContributionController::inbox(and 9 siblings) — resolves a portalsubject(), 401 when absent; per-row subject + tenant + trust-boundary filtering.openconnectorDSO / IwmoIjw / Notificaties / NotifyNl / OpenFormulieren / Peppol / StufZkn inbound,procest CaseFederationController×3,hermiq EgressAuthorize+McpRun,launchpad PublicShareController×2,doriath ApplicationController::create+ApplicationTokenController::exchange,scholiq PaymentTransactionController::callback,shillinq PortalPaymentInitiationController::initiate,softwarecatalog×2,petstore×1.Fix: the rule must distinguish "public but self-authenticating" (signature verify, bearer/portal token, capability token → correct) from "
#[PublicPage]while relying on anIUserSessionsession check" (→ the real mismatch). Only 3 of the 39 findings are theno-admin-required-annotation-with-admin-bodyshape the gate was actually built for, and those were real — fixed in openbuild#127.3. gate-7
no-admin-idor— 14 of 15 findings false, even after #149#149 taught the gate to follow delegation; two blind spots remain.
(a) A guard that returns
bool/nullinstead of throwing or returning 401/403.hermiq×7:AgentVersionController::index/diff→loadAccessibleAgent($id, $userId)→canUserAccessAgent()(private/owner/invited);AgentsController::indexappliescanUserAccessAgent()per row inside the result loop._GUARD_HELPER_NAME_RErequires the name to end inAdmin|Access|Permission|Permitted|Owner|Allowed|Authorised—canUserAccessAgentends inAgent,loadAccessibleAgentstarts withload; and_HELPER_GUARD_BODY_REwants a throw or a 401/403/404, which abool-returning predicate never has.(b) Endpoints with no object identifier at all.
nldesign CatalogController::tokenSets(public catalogue, no args),nldesign ContrastController::evaluate(pure colour math on caller-supplied input),procest AssistantController::availability(boolean feature probe),hermiq AgentsController::create/stats/tools. There is no object to be insecure-direct-referenced; the gate should not ask for a per-object guard where no object is addressed.doriath CxpRelayController::put/getare a capability-URL relay (the 24-charISecureRandompairing id is the bearer credential, payloads are sealed) andSettingsController::getPolicyis a documented read-open-to-all-authenticated. Both false.The one real finding was
docudesk AnonymizationController::updateRelation— genuinely unguarded through four service hops to an unscoped primary-key write. Fixed in docudesk#382. So gate-7's true-positive rate here is 1 in 15 — high enough to be worth keeping, low enough that the noise will train people to skip it.4. gate-14
route-reachability— 22 of 30 findings are AppHost delegationopenbuild(7),shillinq(7),doriath(5),planix(3) are reported as having unrouteddashboard#page,dashboard#catchAll,settings#index/create/load,preferences#getPreference/setPreference. All are supplied byOCA\OpenRegister\AppHost\Routes::standard(). The gate parses only the literal array inappinfo/routes.php, so it cannot see routes contributed by the shared builder.Tell:
docudeskpassed this gate only because it happens to carry a literal$canonicalRoutesfallback array for the openregister-absent case. Passing should not depend on that.Fix: when
appinfo/routes.phpcallsRoutes::standard(), fold inRoutes::canonicalRoutes()before deciding.The other 8 findings are real and worth keeping — see item 5.
5. gate-6
orphan-auth— 2 of 5 falselaunchpad LiveTileController::validateSource— a routed controller method (liveTile#validateSource, POST,appinfo/routes.php:557). Its caller is the NC router, not PHP code.pipelinq LogBerichtenboxAdapter::checkMailbox— implementsBerichtenboxAdapterInterface::checkMailbox; callers go through the interface.Fix: exempt routed controller methods and declared-interface implementations.
The other three are real decidesk#60-shaped orphans — implemented, unit-tested, never wired into any production path — and are being filed against their repos:
procest BewijsstukService::assertMutable,pipelinq PortalTenantService::isSelfSignupAllowed,pipelinq ZgwCoexistenceValidator::validateWritePath.6. The runner is not safe to run twice at once
run-hydra-gates.shwrites ~50 detail logs to hardcoded/tmp/hydra-gate-<name>.logand derives each verdict fromwc -lon that shared file. Two concurrent runs — trivially easy across repos, or a stray backgrounded run nobody noticed — produce numbers that are plausible but wrong, in both directions.This bit me during this exercise: a first measurement pass reported
softwarecatalogwith 7 gate-7 IDOR findings. Runningcheck_no_admin_idor.pydirectly against the same tree returned zero — the 7 belonged tohermiq, running in parallel.procestmoved 22 → 21,softwarecatalog18 → 16,zaakafhandelapp8 → 9 once the runs were isolated. Onlygate-6was immune, because it is the one gate already usingmktemp.Fix: derive a per-run log directory once (
mktemp -d, or honourTMPDIR) and hang every gate's log off it — the pattern gate-6 already uses. Cheap, and it removes a whole class of "measured, but wrong" from every future fleet sweep.Suggested priority
Nothing was suppressed, baselined or excluded anywhere in this exercise.