Blocked fetches not recorded in FetchStore (store-and-forward has no allowlist check; info/refs phase missing recordFetch call)

[coopernetes]

Summary

There are two distinct gaps that cause blocked fetch events to be missing from the Repos page metrics:

Gap 1 — Store-and-forward mode (/push/...) has no URL allowlist enforcement for fetches

registerGitServlet adds no UrlRuleAggregateFilter (or equivalent) to the push-path filter chain.
The only allowlist hook is RepositoryUrlRuleHook, which is wired inside StoreAndForwardReceivePackFactory and runs exclusively in the ReceivePack (push) lifecycle.

Fetches via /push/<provider>/... go through:

1. StoreAndForwardRepositoryResolver — clones/fetches from upstream unconditionally
2. StoreAndForwardUploadPackFactory — creates a bare UploadPack with no rule checks

Neither component consults the UrlRuleRegistry or calls FetchStore.record(), so:

• A repo that is not in the allow list can still be fetched via the store-and-forward path
• No blocked-fetch metric is ever written

Gap 2 — Transparent proxy mode (/proxy/...): /info/refs phase does not call recordFetch

UrlRuleAggregateFilter.applyInfoRefsRules() (the INFO branch in doHttpFilter) calls setResult() when a rule blocks the request, but never calls recordFetch().

git clone / git fetch always hit /info/refs?service=git-upload-pack first. If the repo is not in the allow list, the request is blocked at the discovery handshake and the upload-pack request never arrives — so the recordFetch(request, false) in the upload-pack branch is never reached. The net result is zero rows written to FetchStore for blocked fetches in the common clone/fetch flow.

Affected code

Location	Issue
GitProxyServletRegistrar.registerGitServlet	No UrlRuleAggregateFilter registered on the /push/ path for fetch operations; no FetchStore passed
UrlRuleAggregateFilter.applyInfoRefsRules	recordFetch(request, false) missing for Denied and NotAllowed cases
RepositoryUrlRuleHook	Only handles HttpOperation.PUSH; no fetch-path analogue exists

Expected behaviour

• Fetches via /push/... to a repo not in the allow list should be blocked with an appropriate git error (same as push behaviour).
• Every blocked fetch — regardless of whether it was rejected at the /info/refs phase or the upload-pack phase, and regardless of proxy mode — should be recorded in FetchStore with Result.BLOCKED so the Repos page blocked-fetch count is accurate.

Suggested fix direction

Gap 1: Add a fetch-gate filter on the /push/ path (or extend UrlRuleAggregateFilter to cover it) that checks URL rules for FETCH/INFO operations and calls FetchStore.record() on block. Pass FetchStore through registerGitServlet to support this.

Gap 2: Add recordFetch(request, false) into both the Denied and NotAllowed cases inside applyInfoRefsRules in UrlRuleAggregateFilter.

[coopernetes]

Additional finding: opaque HTTP 403 on /info/refs is a poor UX for the git client

Tested locally against a real blocked repo (torvalds/linux, not in allow rules):

$ git ls-remote http://me:anything@localhost:8080/proxy/github.com/torvalds/linux.git
fatal: unable to access 'http://localhost:8080/proxy/github.com/torvalds/linux.git/': The requested URL returned error: 403

The git client receives a bare Jetty 403 HTML error page. From the user's perspective this is indistinguishable from a misconfigured remote URL or an upstream HTTP error — there is no indication the proxy is the source of the block, and no actionable message.

Root cause

applyInfoRefsRules() in UrlRuleAggregateFilter calls response.sendError(provider.getBlockedInfoRefsStatus()) directly, which produces a plain HTTP error response. The git smart HTTP protocol expects either:

• A valid application/x-git-upload-pack-advertisement response with a pkt-line ERR <message> for the /info/refs discovery phase, or
• A 200 OK with the ERR packet body (as sendGitError already does for upload-pack)

Expected behaviour

The client should print something like:

fatal: remote error: Fetch Blocked - Repository Not Allowed
       Fetches from this repository are not permitted.
       Contact an administrator to add this repository to the allow rules.

Fix direction

Replace the response.sendError(...) call in applyInfoRefsRules with a proper pkt-line ERR response. The existing sendGitError helper already knows how to do this for the INFO/upload-pack case (it writes ERR <message> with the correct content type). The blockedInfoRefsStatus config field becomes irrelevant once the response is git-protocol framed and can be deprecated.