Recurring infrastructure failure in CI
Pattern: `proxy.golang.org INTERNAL_ERROR` during `go mod download` / `go test`
Job type: `run-tests`
Evidence
Impact
Both occurrences caused the entire `run-tests` job to abort before any test ran, producing a false `failure` conclusion on the workflow run. This:
- Inflates the apparent failure rate of unit/integration tests
- Masks real flakiness signal (flaky tests appear to have fewer total runs)
- Causes spurious PR-blocking red checks
Log excerpts
Run 32418052116:
```
cloud.google.com/go/auth@v0.19.0: read "https://proxy.golang.org/cloud.google.com/go/auth/@v/v0.19.0.zip":
stream error: stream ID 241; INTERNAL_ERROR; received from peer
```
Run 32419279874:
```
k8s.io/metrics@v0.36.1: read "https://proxy.golang.org/k8s.io/metrics/@v/v0.36.1.zip":
stream error: stream ID 137; INTERNAL_ERROR; received from peer
```
Triple-check (infra vs test failure)
- Pattern match ✓ — both logs contain `INTERNAL_ERROR` from `proxy.golang.org` during module download
- Timeline ✓ — error appears before the first `=== RUN` line; zero test results in log
- Cross-run consistency ✓ — two separate runs, same failure pattern, different modules (cloud.google.com/go/auth and k8s.io/metrics), indicating the proxy itself is intermittently failing
Suggested fixes
- Add `GONOSUMCHECK=*` + `GOFLAGS=-mod=mod` and retry on proxy error
- Or add `GOPROXY=off` after a successful first download (use module cache) and fail-fast only on genuine source changes
- Consider switching to `GOPROXY=direct` as fallback in CI
Recurring infrastructure failure in CI
Pattern: `proxy.golang.org INTERNAL_ERROR` during `go mod download` / `go test`
Job type: `run-tests`
Evidence
Impact
Both occurrences caused the entire `run-tests` job to abort before any test ran, producing a false `failure` conclusion on the workflow run. This:
Log excerpts
Run 32418052116:
```
cloud.google.com/go/auth@v0.19.0: read "https://proxy.golang.org/cloud.google.com/go/auth/@v/v0.19.0.zip":
stream error: stream ID 241; INTERNAL_ERROR; received from peer
```
Run 32419279874:
```
k8s.io/metrics@v0.36.1: read "https://proxy.golang.org/k8s.io/metrics/@v/v0.36.1.zip":
stream error: stream ID 137; INTERNAL_ERROR; received from peer
```
Triple-check (infra vs test failure)
Suggested fixes