Builder SA not using builder-dockercfg secret for pulling images from internal registry

[midekra]

For some reason when starting builds in Openshift the build pod is not using the correct secret for pulling images from an internal registry. In the registry logs I see that the anonymous user is trying to pull the image instead of the serviceaccount:builder.

Version

[provide output of the openshift version or oc version command]
oc v3.9.0+191fece
kubernetes v1.9.1+a0ce1bc657
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://:8443
openshift v3.9.0+ba7faec-1
kubernetes v1.9.1+a0ce1bc657

Steps To Reproduce

1. Create a simple buildconfig:
   oc new-build --binary=true --name=frontend

2. Create a buildir with a Dockerfile
   cat build/Dockerfile
FROM docker-registry.default.svc:5000/openshift/nginx:latest

3. Run Build
   oc start-build frontend --from-dir=build --follow

Current Result

Pulling image docker-registry.default.svc:5000/openshift/nginx:latest ... Step 1/6 : FROM docker-registry.default.svc:5000/openshift/nginx:latest Trying to pull repository docker-registry.default.svc:5000/openshift/nginx ... Pulling repository docker-registry.default.svc:5000/openshift/nginx error: build error: Error: image openshift/nginx:latest not found error: the build tempus/frontend-7 status is "Failed"

Expected Result

Successfull build.

According to the docs the serviceaccount builder (which runs the build pod) should automatically use the correct secret (builder-dockercfg) when pulling and pushing images. When I manually add a pullsecret:
e.g.:
oc set build-secret --pull bc/frontend builder-dockercfg-<hash>

It works correctly.

Additional Information

Registry logs:

time="2018-10-19T09:39:02.886733502Z" level=error msg="error authorizing context: authorization header required" go.version=go1.9.2 http.request.host="docker-registry.default.svc:5000" http.request.id=c1a72df6-e48e-4af3-81fa-2e80a21f44ce http.request.method=GET http.request.remoteaddr="10.130.2.1:46232" http.request.uri=/v2/ http.request.useragent="docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=15f909ca-5180-4c98-94f0-b7daf61eea59

  | 10.130.2.1 - - [19/Oct/2018:09:39:02 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)"
  | time="2018-10-19T09:39:02.892452453Z" level=debug msg="anonymous token request" go.version=go1.9.2 http.request.host="docker-registry.default.svc:5000" http.request.id=628bff70-b4a4-4842-812e-606679375eae http.request.method=GET http.request.remoteaddr="10.130.2.1:46234" http.request.uri="/openshift/token?scope=repository%3Aopenshift%2Fnginx%3Apull" http.request.useragent="docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=15f909ca-5180-4c98-94f0-b7daf61eea59
  | time="2018-10-19T09:39:02.892483482Z" level=info msg="response completed" go.version=go1.9.2 http.request.host="docker-registry.default.svc:5000" http.request.id=a8568219-a5e6-49ae-bc08-e3ad8288c633 http.request.method=GET http.request.remoteaddr="10.130.2.1:46234" http.request.uri="/openshift/token?scope=repository%3Aopenshift%2Fnginx%3Apull" http.request.useragent="docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" http.response.contenttype=application/json http.response.duration="73.073µs" http.response.status=200 http.response.written=49 instance.id=15f909ca-5180-4c98-94f0-b7daf61eea59
  | 10.130.2.1 - - [19/Oct/2018:09:39:02 +0000] "GET /openshift/token?scope=repository%3Aopenshift%2Fnginx%3Apull HTTP/1.1" 200 49 "" "docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)"
  | time="2018-10-19T09:39:02.898077972Z" level=debug msg="authorizing request" go.version=go1.9.2 http.request.host="docker-registry.default.svc:5000" http.request.id=ca69688d-64d8-4a1d-b3f8-fdf6832d4257 http.request.method=GET http.request.remoteaddr="10.130.2.1:46236" http.request.uri=/v2/openshift/nginx/manifests/latest http.request.useragent="docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=15f909ca-5180-4c98-94f0-b7daf61eea59 vars.name=openshift/nginx vars.reference=latest
  | time="2018-10-19T09:39:02.898284326Z" level=debug msg="Origin auth: checking for access to repository:openshift/nginx:pull" go.version=go1.9.2 http.request.host="docker-registry.default.svc:5000" http.request.id=ca69688d-64d8-4a1d-b3f8-fdf6832d4257 http.request.method=GET http.request.remoteaddr="10.130.2.1:46236" http.request.uri=/v2/openshift/nginx/manifests/latest http.request.useragent="docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=15f909ca-5180-4c98-94f0-b7daf61eea59 openshift.auth.user=anonymous vars.name=openshift/nginx vars.reference=latest
  | time="2018-10-19T09:39:02.899334775Z" level=error msg="OpenShift access denied: User "system:anonymous" cannot get imagestreams/layers.image.openshift.io in project "openshift"" go.version=go1.9.2 http.request.host="docker-registry.default.svc:5000" http.request.id=ca69688d-64d8-4a1d-b3f8-fdf6832d4257 http.request.method=GET http.request.remoteaddr="10.130.2.1:46236" http.request.uri=/v2/openshift/nginx/manifests/latest http.request.useragent="docker/1.13.1 go/go1.9.4 kernel/4.17.10-1.el7.elrepo.x86_64 os/linux arch/amd64 UpstreamClient(go-dockerclient)" instance.id=15f909ca-5180-4c98-94f0-b7daf61eea59 openshift.auth.user=anonymous vars.name=openshift/nginx vars.reference=latest
  | time="2018-10-19T09:39:02.899379268Z" level=error msg="error authorizing context: access denied"

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.