docker registry 404 #876

gimler · 2016-08-28T18:37:47Z

I try to get the registry working but every time i get 404 errors.

docker login registry.gitlab.example.com
Username: abc 
Password: 
Login Succeeded
docker push registry.gitlab.example.com/gimler/docker-php

result

The push refers to a repository [registry.gitlab.example.com/gimler/docker-php]
c56b7dabbc7a: Preparing 
Error: Status 404 trying to push repository gimler/docker-php: "<!DOCTYPE html>\n<html>\n<head>\n  <meta content=\"width=device-width, initial-scale=1, maximum-scale=1\" name=\"viewport\">\n  <title>The page you're looking for could not be found (404)</title>\n  <style>\n    body {\n      color: #666;\n      text-align: center;\n      font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n      margin: auto;\n      font-size: 14px;\n    }\n\n    h1 {\n      font-size: 56px;\n      line-height: 100px;\n      font-weight: normal;\n      color: #456;\n    }\n\n    h2 {\n      font-size: 24px;\n      color: #666;\n      line-height: 1.5em;\n    }\n\n    h3 {\n      color: #456;\n      font-size: 20px;\n      font-weight: normal;\n      line-height: 28px;\n    }\n\n    hr {\n      max-width: 800px;\n      margin: 18px auto;\n      border: 0;\n      border-top: 1px solid #EEE;\n      border-bottom: 1px solid white;\n    }\n\n    img {\n      max-width: 40vw;\n    }\n\n    .container {\n      margin: auto 20px;\n    }\n  </style>\n</head>\n\n<body>\n  <h1>\n    <img src=\"data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjEwIiBoZWlnaHQ9IjIxMCIgdmlld0JveD0iMCAwIDIxMCAyMTAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+CiAgPHBhdGggZD0iTTEwNS4wNjE0IDIwMy42NTVsMzguNjQtMTE4LjkyMWgtNzcuMjhsMzguNjQgMTE4LjkyMXoiIGZpbGw9IiNlMjQzMjkiLz4KICA8cGF0aCBkPSJNMTA1LjA2MTQgMjAzLjY1NDhsLTM4LjY0LTExOC45MjFoLTU0LjE1M2w5Mi43OTMgMTE4LjkyMXoiIGZpbGw9IiNmYzZkMjYiLz4KICA8cGF0aCBkPSJNMTIuMjY4NSA4NC43MzQxbC0xMS43NDIgMzYuMTM5Yy0xLjA3MSAzLjI5Ni4xMDIgNi45MDcgMi45MDYgOC45NDRsMTAxLjYyOSA3My44MzgtOTIuNzkzLTExOC45MjF6IiBmaWxsPSIjZmNhMzI2Ii8+CiAgPHBhdGggZD0iTTEyLjI2ODUgODQuNzM0Mmg1NC4xNTNsLTIzLjI3My03MS42MjVjLTEuMTk3LTMuNjg2LTYuNDExLTMuNjg1LTcuNjA4IDBsLTIzLjI3MiA3MS42MjV6IiBmaWxsPSIjZTI0MzI5Ii8+CiAgPHBhdGggZD0iTTEwNS4wNjE0IDIwMy42NTQ4bDM4LjY0LTExOC45MjFoNTQuMTUzbC05Mi43OTMgMTE4LjkyMXoiIGZpbGw9IiNmYzZkMjYiLz4KICA8cGF0aCBkPSJNMTk3Ljg1NDQgODQuNzM0MWwxMS43NDIgMzYuMTM5YzEuMDcxIDMuMjk2LS4xMDIgNi45MDctMi45MDYgOC45NDRsLTEwMS42MjkgNzMuODM4IDkyLjc5My0xMTguOTIxeiIgZmlsbD0iI2ZjYTMyNiIvPgogIDxwYXRoIGQ9Ik0xOTcuODU0NCA4NC43MzQyaC01NC4xNTNsMjMuMjczLTcxLjYyNWMxLjE5Ny0zLjY4NiA2LjQxMS0zLjY4NSA3LjYwOCAwbDIzLjI3MiA3MS42MjV6IiBmaWxsPSIjZTI0MzI5Ii8+Cjwvc3ZnPgo=\" alt=\"GitLab Logo\" /><br />\n    404\n  </h1>\n  <div class=\"container\">\n    <h3>The page you're looking for could not be found.</h3>\n    <hr />\n    <p>Make sure the address is correct and that the page hasn't moved.</p>\n    <p>Please contact your GitLab administrator if you think this is a mistake.</p>\n  </div>\n</body>\n</html>\n"

docker-compose.yml

postgresql:
  restart: always
  image: sameersbn/postgresql:9.5-1
  environment:
    - DB_USER=root
    - DB_PASS=***
    - DB_NAME=gitlabhq
    - DB_EXTENSION=pg_trgm
  volumes:
    - "$PWD/data/postgresql:/var/lib/postgresql"

gitlab:
  restart: always
  image: sameersbn/gitlab:8.11.2-1
  links:
    - redis:redisio
    - postgresql:postgresql
    - registry:registry
  expose:
    - "80"
  ports:
    - "10022:22"
  environment:
    - VIRTUAL_HOST=gitlab.example.com,registry.gitlab.example.com
    - LETSENCRYPT_HOST=gitlab.example.com,registry.gitlab.example.com
    - LETSENCRYPT_EMAIL=mail@example.com

    - DEBUG=true
    - TZ=Europe/Berlin
    - GITLAB_TIMEZONE=Berlin

    - GITLAB_SECRETS_DB_KEY_BASE=***
    - GITLAB_SECRETS_SECRET_KEY_BASE=***
    - GITLAB_SECRETS_OTP_KEY_BASE=***

    - GITLAB_HOST=gitlab.example.com
    - GITLAB_PORT=443
    - GITLAB_HTTPS=true
    - GITLAB_SSH_PORT=10022
    - GITLAB_RELATIVE_URL_ROOT=

    - GITLAB_REGISTRY_ENABLED=true
    - GITLAB_REGISTRY_HOST=registry.gitlab.example.com
    - GITLAB_REGISTRY_PORT=443
    - GITLAB_REGISTRY_API_URL=http://registry:5000
    - GITLAB_REGISTRY_KEY_PATH=/certs/registry-auth.key

    - GITLAB_BACKUP_SCHEDULE=daily
    - GITLAB_BACKUP_TIME=01:00
  volumes:
    - "$PWD/data/gitlab:/home/git/data"
    - "$PWD/certs:/certs"

redis:
  restart: always
  image: sameersbn/redis:latest
  volumes:
    - "$PWD/data/redis:/var/lib/redis"

gitlab-runner:
  restart: always
  image: gitlab/gitlab-runner:alpine-v1.5.2
  links:
    - gitlab:gitlab
  volumes:
    - "$PWD/config/gitlab-runner:/etc/gitlab-runner"
    - "/var/run/docker.sock:/var/run/docker.sock"

registry:
    restart: always
    image: registry:2.5.0
    volumes:
      - "$PWD/data/registry:/registry"
      - "$PWD/certs:/certs"
    environment:
      - REGISTRY_LOG_LEVEL=debug
      - REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/registry
      - REGISTRY_AUTH_TOKEN_REALM=https://gitlab.example.com/jwt/auth
      - REGISTRY_AUTH_TOKEN_SERVICE=container_registry
      - REGISTRY_AUTH_TOKEN_ISSUER=gitlab-issuer
      - REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE=/certs/registry-auth.crt
      - REGISTRY_STORAGE_DELETE_ENABLED=true

The text was updated successfully, but these errors were encountered:

guybrush · 2016-08-29T00:57:08Z

yep, same here. note that i run this behind traefik (external load-balancer) and thus i did not set SSL_REGISTRY_{KEY,CERT}_PATH - see #714 (comment)

    # https://github.com/sameersbn/docker-gitlab/pull/714#issuecomment-226882702
    # "My current implementation will now work like the normal ssl proxy for gitlab if you didn't provide a cert and key for SSL_REGISTRY_CERT & SSL_REGISTRY_KEY"
    # - SSL_REGISTRY_KEY_PATH=/certs/registry.key
    # - SSL_REGISTRY_CERT_PATH=/certs/registry.crt

gimler · 2016-08-29T04:22:46Z

I use https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion as letsencrypt proxy

solidnerd · 2016-08-29T06:14:33Z

Hey,

if you use an external registry api change your url to GITLAB_REGISTRY_API_URL=https://registry.gitlab.example.com.

gimler · 2016-08-29T07:14:39Z

ok. i get it to work but only on port 5500 set - GITLAB_REGISTRY_PORT=5500 and use SSL_REGISTRY_KEY_PATH and SSL_REGISTRY_CERT_PATH.

Now it works in gitlab-ci.yml but not from external.

docker push registry.gitlab.example.com:5500/gimler/docker-php

return

error parsing HTTP 403 response body: no error details found in HTTP response body: "{\"message\":\"forbidden\",\"status\":\"error\",\"http_status\":403}"

solidnerd · 2016-08-29T07:22:07Z

I use my self an external registry and I don't have these problems. Run the container registry with a minimal configuration env's. Have a look a this example:

GITLAB_REGISTRY_ENABLED=true
GITLAB_REGISTRY_HOST=registry.gitlab.example.com
GITLAB_REGISTRY_API_URL=https://registry.gitlab.example.com
GITLAB_REGISTRY_KEY_PATH=/certs/registry_auth.key
GITLAB_REGISTRY_ISSUER=gitlab-issuer

REGISTRY_LOG_LEVEL=info
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/registry
REGISTRY_AUTH_TOKEN_REALM=https://gitlab.example.com/jwt/auth
REGISTRY_AUTH_TOKEN_SERVICE=container_registry
REGISTRY_AUTH_TOKEN_ISSUER=gitlab-issuer
REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE=/certs/registry_auth.crt
REGISTRY_STORAGE_DELETE_ENABLED=true

solidnerd · 2016-08-29T07:28:47Z

Perhaps your proxy configuration is different.

## Set a variable to help us decide if we need to add the
## 'Docker-Distribution-Api-Version' header.
## The registry always sets this header.
## In the case of nginx performing auth, the header will be unset
## since nginx is auth-ing before proxying.
map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
  'registry/2.0' '';
  default registry/2.0;
}

server {
  listen *:80;
  server_name  registry.gitlab.example.com;
  server_tokens off; ## Don't show the nginx version number, a security best practice
  return 301 https://$http_host:$request_uri;
  access_log  /var/log/nginx/registry.gitlab.example.com/access.log;
  error_log   /var/log/nginx/registry.gitlab.example.com/error.log;
}

server {
  listen *:443 ssl http2;
  server_name registry.gitlab.example.com;

  #Logging
  access_log  /var/log/nginx/registry.gitlab.example.com/access.log;
  error_log   /var/log/nginx/registry.gitlab.example.com/error.log;

  #SSL
  ssl_certificate /etc/letsencrypt/live/registry.gitlab.example.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/registry.gitlab.example.com/privkey.pem;

  # Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
  ssl_protocols TLSv1.1 TLSv1.2;
  ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
  ssl_prefer_server_ciphers on;
  ssl_session_cache shared:SSL:10m;

  # disable any limits to avoid HTTP 413 for large image uploads
  client_max_body_size 0;

  # required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
  chunked_transfer_encoding on;

  location / {
    # Do not allow connections from docker 1.5 and earlier
    # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
    if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*\$" ) {
      return 404;
    }

    ## If $docker_distribution_api_version is empty, the header will not be added.
    ## See the map directive above where this variable is defined.
    add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;

    proxy_pass                          http://gitlab_registry_1:5000;
    proxy_set_header  Host              $http_host;   # required for docker client's sake
    proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
    proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header  X-Forwarded-Proto $scheme;
    proxy_read_timeout                  900;
  }
}

gimler · 2016-08-30T05:38:33Z

I use it as a external registry now all works ;) thanks @solidnerd

guybrush · 2016-08-30T07:40:04Z

yep, using the registry externally works - thanks!

ghost · 2017-02-23T00:55:40Z

@gimler hi! I'm confusing, I have tried to find the file to place external registry. In documentation they mention gitlab.rb but I don´t find. my gitlab is sameersbn 8.16.5 and the registry is 2.4.1.
The error that throws me is :"Error: Status 404 trying to push repository group/name: "<!DOCTYPE html...". The procedure I perform is first log me in, then build the image and the last step is to perform a push where I throw that mistake. please I need help thanks.

julienTocci · 2018-03-14T13:51:20Z

@FranVeraC Did you find a solution? I have the exact same error

gimler · 2018-03-14T19:43:08Z

basically my file was that i have tried to use http://registry:5000 as GITLAB_REGISTRY_API_URL change it to the external url https://registry.gitlab.example.com without the port number fix this form me.

see comment #876 (comment)

nikolicjakov · 2018-09-26T22:50:54Z

I use https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion as letsencrypt proxy

I have found solution to this issue. I am also using "proxy-companion" for letsencrypt certificates and jwilder/nginx-proxy as docker reverse proxy server.

Thing that did the trick is registry nginx port. I have changed port to 80 since all incoming traffic is going to VIRTUAL_PORT=80. As we are hitting our reverse proxy on port 443 traffic is forwarded to VIRTUAL_HOST port and that is 80 of gitlab container. Then we need our registry nginx to listen on port 80 and it should work fine...

So my current config looks like this..

registry_external_url 'https://docker.my-domain.com'

gitlab_rails['registry_enabled'] = true

registry['enable'] = true

registry_nginx['enable'] = true
registry_nginx['listen_https'] = false
registry_nginx['listen_port'] = 80
registry_nginx['proxy_set_headers'] = {
  "Host" => "$http_host",
  "X-Real-IP" => "$remote_addr",
  "X-Forwarded-For" => "$proxy_add_x_forwarded_for",
  "X-Forwarded-Proto" => "https",
  "X-Forwarded-Ssl" => "on"
}

Next problem that you will face is limitation of jwilder/nginx-proxy is the client_max_body_size limit.
You have to increase the body size limit in order to upload docker images to your registry over reverse proxy.
You can do that by creating custom jwilder/nginx-proxy image with client_max_body_size 0; in nginx.conf.
Or you can mount additional proxy config file using -v /path/to/my_proxy.conf:/etc/nginx/conf.d/my_proxy.conf:ro. You can find more information on this here.

After that you should be able to use gitlab docker registry without problems.

gimler closed this as completed Aug 30, 2016

weaktyper mentioned this issue Dec 17, 2016

Gitlab registry - error parsing HTTP 404 response body: invalid character #1025

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker registry 404 #876

docker registry 404 #876

gimler commented Aug 28, 2016

guybrush commented Aug 29, 2016

gimler commented Aug 29, 2016

solidnerd commented Aug 29, 2016

gimler commented Aug 29, 2016

solidnerd commented Aug 29, 2016 •

edited

Loading

solidnerd commented Aug 29, 2016

gimler commented Aug 30, 2016

guybrush commented Aug 30, 2016 •

edited

Loading

ghost commented Feb 23, 2017

julienTocci commented Mar 14, 2018

gimler commented Mar 14, 2018

nikolicjakov commented Sep 26, 2018 •

edited

Loading

docker registry 404 #876

docker registry 404 #876

Comments

gimler commented Aug 28, 2016

guybrush commented Aug 29, 2016

gimler commented Aug 29, 2016

solidnerd commented Aug 29, 2016

gimler commented Aug 29, 2016

solidnerd commented Aug 29, 2016 • edited Loading

solidnerd commented Aug 29, 2016

gimler commented Aug 30, 2016

guybrush commented Aug 30, 2016 • edited Loading

ghost commented Feb 23, 2017

julienTocci commented Mar 14, 2018

gimler commented Mar 14, 2018

nikolicjakov commented Sep 26, 2018 • edited Loading

solidnerd commented Aug 29, 2016 •

edited

Loading

guybrush commented Aug 30, 2016 •

edited

Loading

nikolicjakov commented Sep 26, 2018 •

edited

Loading