Install Local Registry

[j0hnL]

Is your feature request related to a problem? Please describe.
Omnia needs an option to install a local registry

Describe the solution you'd like
a playbook for installing a local registry such as Harbor - https://github.com/goharbor/harbor

Describe alternatives you've considered
local registry - with simple docker container

Additional context

[lwilson]

@j0hnL, what are the dependencies for harbor? Are we already installing all of the necessary services?

[lwilson]

This issue is now dependent on #50

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[lwilson]

cert-manager now works on our Kubernetes deployments (it is being used by Kubeflow as well as the Intel FPGA device plugin).

Should we explore deploying Harbor again @j0hnL? I think it makes the most sense to offer a datacenter-wide registry for containers, so installing it on the appliance probably makes the most sense. Thoughts @sujit-jadhav?

[sujit-jadhav]

@lwilson we can host the datacenter-wide registry for containers on the appliance. We can also host the complete repository for all the rpm packages, github, epel etc. on appliance so that it eliminates the requirement of having internet access for all nodes in the cluster.

[lwilson]

@sujit-jadhav I think that is going to be the right approach for dealing with both the DockerHub bandwidth limits and any air-gapped deployments.

Harbor has a HA deployment with Helm: https://goharbor.io/docs/2.0.0/install-config/harbor-ha-helm/

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[lwilson]

@sujit-jadhav is the plan to deploy local registry in v1.1.0?

[sujit-jadhav]

@lwilson, It is not part of v1.1.0 but it will be part of v1.2.0

[j0hnL]

@sujit-jadhav @lwilson as a temporary (insecure) fix we can setup an internal pull through cache using the Docker registry container: https://docs.docker.com/registry/deploying/

This method is insecure and does not use SSL.

Deploy the registry on the control plane, in this example the IP of control plane is 10.1.1.1

This example registry.yml will use memory for storage. Users will need to fill in their docker credentials to avoid pull limits.

version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
auth:
  htpasswd:
    realm: basic-realm
    path: /etc/registry
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
proxy:
  remoteurl: https://registry-1.docker.io
  username: <<dockerhub username>>
  password: <<dockerhub password>>

Deploying the registry on port 5000

 docker run -d -p 5000:5000 --restart=always --name registry -v `pwd`/registry.yaml registry:2

Update /etc/docker/daemon.json on all compute resources to point to the insecure local pull through cache registry mirror:

{ 
  "insecure-registries":["10.1.1.1:5000"],
  "registry-mirrors": ["http://10.1.1.1:5000"]
}

Restart docker daemon.

systemctl restart docker

[lwilson]

Thanks @j0hnL. What would we need to do to add a self-signed certificate? I think if we could avoid having insecure-registries that would be ideal.

Also, If we were to set the mirror IP/port as variables, it would make this a very flexible solution. Is there any reason to consider using Harbor vs this approach?

[j0hnL]

Just a temporary fix for the release branches if folks hit the pull rate limit of Docker Hub. Harbor should be deployed with the devel release.

What would we need to do to add a self-signed certificate?

Looks like Let's Encrypt might be a good option.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[j0hnL]

in progress

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[j0hnL]

perhaps we can get @sujit-jadhav to give this one some priority

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.