--insecure-registry vs. virtualbox IP

[bfallik]

Hi,

I just encountered a chicken-and-egg problem with minikube.

I've been starting minikube with the command minikube start --insecure-registry 192.168.99.100:5000 followed by docker run -d -p 5000:5000 --restart=always --name registry registry:2. I want to run the registry on the same VM that runs kubernetes to avoid creating another VM just for the registry. However if another virtualbox VM has taken the 192.168.99.100 minikube will get assigned a different address, breaking the insecure-registry flag.

I'm having trouble figuring out a way to predict the registry address so I can use it as an argument to --insecure-registry. One way might be to use a nameserver within minikube but I'd need to reconfigure the minikube VM to use it for name resolution.

Is my only option to create a self-signed cert and drop the insecure registry option? Am I missing a more obvious approach?

Thanks in advance.

brian

[bfallik]

Initially I thought I could work around this by generating self-signed certs and using a name from xip.io, but this requires restarting the docker daemon so it can be passed in the certs to trust them.

[dlorenc]

Hey,

How are you running the registry? Could you access it over localhost, instead of the external VM ip?

[bfallik]

@dlorenc evel $(minikube docker-env); docker run -d -p 5000:5000 --restart=always --name registry registry:2

This is beyond the scope of my understanding of Docker networking but I'm not sure how to teach the registry to bind to localhost on the VM and if kubernetes can pull from localhost.

[dlorenc]

Hmm, I can think of a few ideas. You could run it with --net=host to make it bind to localhost on the VM. Another idea is to run it in Kubernetes following steps like these: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/registry

Then you could create a NodePort load balancer to make it available over localhost.

[bfallik]

I like the kubernetes add-on idea! Are there any minikube-specific instructions needed to enable that add-on or is it just as simple as creating the Pod and services?

[dlorenc]

We have a bug to add support for this here: #432

If you want to create it manually you'll need to remove the kubernetes.io/cluster-service: "true" line from all of the yamls, or the addon-manager will delete the objects right after you create them.

[bfallik]

@dlorenc thanks, the add-on worked perfectly. Here's the config I used:

apiVersion: v1
kind: ReplicationController
metadata:
  name: kube-registry-v0
  namespace: kube-system
  labels:
    k8s-app: kube-registry
    version: v0
spec:
  replicas: 1
  selector:
    k8s-app: kube-registry
    version: v0
  template:
    metadata:
      labels:
        k8s-app: kube-registry
        version: v0
    spec:
      containers:
      - name: registry
        image: registry:2
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
        env:
        - name: REGISTRY_HTTP_ADDR
          value: :5000
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
        volumeMounts:
        - name: registry-volume
          mountPath: /var/lib/registry
        ports:
        - containerPort: 5000
          name: registry
          protocol: TCP
      volumes:
      - name: registry-volume
        hostPath:
          path: /data/registry
---
apiVersion: v1
kind: Service
metadata:
  name: kube-registry
  namespace: kube-system
  labels:
    k8s-app: kube-registry
    kubernetes.io/name: "KubeRegistry"
spec:
  selector:
    k8s-app: kube-registry
  ports:
  - name: registry
    port: 5000
    protocol: TCP
---
apiVersion: v1
kind: Pod
metadata:
  name: kube-registry-proxy
  namespace: kube-system
spec:
  containers:
  - name: kube-registry-proxy
    image: gcr.io/google_containers/kube-registry-proxy:0.3
    resources:
      limits:
        cpu: 100m
        memory: 50Mi
    env:
    - name: REGISTRY_HOST
      value: kube-registry.kube-system.svc.cluster.local
    - name: REGISTRY_PORT
      value: "5000"
    - name: FORWARD_PORT
      value: "5000"
    ports:
    - name: registry
      containerPort: 5000
      hostPort: 5000

[dlorenc]

Going to close this out for now. Please reopen if you have any other trouble!

[bfallik]

@dlorenc the in-cluster registry is working great, no need for any more work here