New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--insecure-registry vs. virtualbox IP #461

Closed
bfallik opened this Issue Aug 8, 2016 · 9 comments

Comments

Projects
None yet
2 participants
@bfallik
Contributor

bfallik commented Aug 8, 2016

Hi,

I just encountered a chicken-and-egg problem with minikube.

I've been starting minikube with the command minikube start --insecure-registry 192.168.99.100:5000 followed by docker run -d -p 5000:5000 --restart=always --name registry registry:2. I want to run the registry on the same VM that runs kubernetes to avoid creating another VM just for the registry. However if another virtualbox VM has taken the 192.168.99.100 minikube will get assigned a different address, breaking the insecure-registry flag.

I'm having trouble figuring out a way to predict the registry address so I can use it as an argument to --insecure-registry. One way might be to use a nameserver within minikube but I'd need to reconfigure the minikube VM to use it for name resolution.

Is my only option to create a self-signed cert and drop the insecure registry option? Am I missing a more obvious approach?

Thanks in advance.

brian

@bfallik

This comment has been minimized.

Contributor

bfallik commented Aug 8, 2016

Initially I thought I could work around this by generating self-signed certs and using a name from xip.io, but this requires restarting the docker daemon so it can be passed in the certs to trust them.

@dlorenc

This comment has been minimized.

Contributor

dlorenc commented Aug 8, 2016

Hey,

How are you running the registry? Could you access it over localhost, instead of the external VM ip?

@bfallik

This comment has been minimized.

Contributor

bfallik commented Aug 8, 2016

@dlorenc evel $(minikube docker-env); docker run -d -p 5000:5000 --restart=always --name registry registry:2

This is beyond the scope of my understanding of Docker networking but I'm not sure how to teach the registry to bind to localhost on the VM and if kubernetes can pull from localhost.

@dlorenc

This comment has been minimized.

Contributor

dlorenc commented Aug 8, 2016

Hmm, I can think of a few ideas. You could run it with --net=host to make it bind to localhost on the VM. Another idea is to run it in Kubernetes following steps like these: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/registry

Then you could create a NodePort load balancer to make it available over localhost.

@bfallik

This comment has been minimized.

Contributor

bfallik commented Aug 8, 2016

I like the kubernetes add-on idea! Are there any minikube-specific instructions needed to enable that add-on or is it just as simple as creating the Pod and services?

@dlorenc

This comment has been minimized.

Contributor

dlorenc commented Aug 8, 2016

We have a bug to add support for this here: #432

If you want to create it manually you'll need to remove the kubernetes.io/cluster-service: "true" line from all of the yamls, or the addon-manager will delete the objects right after you create them.

@bfallik

This comment has been minimized.

Contributor

bfallik commented Aug 9, 2016

@dlorenc thanks, the add-on worked perfectly. Here's the config I used:

apiVersion: v1
kind: ReplicationController
metadata:
  name: kube-registry-v0
  namespace: kube-system
  labels:
    k8s-app: kube-registry
    version: v0
spec:
  replicas: 1
  selector:
    k8s-app: kube-registry
    version: v0
  template:
    metadata:
      labels:
        k8s-app: kube-registry
        version: v0
    spec:
      containers:
      - name: registry
        image: registry:2
        resources:
          limits:
            cpu: 100m
            memory: 100Mi
        env:
        - name: REGISTRY_HTTP_ADDR
          value: :5000
        - name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
          value: /var/lib/registry
        volumeMounts:
        - name: registry-volume
          mountPath: /var/lib/registry
        ports:
        - containerPort: 5000
          name: registry
          protocol: TCP
      volumes:
      - name: registry-volume
        hostPath:
          path: /data/registry
---
apiVersion: v1
kind: Service
metadata:
  name: kube-registry
  namespace: kube-system
  labels:
    k8s-app: kube-registry
    kubernetes.io/name: "KubeRegistry"
spec:
  selector:
    k8s-app: kube-registry
  ports:
  - name: registry
    port: 5000
    protocol: TCP
---
apiVersion: v1
kind: Pod
metadata:
  name: kube-registry-proxy
  namespace: kube-system
spec:
  containers:
  - name: kube-registry-proxy
    image: gcr.io/google_containers/kube-registry-proxy:0.3
    resources:
      limits:
        cpu: 100m
        memory: 50Mi
    env:
    - name: REGISTRY_HOST
      value: kube-registry.kube-system.svc.cluster.local
    - name: REGISTRY_PORT
      value: "5000"
    - name: FORWARD_PORT
      value: "5000"
    ports:
    - name: registry
      containerPort: 5000
      hostPort: 5000
@dlorenc

This comment has been minimized.

Contributor

dlorenc commented Aug 11, 2016

Going to close this out for now. Please reopen if you have any other trouble!

@dlorenc dlorenc closed this Aug 11, 2016

@bfallik

This comment has been minimized.

Contributor

bfallik commented Aug 12, 2016

@dlorenc the in-cluster registry is working great, no need for any more work here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment