file exists error for /etc/kubernetes when running w/ local docker-based kube setup

[jasonbrooks]

Running the "Local Docker-based (recommended starting point)" method from the upstream project, using this Vagrantfile:

$ vagrant up
$ vagrant ssh
$ mkdir wp && cd wp
$ sudo atomic run projectatomic/wordpress-centos7-atomicapp

2015-08-06 17:45:13,588 - kubernetes - INFO - Using namespace default
Traceback (most recent call last):
  File "/usr/bin/atomicapp", line 9, in <module>
    load_entry_point('atomicapp==0.1', 'console_scripts', 'atomicapp')()
  File "/usr/lib/python2.7/site-packages/atomicapp/cli/main.py", line 89, in main
    cli.run()
  File "/usr/lib/python2.7/site-packages/atomicapp/cli/main.py", line 70, in run
    args.func(args)
  File "/usr/lib/python2.7/site-packages/atomicapp/cli/main.py", line 21, in cli_run
    ae.run()
  File "/usr/lib/python2.7/site-packages/atomicapp/run.py", line 187, in run
    self._dispatchGraph()
  File "/usr/lib/python2.7/site-packages/atomicapp/run.py", line 99, in _dispatchGraph
    self._processComponent(component, graph_item)
  File "/usr/lib/python2.7/site-packages/atomicapp/run.py", line 168, in _processComponent
    provider.init()
  File "/usr/lib/python2.7/site-packages/atomicapp/providers/kubernetes.py", line 29, in init
    os.symlink("/host/etc/kubernetes", "/etc/kubernetes")
OSError: [Errno 17] File exists

[aweiteka]

We check for /etc/kubernetes, but if we're in a container and that file is not found we create /host/etc/kubernetes, which is the default kube config dir bindmounted from the host.

@jasonbrooks it doesn't look like you're running from the vagrant box. Do you have an /etc/kubernetes dir on your host or in the vagrant box?

[jasonbrooks]

I'm running from a default centos 7 vagrant box, the Vagrantfile is linked above. There's no /etc/kubernetes in there, because I'm running everything in containers, w/ the project's local Docker-based installation method.

[rtnpro]

@aweiteka @jasonbrooks would exposing /etc/kubernetes from the kubernetes container to the host solve this problem? Or, do you see room for some more elegant solution?

[dustymabe]

@rtnpro I don't think it is possible to bind mount a file that is inside the container to the host (or at least I don't think it is normally done with the docker cli).

@jasonbrooks this is an issue we are going to have to look at more deeply I am going to push this to the next milestone if that is ok.

[jasonbrooks]

I just tried this again, and got a different, but similar error. Creating /etc/kubernetes in my host (the VM configured w/ the vagrantfile above) allowed the process to continue. It's not a priority that this work, more a question of should this / why doesn't this work.

Does atomicapp have to run on a machine w/ an /etc/kubernetes? I've been running it on my kube master always, but have assumed that run atomic foo should work on a regular client w/ kubectl and a valid kubeconfig file as well. Is that right?

[dustymabe]

@jasonbrooks right. I'll try to figure out why it's needed (or even if it's really needed) and get back to you.

[dustymabe]

So I now understand this problem more. Basically our code that creates the symlink will try to create the symlink twice. In the code before we get into the place where it tries to create the symlink it basically looks something like this:

if not os.path.exists(kube_conf_path):
    os.symlink(os.path.join(Utils.getRoot(), kube_conf_path.lstrip("/")), kube_conf_path)

If there are multiple kube artifacts then this code gets run twice and os.path.exists will still return false if the path exists but is a broken symbolic link. This means that os.symlink() will get run twice to create the symlink twice and will fail the 2nd time. We should change the test to os.path.lexists which will return true if something exists even if it is a broken symbolic link.

[dustymabe]

Note this will be handled by #368

[dustymabe]

actually I take it back.. the right approach is just to not create the symlink in the first place if the target doesn't exist.

[dustymabe]

fixed in #368