Weird python behaviors in created bundle

[bdlamprecht]

So, it's me again @udondan...

I came across something else that had me and several other people in the open-source community a little stumped and I'm wondering if you have any ideas on what the core of the problem could be.

I don't recall if I ever explained what the end goal of what I'm trying to accomplish was, so I'll back up and attempt to give an big-picture overview of what I have in mind.

I'm trying to use ansible-silo as a base layer for managing networking devices from Cisco and Juniper using embedded playbooks with a python library called napalm.

Using your instructions in issue #52, I was able to create a docker image with all of the dependencies for napalm>=2.3.0 to be installed correctly.

I can successfully connect to Cisco devices and manage them how I would like to, but when I try and connect to Juniper devices, I get the following error:

fatal: [rtp-vba-a]: FAILED! => {"changed": false, "msg": "cannot connect to device: Cannot import \"napalm.junos\". Is the library installed?"}

While attempting to debug the issue, I accessed the "bundle" using --shell and tried to manually import that library:

[ansible-silo 2.1.1|~/playbooks]$ python
Python 2.7.13 (default, Apr 20 2017, 12:13:37)
[GCC 6.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import napalm.junos
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/silo/userspace/lib/python2.7/site-packages/napalm/junos/__init__.py", line 18, in <module>
    from napalm.junos.junos import JunOSDriver  # noqa
  File "/silo/userspace/lib/python2.7/site-packages/napalm/junos/junos.py", line 32, in <module>
    from jnpr.junos import Device
ImportError: No module named jnpr.junos

However, I can see the library IS in fact there and the directory structure and paths are accurate:

[ansible-silo 2.1.1|~/playbooks]$ echo $PYTHONPATH
/silo/userspace/lib/python2.7/site-packages:/silo/userspace/ansible/lib:

[ansible-silo 2.1.0|~/playbooks]$ ls -l /silo/userspace/lib/python2.7/site-packages | grep -E 'napalm|jnpr'
drwxr-xr-x    3 root     root          4096 Feb 20 17:55 jnpr
drwxr-xr-x    9 root     root          4096 Feb 20 17:55 napalm
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 napalm-2.3.0-py2.7.egg-info

[ansible-silo 2.1.1|~/playbooks]$ ls -l /silo/userspace/lib/python2.7/site-packages/jnpr/junos
total 168
-rw-r--r--    1 root     root          1451 Sep 30 20:32 __init__.py
drwxr-xr-x    3 root     root          4096 Feb 20 17:55 cfg
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 cfgro
-rw-r--r--    1 root     root         11186 Sep 30 20:32 console.py
-rw-r--r--    1 root     root          6876 Sep 30 20:32 decorators.py
-rw-r--r--    1 root     root         51930 Sep 30 20:32 device.py
-rw-r--r--    1 root     root          9145 Sep 30 20:32 exception.py
-rw-r--r--    1 root     root         12747 Sep 30 20:32 factcache.py
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 factory
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 facts
-rw-r--r--    1 root     root          6615 Sep 30 20:32 jxml.py
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 ofacts
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 op
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 resources
-rw-r--r--    1 root     root         15938 Sep 30 20:32 rpcmeta.py
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 transport
drwxr-xr-x    2 root     root          4096 Feb 20 17:55 utils
-rw-r--r--    1 root     root           260 Sep 30 20:32 version.py

[ansible-silo 2.1.1|~/playbooks]$ cat /silo/userspace/lib/python2.7/site-packages/jnpr/junos/version.py
VERSION = "2.1.7"
DATE = "2017-Sep-30"

# Augment with the internal version if present
try:
    from jnpr.junos.internal_version import INTERNAL_VERSION
    VERSION += '+internal.' + str(INTERNAL_VERSION)
except ImportError:
    # No internal version
    pass

What is really strange is that using the Docker image python:2.7.14-alpine3.6 it seems to work just fine:

$ docker run -it python:2.7.14-alpine3.6 /bin/sh
Unable to find image 'python:2.7.14-alpine3.6' locally
2.7.14-alpine3.6: Pulling from library/python
605ce1bd3f31: Pull complete
55018be3009c: Pull complete
918ecf7a26fa: Pull complete
9b782e57edd5: Pull complete
Digest: sha256:cb412d77b3a4eec29243e60585dab19ab40f59b6af17a0420d8771eeb399cd6d
Status: Downloaded newer image for python:2.7.14-alpine3.6

/ # apk --no-cache add make gcc libffi-dev musl-dev linux-headers openssl-dev libxml2-dev libxslt-dev
---SNIP---
/ # pip install napalm
---SNIP---
/ # python
Python 2.7.14 (default, Jan 10 2018, 05:38:41)
[GCC 6.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import napalm.junos

I don't encounter that same errors at all.

Now, as I worked though composing the contents of this issue, I've realized that I can try and take grpn/ansible-silo and base it off of python:2.7.14-alpine3.6 instead of alpine:3.6 and see if that solves the problem I'm encountering.

I'll let you know what I find.

[bdlamprecht]

Yeah, I was able to get my problem resolved by altering the base.Dockerfile so it read like this:

FROM python:2.7.14-alpine3.6

instead of like this:

FROM alpine:3.6

I'm sure that I didn't do it very cleanly, but I'm wondering if you would be opposed to using the python-based release of alpine:3.6 for next release of ansible-silo (version 3)? It seems to be a lot less "funky" in the way python works.

I'd be happy to help you with it if you feel that it would be worthwhile.

[bdlamprecht]

I've been able to solve this by rebuilding grpn/ansible-silo-base:3.0.0 and grpn/ansible-silo:3.0.0.
I'm still working through some issues, but when I'm ready, I will submit a PR.