Make conductor template proxy aware (inject custom ENV)

[alesz]

ISSUE TYPE

• Feature Idea

container.yml

Proposal:

settings:
  conductor_base: centos:7
  conductor_environment:
  - "http_proxy=http://proxy-prod:3128"
  - "https_proxy=http://proxy-prod:3128"
  dev_overrides:
    conductor_environment:
    - "http_proxy=http://proxy-dev:3128"
    - "https_proxy=http://proxy-dev:3128"

SUMMARY

Conductor build fails in explicit proxy environment.
Update ansible-container/container/docker/templates/conductor-dockerfile.j2 to allow adding custom ENV variables

STEPS TO REPRODUCE

ansible-container build

EXPECTED RESULTS

Fetch packages using http_proxy or https_proxy environment variables (if set)

ACTUAL RESULTS

Build fails

[chouseknecht]

Adding a conductor section to settings in container.yml that will include base, environment and volumes attributes. For example:

version: '2'
settings:
    conductor:
        base: 'centos:7'
        volumes:
        - tmp-space:/tmp
        environment:
        - http_proxy=http://proxy-dev:3128
        - https_proxy=http://proxy-dev:3128

We'll still honor conductor_base in settings, so as not to break things.

[MohamedTalhaoui]

Hi,

what is the status here ?
This is documented on the ansible-container official documentation but cannot make it work.

I used:

settings:
  conductor:
    base: centos:7
    environment:
        http_proxy: http://<proxy_host>:<proxy_port>
        https_proxy: http:/<proxy_host>:<proxy_port>

but the environment variables are not set in the base image making the yum update fails:

Step 1/12 : FROM centos:7
 ---> 328edcd84f1b
Step 2/12 : ENV ANSIBLE_CONTAINER 1
 ---> Using cache
 ---> 88c7a62b0093
Step 3/12 : RUN yum update -y &&     yum install -y epel-release &&     yum install -y make gcc git python-devel curl rsync libffi-devel openssl-devel &&     yum remove -y python-chardet &&     yum clean all
 ---> Running in bb1e428d4b8b
Loaded plugins: fastestmirror, ovl


 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=<repoid> ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot find a valid baseurl for repo: base/7/x86_64

Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=container error was
14: curl#7 - "Failed to connect to 2001:4178:5:200::10: Cannot assign requested address"

Thanks.

[chouseknecht]

@MohamedTalhaoui

Looks like I only half-way implemented this. The environment variables get added to the conductor during run, but they're not added during build.

Looking now to see if we can do that...

[chouseknecht]

@MohamedTalhaoui

Feature added. Would appreciate any feedback after you try it out. Thanks!

[MohamedTalhaoui]

Working !!!!
Many thanks @chouseknecht , you saved my day :-)

[alesz]

Thanks!

[shalomb]

@chouseknecht - I'm not sure if this is related. I wanted to try this change out and so built ansible-container from develop but hit a couple of strange errors. This appears to be related to running from source as I did not see this on version 0.9.1

$ git remote -v
origin  https://github.com/ansible/ansible-container.git (fetch)
origin  https://github.com/ansible/ansible-container.git (push)

$ git branch;                                                                                         
* develop
  master

$ git rev-parse --short HEAD
b64cdd4

$ pip list | grep -i -e docker -e container                                                                                                                                  ansible-container (0.9.2rc0, /home/sbhooshi/ansible-container)
docker (2.4.2)
docker-compose (1.8.0)
docker-py (1.9.0)
docker-pycreds (0.2.1)
dockerpty (0.4.1)

$ ansible-container build 
Building Docker Engine context...
Starting Docker build of Ansible Container Conductor image (please be patient)...
Parsing conductor CLI args.
Traceback (most recent call last):
  File "/usr/local/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container==0.9.1', 'console_scripts', 'conductor')()
  File "/usr/local/lib/python2.7/dist-packages/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/container/cli.py", line 360, in conductor_commandline
    **params)
  File "/usr/local/lib/python2.7/dist-packages/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/container/core.py", line 610, in conductorcmd_build
    engine = load_engine(['BUILD'], engine_name, project_name, services, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/container/utils/loader.py", line 16, in load_engine
    package='container')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/dist-packages/container/docker/engine.py", line 44, in <module>
    container.__version__
ImportError: You must install Ansible Container with Docker(tm) support. Try:
pip install ansible-container==0.9.1[docker]
Conductor terminated. Cleaning up.      command_rc=1 conductor_id=ad46a574e6e39477a6db11d2969c3a5668bb67194722832f31e0ea64b8149204 save_container=False
ERROR   Conductor exited with status 1

So I went through the motions of uninstalling all ansible/docker pip packages and then reinstalled ansible-container from source .. this time I see conductor creating the docker instances/images, etc yet it still bombs out after a few minutes with the same exception.

$ sudo -HE pip install --upgrade --force-reinstall .[docker]

$ cd /path/to/my/project

$ ansible-container build 
Building Docker Engine context...
Starting Docker build of Ansible Container Conductor image (please be patient)...
Parsing conductor CLI args.
Traceback (most recent call last):
  File "/usr/local/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container==0.9.1', 'console_scripts', 'conductor')()
  File "/usr/local/lib/python2.7/dist-packages/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/container/cli.py", line 360, in conductor_commandline
    **params)
  File "/usr/local/lib/python2.7/dist-packages/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/container/core.py", line 610, in conductorcmd_build
    engine = load_engine(['BUILD'], engine_name, project_name, services, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/container/utils/loader.py", line 16, in load_engine
    package='container')
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/local/lib/python2.7/dist-packages/container/docker/engine.py", line 44, in <module>
    container.__version__
ImportError: You must install Ansible Container with Docker(tm) support. Try:
pip install ansible-container==0.9.1[docker]
Conductor terminated. Cleaning up.      command_rc=1 conductor_id=ad46a574e6e39477a6db11d2969c3a5668bb67194722832f31e0ea64b8149204 save_container=False
ERROR   Conductor exited with status 1

:|

What I notice is that the version in the error message

pip install ansible-container==0.9.1[docker]

does not match the version pip or even ansible-container reports

$ pip list | grep -i ansible
ansible (2.3.2.0)
ansible-container (0.9.2rc0, /home/sbhooshi/ansible-container)

$ ansible-container version
Ansible Container, version 0.9.2rc0

Hmm ...

[shalomb]

Disregard the comment I had here previously - wrong ticket.