Skip to content
This repository has been archived by the owner on Mar 23, 2019. It is now read-only.

Doesn't install roles automatically #658

Closed
drym3r opened this issue Jul 21, 2017 · 11 comments
Closed

Doesn't install roles automatically #658

drym3r opened this issue Jul 21, 2017 · 11 comments
Assignees
@chouseknecht
Labels
bug

Comments

@drym3r
Copy link

drym3r commented Jul 21, 2017

ISSUE TYPE
  • Bug Report
container.yml
version: '2'
settings:
  conductor_base: 'blablabla.bla.ecr.eu-west-1.amazonaws.com/test'
  project_name  : 'whatever'
services:
  vars:
    deploy_dir: /opt/app/
  base:
    roles       :
      - ansible-role-jenkins

    user        : 'java'
    from        : 'blablabla.bla.ecr.eu-west-1.amazonaws.com/test:latest'
    command     :
      - sleep 1
OS / ENVIRONMENT
Linux, notebook, 4.9.0-3-amd64, #1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26), x86_64
2.7.13 (default, Jan 19 2017, 14:48:08) 
[GCC 6.3.0 20170118] /home/somename/.virtualenvs/container/bin/python
{
  "ContainersPaused": 0, 
  "Labels": null, 
  "CgroupDriver": "cgroupfs", 
  "ContainersRunning": 0, 
  "ContainerdCommit": {
    "Expected": "cfb82a876ecc11b5ca0977d1733adbe58599088a", 
    "ID": "cfb82a876ecc11b5ca0977d1733adbe58599088a"
  }, 
  "InitBinary": "docker-init", 
  "NGoroutines": 26, 
  "Swarm": {
    "ControlAvailable": false, 
    "NodeID": "", 
    "Error": "", 
    "RemoteManagers": null, 
    "LocalNodeState": "inactive", 
    "NodeAddr": ""
  }, 
  "LoggingDriver": "json-file", 
  "OSType": "linux", 
  "HttpProxy": "", 
  "Runtimes": {
    "runc": {
      "path": "docker-runc"
    }
  }, 
  "DriverStatus": [
    [
      "Backing Filesystem", 
      "extfs"
    ], 
    [
      "Supports d_type", 
      "true"
    ], 
    [
      "Native Overlay Diff", 
      "true"
    ]
  ], 
  "OperatingSystem": "Debian GNU/Linux 9 (stretch)", 
  "Containers": 95, 
  "HttpsProxy": "", 
  "BridgeNfIp6tables": true, 
  "MemTotal": 16760045568, 
  "SecurityOptions": [
    "name=seccomp,profile=default"
  ], 
  "Driver": "overlay2", 
  "IndexServerAddress": "https://index.docker.io/v1/", 
  "ClusterStore": "", 
  "InitCommit": {
    "Expected": "949e6fa", 
    "ID": "949e6fa"
  }, 
  "Isolation": "", 
  "SystemStatus": null, 
  "OomKillDisable": true, 
  "ClusterAdvertise": "", 
  "SystemTime": "2017-07-21T13:42:34.149115394+02:00", 
  "Name": "notebook", 
  "CPUSet": true, 
  "RegistryConfig": {
    "AllowNondistributableArtifactsCIDRs": [], 
    "Mirrors": [], 
    "IndexConfigs": {
      "docker.io": {
        "Official": true, 
        "Name": "docker.io", 
        "Secure": true, 
        "Mirrors": []
      }
    }, 
    "AllowNondistributableArtifactsHostnames": [], 
    "InsecureRegistryCIDRs": [
      "127.0.0.0/8"
    ]
  }, 
  "DefaultRuntime": "runc", 
  "ContainersStopped": 95, 
  "NCPU": 8, 
  "NFd": 17, 
  "Architecture": "x86_64", 
  "KernelMemory": true, 
  "CpuCfsQuota": true, 
  "Debug": false, 
  "ID": "YEYW:FXKT:2HOB:S242:BQCA:E3OB:ADSH:CU6E:JUVN:S66Y:MONE:FNUB", 
  "IPv4Forwarding": true, 
  "KernelVersion": "4.9.0-3-amd64", 
  "BridgeNfIptables": true, 
  "NoProxy": "", 
  "LiveRestoreEnabled": false, 
  "ServerVersion": "17.06.0-ce", 
  "CpuCfsPeriod": true, 
  "ExperimentalBuild": false, 
  "MemoryLimit": true, 
  "SwapLimit": false, 
  "Plugins": {
    "Volume": [
      "local"
    ], 
    "Network": [
      "bridge", 
      "host", 
      "macvlan", 
      "null", 
      "overlay"
    ], 
    "Authorization": null, 
    "Log": [
      "awslogs", 
      "fluentd", 
      "gcplogs", 
      "gelf", 
      "journald", 
      "json-file", 
      "logentries", 
      "splunk", 
      "syslog"
    ]
  }, 
  "Images": 24, 
  "DockerRootDir": "/var/lib/docker", 
  "NEventsListener": 0, 
  "CPUShares": true, 
  "RuncCommit": {
    "Expected": "2d41c047c83e09a6d61d464906feb2a2f3c52aa4", 
    "ID": "2d41c047c83e09a6d61d464906feb2a2f3c52aa4"
  }
}
{
  "KernelVersion": "4.9.0-3-amd64", 
  "Arch": "amd64", 
  "BuildTime": "2017-06-23T21:16:12.825866885+00:00", 
  "ApiVersion": "1.30", 
  "Version": "17.06.0-ce", 
  "MinAPIVersion": "1.12", 
  "GitCommit": "02c1d87", 
  "Os": "linux", 
  "GoVersion": "go1.8.3"
}
SUMMARY

When the roles aren't installed, ansible-container doesn't install them. I've tried different ways of installing them and I've find that the ones on galaxy are the only that work.

STEPS TO REPRODUCE
echo "- src: git+https://github.com/geerlingguy/ansible-role-jenkins" >> requirements.yml
ansible-container --debug build

The role is installed when executing:

ansible-galaxy install -r requirements.yml
EXPECTED RESULTS

I expect that ansible-container installs the role when it doesn't find it on it's path. I've used this role just as an example, it doesn't work with any other. It seems to only work when the role is on ansible-galaxy.

ACTUAL RESULTS

It doesn't install the roles automatically.

Traceback (most recent call last):
  File "/usr/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container', 'console_scripts', 'conductor')()
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/cli.py", line 370, in conductor_commandline
    conductor_config = AnsibleContainerConductorConfig(list_to_ordereddict(containers_config))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/config.py", line 269, in __init__
    self._process_services()
  File "/_ansible/container/config.py", line 329, in _process_services
    role_metadata = get_metadata_from_role(role_name)
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/utils/__init__.py", line 271, in get_metadata_from_role
    return get_content_from_role(role_name, os.path.join('meta', 'container.yml'))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/utils/__init__.py", line 260, in get_content_from_role
    role_path = resolve_role_to_path(role_name)
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/utils/__init__.py", line 209, in resolve_role_to_path
    loader=loader)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/role/include.py", line 59, in load
    return ri.load_data(data, variable_manager=variable_manager, loader=loader)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/base.py", line 242, in load_data
    ds = self.preprocess_data(ds)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/role/definition.py", line 94, in preprocess_data
    (role_name, role_path) = self._load_role_path(role_name)
  File "/usr/lib/python2.7/site-packages/ansible/playbook/role/definition.py", line 187, in _load_role_path
    raise AnsibleError("the role '%s' was not found in %s" % (role_name, ":".join(role_search_paths)), obj=self._ds)
ansible.errors.AnsibleError: the role 'ansible-role-jenkins' was not found in ./roles:/src/roles:/etc/ansible/roles:.
@chouseknecht
Copy link
Contributor

@drym3r:

Thanks for using Ansible Container, and for taking the time to share your feedback!

Not sure which version of Ansible Container you're using. I made a change in develop yesterday that fixed this same issue when using --project-path. See #655.

Tested this morning for the same with no --project-path, and I can confirm that it works as expected. Roles in requirements.yml do get installed.

I assume that you're not running from source, in which case, can you uninstall Ansible Container, and install from source. There's a guide here, if you need assistance.

Thanks!

@drym3r
Copy link
Author

drym3r commented Jul 24, 2017 •
edited

@chouseknecht I just pulled and retried with same results. The last commit is 601dae5.

To add more information, I have the roles_path = roles line on the ansible.cfg file of the project. When building, it returns this error:

You must give at least one requirement to install (see "pip help install")

 [WARNING]: - ansible-role-jenkins was NOT installed successfully: error
executing: git clone https://github.com/geerlingguy/ansible-role-jenkins
ansible-role-jenkins
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.

If I can do more to debug it, just ask for it.

@chouseknecht
Copy link
Contributor

@drym3r

You shouldn't need anything in ansible.cfg related to roles_path. If you need to add a custom path, use the --roles-path option on the build command.

The fact that you're seeing an error is proof that it's at least attempting to now install the jenkins role.

Notice in the steps to reproduce above, you're missing the .git at the end of the URL. Please confirm that your requirements.yml file contains the following:

- src: git+https://github.com/geerlingguy/ansible-role-jenkins.git

Or, you could set it to the following as well:

- geerlingguy.jenkins

@drym3r
Copy link
Author

drym3r commented Jul 25, 2017

You shouldn't need anything in ansible.cfg related to roles_path. If you need to add a custom path, use the --roles-path option on the build command.

Is just a habit I guess.

The fact that you're seeing an error is proof that it's at least attempting to now install the jenkins role.

It is indeed attempting to clone the jenkins role, but is failing (and I can't see why). When executing with debug, the relevant lines are:

` [WARNING]: - ansible-role-jenkins was NOT installed successfully: error
executing: git clone https://github.com/geerlingguy/ansible-role-jenkins.git
ansible-role-jenkins

ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
`

If I execute the command git clone https://github.com/geerlingguy/ansible-role-jenkins.git ansible-role-jenkins manually it works with no problem.

Notice in the steps to reproduce above, you're missing the .git at the end of the URL. Please confirm that your requirements.yml file contains the following:

Confirmed that the .git url is correct. As said, the command works correctly when executed manually.

@chouseknecht
Copy link
Contributor

I would try running this without roles_path set in ansible.cfg, if you have not already done so.

Another thing I would try is removing this role (and any others) from requirements.yml, and letting the conductor build complete. This will give you a conductor image to play with. Then run something like:

$ cd <your-project>
$ docker run -d --name foo -v ${PWD}:/src <your-project>-conductor sleep 1d

The idea (and the above syntax may not be perfect) is to create a running conductor. Then docker exec -it foo /bin/bash into it, set your working directory to /src, and run the ansible-galaxy command. With that you should be able to discover what the issue is. I think ansible-galaxy has a -v, or some option, that provides more debugging output.

@drym3r
Copy link
Author

drym3r commented Jul 28, 2017

Ok, I'll try next week, thanks for your help.

@chouseknecht chouseknecht self-assigned this Jul 28, 2017
@marcusianlevine
Copy link
Contributor

@chouseknecht I am testing out the latest ansible/develop commit before merging in the new secrets behavior to my fork, and I just experienced a very similar issue, with roles in requirements.yml not being available to the conductor (debian:jessie base) during build:

Traceback (most recent call last):
  File "/usr/local/bin/conductor", line 11, in <module>
    load_entry_point('ansible-container', 'console_scripts', 'conductor')()
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/cli.py", line 382, in conductor_commandline
    conductor_config = AnsibleContainerConductorConfig(list_to_ordereddict(containers_config))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/config.py", line 292, in __init__
    self._process_services()
  File "/_ansible/container/config.py", line 352, in _process_services
    role_metadata = get_metadata_from_role(role_name)
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/utils/__init__.py", line 271, in get_metadata_from_role
    return get_content_from_role(role_name, os.path.join('meta', 'container.yml'))
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/utils/__init__.py", line 260, in get_content_from_role
    role_path = resolve_role_to_path(role_name)
  File "/_ansible/container/__init__.py", line 19, in __wrapped__
    return fn(*args, **kwargs)
  File "/_ansible/container/utils/__init__.py", line 209, in resolve_role_to_path
    loader=loader)
  File "/usr/local/lib/python2.7/dist-packages/ansible/playbook/role/include.py", line 59, in load
    return ri.load_data(data, variable_manager=variable_manager, loader=loader)
  File "/usr/local/lib/python2.7/dist-packages/ansible/playbook/base.py", line 242, in load_data
    ds = self.preprocess_data(ds)
  File "/usr/local/lib/python2.7/dist-packages/ansible/playbook/role/definition.py", line 94, in preprocess_data
    (role_name, role_path) = self._load_role_path(role_name)
  File "/usr/local/lib/python2.7/dist-packages/ansible/playbook/role/definition.py", line 187, in _load_role_path
    raise AnsibleError("the role '%s' was not found in %s" % (role_name, ":".join(role_search_paths)), obj=self._ds)
ansible.errors.AnsibleError: the role 'marcusianlevine.django-container' was not found in ./roles:/src/roles:/etc/ansible/roles:.

I don't pass my roles path in ansible.cfg or even through the CLI, I just have a few roles from Galaxy in my requirements.yml:

- marcusianlevine.nginx-container
- marcusianlevine.django-container

After running the build in debug mode with no cache, I found that my conductor image was successfully installing the roles, but to a location where ansible-container does not look: /root/.ansible/roles

Installed /_ansible
Processing dependencies for ansible-container==0.9.2rc0
Finished processing dependencies for ansible-container==0.9.2rc0
- downloading role 'kubernetes-modules', owned by ansible
- downloading role from https://github.com/ansible/ansible-kubernetes-modules/archive/master.tar.gz
- extracting kubernetes-modules to /root/.ansible/roles/kubernetes-modules
- kubernetes-modules (master) was installed successfully
 ---> 7107c422a2b6
Removing intermediate container aa8d6626d9af
Step 8/11 : COPY /build-src /_ansible/build
 ---> 95801ee71a92
Removing intermediate container edc0723667c8
Step 9/11 : RUN ( test -f /_ansible/build/ansible-requirements.txt && pip install --no-cache-dir -r /_ansible/build/ansible-requirements.txt|| true ) &&     ( test -f /_ansible/build/requirements.yml && ansible-galaxy install -p /etc/ansible/roles -r /_ansible/build/requirements.yml || true ) &&     ( test -f /_ansible/build/ansible.cfg && cp /_ansible/build/ansible.cfg /etc/ansible/ansible.cfg || true)
 ---> Running in 107a0451570b
- downloading role 'nginx-container', owned by marcusianlevine
- downloading role from https://github.com/marcusianlevine/nginx-container/archive/master.tar.gz
- extracting marcusianlevine.nginx-container to /root/.ansible/roles/marcusianlevine.nginx-container
- marcusianlevine.nginx-container (master) was installed successfully
- downloading role 'django-container', owned by marcusianlevine
- downloading role from https://github.com/marcusianlevine/ansible-django-container/archive/master.tar.gz
- extracting marcusianlevine.django-container to /root/.ansible/roles/marcusianlevine.django-container
- marcusianlevine.django-container (master) was installed successfully
 ---> a2c45a8f7789
Removing intermediate container 107a0451570b
Step 10/11 : VOLUME /usr
 ---> Running in fecb8a00f2ce
 ---> ee6d7dd44e51
Removing intermediate container fecb8a00f2ce
Step 11/11 : VOLUME /lib
 ---> Running in 8a9ddbbdff7b
 ---> ef8131c60214
Removing intermediate container 8a9ddbbdff7b
{"aux": {"ID": "sha256:ef8131c60214a72c33b112a693aef3ddac78978a8cc848affbea16c5184ae815"}}
Successfully built ef8131c60214
Successfully tagged ckmdashboards-conductor:latest

I was able to address this issue by adding ~/.ansible/roles to the default set of roles paths supplied to the conductor. Will submit a pull request with the change shortly

marcusianlevine pushed a commit to marcusianlevine/ansible-container that referenced this issue Jul 30, 2017
fixes ansible#658 by adding ~/.ansible/roles to conductor ANSIBLE_ROL…
edf17ac 
…ES_PATH
@marcusianlevine marcusianlevine mentioned this issue Jul 30, 2017
Closed
@drym3r
Copy link
Author

drym3r commented Jul 31, 2017

I confirm that trying it without the line on ansible.cfg returns same error. Didn''t try the docker thing, since another person reported the error it. If you need it anyway I'll do it.

@marcusianlevine marcusianlevine mentioned this issue Jul 31, 2017
Closed
marcusianlevine pushed a commit to marcusianlevine/ansible-container that referenced this issue Aug 1, 2017
fixes ansible#658 by adding ~/.ansible/roles to conductor ANSIBLE_ROL…
e5ef102 
…ES_PATH
@marcusianlevine
Copy link
Contributor

@drym3r my issue was patched by an upstream PR in Ansible core, see #673

Are you still encountering this issue with the latest commit?

@chouseknecht
Copy link
Contributor

@drym3r

Closing this, as I think we have it working now. If I'm wrong, please let me know, and we'll re-open it.

Thanks, everyone!

@drym3r
Copy link
Author

drym3r commented Aug 11, 2017 •
edited

@chouseknecht Just FYI, yeah it works, thank you very much!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@chouseknecht chouseknecht

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@chouseknecht @marcusianlevine @drym3r