New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failures with ansible 2.2.1.0 #3111

Closed
sdodson opened this Issue Jan 18, 2017 · 51 comments

Comments

Projects
None yet
@sdodson
Member

sdodson commented Jan 18, 2017

We're seeing yaml rendering issues with our code and 2.2.1.0. Need to sort out if we need to fix something or if ansible will be fixing things.

Upstream references ansible/ansible#20290 ansible/ansible#20253

    TASK [openshift_master : set_fact] *********************************************
    task path: /home/rdu/sdodson/git/Openshift/openshift-ansible/roles/openshift_master/tasks/main.yml:146
    An exception occurred during task execution. The full traceback is:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
        res = self._execute()
      File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 443, in _execute
        self._task.post_validate(templar=templar)
      File "/usr/lib/python2.7/site-packages/ansible/playbook/task.py", line 248, in post_validate
        super(Task, self).post_validate(templar)
      File "/usr/lib/python2.7/site-packages/ansible/playbook/base.py", line 373, in post_validate
        value = templar.template(getattr(self, name))
      File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 427, in template
        disable_lookups=disable_lookups,
      File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 383, in template
        disable_lookups=disable_lookups,
      File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 583, in do_template
        res = j2_concat(rf)
      File "<template>", line 9, in root
      File "/home/rdu/sdodson/git/Openshift/openshift-ansible/filter_plugins/openshift_master.py", line 488, in translate_idps
        return yaml.safe_dump([idp.to_dict() for idp in idp_list], default_flow_style=False)
      File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 218, in safe_dump
        return dump_all([data], stream, Dumper=SafeDumper, **kwds)
      File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 190, in dump_all
        dumper.represent(data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 28, in represent
        node = self.represent_data(data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 57, in represent_data
        node = self.yaml_representers[data_types[0]](self, data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 215, in represent_list
        return self.represent_sequence(u'tag:yaml.org,2002:seq', data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 101, in represent_sequence
        node_item = self.represent_data(item)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 57, in represent_data
        node = self.yaml_representers[data_types[0]](self, data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 223, in represent_dict
        return self.represent_mapping(u'tag:yaml.org,2002:map', data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 123, in represent_mapping
        node_value = self.represent_data(item_value)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 67, in represent_data
        node = self.yaml_representers[None](self, data)
      File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 247, in represent_undefined
        raise RepresenterError("cannot represent an object: %s" % data)
    RepresenterError: cannot represent an object: htpasswd_auth
     
    fatal: [ose3-master.example.com]: FAILED! => {
        "failed": true,
        "msg": "Unexpected failure during module execution.",
        "stdout": ""
    }
@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Jan 18, 2017

Member

@abutcher had some proposed changes that may address this
CC: @tdawson @detiber

Member

sdodson commented Jan 18, 2017

@abutcher had some proposed changes that may address this
CC: @tdawson @detiber

@abutcher

This comment has been minimized.

Show comment
Hide comment
@abutcher

abutcher Jan 18, 2017

Member

Casting ansible.vars.unsafe_proxy.AnsibleUnsafeText to string in the IdentityProviderBase class allowed this to continue. WIP commit here abutcher@8d3714f

Member

abutcher commented Jan 18, 2017

Casting ansible.vars.unsafe_proxy.AnsibleUnsafeText to string in the IdentityProviderBase class allowed this to continue. WIP commit here abutcher@8d3714f

@marekjelen

This comment has been minimized.

Show comment
Hide comment
@marekjelen

marekjelen Jan 20, 2017

Member

Hi, trying to install v1.4.0 cluster and Ansible is failing because of

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: htpasswd_auth

I have tried to checkout release-1.4 branch as well as openshift-ansible-3.4.55-1 tag.

Is there some workaround that would allow me to finish the installation process except of downgrading Ansible itself?

thanks

Member

marekjelen commented Jan 20, 2017

Hi, trying to install v1.4.0 cluster and Ansible is failing because of

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: htpasswd_auth

I have tried to checkout release-1.4 branch as well as openshift-ansible-3.4.55-1 tag.

Is there some workaround that would allow me to finish the installation process except of downgrading Ansible itself?

thanks

@detiber

This comment has been minimized.

Show comment
Hide comment
@detiber

detiber Jan 20, 2017

Contributor

@marekjelen I believe you would need to either need to use the commit that @abutcher linked to above. Ansible is working on a new build that resolves the problem, so I do not expect that we will ship a workaround.

Contributor

detiber commented Jan 20, 2017

@marekjelen I believe you would need to either need to use the commit that @abutcher linked to above. Ansible is working on a new build that resolves the problem, so I do not expect that we will ship a workaround.

@abutcher

This comment has been minimized.

Show comment
Hide comment
@abutcher

abutcher Jan 20, 2017

Member

Playing whack-a-mole with openshift-ansible workarounds. The next place we hit the issue is within the to_padded_yaml filter.

TASK [openshift_master : Create master config] *********************************
task path: /home/abutcher/rhat/openshift-ansible/roles/openshift_master/tasks/main.yml:150
fatal: [master1.abutcher.com]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "backup": true, 
            "dest": "/etc/origin/master/master-config.yaml", 
            "group": "root", 
            "mode": 384, 
            "owner": "root", 
            "src": "master.yaml.v1.j2"
        }, 
        "module_name": "template"
    }, 
    "msg": "AnsibleFilterError: Failed to convert: cannot represent an object: BuildDefaults"
}
Member

abutcher commented Jan 20, 2017

Playing whack-a-mole with openshift-ansible workarounds. The next place we hit the issue is within the to_padded_yaml filter.

TASK [openshift_master : Create master config] *********************************
task path: /home/abutcher/rhat/openshift-ansible/roles/openshift_master/tasks/main.yml:150
fatal: [master1.abutcher.com]: FAILED! => {
    "changed": false, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "backup": true, 
            "dest": "/etc/origin/master/master-config.yaml", 
            "group": "root", 
            "mode": 384, 
            "owner": "root", 
            "src": "master.yaml.v1.j2"
        }, 
        "module_name": "template"
    }, 
    "msg": "AnsibleFilterError: Failed to convert: cannot represent an object: BuildDefaults"
}
@ArminMW

This comment has been minimized.

Show comment
Hide comment
@ArminMW

ArminMW Jan 21, 2017

I can confirm that downgrading ansible-2.2.1.0-1.el7.noarch.rpm from epel-release-7-9 to ansible-2.2.0.0-4.el7.noarch.rpm from epel-release-7-8 is a workaround for a successful openshift-ansible v1.4 installation. Regards, Armin.

ArminMW commented Jan 21, 2017

I can confirm that downgrading ansible-2.2.1.0-1.el7.noarch.rpm from epel-release-7-9 to ansible-2.2.0.0-4.el7.noarch.rpm from epel-release-7-8 is a workaround for a successful openshift-ansible v1.4 installation. Regards, Armin.

@amdonov

This comment has been minimized.

Show comment
Hide comment
@amdonov

amdonov Jan 21, 2017

@ArminMW Dumb question, but where did you find the old version? I'm not seeing it in EPEL.

amdonov commented Jan 21, 2017

@ArminMW Dumb question, but where did you find the old version? I'm not seeing it in EPEL.

@wicksy

This comment has been minimized.

Show comment
Hide comment
@wicksy

wicksy Jan 21, 2017

@amdonov Just run into exactly the same issue. Installed 2.2.0.0 as a pip.

wicksy commented Jan 21, 2017

@amdonov Just run into exactly the same issue. Installed 2.2.0.0 as a pip.

@ArminMW

This comment has been minimized.

Show comment
Hide comment
@ArminMW

ArminMW Jan 21, 2017

I found the downlevel ansible in an epel mirror that seems to be lagging behind:
http://ftp.uni-hannover.de/pub/mirror/linux/epel/7Server/x86_64/a/ansible-2.2.0.0-4.el7.noarch.rpm

ArminMW commented Jan 21, 2017

I found the downlevel ansible in an epel mirror that seems to be lagging behind:
http://ftp.uni-hannover.de/pub/mirror/linux/epel/7Server/x86_64/a/ansible-2.2.0.0-4.el7.noarch.rpm

@amdonov

This comment has been minimized.

Show comment
Hide comment
@amdonov

amdonov Jan 21, 2017

@wicksy Thanks for the tip. I took your advice and did something similar to the following. (Hope I'm not missing anything)

sudo yum -y erase ansible
sudo yum install -y "@development Tools" python2-pip openssl-devel python-devel
sudo pip install -Iv ansible==2.2.0.0

Before going that route I tried applying the patch that closed the Ansible issue. I think that additional work will need to happen to the openshift playbooks to work going forward.

amdonov commented Jan 21, 2017

@wicksy Thanks for the tip. I took your advice and did something similar to the following. (Hope I'm not missing anything)

sudo yum -y erase ansible
sudo yum install -y "@development Tools" python2-pip openssl-devel python-devel
sudo pip install -Iv ansible==2.2.0.0

Before going that route I tried applying the patch that closed the Ansible issue. I think that additional work will need to happen to the openshift playbooks to work going forward.

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Jan 22, 2017

Member

We've also got ansible-2.2.0.0 in the centos paas sig common repo http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/common/ansible-2.2.0.0-1.el7.noarch.rpm If you choose to downgrade to 2.2.0.0 it should be noted that version is vulnerable to CVE-2016-9587, see https://bugzilla.redhat.com/show_bug.cgi?id=1404378

Member

sdodson commented Jan 22, 2017

We've also got ansible-2.2.0.0 in the centos paas sig common repo http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/common/ansible-2.2.0.0-1.el7.noarch.rpm If you choose to downgrade to 2.2.0.0 it should be noted that version is vulnerable to CVE-2016-9587, see https://bugzilla.redhat.com/show_bug.cgi?id=1404378

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Jan 22, 2017

Member

We'll put a workaround in unless we're sure that we'll have a fixed version of ansible available soon.

Member

sdodson commented Jan 22, 2017

We'll put a workaround in unless we're sure that we'll have a fixed version of ansible available soon.

@leifmadsen

This comment has been minimized.

Show comment
Hide comment
@leifmadsen

leifmadsen Jan 23, 2017

Just another confirmation that 2.2.0.0 downgrade results in a pass for me as well. Thanks for linking to the Ansible issues to track.

Just another confirmation that 2.2.0.0 downgrade results in a pass for me as well. Thanks for linking to the Ansible issues to track.

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Jan 25, 2017

Member

Referenced PR is an incomplete fix fwiw.

Member

sdodson commented Jan 25, 2017

Referenced PR is an incomplete fix fwiw.

@ioggstream

This comment has been minimized.

Show comment
Hide comment
@ioggstream

ioggstream Jan 27, 2017

Contributor

Downgrading works. I strongly suggest:

  • adding an entry in yum versionlock
  • put a note in the documentation
Contributor

ioggstream commented Jan 27, 2017

Downgrading works. I strongly suggest:

  • adding an entry in yum versionlock
  • put a note in the documentation
@anosulchik

This comment has been minimized.

Show comment
Hide comment
@anosulchik

anosulchik Jan 31, 2017

+1 for ansible 2.2.1.1 in EPEL repo. thanks

+1 for ansible 2.2.1.1 in EPEL repo. thanks

@detiber

This comment has been minimized.

Show comment
Hide comment
@detiber

detiber Feb 1, 2017

Contributor

@anosulchik are you saying you are seeing the same errors with 2.2.1.1?

Contributor

detiber commented Feb 1, 2017

@anosulchik are you saying you are seeing the same errors with 2.2.1.1?

@anosulchik

This comment has been minimized.

Show comment
Hide comment
@anosulchik

anosulchik Feb 1, 2017

@detiber Yes. Looks like EPEL repo now contains 2.2.1.0 as latest Ansible instead of 2.2.1.1. 2.2.1.0 works just fine.

@detiber Yes. Looks like EPEL repo now contains 2.2.1.0 as latest Ansible instead of 2.2.1.1. 2.2.1.0 works just fine.

@jhcook

This comment has been minimized.

Show comment
Hide comment
@jhcook

jhcook Feb 4, 2017

Contributor

I've run in to this issue on macOS with virtualenv and sorted it by:
pip install ansible==2.2.0.0 --upgrade

Contributor

jhcook commented Feb 4, 2017

I've run in to this issue on macOS with virtualenv and sorted it by:
pip install ansible==2.2.0.0 --upgrade

@vbartoni

This comment has been minimized.

Show comment
Hide comment
@vbartoni

vbartoni Feb 7, 2017

Would someone know if Ansible is willing to provide a fix for this in the near future (maybe with 2.2.2?), or something is possible to be done on this side? This issue is kind of painful - using openshift-on-openstack to install Origin, and fetching all packages from CentOS repos. Thanks!

vbartoni commented Feb 7, 2017

Would someone know if Ansible is willing to provide a fix for this in the near future (maybe with 2.2.2?), or something is possible to be done on this side? This issue is kind of painful - using openshift-on-openstack to install Origin, and fetching all packages from CentOS repos. Thanks!

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Feb 7, 2017

Member
Member

sdodson commented Feb 7, 2017

@jospradlin

This comment has been minimized.

Show comment
Hide comment
@jospradlin

jospradlin Feb 7, 2017

Just downgrade Ansible to from 2.2.1 to 2.2.0 using the YUM command below. (no other commands required)

# yum downgrade ansible

jospradlin commented Feb 7, 2017

Just downgrade Ansible to from 2.2.1 to 2.2.0 using the YUM command below. (no other commands required)

# yum downgrade ansible

@vbartoni

This comment has been minimized.

Show comment
Hide comment
@vbartoni

vbartoni Feb 7, 2017

+1 thanks @jospradlin, great trick ;-)

vbartoni commented Feb 7, 2017

+1 thanks @jospradlin, great trick ;-)

@vnugent

This comment has been minimized.

Show comment
Hide comment
@vnugent

vnugent Feb 8, 2017

Contributor

downgraded to ansible 2.2.0.0 but now I'm hitting ansible/ansible#14143

env: Fedora 25
git describe: openshift-ansible-3.5.3-1-89-g405bd70

TASK [openshift_examples : Remove defunct quickstart templates from openshift namespace] ***
task path: /root/openshift-ansible/roles/openshift_examples/tasks/main.yml:90
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/commands/command.py
<172.18.17.19> ESTABLISH SSH CONNECTION FOR USER: root
<172.18.17.19> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAut
hentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -
o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r 172.18.17.19 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansib
le/tmp/ansible-tmp-1486578115.75-261880374043026 `" && echo ansible-tmp-1486578115.75-261880374043026="` echo $HOME/.ansible/t
mp/ansible-tmp-1486578115.75-261880374043026 `" ) && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 91, in run
    item_results = self._run_loop(items)
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 274, in _run_loop
    res = self._execute(variables=task_vars)
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 490, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 574, in _execute_module
    tmp = self._make_tmp_path(remote_user)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 222, in _make_tmp_path
    result = self._low_level_execute_command(cmd, sudoable=False)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 754, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 670, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 598, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 434, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [172.18.17.19]: FAILED! => {
    "failed": true, 
    "msg": "Unexpected failure during module execution.", 
    "stdout": ""
}
        to retry, use: --limit @/root/openshift-ansible/playbooks/byo/config.retry

PLAY RECAP *********************************************************************
172.18.17.19               : ok=265  changed=10   unreachable=0    failed=1   
172.18.17.27               : ok=60   changed=0    unreachable=0    failed=0   
localhost                  : ok=9    changed=0    unreachable=0    failed=0 
Contributor

vnugent commented Feb 8, 2017

downgraded to ansible 2.2.0.0 but now I'm hitting ansible/ansible#14143

env: Fedora 25
git describe: openshift-ansible-3.5.3-1-89-g405bd70

TASK [openshift_examples : Remove defunct quickstart templates from openshift namespace] ***
task path: /root/openshift-ansible/roles/openshift_examples/tasks/main.yml:90
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/commands/command.py
<172.18.17.19> ESTABLISH SSH CONNECTION FOR USER: root
<172.18.17.19> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAut
hentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -
o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r 172.18.17.19 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/.ansib
le/tmp/ansible-tmp-1486578115.75-261880374043026 `" && echo ansible-tmp-1486578115.75-261880374043026="` echo $HOME/.ansible/t
mp/ansible-tmp-1486578115.75-261880374043026 `" ) && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 91, in run
    item_results = self._run_loop(items)
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 274, in _run_loop
    res = self._execute(variables=task_vars)
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 490, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 574, in _execute_module
    tmp = self._make_tmp_path(remote_user)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 222, in _make_tmp_path
    result = self._low_level_execute_command(cmd, sudoable=False)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 754, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 670, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 598, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 434, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [172.18.17.19]: FAILED! => {
    "failed": true, 
    "msg": "Unexpected failure during module execution.", 
    "stdout": ""
}
        to retry, use: --limit @/root/openshift-ansible/playbooks/byo/config.retry

PLAY RECAP *********************************************************************
172.18.17.19               : ok=265  changed=10   unreachable=0    failed=1   
172.18.17.27               : ok=60   changed=0    unreachable=0    failed=0   
localhost                  : ok=9    changed=0    unreachable=0    failed=0 

BorisB2015 pushed a commit to BorisB2015/azure-quickstart-templates that referenced this issue Feb 14, 2017

@DanyC97

This comment has been minimized.

Show comment
Hide comment
@DanyC97

DanyC97 Feb 20, 2017

Contributor

@sdodson can you please confirm if the ansible verion in centos-openshift-origin repo is been patch to cover the above issue?

in epel i see version 2.2.1.0-1.el7
in centos-openshift-origin repo i see 2.2.1.0-2.el7

Contributor

DanyC97 commented Feb 20, 2017

@sdodson can you please confirm if the ansible verion in centos-openshift-origin repo is been patch to cover the above issue?

in epel i see version 2.2.1.0-1.el7
in centos-openshift-origin repo i see 2.2.1.0-2.el7

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Feb 20, 2017

Member

Yes, 2.2.1.0-2.el7 is patched, there may be openshift-ansible code paths that still have problems, if you're using 2.2.1.0-2.el7 and hit a problem with latest master lets get an issue open for those.

Member

sdodson commented Feb 20, 2017

Yes, 2.2.1.0-2.el7 is patched, there may be openshift-ansible code paths that still have problems, if you're using 2.2.1.0-2.el7 and hit a problem with latest master lets get an issue open for those.

@DanyC97

This comment has been minimized.

Show comment
Hide comment
@DanyC97

DanyC97 Feb 21, 2017

Contributor

sure will do indeed, thanks @sdodson !

Contributor

DanyC97 commented Feb 21, 2017

sure will do indeed, thanks @sdodson !

@OldCrowEW

This comment has been minimized.

Show comment
Hide comment
@OldCrowEW

OldCrowEW Feb 22, 2017

pip install ansible==2.2.0.0 --upgrade fixes for me

pip install ansible==2.2.0.0 --upgrade fixes for me

@esuomi

This comment has been minimized.

Show comment
Hide comment
@esuomi

esuomi Feb 28, 2017

Hi,

We also were hit by this issue earlier this week with rather cryptic error:

TASK [openshift_master : set_fact] *********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: claim
fatal: [10.27.41.51]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}

and indeed downgrading to 2.2.0.0 as @OldCrowEW suggested worked for us too.

esuomi commented Feb 28, 2017

Hi,

We also were hit by this issue earlier this week with rather cryptic error:

TASK [openshift_master : set_fact] *********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: claim
fatal: [10.27.41.51]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}

and indeed downgrading to 2.2.0.0 as @OldCrowEW suggested worked for us too.

@jberkus

This comment has been minimized.

Show comment
Hide comment
@jberkus

jberkus Mar 1, 2017

Incidentally, ansible 2.2.0.0 is no longer available as a package for Fedora 25. So at this point, there is no packaged version of ansible for Fedora 25 which works with Openshift-Ansible. (and replacing it via Pip is kinda painful, because it requires rebuilding the cryptography module).

jberkus commented Mar 1, 2017

Incidentally, ansible 2.2.0.0 is no longer available as a package for Fedora 25. So at this point, there is no packaged version of ansible for Fedora 25 which works with Openshift-Ansible. (and replacing it via Pip is kinda painful, because it requires rebuilding the cryptography module).

@DanyC97

This comment has been minimized.

Show comment
Hide comment
@DanyC97

DanyC97 Mar 1, 2017

Contributor

@jberkus if you read what i mentioned few comments above you can get the patched version from centos-openshift-origin and you are all good

Contributor

DanyC97 commented Mar 1, 2017

@jberkus if you read what i mentioned few comments above you can get the patched version from centos-openshift-origin and you are all good

@jberkus

This comment has been minimized.

Show comment
Hide comment
@jberkus

jberkus Mar 1, 2017

@DanyC97 well, I used a Fedora 24 container, which worked. But it does show that there's a deadline on fixing this one way or the other.

jberkus commented Mar 1, 2017

@DanyC97 well, I used a Fedora 24 container, which worked. But it does show that there's a deadline on fixing this one way or the other.

@marconesns

This comment has been minimized.

Show comment
Hide comment
@marconesns

marconesns Mar 3, 2017

Hello everyone,

It was like same problem, I decided to configure the master server in hours, in the case in a CentOS 7 with chrony. I have the version of ansbile 2.2.1.0

Hello everyone,

It was like same problem, I decided to configure the master server in hours, in the case in a CentOS 7 with chrony. I have the version of ansbile 2.2.1.0

@detiber

This comment has been minimized.

Show comment
Hide comment
@detiber

detiber Mar 3, 2017

Contributor

@jberkus @maxamillion the current situation is far from ideal by any means.

To give a bit more context:

  • After the initial issue was discovered, we started playing whack-a-mole with trying to work around the issue in openshift-ansible before realizing it was a bug in Ansible.
  • The Ansible team provided us with a patched 2.2.1 build that resolved the issues we were seeing and we expected a 2.2.1.1 release.
  • As an interim fix for RHEL and CentOS releases (which are both shipped via separate repos from the main OS repo) we worked out an agreement with the Ansible team to ship the patched 2.2.1 release to the OpenShift repos.
  • Because 2.2.2.0 appeared to be imminent and contained the same fix, both the OpenShift and the Ansible teams agreed to wait on 2.2.2
  • Ansible 2.2.2.0 has been delayed while trying to address other issues that have surfaced as a result of the security fixes contained in 2.2.1.0. While these did not affect openshift-ansible, but we're still breaking production uses of Ansible.

I'm wondering if it would make sense to deliver OpenShift and openshift-ansible through a separate repo (or ideally repos, where each version is a separate repo) similar to how we are shipping in RHEL and CentOS. I think there are quite a few different reasons for why, but in this particular case, it would have allowed us to ship the patched Ansible version without affecting non-OpenShift users.

Contributor

detiber commented Mar 3, 2017

@jberkus @maxamillion the current situation is far from ideal by any means.

To give a bit more context:

  • After the initial issue was discovered, we started playing whack-a-mole with trying to work around the issue in openshift-ansible before realizing it was a bug in Ansible.
  • The Ansible team provided us with a patched 2.2.1 build that resolved the issues we were seeing and we expected a 2.2.1.1 release.
  • As an interim fix for RHEL and CentOS releases (which are both shipped via separate repos from the main OS repo) we worked out an agreement with the Ansible team to ship the patched 2.2.1 release to the OpenShift repos.
  • Because 2.2.2.0 appeared to be imminent and contained the same fix, both the OpenShift and the Ansible teams agreed to wait on 2.2.2
  • Ansible 2.2.2.0 has been delayed while trying to address other issues that have surfaced as a result of the security fixes contained in 2.2.1.0. While these did not affect openshift-ansible, but we're still breaking production uses of Ansible.

I'm wondering if it would make sense to deliver OpenShift and openshift-ansible through a separate repo (or ideally repos, where each version is a separate repo) similar to how we are shipping in RHEL and CentOS. I think there are quite a few different reasons for why, but in this particular case, it would have allowed us to ship the patched Ansible version without affecting non-OpenShift users.

@jberkus

This comment has been minimized.

Show comment
Hide comment
@jberkus

jberkus Mar 4, 2017

Well, personally, I think we should ship an Openshift-ansible container which has all the bits.

jberkus commented Mar 4, 2017

Well, personally, I think we should ship an Openshift-ansible container which has all the bits.

@detiber

This comment has been minimized.

Show comment
Hide comment
@detiber

detiber Mar 4, 2017

Contributor

@jberkus kind of like this: #3382 ?

There are some tricky problems with containerizing the install bits, though. The two biggest being:

  • Getting all the remote access config in place. ssh keys, ssh config, proxy config, etc
  • Having a containerized install that works on one of the hosts being installed

The second reason is the biggest reason we haven't shipped something supported to date. Many users don't seem to like the idea of running the "installer" on a separate host and running a docker container on a host that you are managing the docker daemon and requiring a docker restart kind of kills your install.

Contributor

detiber commented Mar 4, 2017

@jberkus kind of like this: #3382 ?

There are some tricky problems with containerizing the install bits, though. The two biggest being:

  • Getting all the remote access config in place. ssh keys, ssh config, proxy config, etc
  • Having a containerized install that works on one of the hosts being installed

The second reason is the biggest reason we haven't shipped something supported to date. Many users don't seem to like the idea of running the "installer" on a separate host and running a docker container on a host that you are managing the docker daemon and requiring a docker restart kind of kills your install.

@jberkus

This comment has been minimized.

Show comment
Hide comment
@jberkus

jberkus Mar 5, 2017

I generally solve the first problem by mounting my .ssh into the container. Not sure about proxy config, though, hasn't been an issue I had to solve.

Yes, self-bootstrapping by being able to run the installer on one of the target hosts would be ideal. However, until we solve that problem, a container we can run elsewhere would still be an improvement.

jberkus commented Mar 5, 2017

I generally solve the first problem by mounting my .ssh into the container. Not sure about proxy config, though, hasn't been an issue I had to solve.

Yes, self-bootstrapping by being able to run the installer on one of the target hosts would be ideal. However, until we solve that problem, a container we can run elsewhere would still be an improvement.

@rastakajakwanna

This comment has been minimized.

Show comment
Hide comment
@rastakajakwanna

rastakajakwanna Mar 6, 2017

I already use containerized Ansible-controller and it really needs a lot of tweaking to cover e.g. ansible-vault secrets, .ssh/{config,id_rsa} (ssh bastion host is not an issue), aws api credentials and all possible adjustments to /etc/ansible/ansible.cfg. However, it's worth it, especially in situations like this.

Btw, how do you solve such situations within Ansible Tower?? Would you be ever able to automate OCP installation with CloudForms and Tower?

PS: We just ran into this issue...

rastakajakwanna commented Mar 6, 2017

I already use containerized Ansible-controller and it really needs a lot of tweaking to cover e.g. ansible-vault secrets, .ssh/{config,id_rsa} (ssh bastion host is not an issue), aws api credentials and all possible adjustments to /etc/ansible/ansible.cfg. However, it's worth it, especially in situations like this.

Btw, how do you solve such situations within Ansible Tower?? Would you be ever able to automate OCP installation with CloudForms and Tower?

PS: We just ran into this issue...

@Delmonte3161

This comment has been minimized.

Show comment
Hide comment
@Delmonte3161

Delmonte3161 Mar 6, 2017

I'm having an issue and not sure if it is the same or not.

I am using:
ansible.noarch 2.2.1.0-1.el7 @epel

Does this version have the fix or not?

The issue I am seeing is during: TASK [openshift_master : set_fact]
Error message contains: cannot represent an object: allow_all

Is this the result of the same bug or do I have a unique issue?

I'm having an issue and not sure if it is the same or not.

I am using:
ansible.noarch 2.2.1.0-1.el7 @epel

Does this version have the fix or not?

The issue I am seeing is during: TASK [openshift_master : set_fact]
Error message contains: cannot represent an object: allow_all

Is this the result of the same bug or do I have a unique issue?

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Mar 7, 2017

Member

@Delmonte3161 That's the same problem. You can get ansible-2.2.1.0-2.el7 from the Centos PaaS SIG repo at http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/

Member

sdodson commented Mar 7, 2017

@Delmonte3161 That's the same problem. You can get ansible-2.2.1.0-2.el7 from the Centos PaaS SIG repo at http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/

esuomi added a commit to esuomi/dotfiles that referenced this issue Mar 7, 2017

Replace brew managed Ansible with virtualenv specific Ansible
Ansible has multiple issues running OpenStack/OpenShift related scripts between point releases; this is definitely not the first time so unfortunately the only way around the issues is to downgrade to specific version.

See also: openshift/openshift-ansible#3111
@jhadvig

This comment has been minimized.

Show comment
Hide comment
@jhadvig

jhadvig Mar 7, 2017

Member

@sdodson so I've installed the ansible-2.2.1.0-2.el7 from repo you suggested but I still can see the issue

TASK [openshift_master : set_fact] *********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: allow_all
fatal: [localhost]: FAILED! => {
"failed": true
}
MSG:
Unexpected failure during module execution.

Member

jhadvig commented Mar 7, 2017

@sdodson so I've installed the ansible-2.2.1.0-2.el7 from repo you suggested but I still can see the issue

TASK [openshift_master : set_fact] *********************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: RepresenterError: cannot represent an object: allow_all
fatal: [localhost]: FAILED! => {
"failed": true
}
MSG:
Unexpected failure during module execution.

@jberkus

This comment has been minimized.

Show comment
Hide comment
@jberkus

jberkus Mar 7, 2017

@rastakajakwanna oh, hey, how did you deal with secrets?

jberkus commented Mar 7, 2017

@rastakajakwanna oh, hey, how did you deal with secrets?

@rastakajakwanna

This comment has been minimized.

Show comment
Hide comment
@rastakajakwanna

rastakajakwanna Mar 8, 2017

@jberkus some of them are parsed from known locations and exposed as env vars in the container, ansible-vault secret is mounted to the container user home location, that's the easiest solution.

@jberkus some of them are parsed from known locations and exposed as env vars in the container, ansible-vault secret is mounted to the container user home location, that's the easiest solution.

obnoxxx added a commit to obnoxxx/vagrant-heketi that referenced this issue Mar 9, 2017

openshift: Use ansible 2.2.1.0-2 from the paas sig repo
This build contains a fix for ansible that lets the installation
of openshift (via openshift-ansible) pass.

See: openshift/openshift-ansible#3111
@jdeglopper

This comment has been minimized.

Show comment
Hide comment
@jdeglopper

jdeglopper Mar 21, 2017

For Fedora 25, ansible 2.2.2.0 is available in Fedora Rawhide as ansible-2.2.2.0-0.1.rc1.fc26.noarch.rpm , which appears to correctly run the playbook.

For Fedora 25, ansible 2.2.2.0 is available in Fedora Rawhide as ansible-2.2.2.0-0.1.rc1.fc26.noarch.rpm , which appears to correctly run the playbook.

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Mar 21, 2017

Member

Yes, as soon as 2.2.2.0 is available we can put this whole mess behind us. Unfortunately that's happened slower than expected.

Member

sdodson commented Mar 21, 2017

Yes, as soon as 2.2.2.0 is available we can put this whole mess behind us. Unfortunately that's happened slower than expected.

@mkungla

This comment has been minimized.

Show comment
Hide comment
@mkungla

mkungla Mar 27, 2017

For Fedora 25 had to go for rawhide too which solved the issue.

dnf install -y fedora-repos-rawhide
dnf install -y --enablerepo rawhide --best --allowerasing ansible

mkungla commented Mar 27, 2017

For Fedora 25 had to go for rawhide too which solved the issue.

dnf install -y fedora-repos-rawhide
dnf install -y --enablerepo rawhide --best --allowerasing ansible
@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Mar 28, 2017

Member

Ansible 2.2.2.0 has been released, it should be available in F25 and EPEL shortly. OCP channels will be updated after we've validated this version.

Member

sdodson commented Mar 28, 2017

Ansible 2.2.2.0 has been released, it should be available in F25 and EPEL shortly. OCP channels will be updated after we've validated this version.

@jberkus

This comment has been minimized.

Show comment
Hide comment
@jberkus

jberkus Mar 30, 2017

Looking forward to it. When it's available, we can close this.

jberkus commented Mar 30, 2017

Looking forward to it. When it's available, we can close this.

@x1957

This comment has been minimized.

Show comment
Hide comment
@x1957

x1957 Apr 15, 2017

same problem with ansible 2.2.2.0

An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud02.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<c4-container-cloud06.bj> ESTABLISH SSH CONNECTION FOR USER: root
<c4-container-cloud06.bj> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r c4-container-cloud06.bj '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<c4-container-cloud07.bj> ESTABLISH SSH CONNECTION FOR USER: root
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud04.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
<c4-container-cloud07.bj> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r c4-container-cloud07.bj '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud03.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<c4-container-cloud08.bj> ESTABLISH SSH CONNECTION FOR USER: root
<c4-container-cloud08.bj> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r c4-container-cloud08.bj '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud05.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud07.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud06.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud08.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud01.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.

x1957 commented Apr 15, 2017

same problem with ansible 2.2.2.0

An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud02.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<c4-container-cloud06.bj> ESTABLISH SSH CONNECTION FOR USER: root
<c4-container-cloud06.bj> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r c4-container-cloud06.bj '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<c4-container-cloud07.bj> ESTABLISH SSH CONNECTION FOR USER: root
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud04.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
<c4-container-cloud07.bj> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r c4-container-cloud07.bj '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud03.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
Using module file /usr/lib/python2.7/site-packages/ansible/modules/core/system/setup.py
<c4-container-cloud08.bj> ESTABLISH SSH CONNECTION FOR USER: root
<c4-container-cloud08.bj> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/ansible-ssh-%h-%p-%r c4-container-cloud08.bj '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud05.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud07.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud06.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud08.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 502, in _execute
    result = self._handler.run(task_vars=variables)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 33, in run
    results = merge_hash(results, self._execute_module(tmp=tmp, task_vars=task_vars))
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 653, in _execute_module
    res = self._low_level_execute_command(cmd, sudoable=sudoable, in_data=in_data)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/action/__init__.py", line 777, in _low_level_execute_command
    rc, stdout, stderr = self._connection.exec_command(cmd, in_data=in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 674, in exec_command
    return_tuple = self._exec_command(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 599, in _exec_command
    (returncode, stdout, stderr) = self._run(cmd, in_data, sudoable=sudoable)
  File "/usr/lib/python2.7/site-packages/ansible/plugins/connection/ssh.py", line 437, in _run
    rfd, wfd, efd = select.select(rpipes, [], [], timeout)
ValueError: filedescriptor out of range in select()
fatal: [c4-container-cloud01.bj]: FAILED! => {
    "failed": true
}

MSG:

Unexpected failure during module execution.

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Apr 19, 2017

Member

@x1957 Can you please open a new issue for that, that doesn't look like any of the other errors in this issue and it'd be helpful to gather the output with -vvvv option and include the tasks leading up to this failure.

Member

sdodson commented Apr 19, 2017

@x1957 Can you please open a new issue for that, that doesn't look like any of the other errors in this issue and it'd be helpful to gather the output with -vvvv option and include the tasks leading up to this failure.

@sdodson sdodson closed this Apr 19, 2017

@sdodson

This comment has been minimized.

Show comment
Hide comment
@sdodson

sdodson Apr 19, 2017

Member

Closing this because Ansible 2.2.2.0 is widely available, if you find any new issues please open a new unique issue and include the output with -vvv and the task.

Member

sdodson commented Apr 19, 2017

Closing this because Ansible 2.2.2.0 is widely available, if you find any new issues please open a new unique issue and include the output with -vvv and the task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment