apt_repository never updates the APT cache on retry

[git-harry]

ISSUE TYPE

• Bug Report

COMPONENT NAME

apt_repository

ANSIBLE VERSION

ansible 2.4.3.0
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

CONFIGURATION

OS / ENVIRONMENT

Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial

SUMMARY

On the first attempt apt_repository updates the configuration file and then tries to update the cache. If the cache update fails, and the task is set to retry, a second attempt is made to perform the task. This second attempt always succeeds however the cache is never updated.

The relevant block of code appears to be https://github.com/ansible/ansible/blob/v2.4.3.0-1/lib/ansible/modules/packaging/os/apt_repository.py#L540-L547. On the first attempt the configuration changes are saved, on the second attempt no changes are detected and so this block is skipped preventing the cache update being retried.

STEPS TO REPRODUCE

- hosts: localhost
  tasks:
    - apt_repository:
        repo: "deb http://repo.example.com/foo bar baz"
        update_cache: yes
      register: add_repo
      until: add_repo|success
      retries: 3

EXPECTED RESULTS

When the task is retried it should attempt to update the cache. With the above example the task should fail having exhausted all retries given the repository specified does not exist.

ACTUAL RESULTS

The task succeeds on the first retry even though the cache hasn't been updated.

# ansible-playbook test.yml -vvvv
ansible-playbook 2.4.3.0
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
No config file found; using defaults
setting up inventory plugins
 [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source

 [WARNING]: No inventory was parsed, only implicit localhost is available

 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Loading callback plugin default of type stdout, v2.0 from /usr/local/lib/python2.7/dist-packages/ansible/plugins/callback/default.pyc

PLAYBOOK: test.yml *****************************************************************************************************************************************************************************************
1 plays in test.yml

PLAY [localhost] *******************************************************************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************************************************
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/system/setup.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635 `" && echo ansible-tmp-1519337361.1-12167561870635="` echo /root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpUtD8Aj TO /root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635/setup.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635/ /root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635/setup.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635/setup.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1519337361.1-12167561870635/" > /dev/null 2>&1 && sleep 0'
ok: [localhost]
META: ran handlers

TASK [apt_repository] **************************************************************************************************************************************************************************************
task path: /root/test.yml:3
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/packaging/os/apt_repository.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: root
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250 `" && echo ansible-tmp-1519337362.87-19000655109250="` echo /root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpeeIaHe TO /root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250/apt_repository.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250/ /root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250/apt_repository.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250/apt_repository.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1519337362.87-19000655109250/" > /dev/null 2>&1 && sleep 0'
FAILED - RETRYING: apt_repository (3 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_eGGoS0/ansible_module_apt_repository.py\", line 556, in <module>\n    main()\n  File \"/tmp/ansible_eGGoS0/ansible_module_apt_repository.py\", line 544, in main\n    cache.update()\n  File \"/usr/lib/python2.7/dist-packages/apt/cache.py\", line 456, in update\n    raise FetchFailedException()\napt.cache.FetchFailedException\n", 
    "module_stdout": "", 
    "msg": "MODULE FAILURE", 
    "rc": 0, 
    "retries": 4
}
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/packaging/os/apt_repository.py
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762 `" && echo ansible-tmp-1519337371.12-4389313985762="` echo /root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpiebQtl TO /root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762/apt_repository.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762/ /root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762/apt_repository.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762/apt_repository.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1519337371.12-4389313985762/" > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "attempts": 2, 
    "changed": false, 
    "diff": {}, 
    "invocation": {
        "module_args": {
            "codename": null, 
            "filename": null, 
            "install_python_apt": true, 
            "mode": null, 
            "repo": "deb http://repo.example.com/foo bar baz", 
            "state": "present", 
            "update_cache": true, 
            "validate_certs": true
        }
    }, 
    "repo": "deb http://repo.example.com/foo bar baz", 
    "state": "present"
}
META: ran handlers
META: ran handlers

PLAY RECAP *************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0

[ansibot]

Files identified in the description:

• lib/ansible/modules/packaging/os/apt_repository.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @sashka
click here for bot help

[avoidik]

Same issue here, it seems that whole apt-* cache thing is broken, because it fails even with:

- name: refresh apt
  action: "apt"
  args:
    update_cache: yes
  register: cache_result
  until: cache_result|succeeded
  retries: 4

With

ansible-playbook 2.4.3.0

[resmo]

this issue is address in the following PR and also implements multiple repos support.

resolved_by_pr #56951