Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Log the exception when the update cache command fails #77941

Closed
1 task done
guenhter opened this issue Jun 1, 2022 · 6 comments · Fixed by #83230
Closed
1 task done

Log the exception when the update cache command fails #77941

guenhter opened this issue Jun 1, 2022 · 6 comments · Fixed by #83230
Labels
affects_2.14 feature This issue/PR relates to a feature request. has_pr This issue has an associated PR. module This issue/PR relates to a module. P3 Priority 3 - Approved, No Time Limitation support:core This issue/PR relates to code supported by the Ansible Engineering Team.

Comments

@guenhter
Copy link

guenhter commented Jun 1, 2022 •
edited

Summary

The apt module should get some more logging in case of errors

Issue Type

Feature Idea

Component Name

lib/ansible/modules/apt.py

Additional Information

The apt python package sometimes fails on cache.update() with an error which is not extractable by err = to_native(e).
This is then quite unfortunate, because the error message in the log is them simply: Failed to update apt cache: unknown reason.

I'd suggest that when the cache.update() throws an exception that that exception is logged (to the debug log or so), so that it is possible to get an understanding what went wrong with cache.update().

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@ansibot
Copy link
Contributor

ansibot commented Jun 1, 2022

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibot ansibot added affects_2.14 feature This issue/PR relates to a feature request. module This issue/PR relates to a module. needs_triage Needs a first human triage before being processed. support:core This issue/PR relates to code supported by the Ansible Engineering Team. labels Jun 1, 2022
@jborean93 jborean93 added P3 Priority 3 - Approved, No Time Limitation and removed needs_triage Needs a first human triage before being processed. labels Jun 2, 2022
@goretkin
Copy link

goretkin commented Jan 9, 2024

I ran into this issue where ansible effectively swallowed the following error:

user@org_docker:/workspaces/org$ sudo apt-get update
[sudo] password for ggoretkin: 
[...]
Reading package lists... Done                              
E: Repository 'http://packages.cloud.google.com/apt cloud-sdk InRelease' changed its 'Origin' value from 'cloud-sdk' to 'namespaces/google.com:cloudsdktool/repositories/cloud-sdk'
E: Repository 'http://packages.cloud.google.com/apt cloud-sdk InRelease' changed its 'Label' value from 'cloud-sdk' to 'namespaces/google.com:cloudsdktool/repositories/cloud-sdk'
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
user@org_docker:/workspaces/org$ echo $?
100

There are other cases where apt.py doesn't generate a detailed error compared to the apt-get CLI, for example this case, reproduced below:

Assuming there is no package server running on localhost:

$ echo 'deb [trusted=yes] http://localhost:8000/ bogus-test-repo main' | sudo tee /etc/apt/sources.list.d/placeholder-test-repo.list

and then

$ python -c "import apt; c = apt.Cache(); c.update()"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/apt/cache.py", line 575, in update
    raise FetchFailedException()
apt.cache.FetchFailedException

and compare with

$ sudo apt-get update
Ign:1 http://localhost:8000/ bogus-test-repo InRelease
Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease
Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:4 http://archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease
Ign:1 http://localhost:8000/ bogus-test-repo InRelease
Ign:1 http://localhost:8000/ bogus-test-repo InRelease
Err:1 http://localhost:8000/ bogus-test-repo InRelease
  Could not connect to localhost:8000 (127.0.0.1). - connect (111: Connection refused)
Reading package lists... Done
W: Failed to fetch http://localhost:8000/dists/bogus-test-repo/InRelease Could not connect to localhost:8000 (127.0.0.1). - connect (111: Connection refused)
W: Some index files failed to download. They have been ignored, or old ones used instead.

Note specifically a recommendation made on that thread:

It's generally not advisable to use the "apt" module as it is badly abstracted, use "apt_pkg" instead.

Akasurde added a commit to Akasurde/ansible that referenced this issue May 9, 2024
@Akasurde
apt_*: add more info message while updating apt cache
1e8df27 
Fixes: ansible#77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
@Akasurde Akasurde mentioned this issue May 9, 2024
Merged
@Akasurde
Copy link
Member

Akasurde commented May 9, 2024 

...
[WARNING]: Failed to update cache after retry 1, retrying
[WARNING]: Sleeping for 1 seconds, before attempting to refresh the cache again
[WARNING]: Failed to update cache after retry 2, retrying
[WARNING]: Sleeping for 2 seconds, before attempting to refresh the cache again
[WARNING]: Failed to update cache after retry 3, retrying
[WARNING]: Sleeping for 4 seconds, before attempting to refresh the cache again
[WARNING]: Failed to update cache after retry 4, retrying
[WARNING]: Sleeping for 8 seconds, before attempting to refresh the cache again
[WARNING]: Failed to update cache after retry 5, retrying
[WARNING]: Sleeping for 12 seconds, before attempting to refresh the cache again
localhost | FAILED! => {
...

@ansibot ansibot added the has_pr This issue has an associated PR. label May 9, 2024
Akasurde added a commit to Akasurde/ansible that referenced this issue May 9, 2024
@Akasurde
apt_*: add more info message while updating apt cache
5481467 
Fixes: ansible#77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Akasurde added a commit to Akasurde/ansible that referenced this issue May 10, 2024
@Akasurde
apt_*: add more info message while updating apt cache
3d3f18f 
Fixes: ansible#77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Akasurde added a commit to Akasurde/ansible that referenced this issue May 10, 2024
@Akasurde
apt_*: add more info message while updating apt cache
e9f8bf2 
Fixes: ansible#77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
@guenhter
Copy link
Author

I did a few experiments with the python library itself and the situation is not better there. Before this can be tackled in ansible, the python apt lib must be extended fist. Closing this for the moment.

@Akasurde
Copy link
Member

@guenhter Do you think #77941 (comment) is better than nothing?

@guenhter
Copy link
Author

Although it doesn't contain the actual reason, it is definitely better then nothing.

Akasurde added a commit to Akasurde/ansible that referenced this issue May 23, 2024
@Akasurde
apt_*: add more info message while updating apt cache
504da67 
Fixes: ansible#77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Akasurde added a commit to Akasurde/ansible that referenced this issue May 23, 2024
@Akasurde
apt_*: add more info message while updating apt cache
0a7fe12 
Fixes: ansible#77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Akasurde added a commit that referenced this issue May 23, 2024
@Akasurde
apt_*: add more info message while updating apt cache (#83230)
3af9d0f 
Fixes: #77941

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
@ansible ansible locked and limited conversation to collaborators May 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
affects_2.14 feature This issue/PR relates to a feature request. has_pr This issue has an associated PR. module This issue/PR relates to a module. P3 Priority 3 - Approved, No Time Limitation support:core This issue/PR relates to code supported by the Ansible Engineering Team.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

apt_*: add more info message while updating apt cache Akasurde/ansible
5 participants
@goretkin @Akasurde @ansibot @jborean93 @guenhter