Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[4.8] libct/cg/sd: reconnect and retry on dbus connection error #50

Closed
wants to merge 5 commits into from

Conversation

kolyshkin
Copy link
Collaborator

This is a backport of upstream PR opencontainers/runc#2923.

It is mostly straightforward, only a few minor merge conflicts due to missing upstream commits:

Original description follows.


In case the dbus daemon is ever restarted, the connection is no longer valid and every operation
fails. This is a minor concern for short-lived runc, but much more of a issue in case there is
a long-running daemon (e.g. cri-o) is using runc's libcontainer, as the connection is never
retried and the only remedy is to restart the daemon.

The solution to the above is to check the errors returned for dbus: connection closed by user
error, and try to re-connect on that. This is what PR #2862 does.

This is a carry of #2862, implementing the idea of retry-in-place (first described
at opencontainers/runc#2862 (comment) and opencontainers/runc#2862 (comment)) on top of what it does.

For more info, see commit messages as well as #2862.

Fixes:

wzshiming and others added 5 commits April 29, 2021 16:53
[@kolyshkin: documentation nits]

Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit cdbed6f)

[minor merge conflict due to missing upstream commit 73f22e7]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Generalize isUnitExists as isDbusError, and use errors.As while at it
(which can handle wrapped errors as well).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit bacfc2c)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
[@kolyshkin: doc nits, use dbus.ErrClosed and isDbusError]

Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 15fee98)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Shiming Zhang <wzshiming@foxmail.com>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 6122bc8)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Instead of reconnecting to dbus after some failed operations, and
returning an error (so a caller has to retry), reconnect AND retry
in place for all such operations.

This should fix issues caused by a stale dbus connection after e.g.
a dbus daemon restart.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
(cherry picked from commit 47ef9a1)

[Minor merge conflicts due to missing upstream commits
52390d6 and af521ed.]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
@kolyshkin kolyshkin changed the base branch from master to rhaos-4.8 April 30, 2021 00:04
@kolyshkin
Copy link
Collaborator Author

wrong repo 🤦🏻

@kolyshkin kolyshkin closed this Apr 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants