Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves tolerance of db connection failures in pulpcore-content #1701

Merged
merged 2 commits into from
Nov 5, 2021

Conversation

dkliban
Copy link
Member

@dkliban dkliban commented Oct 27, 2021

This patch moves Handler._reset_db_connection() call to get_base_paths_blocking(). This ensures that the
connection is reset inside the same thread that is used to perform ORM calls. This fixes a bug where the
/pulp/content/ page would return a 500 error after the connection the database is closed due to a network
problem or a database restart.

A call to Handler._reset_db_connection() has also been added to the _heartbeat() co-routine. This ensures
that the heartbeat continues to be recorded in the database even if the db connection was closed.

fixes: #9515
https://pulp.plan.io/issues/9515

@pulpbot
Copy link
Member

pulpbot commented Oct 27, 2021

Attached issue: https://pulp.plan.io/issues/9515

[noissue]
@dkliban dkliban changed the title Moves Handler._reset_db_connection() call to get_base_paths_blocking() Improves tolerance of db connection failures in pulpcore-content Nov 4, 2021
This patch moves Handler._reset_db_connection() call to get_base_paths_blocking(). This ensures that the
connection is reset inside the same thread that is used to perform ORM calls. This fixes a bug where the
/pulp/content/ page would return a 500 error after the connection the database is closed due to a network
problem or a database restart.

A call to Handler._reset_db_connection() has also been added to the _heartbeat() co-routine. This ensures
that the heartbeat continues to be recorded in the database even if the db connection was closed.

fixes: #9515
https://pulp.plan.io/issues/9515
class MockException(Exception):
pass

mock_get_or_create.side_effect = [InterfaceError(), OperationalError(), MockException()]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The triplet!

Copy link
Member

@bmbouter bmbouter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Thanks!

Copy link
Member

@goosemania goosemania left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@dkliban dkliban merged commit 56f5e38 into pulp:3.14 Nov 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants