Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix attrd race condition when shutting down #3178

Merged
merged 4 commits into from
Aug 3, 2023

Conversation

kgaillot
Copy link
Contributor

@kgaillot kgaillot commented Aug 3, 2023

No description provided.

@kgaillot
Copy link
Contributor Author

kgaillot commented Aug 3, 2023

@clumens, can you review this when you get a chance? Thanks

This addresses a race condition that can occur when the DC and the attribute
writer are different nodes, and shutting down at the same time. When the DC
controller leaves its Corosync process group, the remaining nodes erase its
transient node attributes (including "shutdown") from the CIB. However if the
(former) DC's attrd is still up, it can win the attribute writer election
called after the original writer leaves. As the election winner, it writes out
all its attributes to the CIB, including "shutdown". The next time it rejoins
the cluster, it will be immediately shut down.

Fixes T138
This makes it more obvious that each caller has considered whether to include
shutdown requests in its check.
@kgaillot
Copy link
Contributor Author

kgaillot commented Aug 3, 2023

Updated for review and threw one more commit on top

tools/attrd_updater.c Outdated Show resolved Hide resolved
@clumens
Copy link
Contributor

clumens commented Aug 3, 2023

Patches look good. ctslab is looking good through 65 runs, including multiple NearQuorumPoint runs. I have it set to run 150 tests, so I guess we'll know tomorrow.

@kgaillot kgaillot merged commit 52694e6 into ClusterLabs:main Aug 3, 2023
1 check was pending
@kgaillot kgaillot deleted the attrd branch August 3, 2023 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants