Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node consistency compromised during shutdown #353

Closed
mkaruza opened this issue Jun 14, 2019 · 1 comment
Closed

Node consistency compromised during shutdown #353

mkaruza opened this issue Jun 14, 2019 · 1 comment

Comments

@mkaruza
Copy link
Contributor

mkaruza commented Jun 14, 2019

MySQL 5.7 shutdown procedure does not wait till all applier threads are closed. If applier is applying write set during initiated shutdown it could happen that node consistency is compromised because write set is not applied resulting in node abort.

...
2019-05-29T05:00:35.889463Z 0 [Note] Shutting down slave threads
2019-05-29T05:00:36.145648Z 2 [Warning] WSREP: Failed to apply app buffer: seqno: 547980330, status: 1
at galera/src/trx_handle.cpp:apply():353
Retrying 2th time
2019-05-29T05:00:36.145671Z 2 [Note] WSREP: applier has been aborted, skipping apply_rbr: 547980330
2019-05-29T05:00:36.176279Z 2 [Warning] WSREP: Failed to apply app buffer: seqno: 547980330, status: 1
at galera/src/trx_handle.cpp:apply():353
Retrying 3th time
2019-05-29T05:00:36.176302Z 2 [Note] WSREP: applier has been aborted, skipping apply_rbr: 547980330
2019-05-29T05:00:36.251995Z 2 [Warning] WSREP: Failed to apply app buffer: seqno: 547980330, status: 1
at galera/src/trx_handle.cpp:apply():353
Retrying 4th time
2019-05-29T05:00:36.252027Z 2 [Note] WSREP: applier has been aborted, skipping apply_rbr: 547980330
2019-05-29T05:00:36.369721Z 2 [ERROR] WSREP: Failed to apply trx: source: 8ec31469-8109-11e9-9595-3a236f1a3e8c version: 4 local: 0 state: APPLYING flags: 1 conn_id: 1951034 trx_id: 140947703 seqnos (l: 41027348, g: 547980330, s: 547980135, d: 547980098, ts: 10797851157436590)
2019-05-29T05:00:36.369780Z 2 [ERROR] WSREP: Failed to apply trx 547980330 4 times
2019-05-29T05:00:36.369795Z 2 [ERROR] WSREP: Node consistency compromised, aborting...
2019-05-29T05:00:36.369814Z 2 [Note] WSREP: /usr/sbin/mysqld: Terminated.
...

@mkaruza mkaruza self-assigned this Jun 14, 2019
@mkaruza
Copy link
Contributor Author

mkaruza commented Jul 19, 2019

This should be fixed in upcoming 5.7.26-25.18

@mkaruza mkaruza closed this as completed Jul 19, 2019
@temeo temeo added this to the 5.6.44-25.26 5.7.26-25.18 milestone Jul 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants