New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Galera reads non-existent values from empty gvwstate.dat #354

Open
GeoffMontee opened this Issue Sep 29, 2015 · 5 comments

Comments

Projects
None yet
6 participants
@GeoffMontee

GeoffMontee commented Sep 29, 2015

A server somehow ended up with an empty gvwstate.dat file. When the MySQL service was restarted, Galera seemed to believe that it successfully restored the viewstate from the file. The viewstate is obviously invalid, so Galera fails with a non-intuitive error "invalid UUID: 00000000."

For example, let's truncate gvwstate.dat and then attempt to start MySQL:

[ec2-user@ip-172-31-31-73 ~]$ sudo cp /dev/null /var/lib/mysql/gvwstate.dat
[ec2-user@ip-172-31-31-73 ~]$ sudo service mysql start
Starting MySQL... ERROR!

Then the error log shows:

150929 11:51:36 [Note] WSREP: restore pc from disk successfully
150929 11:51:36 [Note] WSREP: GMCast version 0
150929 11:51:36 [Note] WSREP: (00000000, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
150929 11:51:36 [Note] WSREP: (00000000, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
150929 11:51:36 [ERROR] WSREP: failed to open gcomm backend connection: 131: invalid UUID: 00000000 (FATAL)
         at gcomm/src/pc.cpp:PC():309
150929 11:51:36 [ERROR] WSREP: gcs/src/gcs_core.cpp:long int gcs_core_open(gcs_core_t*, const char*, const char*, bool)():206: Failed to open backend connection: -131 (State not recoverable)
150929 11:51:36 [ERROR] WSREP: gcs/src/gcs.cpp:long int gcs_open(gcs_conn_t*, const char*, const char*, bool)():1379: Failed to open channel 'my_wsrep_cluster' at 'gcomm://172.31.31.72,172.31.31.73,172.31.20.230': -131 (State not recoverable)
150929 11:51:36 [ERROR] WSREP: gcs connect failed: State not recoverable
150929 11:51:36 [ERROR] WSREP: wsrep::connect() failed: 7
150929 11:51:36 [ERROR] Aborting
@wk8

This comment has been minimized.

Show comment
Hide comment
@wk8

wk8 Oct 7, 2015

Just hit that too.

wk8 commented Oct 7, 2015

Just hit that too.

@zeki893

This comment has been minimized.

Show comment
Hide comment
@zeki893

zeki893 Jan 11, 2016

ya i just hit it. i renamed gvwstate.dat to gvwstate.dat.bak and restarted mysql. it joined the cluster after that.

zeki893 commented Jan 11, 2016

ya i just hit it. i renamed gvwstate.dat to gvwstate.dat.bak and restarted mysql. it joined the cluster after that.

@dhivin94

This comment has been minimized.

Show comment
Hide comment
@dhivin94

dhivin94 Dec 22, 2016

go to

cd /var/lib/mysql

sudo mv  grastate.bat    grastate.bat.bck

sudo mv  gvwstate.bat   gvwstate.bat.bck

sudo systemctl start mysql

check log for more info

dhivin94 commented Dec 22, 2016

go to

cd /var/lib/mysql

sudo mv  grastate.bat    grastate.bat.bck

sudo mv  gvwstate.bat   gvwstate.bat.bck

sudo systemctl start mysql

check log for more info

@AhmadFazliIsmail

This comment has been minimized.

Show comment
Hide comment
@AhmadFazliIsmail

AhmadFazliIsmail May 24, 2017

Just to make correction on your comment;

`cd /var/lib/mysql

sudo mv grastate.bat grastate.bat.bck

sudo mv gvwstate.bat gvwstate.bat.bck`

You wrote grastate.bat and gvwstate.bat , but the actual filenames are grastate.dat and gvwstate.dat

AhmadFazliIsmail commented May 24, 2017

Just to make correction on your comment;

`cd /var/lib/mysql

sudo mv grastate.bat grastate.bat.bck

sudo mv gvwstate.bat gvwstate.bat.bck`

You wrote grastate.bat and gvwstate.bat , but the actual filenames are grastate.dat and gvwstate.dat

@dhivin94

This comment has been minimized.

Show comment
Hide comment
@dhivin94

dhivin94 May 24, 2017

thanks for your correction

dhivin94 commented May 24, 2017

thanks for your correction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment