Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server temporarily suspended and 4 months of data lost #5823

Open
PCPisChill opened this issue Feb 2, 2019 · 1 comment
Open

Server temporarily suspended and 4 months of data lost #5823

PCPisChill opened this issue Feb 2, 2019 · 1 comment

Comments

@PCPisChill
Copy link

PCPisChill commented Feb 2, 2019

2 days ago my host suspended my server for having "20k+ UNREPLIED conntrack sessions", when I contacted them they unsuspected it and I logged in to the server to find it rebooted. I started everything up, and noticed that the data redis was using was almost 4 months old (from when I first setup the server).

I had the default save config

save 900 1
save 300 10
save 60 10000

appendonly was off, but I've since turned it on with appendfsync everysec. locate *.rdb only finds the /var/lib/redis/dump.rdb and locate *.aof only finds the newly created /var/lib/redis/appendonly.aof

Checking the logs from a week ago up until 2 days ago when the server was suspended, I don't see any DB saved on disk in the logs like I do in my most recent one after re-setting everything up, it consists entirely of this:

10899:S 20 Jan 01:02:03.578 # Error condition on socket for SYNC: Connection refused
10899:S 20 Jan 01:02:04.434 * Connecting to MASTER IP:PORT
10899:S 20 Jan 01:02:04.435 * MASTER <-> SLAVE sync started
10899:S 20 Jan 01:02:04.583 # Error condition on socket for SYNC: Connection refused
10899:S 20 Jan 01:02:05.444 * Connecting to MASTER IP:PORT
10899:S 20 Jan 01:02:05.444 * MASTER <-> SLAVE sync started

with 1 10899:S 28 Jan 00:08:40.426 # Timeout connecting to the MASTER...

When re-setting everything up I did re-chown and re-chmod /var/lib/redis and dump.rdb so maybe that was the issue? or could have it of been something else?

@ArkayZheng
Copy link
Contributor

Hello @PCPisChill
1 It seems like your slave node have problem connecting your master node. So please check the slaveof IP:PORT and masterauth in redis.conf on your slave node.
2 The log information DB saved on disk will not show out at once when persistence is triggered. You can set loglevel debug for more information or read the source code: int rdbSave in rdb.c.
3 Usually persistence can be triggered by these items: new data\MASTER <-> SLAVE sync\exit in the correct way. So make sure that you have done at least one of them successfully.
Hope these would help a little.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants