New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support backup of v2 and v3 stores #7002

Closed
heyitsanthony opened this Issue Dec 14, 2016 · 5 comments

Comments

3 participants
@heyitsanthony
Contributor

heyitsanthony commented Dec 14, 2016

Right now etcd backups must choose between v2's backup and v3's snapshot. For systems that store v2 and v3 data at once, there is no way to fully back up the cluster. This behavior is somewhat unexpected since presumably a backup of the cluster should restore all the keys that were on the system.

@raoofm

This comment has been minimized.

Contributor

raoofm commented Mar 30, 2017

I think this should be a really high priority rather than just an enhancement. It puts all the existing v2 users who moved to v3 without migrating data (or with migrating data but not all clients migrated yet) at a huge risk.

@r7vme

This comment has been minimized.

r7vme commented Jul 28, 2017

i've just experimented, after reading blog post from @mattymo , and found out that v2 backup actually contains both v2 and v3 data.

I have etcd 3.2.0-rc.0+git

To check

Start etcd

mkdir /tmp/etcd-data
etcd -debug -data-dir /tmp/etcd-data

Create v2 and v3 data

for i in `seq 1 10`; do curl http://127.0.0.1:2379/v2/keys/message$i -XPUT -d value="Hello world"; done
# create v3 foo bar (base64 encoded)
curl -L http://127.0.0.1:2379/v3alpha/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'

Create backup

mkdir /tmp/etcd-backup
etcdctl backup --data-dir /tmp/etcd-data --backup-dir /tmp/etcd-backup

Stop existing etcd and start new one from backup

etcd -debug -data-dir /tmp/etcd-backup --force-new-cluster

Check values

~|⇒ etcdctl ls                                                 
/message4
/message5
/message8
/message9
/message2
/message3
/message7
/message10
/message1
/message6
~|⇒ ETCDCTL_API=3 etcdctl get "" --prefix=true 
foo
bar

Is this smth unexpected?

@heyitsanthony

This comment has been minimized.

Contributor

heyitsanthony commented Jul 28, 2017

@r7vme there's probably v3 entries in the wal that are being copied over and then replayed back into the v3 store. It's not an actual backup

@raoofm

This comment has been minimized.

Contributor

raoofm commented Aug 1, 2017

@xiang90 @heyitsanthony Should this issue be closed in favor of #6925. Then the users can migrate v2 data to v3 and still support v2 clients. So the current problem of taking both v2 and v3 backup as a whole can be dropped as we would then need only v3 backup.

@heyitsanthony

This comment has been minimized.

Contributor

heyitsanthony commented Sep 5, 2017

Fixed by #8479

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment