backup object seems to be missing actual kv pairs

[rusenask]

Trying to use backup operator but it doesn't seem to be getting any data extracted into the backup object.

Here are our bucket contents:

Currently there are quite a lot of objects in etcd:

/ # etcdctl ls -r /dotmesh.io | wc -l
9629

Backup operator logs:

➜  etcd-operator git:(master) ✗ kubectl logs etcd-backup-operator-5758d6f855-m8n2k -f
time="2018-12-05T13:22:57Z" level=info msg="Go Version: go1.10"
time="2018-12-05T13:22:57Z" level=info msg="Go OS/Arch: linux/amd64"
time="2018-12-05T13:22:57Z" level=info msg="etcd-backup-operator Version: 0.9.3+git"
time="2018-12-05T13:22:57Z" level=info msg="Git SHA: b3b2f83"
time="2018-12-05T13:23:14Z" level=info msg="Event(v1.ObjectReference{Kind:\"Endpoints\", Namespace:\"dotmesh\", Name:\"etcd-backup-operator\", UID:\"3a48523f-e439-11e8-94c8-42010a9a0132\", APIVersion:\"v1\", ResourceVersion:\"48584655\", FieldPath:\"\"}): type: 'Normal' reason: 'LeaderElection' etcd-backup-operator-5758d6f855-m8n2k became leader"
time="2018-12-05T13:23:14Z" level=info msg="starting backup controller" pkg=controller
time="2018-12-05T13:24:09Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster-client:2379 revision (1)"
time="2018-12-05T13:30:10Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster-client:2379 revision (1)"
time="2018-12-05T13:52:08Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster-client:2379 revision (1)"
time="2018-12-05T13:57:59Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster:2379 revision (1)"
time="2018-12-05T14:00:09Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster-client:2379 revision (1)"
time="2018-12-05T14:30:11Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster-client:2379 revision (1)"
time="2018-12-05T15:00:11Z" level=info msg="getMaxRev: endpoint http://dotmesh-etcd-cluster-client:2379 revision (1)"

While the backup object is binary, I can see via cat that it only contains peer and client URLs of the members.

[hexfusion]

Currently there are quite a lot of objects in etcd:
/ # etcdctl ls -r /dotmesh.io | wc -l
9629

etcdctl ls is against the etcd v2 store backup operator is backing up and restoring v3 store.

What is output of

ETCDCTL_API=3 etcdctl get --prefix=true "" /dotmesh.io | wc -l

[rusenask]

hmmm, so this is the situation:

/ # etcdctl ls -r /dotmesh.io | wc -l
9629
/ # ETCDCTL_API=3 etcdctl get --prefix=true "" /dotmesh.io | wc -l
Error:  too many arguments, only accept one argument when `--prefix` or `--from-key` is set.
0
/ # ETCDCTL_API=3 etcdctl get --prefix=true /dotmesh.io | wc -l
0
/ # ETCDCTL_API=3 etcdctl get --prefix=true "/dotmesh.io" | wc -l
0
/ # ETCDCTL_API=3 etcdctl get --prefix=true "" | wc -l
0

I suspect that the project uses v2 API then, will need to investigate :|

[hexfusion]

I suspect that the project uses v2 API then, will need to investigate :|

Right, I am going to close this as the issue seems to be v2 data which is not supported. If you find this is not the case or have additional information please add to this issue and we can reopen if necessary.