`ls` command is missing in etcdctl v3

[armstrongli]

The ls command is missing in etcdctl v3 APIs. Hope it can be back cause etcdctl stores is incompatible between v2 and v3.
e.g.

$ ./etcdctl set z 1
1
$ ETCDCTL_API=3 ./etcdctl get z
$ ./etcdctl get z
1

And after data migration, I can't find what the latest data is. Neither can I debug my business.

[xiang90]

You can use ETCDCTL_API=3 ./etcdctl get z --prefix to achieve similar goal. There is no directory in v3, thus no ls.

[armstrongli]

The ETCDCTL_API=3 ./etcdctl get z --prefix will show all keys and values which match the prefix. It's really not friendly to operators to check the keys in each level.
e.g.
I want to know how many third party resources created in kubernetes, I need to know the exact path of the keys' prefix. All the values are returned at the same time.

[xiang90]

I want to know how many third party resources created in kubernetes

You should use k8s tool set to do it, not etcdctl. etcd3 does not understand any path structure. we do not plan to add that feature in core etcd either.

[armstrongli]

We got this issue when we upgrade kubernetes from 1.2 to 1.3:
All third party resources should be rebuilt to match the none-namespace resources definition.
We need to delete the old resources from etcd directly and use new kubernetes API to rebuild those third party resources. The new kubernetes third party resource API can't get the old third party resources any more.
For such cases, we have to do operations directly on etcd as cluster operator.

[xiang90]

All third party resources should be rebuilt to match the none-namespace resources definition.
We need to delete the old resources from etcd directly and use new kubernetes API to rebuild those third party resources. The new kubernetes third party resource API can't get the old third party resources any more.

Open an issue in k8s repo. Upgrading etcd should not break any k8s feature.

[armstrongli]

Thanks :)

[Sartner]

You can use ETCDCTL_API=3 ./etcdctl get z --prefix to achieve similar goal. There is no directory in v3, thus no ls.

But how to list all keys ? @xiang90

[armstrongli]

@Sartner
ETCDCTL_API=3 ./etcdctl get / --prefix --keys-only

[gmcquillan]

I recognize that the underlying serialization and replication of data in etcd3 has very different semantics than it did with etcd2, however people still conceive of data as a hierarchical tree.

Removing the commands to explore that hierarchy from the core of command-line client seems like a mistake to me.

There's an additional 40 characters needed for one of the most common operations an administrator could possibly have in etcd -- and it's not obvious without a google search how to list keys as you might've done before.

Is there a reason to make the UX for this much, much harder to use? Is querying partial keys (i.e. using ls) really that unsafe?

[heyitsanthony]

@gmcquillan ls was not removed; it was never supported in v3 in the first place. Use the v2 backend or the v2 emulation layer if you want a directory hierarchy.

[gmcquillan]

@heyitsanthony Thanks for responding. Those are two good choices.

I'm just concerned that in v4, or v5, or v6 those kinds of feature bridges won't be available.

I want to communicate that, like the original author of this issue and the 30 people who've found it and thumbs-up'ed the answer, this change (between v2 and v3) kinda violates our expectations about the UX. If it were my project, I'd want to know.

Thanks.

[heyitsanthony]

@gmcquillan we're trying to support sane migration paths with data model emulation. v4 will likely be a cleanup of v3, it won't be as much of a conceptual leap. There's no ls in etcdctl v3 because it'd be very expensive to support with the v3 data model and having special support for it would reduce performance for applications that don't need it.

Since v3 was a major revision bump, going by semver, all bets are off in terms of expectations of command carry-over. This shouldn't come as a surprise.

[peterwillcn]

ETCDCTL_API=3 etcdctl --endpoints=http://localhost:2379 get / --prefix --keys-only

[SamuelMarks]

I've seen seriously hierarchical systems built on etcdv2; like DNS systems. So +1 for supporting hierarchies again.

because it'd be very expensive to support with the v3 data model and having special support for it would reduce performance for applications that don't need it.

@heyitsanthony - How about hiding it behind a compiler flag? - We're in the Go world here, so it's possible without any efficiency drop.

(just don't know if the etcd people want to maintain two paradigms; it's like having InnoDB and TokuDB in the MySQL world. Abstracting out the engines into those two then letting the communities split could be a long-term maintenance option…)