New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`ls` command is missing in etcdctl v3 #6904

Closed
armstrongli opened this Issue Nov 28, 2016 · 16 comments

Comments

9 participants
@armstrongli
Copy link

armstrongli commented Nov 28, 2016

The ls command is missing in etcdctl v3 APIs. Hope it can be back cause etcdctl stores is incompatible between v2 and v3.
e.g.

$ ./etcdctl set z 1
1
$ ETCDCTL_API=3 ./etcdctl get z
$ ./etcdctl get z
1

And after data migration, I can't find what the latest data is. Neither can I debug my business.

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Nov 28, 2016

You can use ETCDCTL_API=3 ./etcdctl get z --prefix to achieve similar goal. There is no directory in v3, thus no ls.

@xiang90 xiang90 closed this Nov 28, 2016

@armstrongli

This comment has been minimized.

Copy link
Author

armstrongli commented Nov 29, 2016

The ETCDCTL_API=3 ./etcdctl get z --prefix will show all keys and values which match the prefix. It's really not friendly to operators to check the keys in each level.
e.g.
I want to know how many third party resources created in kubernetes, I need to know the exact path of the keys' prefix. All the values are returned at the same time.

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Nov 29, 2016

I want to know how many third party resources created in kubernetes

You should use k8s tool set to do it, not etcdctl. etcd3 does not understand any path structure. we do not plan to add that feature in core etcd either.

@armstrongli

This comment has been minimized.

Copy link
Author

armstrongli commented Nov 29, 2016

We got this issue when we upgrade kubernetes from 1.2 to 1.3:
All third party resources should be rebuilt to match the none-namespace resources definition.
We need to delete the old resources from etcd directly and use new kubernetes API to rebuild those third party resources. The new kubernetes third party resource API can't get the old third party resources any more.
For such cases, we have to do operations directly on etcd as cluster operator.

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Nov 29, 2016

All third party resources should be rebuilt to match the none-namespace resources definition.
We need to delete the old resources from etcd directly and use new kubernetes API to rebuild those third party resources. The new kubernetes third party resource API can't get the old third party resources any more.

Open an issue in k8s repo. Upgrading etcd should not break any k8s feature.

@armstrongli

This comment has been minimized.

Copy link
Author

armstrongli commented Nov 29, 2016

Thanks :)

@Sartner

This comment has been minimized.

Copy link

Sartner commented Jan 21, 2017

You can use ETCDCTL_API=3 ./etcdctl get z --prefix to achieve similar goal. There is no directory in v3, thus no ls.

But how to list all keys ? @xiang90

@armstrongli

This comment has been minimized.

Copy link
Author

armstrongli commented Jan 22, 2017

@Sartner
ETCDCTL_API=3 ./etcdctl get / --prefix --keys-only

@gmcquillan

This comment has been minimized.

Copy link

gmcquillan commented Sep 14, 2017

I recognize that the underlying serialization and replication of data in etcd3 has very different semantics than it did with etcd2, however people still conceive of data as a hierarchical tree.

Removing the commands to explore that hierarchy from the core of command-line client seems like a mistake to me.

There's an additional 40 characters needed for one of the most common operations an administrator could possibly have in etcd -- and it's not obvious without a google search how to list keys as you might've done before.

Is there a reason to make the UX for this much, much harder to use? Is querying partial keys (i.e. using ls) really that unsafe?

@heyitsanthony

This comment has been minimized.

Copy link
Contributor

heyitsanthony commented Sep 14, 2017

@gmcquillan ls was not removed; it was never supported in v3 in the first place. Use the v2 backend or the v2 emulation layer if you want a directory hierarchy.

@gmcquillan

This comment has been minimized.

Copy link

gmcquillan commented Sep 14, 2017

@heyitsanthony Thanks for responding. Those are two good choices.

I'm just concerned that in v4, or v5, or v6 those kinds of feature bridges won't be available.

I want to communicate that, like the original author of this issue and the 30 people who've found it and thumbs-up'ed the answer, this change (between v2 and v3) kinda violates our expectations about the UX. If it were my project, I'd want to know.

Thanks.

@heyitsanthony

This comment has been minimized.

Copy link
Contributor

heyitsanthony commented Sep 14, 2017

@gmcquillan we're trying to support sane migration paths with data model emulation. v4 will likely be a cleanup of v3, it won't be as much of a conceptual leap. There's no ls in etcdctl v3 because it'd be very expensive to support with the v3 data model and having special support for it would reduce performance for applications that don't need it.

Since v3 was a major revision bump, going by semver, all bets are off in terms of expectations of command carry-over. This shouldn't come as a surprise.

@peterwillcn

This comment has been minimized.

Copy link

peterwillcn commented Sep 16, 2017

ETCDCTL_API=3 etcdctl --endpoints=http://localhost:2379 get / --prefix --keys-only

@SamuelMarks

This comment has been minimized.

Copy link
Contributor

SamuelMarks commented Nov 30, 2017

I've seen seriously hierarchical systems built on etcdv2; like DNS systems. So +1 for supporting hierarchies again.

because it'd be very expensive to support with the v3 data model and having special support for it would reduce performance for applications that don't need it.

@heyitsanthony - How about hiding it behind a compiler flag? - We're in the Go world here, so it's possible without any efficiency drop.

(just don't know if the etcd people want to maintain two paradigms; it's like having InnoDB and TokuDB in the MySQL world. Abstracting out the engines into those two then letting the communities split could be a long-term maintenance option…)

@josue

This comment has been minimized.

Copy link

josue commented Dec 4, 2017

On environment:

  • macOS v10.13.1 (High Sierra)
  • etcd v3.2.10 (install via brew install etcd)

The command get / --prefix will return an empty response. Instead you can do the following:

ETCDCTL_API=3 etcdctl get "" --prefix --keys-only | sed '/^\s*$/d'

^ using the empty quote will returns all keys then pipe it to removes empty lines.

@tantjp

This comment has been minimized.

Copy link

tantjp commented Jul 24, 2018

@xiang90 Hi, I would like to get all keys by rest api, can you do me a favor?thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment