Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: Add Etcd v3 protocol support via api gRPC-gateway #1162

Open
wants to merge 12 commits into
base: master
from

Conversation

@CyberDem0n
Copy link
Member

CyberDem0n commented Sep 2, 2019

The only python-etcd3 client working directly via gRPC still supports only a single endpoint, what is not very nice for high-availability.

Since Patroni is already using a heavily hacked version of python-etcd with smart retries and auto-discovery out-of-the-box, I decided to enhance the existing code with limited support of v3 protocol via gRPC-gateway.

Currently it passes all integration tests (behave). Unit tests are missing.

Unfortunately, watches via gRPC-gateway requires us to open and keep the second connection to the etcd.

Known limitations:

  • The very minimal supported version is 3.0.4. On earlier versions transactions don't work due to bugs in grpc-gateway. Without transactions we can't do atomic operations, i.e. leader locks.
  • Watches work only starting from 3.1.0
  • Authentication works only starting from 3.3.0
  • gRPC-gateway does not support authentication using TLS Common Name. This is because gRPC-proxy terminates TLS from its client so all the clients share a cert of the proxy: https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/authentication.md#using-tls-common-name
CyberDem0n added 7 commits Sep 2, 2019
The only python-etcd3 client working directly via gRPC still supports
only a single endpoint, what is not very nice for high-availability.

Since Patroni is already using a heavily hacked version of python-etcd
with smart retries and auto-discovery out-of-the-box, I decided to
enhance the existing code with a limited support of v3 protocol via
grpc-gateway.

Currently it passes all integration tests (behave). Unit tests are
missing.  Watch support is also missing. I.e. if the leader lease/key
disappeared from the Etcd, nodes will not be notified immediately, but
will learn about this fact only when running the next HA loop.
Effectively that means that the failover will be delayed up to
`loop_wait` seconds.
Implementing the watch support should not be hard, but due to a
streaming nature it will require a separate connection to the Etcd.
P.S. watch requests via grpc-gateway are broken on 3.0.X, but this is
probably not a big deal, since the current version is 3.4.0
it is possible to use it only starting from 3.3.0
@CyberDem0n CyberDem0n changed the title POC: Add Etcd v3 protocol support via api grpc-gateway POC: Add Etcd v3 protocol support via api gRPC-gateway Sep 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
1 participant
You can’t perform that action at this time.