Previous change logs can be found at CHANGELOG-3.4.
The minimum recommended etcd versions to run in production are 3.2.28+, 3.3.18+, and 3.4.2+.
See code changes and v3.5 upgrade guide for any breaking changes.
- v3.5.0 (2020 TBD), see code changes.
- v3.5.0-rc.1 (2020 TBD), see code changes.
- v3.5.0-rc.0 (2020 TBD), see code changes.
Again, before running upgrades from any previous release, please make sure to read change logs below and v3.5 upgrade guide.
go.etcd.io/etcd
Go packages have moved togo.etcd.io/etcd/v3
to follow the Go modules conventions- Changed behavior of clienv3 API MemberList.
- Previously, it is directly served with server's local data, which could be stale.
- Now, it is served with linearizable guarantee. If the server is disconnected from quorum,
MemberList
call will fail.
- gRPC gateway only supports
/v3
endpoint.- Deprecated
/v3beta
. curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'
does work in v3.5. Usecurl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'
instead.
- Deprecated
etcd --experimental-enable-v2v3
flag has been deprecated. Useetcd --enable-v2v3
instead.- Change
etcd --experimental-enable-v2v3
flag toetcd --enable-v2v3
; v2 storage emulation is now stable.
- Change
etcd --experimental-backend-bbolt-freelist-type
flag has been deprecated. Useetcd --backend-bbolt-freelist-type
instead. The default type is hashmap and it is stable now.etcd --debug
flag has been deprecated. Useetcd --log-level=debug
instead.- Remove
embed.Config.Debug
. etcd --log-output
flag has been deprecated. Useetcd --log-outputs
instead.etcd --logger=zap --log-outputs=stderr
is now the default.etcd --logger=capnslog
flag value has been deprecated.etcd --logger=zap --log-outputs=default
flag value is not supported..- Use
etcd --logger=zap --log-outputs=stderr
. - Or, use
etcd --logger=zap --log-outputs=systemd/journal
to send logs to the local systemd journal. - Previously, if etcd parent process ID (PPID) is 1 (e.g. run with systemd),
etcd --logger=capnslog --log-outputs=default
redirects server logs to local systemd journal. And if write to journald fails, it writes toos.Stderr
as a fallback. - However, even with PPID 1, it can fail to dial systemd journal (e.g. run embedded etcd with Docker container). Then, every single log write will fail and fall back to
os.Stderr
, which is inefficient. - To avoid this problem, systemd journal logging must be configured manually.
- Use
etcd --log-outputs=stderr
is now the default.etcd --log-package-levels
flag forcapnslog
has been deprecated. Now,etcd --logger=zap --log-outputs=stderr
is the default.[CLIENT-URL]/config/local/log
endpoint has been deprecated, as isetcd --log-package-levels
flag.curl http://127.0.0.1:2379/config/local/log -XPUT -d '{"Level":"DEBUG"}'
won't work.- Please use
etcd --logger=zap --log-outputs=stderr
instead.
- Deprecated
etcd_debugging_mvcc_db_total_size_in_bytes
Prometheus metric. Useetcd_mvcc_db_total_size_in_bytes
instead. - Deprecated
etcd_debugging_mvcc_put_total
Prometheus metric. Useetcd_mvcc_put_total
instead. - Deprecated
etcd_debugging_mvcc_delete_total
Prometheus metric. Useetcd_mvcc_delete_total
instead. - Deprecated
etcd_debugging_mvcc_txn_total
Prometheus metric. Useetcd_mvcc_txn_total
instead. - Deprecated
etcd_debugging_mvcc_range_total
Prometheus metric. Useetcd_mvcc_range_total
instead. - Master branch
/version
outputs3.5.0-pre
, instead of3.4.0+git
. - Changed
proxy
package function signature to support structured logger.- Previously,
NewClusterProxy(c *clientv3.Client, advaddr string, prefix string) (pb.ClusterServer, <-chan struct{})
, nowNewClusterProxy(lg *zap.Logger, c *clientv3.Client, advaddr string, prefix string) (pb.ClusterServer, <-chan struct{})
. - Previously,
Register(c *clientv3.Client, prefix string, addr string, ttl int)
, nowRegister(lg *zap.Logger, c *clientv3.Client, prefix string, addr string, ttl int) <-chan struct{}
. - Previously,
NewHandler(t *http.Transport, urlsFunc GetProxyURLs, failureWait time.Duration, refreshInterval time.Duration) http.Handler
, nowNewHandler(lg *zap.Logger, t *http.Transport, urlsFunc GetProxyURLs, failureWait time.Duration, refreshInterval time.Duration) http.Handler
.
- Previously,
- Changed
pkg/flags
function signature to support structured logger.- Previously,
SetFlagsFromEnv(prefix string, fs *flag.FlagSet) error
, nowSetFlagsFromEnv(lg *zap.Logger, prefix string, fs *flag.FlagSet) error
. - Previously,
SetPflagsFromEnv(prefix string, fs *pflag.FlagSet) error
, nowSetPflagsFromEnv(lg *zap.Logger, prefix string, fs *pflag.FlagSet) error
.
- Previously,
- Changed behavior on existing dir permission.
- Previously, the permission was not checked on existing data directory and the directory used for automatically generating self-signed certificates for TLS connections with clients. Now a check is added to make sure those directories, if already exist, has a desired permission of 700 on Linux and 777 on Windows.
- Add
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
andTLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
toetcd --cipher-suites
. - Changed the format of WAL entries related to auth for not keeping password as a plain text.
- Add third party Security Audit Report.
See List of metrics for all metrics per release.
Note that any etcd_debugging_*
metrics are experimental and subject to change.
- Deprecated
etcd_debugging_mvcc_db_total_size_in_bytes
Prometheus metric. Useetcd_mvcc_db_total_size_in_bytes
instead. - Deprecated
etcd_debugging_mvcc_put_total
Prometheus metric. Useetcd_mvcc_put_total
instead. - Deprecated
etcd_debugging_mvcc_delete_total
Prometheus metric. Useetcd_mvcc_delete_total
instead. - Deprecated
etcd_debugging_mvcc_txn_total
Prometheus metric. Useetcd_mvcc_txn_total
instead. - Deprecated
etcd_debugging_mvcc_range_total
Prometheus metric. Useetcd_mvcc_range_total
instead. - Add
etcd_debugging_mvcc_current_revision
Prometheus metric. - Add
etcd_debugging_mvcc_compact_revision
Prometheus metric. - Change
etcd_cluster_version
Prometheus metrics to include only major and minor version. - Add
etcd_debugging_mvcc_total_put_size_in_bytes
Prometheus metric. - Add
etcd_server_client_requests_total
with"type"
and"client_api_version"
labels. - Add
etcd_wal_write_bytes_total
. - Add
etcd_debugging_auth_revision
. - Add
os_fd_used
andos_fd_limit
to monitor current OS file descriptors.
etcd --enable-v2v3
flag is now stable.etcd --experimental-enable-v2v3
has been deprecated.- Added more v2v3 integration tests.
etcd --enable-v2=true --enable-v2v3=''
by default, to enable v2 API server that is backed by v2 store.etcd --enable-v2=true --enable-v2v3=/aaa
to enable v2 API server that is backed by v3 storage.etcd --enable-v2=false --enable-v2v3=''
to disable v2 API server.etcd --enable-v2=false --enable-v2v3=/aaa
to disable v2 API server. TODO: error?- Add
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
andTLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
toetcd --cipher-suites
. - Automatically create parent directory if it does not exist (fix issue#9609).
- v4.0 will configure
etcd --enable-v2=true --enable-v2v3=/aaa
to enable v2 API server that is backed by v3 storage.
- [
etcd --backend-bbolt-freelist-type
] flag is now stable.etcd --experimental-backend-bbolt-freelist-type
has been deprecated.
- Support rollback/downgrade.
- Deprecate v2 apply on cluster version. Use v3 request to set cluster version and recover cluster version from v3 backend.
- Fix corruption bug in defrag.
- Fix quorum protection logic when promoting a learner.
- Improve peer corruption checker to work when peer mTLS is enabled.
- Log
[CLIENT-PORT]/health
check in server side. - Improve compaction performance when latest index is greater than 1-million.
- Refactor consistentindex.
- Add log when etcdserver failed to apply command.
- Improve count-only range performance.
- Remove redundant storage restore operation to shorten the startup time.
- With 40 million key test data,it can shorten the startup time from 5 min to 2.5 min.
- Fix deadlock bug in mvcc.
- Fix inconsistency between WAL and server snapshot.
- Previously, server restore fails if it had crashed after persisting raft hard state but before saving snapshot.
- See etcd-io#10219 for more.
- Add missing CRC checksum check in WAL validate method otherwise causes panic.
- See etcd-io#11918.
- Improve logging around snapshot send and receive.
- Push down RangeOptions.limit argv into index tree to reduce memory overhead.
- Add reason field for /health response.
- Add
--unsafe-no-fsync
flag.- Setting the flag disables all uses of fsync, which is unsafe and will cause data loss. This flag makes it possible to run an etcd node for testing and development without placing lots of load on the file system.
- Add etcd --auth-token-ttl flag to customize
simpleTokenTTL
settings. - Improve
runtime.FDUsage
call pattern to reduce objects malloc of Memory Usage and CPU Usage. - Improve mvcc.watchResponse channel Memory Usage.
- Log expensive request info in UnaryInterceptor.
- Fix invalid Go type in etcdserverpb.
- Improve healthcheck by using v3 range request and its corresponding timeout.
- Add
etcd --experimental-watch-progress-notify-interval
flag to make watch progress notify interval configurable. - Fix server panic in slow writes warnings.
- Fixed via PR#12238.
- Remove
embed.Config.Debug
.- Use
embed.Config.LogLevel
instead.
- Use
- Add
embed.Config.ZapLoggerBuilder
to allow creating a custom zap logger. - Replace global
*zap.Logger
with etcd server logger object.
- Remove excessive watch cancel logging messages.
- Add
TryLock
method toclientv3/concurrency/Mutex
. A non-blocking method onMutex
which does not wait to get lock on the Mutex, returns immediately if Mutex is locked by another session. - Fix client balancer failover against multiple endpoints.
- Fix IPv6 endpoint parsing in client.
- Fix errors caused by grpc changing balancer/resolver API. This change is compatible with grpc >= v1.26.0, but is not compatible with < v1.26.0 version.
- Use ServerName as the authority after bumping to grpc v1.26.0. Remove workaround in #11184.
- Fix
"hasleader"
metadata embedding.- Previously,
clientv3.WithRequireLeader(ctx)
was overwriting existing context keys.
- Previously,
- Fix watch leak caused by lazy cancellation. When clients cancel their watches, a cancel request will now be immediately sent to the server instead of waiting for the next watch event.
- Make sure save snapshot downloads checksum for integrity checks.
- Fix memory leak in follower nodes.
- Make sure grant/revoke won't be applied repeatedly after restarting etcd.
- Add
etcd_wal_write_bytes_total
. - Handle out-of-range slice bound in
ReadAll
and entry limit indecodeRecord
.
- Fix
etcdctl member add
command to prevent potential timeout. (PR#11194 and PR#11638) - Add
etcdctl watch --progress-notify
flag. - Add
etcdctl auth status
command to check if authentication is enabled - Add
etcdctl get --count-only
flag for output typefields
. - Add
etcdctl member list -w=json --hex
flag to print memberListResponse in hex format json.
- gRPC gateway only supports
/v3
endpoint.- Deprecated
/v3beta
. curl -L http://localhost:2379/v3beta/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'
does work in v3.5. Usecurl -L http://localhost:2379/v3/kv/put -X POST -d '{"key": "Zm9v", "value": "YmFy"}'
instead.
- Deprecated
- Fix
panic on error
for metrics handler. - Add gRPC keepalive related flags
grpc-keepalive-min-time
,grpc-keepalive-interval
andgrpc-keepalive-timeout
. - Fix grpc watch proxy hangs when failed to cancel a watcher .
- Add metrics handler for grpcproxy self.
- Add health handler for grpcproxy self.
- Fix NoPassword check when adding user through GRPC gateway (issue#11414)
- Fix bug where some auth related messages are logged at wrong level
- Fix a data corruption bug by saving consistent index.
- Improve checkPassword performance.
- Add authRevision field in AuthStatus.
- Add
/v3/auth/status
endpoint to check if authentication is enabled - Add
Linearizable
field toetcdserverpb.MemberListRequest
.
- Upgrade
google.golang.org/grpc
fromv1.23.0
tov1.26.0
. - Upgrade
go.uber.org/zap
fromv1.14.1
tov1.15.0
.
- The etcd team has added, a well defined and openly discussed, project governance.