You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi Folks,
I have an application what writes large values to etcd (can be larger than 1MB), and I believe I am running into #12548 where the etcd server consumes a lot of memory. For my application, I would like to limit the maximum memory consumption of etcd to something lower, like 3GB.
However, when I reduce DefaultSnapshotCatchUpEntries to 100, the memory consumption is seems to stay below 3 GB.
In the code, there is a comment justifying the 5000 value for DefaultSnapshotCatchUpEntries, and I am curious when this value comes into play. What is the drawback if DefaultSnapshotCatchUpEntries is too small? Will a follower that takes a long time to do its local snapshot need to retrieve the entire snapshot from the leader instead of catching up on only the recent log entries?
// DefaultSnapshotCatchUpEntries is the number of entries for a slow follower
// to catch-up after compacting the raft storage entries.
// We expect the follower has a millisecond level latency with the leader.
// The max throughput is around 10K. Keep a 5K entries is enough for helping
// follower to catch up.
DefaultSnapshotCatchUpEntries uint64 = 5000
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi Folks,
I have an application what writes large values to etcd (can be larger than 1MB), and I believe I am running into #12548 where the etcd server consumes a lot of memory. For my application, I would like to limit the maximum memory consumption of etcd to something lower, like 3GB.
Even when I set
the RSS memory consumption can exceed 6+GB.
However, when I reduce DefaultSnapshotCatchUpEntries to 100, the memory consumption is seems to stay below 3 GB.
In the code, there is a comment justifying the 5000 value for DefaultSnapshotCatchUpEntries, and I am curious when this value comes into play. What is the drawback if DefaultSnapshotCatchUpEntries is too small? Will a follower that takes a long time to do its local snapshot need to retrieve the entire snapshot from the leader instead of catching up on only the recent log entries?
thank you
Beta Was this translation helpful? Give feedback.
All reactions