Delay fork() if another instance in the same box is fork()ing already

[antirez]

When multiple Redis instances are running on the same box, it is a good idea for them to coordinate so that different instances will persist on disk at different times, avoiding to create a child process and to do a lot of disk I/O at the same time. While in general running multiple very busy Redis instances in multi-tenancy is anyway more complex to get right, this alone could reduce significantly the problems that normally users face. This is especially important in Redis Cluster where it happens often (even if not always a good idea) that multiple cluster nodes are grouped in the same physical systems or VMs.

In order to implement such a feature, some kind of inter-process locking primitive should be used, possibly the one posing the lowest latency concerns. Also instances should be able to detect if a lock is no longer valid because the instance locking the resource is not operating correctly, or crashed, or any other problem, and in that case re-acquire the lock. In the simple case of a lock file, the process holding the lock could refresh the time of the lock as the lock file content so that stale lock detection could be simple.

The feature would be disabled by default. Potentially the lock name could be configured so that different instances groups could be locked independently of others. This is useful, for instance, in the case of multiple disks in the same machine.

[mp911de]

You should consider multi-disk systems in your thoughts. If you have as many Redis processes as disks, then you don't run necessarily into I/O contention.

[antirez]

@mp911de thanks, comment updated to reflect this. Specifying the lock name allows to group instances. Also the feature is off by default.

[dvirsky]

@mp911de it still creates a CPU and RAM overhead that is considerable. Perhaps it should be tunable what the maximum allowed number of concurrent forks at once.

[wooparadog]

@dvirsky Using a named lock can limit concurrency.

--
edit: limit to the number of locks.

[dvirsky]

@antirez another thing to keep in mind - multiple containers running a single redis instance each - all on the same machine. So just using some filesystem based thingie might be harder. Not sure what the optimal solution is, but just something to keep in mind when considering trade-offs.

[antirez]

@dvirsky yep... good point. Maybe it's possible to create a shared memory chunk or alike. There is to evaluate all the IPC possibilities.