An experimental forking sequence server with asynchronous persistence, written in Ruby by Paul Annesley.
Simply tracks and increments named sequences, for example as a more scalable replacement for database auto increment or sequence functionality.
Extensive use of Unix system calls like fork(), pipe() and select() is made, as inspired by the Unicorn HTTP server.
A functional work in progress.
- Known to run on Ubuntu 9.10
- Known to not yet run on Mac OS X
Eventually any platform on which Ruby's fork(), pipe(), select(), trap() etc methods function correctly should be supported.
The master process:
- creates a listening TCP socket,
- forks one 'writer' process for async disk persistence,
- forks N 'workers' to handle inbound connections,
- monitors pipes into each worker for IPC commands.
- manages sequences, pre-reserving blocks of sequences on disk.
The worker processes:
- accept an inbound connection,
- receive a command e.g. 'next example' from the client,
- send a light IPC command to the master requesting the next number in the sequence named 'example',
- pass the response back to the client.
The writer process:
- waits for light IPC commands instructing it to async write sequences to files.
Sequences are first pre-reserved in configurable sized blocks on disk, and then tracked in memory. For example on the first request for sequence named 'example', the number 100 (configurable) will be written to the 'example' sequence file, but the number 1 will be served to the client.
Subsequent requests for the 'example' sequence will be incremented in memory only, until a threshold is reached and another block of 100 is written to disk. The current actual value is only ever written to disk during a clean shutdown.
By this mechanism, a crash would result in a sequence on disk beyond the current actual value, causing usually-safe gaps, rather than often-catastrophic overlapping.