Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Collate high-volume logs for improved throughput #3662
This PR is intended to improve throughput when we have extremely high volumes of logging. The idea is to batch up logs into a buffer before collating them and pushing to the log output in a single action.
We also have a timeout mechanism, so even if the buffer is not filled, after a short time (1ms), we'll flush it to the output anyway. This avoids having logs sat in a buffer for ages.
I've had to add handling for panic recovery and os interrupts in ServerMain, as without these short delays, we'd miss seeing the last of any logs we wrote.
Based on developer testing, we'd expect to see double the write throughput compared to the 2.1 default config, whilst still logging everything, although still slightly short of the 2.0 default (HTTP only) logging.