When a buffer gets resized and needs to drop data, the data is dropped, but the maximal value of the buffer is never brought down. This patch makes it so the function works as expected. This fix also removes all the unused conversion functions that were used in a past deploy.
Changing the allocator strategy for binary_alloc to do aobf rather than bf (+MBas aobf). Depending on how all of the binaries are allocated, this could make new allocations favor the same carrier. This will only add a small cpu overhead when allocating new binaries. This could however worsen our utilization even more if we are unlucky, it will however make allocations faster and more in the same area in the expected case, which should reduce fragmentation and untraceable leaks. We're also decreasing the size of our mbcs. Right now we have smbcs set to 256 kb and lmbcs at 5 MB (rounded up to 8MB as ERTS only allocates multiples of 2) and an average multi-block carrier size of 7.78 MB. We try to set +MBlmbcs 512 so that we get many more carriers and thus increase the chance that it can be returned to the OS. These two options have been recommended by members of the Erlang/OTP team to reduce the passive memory leaks due to allocation patterns compared to our peculiar use cases for log messages.
batches io:format calls into a buffer process to reduce the number of calls required and also do overload protection automatically. Experimental material to see if it helps with performance, given the buffering and load shedding allow to go asynchronous on log messages without loosing too much data.
When network or drains show bad behaviour temporarily, low timeouts we currently have (1 second) ends up killing connections and raising the retry count of frames. When massive losses are seen, it makes it difficult to put the blame on logplex's speed at sending logs, or the drains consumption (or network). By raising the timeout a bit, we should reduce the reconnection rate and at the same time make it harder to blame logplex (as an individual node) for the problems. This should not have a super significant impact on the drop rate, however, but possibly a noticeable one.