runtime: performance degradation after upgrade from Go 1.4 to Go 1.7 #18805
This is yet another trouble with GC/GOGC in 1.4+ releases. Although this issue might look like a duplicate of #12228, #14189, #14161, I will try to provide some additional details that should help you to build a better technology.
Couchbase Server has a query engine written in Go. After using Go 1.4 for a long time we decided to switch to Go 1.7. Unfortunately, the throughput of some queries has dropped by 60% after upgrade.
In order to explain the issue, let me introduce two workloads:
Q1 throughput dropped by 60% after upgrade to Go 1.7, while Q2 throughput remained almost the same.
We ran several experiments with GOGC, we also tried Go 1.8 RC2 a couple days ago. The following spreadsheet summarizes the most important results:
Here are some observations and concerns:
The last experiment (Q1 after Q2) is very important. It shows how the relative nature of GOGC hurts use cases with small live sizes and large amount of garbage. We even had a joke that we should allocate and block a large array in order to achieve a better performance.
It seems that Xmx model might have worked better for us.
Go 1.8 with GOGC=200 provides a sufficient improvement and we consider this issue resolved. Let me know if you need any additional information (e.g., gctrace output). Otherwise, feel free to close the ticket;)