GC not freeing short living variables under some circumstances #2436
Comments
|
Additionally I ran the same tests on Linux (Windows 10 WSL1). Thus here are the results for Linux WSL1, SN 0.4.1, LTO thin, release-fast, using either:
Also the third example was a lot faster compared to the two first ones in all different configurations. |
|
And here is another series of example with Zones. Example 1, no call to var sum = 0
val nIters = 3000000
Zone { implicit z =>
var i = 0
while (i < nIters) {
var j = 1
while (j < 10) {
val buffer = new Array[Byte](1600)
sum += buffer(0).toInt
j += 1
}
i += 1
}
}Example 2, call to var sum = 0
val nIters = 3000000
Zone { implicit z =>
var i = 0
while (i < nIters) {
var j = 1
while (j < 10) {
val buffer = new Array[Byte](1600)
sum += buffer(0).toInt
j += 1
}
alloc[CInt] // additional call compared to the previous example
i += 1
}
}Example 3, a while loop containing only a call to val nIters = 3000000
Zone { implicit z =>
var i = 0
while (i < nIters) {
alloc[CInt]
i += 1
}
}Example 4, combination of examples 1 and 3: var sum = 0
val nIters = 3000000
Zone { implicit z =>
var i = 0
while (i < nIters) {
var j = 1
while (j < 10) {
val buffer = new Array[Byte](1600)
sum += buffer(0).toInt
j += 1
}
i += 1
}
}
Zone { implicit z =>
var i = 0
while (i < nIters) {
alloc[CInt]
i += 1
}
}Here are the results I get on Linux WSL1, SN 0.4.1, LTO thin, immix, release-fast:
I'm surprised by the used memory in the second example, which is really exploding. |
|
A last example to show that this issue can be very important, when the operation performed inside the while generates more garbage: val nIters = 3000000
val email = "miles.davis at anymail.com"
val objsBuffer = new ArrayBuffer[Object](nIters)
var i = 0
while (i < nIters) {
val fixedEmail = email.replaceAll(raw"\sat\s","@")
objsBuffer += new Object()
i += 1
}Here the problem is that By comparison, running the same example while commenting the line Note that regex memory issues were reported before: |
|
Another reproduction of this problem, kind of similar to the last comment, but without the Regex. I'm not entirely sure how much memory the second example takes, because I'm using a VM with limited memory. Tested with immix, both in debug and in release-full. // Split loops: ~ 150MB
def main(args: Array[String]): Unit = {
val nIters = 3000000
val objsBuffer = Array.ofDim[Object](nIters)
var num = 0
var i = 0
while (i < nIters) {
num = Array.fill(100)(scala.util.Random.nextInt).sum
i += 1
}
i = 0
while (i < nIters) {
objsBuffer(i) = new Object()
i += 1
}
println(num)
println(objsBuffer.##)
} // Single loop: Quickly reaches 1.5GB usage
def main(args: Array[String]): Unit = {
val nIters = 3000000
val objsBuffer = Array.ofDim[Object](nIters)
var num = 0
var i = 0
while (i < nIters) {
num = Array.fill(100)(scala.util.Random.nextInt).sum
objsBuffer(i) = new Object()
i += 1
}
println(num)
println(objsBuffer.##)
} |
|
Ran the previous code with GC logs (with Good logs:Bad logs:The second version seems to have much less collects, however it grows the heap on every collect... It also seems to have way more recycled blocks. |
I'm wondering if this could be explained by an issue in the current implementation of the escape analysis system. |
|
I just tried the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have observed that SN GC (tried on immix and commix) is not always freeing short living variables. This can be annoying because it can increase very quickly the total used memory of the program process.
Let's take this simple while loop example doing a high number of iterations:
In this context the value called
buffer, which is used o compute a total sum (not meaningful though in this example), is well freed after each iteration.However if one adds a global allocation like this
charsBuffer, then is totally changing the behavior of the GC:As a reference it is also possible to check the memory used when only appending to the
charsBufferalone:As a summary here are the different memory peaks of the 3 previous examples (SN 0.4.1, Windows 10, immix, LTO thin, release-fast):
What I would expect in the case of the second example would be to have a memory peak close to the third example (81 MB).
It looks like all intermediate
buffervalues were not freed by the GC.@ekrich mentioned on Discord that it could be a memory fragmentation issue, but it is not totally clear to me.
The text was updated successfully, but these errors were encountered: