realloc leaks #4818

Closed
Tormund opened this Issue Sep 22, 2016 · 8 comments

Projects

None yet

5 participants

@Tormund
Tormund commented Sep 22, 2016 edited

realloc leaks memory on windows, when application compiled in 32bit
Leak appears only if buffer size growth and I got out of memory, for example:

const BUFFER_SIZE = 5000
var buffer = cast[ptr uint16](alloc(BUFFER_SIZE))

var total_size: int64 = 0
for i in 0 .. 4_000:
    let size = BUFFER_SIZE * i
    total_size += size.int64
    buffer = cast[ptr uint16](realloc(buffer, size))

dealloc(buffer)

But if I replace nim's alloc, realloc, dealloc to c_malloc, c_realloc, c_free, I don't have out of memory. Like this:

const BUFFER_SIZE = 5000
var buffer = cast[ptr uint16](c_malloc(BUFFER_SIZE))

var total_size: int64 = 0
for i in 0 .. 4_000:
    let size = BUFFER_SIZE * i
    total_size += size.int64
    buffer = cast[ptr uint16](c_realloc(buffer, size.csize))

c_free(buffer)
@Tormund Tormund referenced this issue in yglukhov/sound Sep 22, 2016
Merged

realloc leaks #12

@kirbyfan64
Contributor

http://stackoverflow.com/a/4548441/2097780

No clue about the out-of-memory issue, but the leak itself isn't a Nim bug.

@yglukhov
Member

Nim realloc != c realloc, and has nothing to do with it. Most definitely looks like a Nim bug.

@cheatfate
Contributor
cheatfate commented Sep 23, 2016 edited

@Tormund, please add doAssert(buffer != nil) because you don't check anything in your c_xxxxxx example, and c_realloc can work even in such case.

@Tormund
Tormund commented Sep 23, 2016 edited

@cheatfate. I do mistake in sample code, to try realloc with zero buffer size (got assertion) in first iteration.
So this sample with c_realloc work's perfectly:

const BUFFER_SIZE = 5000
var buffer = cast[ptr uint16](c_malloc(BUFFER_SIZE))

var total_size: int64 = 0
for i in 1 .. 4_000:
    let size = BUFFER_SIZE * i
    total_size += size.int64
    buffer = cast[ptr uint16](c_realloc(buffer, size.csize))
    doAssert(buffer != nil)
c_free(buffer)

But same code with Nim realloc still doesn't work, and I don't got my assertion(doAssert(buffer != nil)). Looks like app execution stoped somewhere from realloc proc.

@Araq
Member
Araq commented Sep 24, 2016

realloc does deallocate and the problem is likely address space fragmentation.

@cheatfate
Contributor

I think problem not in fragmentation, but in algorithm of nim's realloc. At one moment of time it tries to allocate on big memory block, while holding another one big memory block (before copyMem). And c_realloc don't do this.

@yglukhov
Member

@cheatfate please note, that example code allocates maximum 20MB. Even multiplied by 2 (old block being held while allocating new block) thats still nothing and should not cause out of mem.

@cheatfate
Contributor

Ok, this is bug, on my Windows x64 it eats 20Gb of memory.

@Araq Araq added the High Priority label Oct 21, 2016
@Araq Araq added a commit that closed this issue Dec 18, 2016
@Araq Araq fixes #4818 39ca8b8
@Araq Araq closed this in 39ca8b8 Dec 18, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment