Skip to content
This repository

Parrot chooses to panic instead of GC sweep #795

Open
pmichaud opened this Issue July 02, 2012 · 3 comments

5 participants

Patrick R. Michaud Andrew Whitworth Duke Leto chromatic Nick Wellnhofer
Patrick R. Michaud
Owner

Much of this ticket uses the following snippet of PIR code as
a basic test:

    $I0 = 0
  loop:
    unless $I0 < N goto done
    $P0 = new ['ResizablePMCArray']
    inc $I0
    goto loop
  done:

What this loop does is perform N allocations of a ResizablePMCArray (RPA).
At the end of the loop, only the last RPA is "active" -- the rest are available to
be garbage collected.

We can put this into a larger program (copy below) that allows
us to run the loop for a given value of N and report the resulting
memory usage statistics. So, to find out the amount of memory Parrot
needs to run this program when N==1 (allocating exactly one RPA):

pmichaud@kiwi:~/p6/parrot$ ./parrot gc.pir 1
gc_mark_runs=0
gc_collect_runs=0
total_pmcs=2040
active_pmcs=1999
total_mem_alloc=663552
total_mem_used=318342
pmichaud@kiwi:~/p6/parrot$ 

Running with N==1, Parrot allocated a total of 663,552 bytes of
memory, and is "using" 318,342 bytes.

Now let's have the OS limit the program to 64MB of memory and run it again:

pmichaud@kiwi:~/p6/parrot$ ulimit -Sv 64000     # limit processes to 64MB
pmichaud@kiwi:~/p6/parrot$ ./parrot gc.pir 1
gc_mark_runs=0
gc_collect_runs=0
total_pmcs=2040
active_pmcs=1999
total_mem_alloc=663552
total_mem_used=318342
pmichaud@kiwi:~/p6/parrot$ 

We get the same results, as we would expect. But watch what
happens when we get the loop to create and release a million RPAs:

pmichaud@kiwi:~/p6/parrot$ ulimit -v
64000
pmichaud@kiwi:~/p6/parrot$ ./parrot gc.pir 1000000
Failed allocation of 4096 bytes
Parrot VM: PANIC: Out of mem!
C file src/gc/alloc_memory.c, line 75
Parrot file (not available), line (not available)
[...]

Parrot chooses to panic rather than perform a GC that
would recover the needed space to allocate a PMC.

Note that this program shouldn't actually need anywhere close
to 64MB of space to run; at any point in time there's at most 1 or
2 RPA PMCs "active".

The full program used is:

pmichaud@kiwi:~/p6/parrot$ cat gc.pir

## This program reads a number from the command line,
## then creates that number of RPAs via a loop and reports
## interpinfo memory statistics at the end.  The RPAs
## are created with replacement, so that only one RPA
## is actually "alive" at any given time.

.include 'interpinfo.pasm'

.sub 'main' :main
    .param pmc args

    .local int N
    N = args[1]

    $I0 = 0
  loop:
    unless $I0 < N goto done
    $P0 = new ['ResizablePMCArray']
    inc $I0
    goto loop
  done:

    # Now display the current memory consumption statistics.
    print "gc_mark_runs="
    $I0 = interpinfo .INTERPINFO_GC_MARK_RUNS
    say $I0

    print "gc_collect_runs="
    $I0 = interpinfo .INTERPINFO_GC_COLLECT_RUNS
    say $I0

    print "total_pmcs="
    $I0 = interpinfo .INTERPINFO_TOTAL_PMCS
    say $I0

    print "active_pmcs="
    $I0 = interpinfo .INTERPINFO_ACTIVE_PMCS
    say $I0

    print "total_mem_alloc="
    $I0 = interpinfo .INTERPINFO_TOTAL_MEM_ALLOC
    say $I0

    print "total_mem_used="
    $I0 = interpinfo .INTERPINFO_TOTAL_MEM_USED
    say $I0
.end

I can get a similar result using a program that simply makes
subroutine calls and never creates any PMCs directly:

pmichaud@kiwi:~/p6/parrot$ cat gc-sub.pir
.sub 'main' :main
    .param pmc args

    .local int N
    N = args[1]

    $I0 = 0
  loop:
    unless $I0 < N goto done
    'no-op'()
    inc $I0
    goto loop
  done:
.end

.sub 'no-op'
    noop
.end

pmichaud@kiwi:~/p6/parrot$ ./parrot gc-sub.pir 1000000
pmichaud@kiwi:~/p6/parrot$ ulimit -Sv 64000
pmichaud@kiwi:~/p6/parrot$ ./parrot gc-sub.pir 1000000
Failed allocation of 4096 bytes
Parrot VM: PANIC: Out of mem!
C file src/gc/alloc_memory.c, line 105
Parrot file (not available), line (not available)

Pm

Duke Leto
Owner
leto commented July 02, 2012

Thanks for this well-written issue with example code! This behavior from the Parrot GC is not acceptable.

chromatic

Sure, only one RPA is alive at any given time, but without reference counting, the only other way to determine the lifecycle of a given gcable is to perform exhaustive automated escape analysis. With so much of Parrot written in C, that's infeasible.

With that said, if you're willing to perform some manual escape analysis in tight loops, a combination of the needs_destroy opcode and sweep 0 (lazy sweep) will work, once someone implements lazy sweep of early destruction PMCs in the GMS GC (say in the chromatic/early_pmc_gc branch).

Nick Wellnhofer
Collaborator

There are other solutions to this problem. For example the GC thresholds we had in the old non-generational mark-and-sweep collector. I'm pretty sure we once had a test that checked that code very similiar to pmichaud's examples never uses more than some MBs of RAM.

You could also run a GC whenever malloc runs out of memory, but this is pretty tricky because then you can't allocate additional memory during GC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.