Support for gc.freeze() for apps that use preloading

[gcbirzan]

In python 3.7 (still alpha), a new API was added: gc.freeze. The rationale and advantages (and caveats) are explained in the ticket.

Basically, the garbage collector touches a lot of objects and removes a lot of the benefits of preload, this API allows you to stop that and really share memory between processes.

[hingston]

#1566

[berkerpeksag]

See also Instagram's post on gc.freeze() at https://engineering.instagram.com/copy-on-write-friendly-python-garbage-collection-ad6ed5233ddf.

[Somewater]

I don't understand how to use it. Does fix already in 3.7 release?
I did some test script:
https://gist.github.com/Somewater/40d7a808d1efd7b2c77f22f8bcb73553
I expected that the script allocates 1-2 GB of memory at the beginning and does not require more memory in the process of work. But it does - just after start of iterations in workers:
http://pix.toile-libre.org/?img=1536607288.png
What I did wrong?

[tilgovi]

@Somewater this is not implemented yet in Gunicorn.

[tilgovi]

Oh, my fault. I thought you were asking about Gunicorn. Please keep general questions about gc.freeze() in Python to other forums. This ticket is about using it in Gunicorn.

[PetrochukM]

What does gunicorn need to do to support gc.freeze? Can we just add gc.freeze after loading large objects globally in the app?

[tilgovi]

@PetrochukM for the benefits to be large, the application probably needs to be using --preload. Then you could test it out by calling gc.freeze() in a Gunicorn hook, such as on_starting or when_ready.

For maximum benefit, we might want to pause garbage collection in the master process, but I suspect Gunicorn will need some changes to make that safe or else the master process will continue to grow in memory as workers restart.

Please post any results if you do experiment!

[aisk]

@tilgovi Hi I think users can call gc.freeze() by hand but most people do not know it because it's hidden so deep.

I think it's cool to have an option do enable it, for users can found it and know this feature. Furthermore, maybe we can enable it automacally if --preload is specified?

[tilgovi]

@aisk yes, that's why I'm encouraging experimentation with this by anyone who does feel comfortable doing this by hand. It would definitely be a better experience for users if it's automatic, but for that to be possible we have to be convinced that there are no bad effects. For that we need people to test and for that we don't need any code changes, just people who are willing to add some calls to their hooks.

[PetrochukM]

@tilgovi Thank you!

I was wondering if there is anything special that was needed, and it sounds like nothing special is required apart from running --preload.

I am testing a change with gc.freeze. Do you have any recommendations on measuring shared memory, to see if it actually works?

[tilgovi]

@PetrochukM I do not, but I'm very interested to hear what you find! Thanks for trying it out.

[jab]

https://pythonspeed.com/products/filmemoryprofiler/?

[aisk]

FYI @jab I have some projects in company, which need a big readonly shared dict on start up, after gc.freeze, the memory is reduced as expected.

[joekohlsdorf]

It can be done with the following --config:

import gc

preload_app = True
workers = 1
worker_class = "sync"

# disable GC in master as early as possible
gc.disable()

def when_ready(server):
    # freeze objects after preloading app
    gc.freeze()
    print("Objects frozen in perm gen: ", gc.get_freeze_count())

def post_fork(server, worker):
    # reenable GC on worker
    gc.enable()

When testing this you'll see the effect with time. The memory usage of worker processes will initially be the same as without this configuration but it should grow slower when you hit the workers with traffic because GC won't touch shared memory areas.

Note that Instagram did this because they have low-memory servers (the original post shows 32 CPUs and 32GB of memory), today you won't find this type of offering from any cloud provider. You save some memory and some CPU cycles because GC doesn't touch these memory areas but don't expect much from it. Leaving GC disabled is probably not an option, in my case worker RSS skyrockets eating up all the benefits from it, just like Instagram had to give up on it.
I'm not sure how they managed to share 250MB of memory, I'm working with a huge app and at most I see 50MB. My guess is that they do some magic in preloading, Django is ultra lazy and preloading itself won't do much.