Reduce memory used by ActiveSupport::Callbacks #49728
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, callbacks which were shared with subclasses would not share between them the procs generated in
Filters::Before
andFilters::After
. This was also lazily generated so would cause memory growth after an application is booted (and not sharable between workers via Cope on Write).This was because we would rebuild the objects used to invoke the callbacks via
CallbackChain#compile
, so any difference in the callback chain would result in all of the callback procs being rebuilt.This commit changes before and after callbacks (but not around!) to be shared between all subclasses of where it was defined. This is done by changing
Filters::Before
andFilters::After
to plain classes which respond to call instead of generating procs (which wasn't strictly necessary but was easier to implement, and also results in simpler objects which use less memory). These objects avoid referencing and tied to a specific callback sequence and so can be memoized and reused.This has the most impact on applications with many Controllers, and many callbacks in the
ApplicationController
(or similar).I also took this opportunity to merge together all the different forms of procs generated (halting, halting_and_conditional, conditional, simple) into one form with if statements. There isn't any significant performance benefit from the specialization previously being done.
I didn't make this eagerly build the callable filter objects, but that's probably worth doing in a follow-up.
I think @byroot will appreciate this 馃槉
Reproduction
This is a self contained app which created 1000 controllers all inheriting from an
ApplicationController
with 50 before callbacks (numbers which are pretty comparable to GitHub's largest application), and makes one request to each controller going through the whole stack. Measuring live objects and RSS we see:https://gist.github.com/jhawthorn/1540799e0d5c0a4be7f929fa6edd8d5b
main
this branch
This removes half of the memory growth from boot to post-request-serving (most of the remainder being I believe callcaches).