Use weak references in descendants tracker

[ebeigarts]

This PR will make descendants tracker use weak references and weak map, so that unused subclasses (unused anonymous subclasses) can be garbage collected, for more details you can see #31395. /cc @matthewd

Ideally we would need WeekKeyMap and WeekSet/WeekArray, but the ref gem only provides WeekKeyMap.

Fixes #31395.

Benchmark results:

Original:
              Parent    352.629k (± 6.2%) i/s -      1.774M in   5.052777s
               Child    661.191k (± 6.6%) i/s -      3.329M in   5.062953s
          Grandchild      1.834M (± 5.0%) i/s -      9.151M in   5.001540s

           Class.new    348.889k (±29.5%) i/s -      1.443M in   5.070915s

Weak map (ref gem):
              Parent    131.917k (± 4.9%) i/s -    664.962k in   5.054227s
               Child    212.329k (±12.9%) i/s -      1.058M in   5.081918s
          Grandchild    848.726k (±14.1%) i/s -      4.152M in   5.013853s

           Class.new     75.910k (±29.9%) i/s -    280.782k in   5.172886s

Weak ref array:
              Parent    261.569k (± 6.5%) i/s -      1.314M in   5.045141s
               Child    510.892k (± 4.6%) i/s -      2.581M in   5.064146s
          Grandchild      1.813M (± 4.8%) i/s -      9.048M in   5.002708s

           Class.new      3.376k (±186.0%) i/s -      6.182k in   5.028783s

According to this class creation would be 4.5x times slower and .descendants calls would be 2.6x times slower. The performance could be improved by writing WeekSet and maybe improving WeekKeyMap.

Benchmark examples:

• https://gist.github.com/2bf37ac7b1b2ed1f57dc63e5682655db
• https://gist.github.com/c160f9cf698b5d5e31685deeb50b5cb3

[matthewd]

This isn't worth adding a dependency for. I thought the plan was to use WeakRef?

[ebeigarts]

The ref gem uses WeakRef underhood for MRI.

Using just WeakRef there could be situations where the descendants array contains a lot of dead WeakRef's (and would eat up small amount of memory) and we would need to somehow cleanup the array of those dead references (which ref does for maps with finalizers) or we could just leave them there and when calling descendants we would just skip the dead references.

I can also try to implement DescendantsMap without using ref gem (~160 lines).

[matthewd]

I'd like to try for something more like the array-of-weakref. As long as we're aware of threading concerns, we should be able to clean it out when adding new entries. That leaves a degenerate case where it doesn't shrink if no new descendants are being defined... but at that point the list isn't growing either.

[ebeigarts]

Sounds good enough, I will try that.

[ebeigarts]

@matthewd I have updated the implementation to use WeakRef.

Benchmarks:

              Parent    261.569k (± 6.5%) i/s -      1.314M in   5.045141s
               Child    510.892k (± 4.6%) i/s -      2.581M in   5.064146s
          Grandchild      1.813M (± 4.8%) i/s -      9.048M in   5.002708s

           Class.new      3.376k (±186.0%) i/s -      6.182k in   5.028783s

Class.new benchmark is much slower because if the descendants array is large, then the cleaning of the dead refs take a lot of time.

Without the cleanup! it is

           Class.new    111.620k (±44.2%) i/s -    443.080k in  10.032368s

[ebeigarts]

There is one case, where this will not work:

class Parent; extend ActiveSupport::DescendantsTracker end
child_klass = Class.new(Parent)
grandchild_klass = Class.new(child_klass)

In this example child_klass will not be garbage collected, because child_klass is registred as the key in @@direct_descendants, because it had descendants.

I think most people will not inherit an anonymous class from another anonymous class.

[FutoRicky]

Any update on this?

[simi]

Hello. I'm also interested in this fix since if I understand it well it allows you to safely use anonymous ActiveRecord::Base inherits which can be handy for temporary tables.

Is there anything I can help with?

[ebeigarts]

I have rebased this against master. @matthewd can you take a look at this?

[matthewd]

Thanks for your patience, and sorry this fell off the radar 😔

If you can just make codeclimate happy (with our minimum ruby, do/rescue/end is now valid), I think it's good to

[ebeigarts]

@matthewd codeclimate should be happy now :)

[simi]

I think this is worth changelog entry as well.

[ebeigarts]

@simi @matthewd yeah, changelog would be nice, do I need to update changelog myself or this will be done by the rails core? (I think it's not mentioned in the CONTRIBUTING.md)

[simi]

I think it is common to write changelog entry on your own.

[ebeigarts]

I have updated the changelog, @matthewd can we merge this?

[matthewd]

Thanks again for persevering on this! (And for the poke, @simi)