Finer-grained inline constant cache invalidation #5433

kddnewton · 2022-01-12T19:50:52Z

Current behavior

Caches depend on a global counter. All constant mutations cause caches to be invalidated.

class A
  B = 1
end

def foo
  A::B # inline cache depends on global counter
end

foo # populate inline cache
foo # hit inline cache

C = 1 # global counter increments, all caches are invalidated

foo # misses inline cache due to `C = 1`

Proposed behavior

Caches depend on name components. Only constant mutations with corresponding names will invalidate the cache.

class A
  B = 1
end

def foo
  A::B # inline cache depends constants named "A" and "B"
end

foo # populate inline cache
foo # hit inline cache

C = 1 # caches that depend on the name "C" are invalidated

foo # hits inline cache because IC only depends on "A" and "B"

Examples of breaking the new cache:

module C
  # Breaks `foo` cache because "A" constant is set and the cache in foo depends
  # on "A" and "B"
  class A; end
end

B = 1

We expect the new cache scheme to be invalidated less often because names aren't frequently reused. With the cache being invalidated less, we can rely on its stability more to keep our constant references fast and reduce the need to throw away generated code in YJIT.

Benchmarks

The following benchmark (included in this pull request) performs about 2x faster than master.

CONSTANT1 = 1
CONSTANT2 = 1
CONSTANT3 = 1
CONSTANT4 = 1
CONSTANT5 = 1

def constants
  [CONSTANT1, CONSTANT2, CONSTANT3, CONSTANT4, CONSTANT5]
end

500_000.times do
  constants
  INVALIDATE = true
end

In terms of macro benchmarks, I ran with this code on railsbench and there was not a statistically significant different in startup time or overall runtime performance.

In terms of memory, this includes an increase in VM size by about 500KiB when running on railsbench. This is because we're now tracking cache associations ({ ID => IC[] }) on the VM to know how to invalidate specific caches when constants change.

Ticket: [Feature #18589]

maximecb · 2022-01-12T22:13:20Z

vm.c

+static int
+vm_stat_constant_cache_i(st_data_t id, st_data_t table, st_data_t constant_cache)
+{
+    st_index_t size = ((st_table *) table)->num_entries;
+    rb_hash_aset((VALUE) constant_cache, ID2SYM((ID) id), LONG2NUM(size));
+    return ST_CONTINUE;
+}


Maybe I'm being silly but what does this function do? 😅 could use a comment

maximecb · 2022-01-12T22:17:04Z

This PR ended up being bigger than I thought 😅 but it looks quite good 👍

Are the failing tests related to the changes made here?

nobu · 2022-01-13T02:11:33Z

include/ruby/internal/intern/vm.h

 */
-void rb_clear_constant_cache(void);
+void rb_clear_constant_cache(ID id);


We should move this declaration to include/ruby/backward.h, and mark as RBIMPL_ATTR_DEPRECATED_INTERNAL.

Or even, let rb_clear_constant_cache be a no-op function and have ID-taking variant in a different name.

If it's internal and we've removed all usage of this function, do we still need to mark it as deprecated?

This declaration has been visible from extension libraries in the wild. Deleting it could break them. It is not a bad idea to tell them that we want to delete it, before actually doing so.

I agree with @shyouhei that you should make the old one as deprecated and give the new one which takes an id parameter a different name.

vm.c

iseq.c

class.c

michalmalecki

CPU impact - can it regress?

michalmalecki · 2022-03-08T09:20:33Z

iseq.c

+    size = body->iseq_size;
+    code = body->iseq_encoded;
+
+    for (index = start_index; index < size;) {


In addition to extra memory consumed, is there measurable time penalty for doing more precise cache invalidation? commenting here, since this is the only loop I found.

@michalmalecki see discussion here https://bugs.ruby-lang.org/issues/18589

Thanks, make sense. Sorry for missing it in the first place

Current behavior - caches depend on a global counter. All constant mutations cause caches to be invalidated. ```ruby class A B = 1 end def foo A::B # inline cache depends on global counter end foo # populate inline cache foo # hit inline cache C = 1 # global counter increments, all caches are invalidated foo # misses inline cache due to `C = 1` ``` Proposed behavior - caches depend on name components. Only constant mutations with corresponding names will invalidate the cache. ```ruby class A B = 1 end def foo A::B # inline cache depends constants named "A" and "B" end foo # populate inline cache foo # hit inline cache C = 1 # caches that depend on the name "C" are invalidated foo # hits inline cache because IC only depends on "A" and "B" ``` Examples of breaking the new cache: ```ruby module C # Breaks `foo` cache because "A" constant is set and the cache in foo depends # on "A" and "B" class A; end end B = 1 ``` We expect the new cache scheme to be invalidated less often because names aren't frequently reused. With the cache being invalidated less, we can rely on its stability more to keep our constant references fast and reduce the need to throw away generated code in YJIT.

Co-authored-by: John Hawthorn <john@hawthorn.email>

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>

kddnewton requested review from maximecb, tenderlove and XrXr as code owners January 12, 2022 19:50

maximecb reviewed Jan 12, 2022

View reviewed changes

nobu reviewed Jan 13, 2022

View reviewed changes

XrXr reviewed Jan 13, 2022

View reviewed changes

vm.c Outdated Show resolved Hide resolved

kddnewton force-pushed the constant-invalidation branch 2 times, most recently from 572c60b to f8bda4f Compare March 8, 2022 18:57

jhawthorn reviewed Mar 10, 2022

View reviewed changes

iseq.c Show resolved Hide resolved

kddnewton force-pushed the constant-invalidation branch from d2b813c to 44149ff Compare March 10, 2022 02:25

nobu reviewed Mar 15, 2022

View reviewed changes

class.c Outdated Show resolved Hide resolved

michalmalecki reviewed Mar 17, 2022

View reviewed changes

kddnewton force-pushed the constant-invalidation branch from 5f3ab7d to aa39a4f Compare March 24, 2022 14:03

kddnewton and others added 3 commits March 24, 2022 12:13

Delete ID from constant cache table if it becomes empty on ISEQ free

a783a5e

Co-authored-by: John Hawthorn <john@hawthorn.email>

Update formatting per feedback

50561e4

Co-authored-by: Nobuyoshi Nakada <nobu@ruby-lang.org>

kddnewton force-pushed the constant-invalidation branch from aa39a4f to 50561e4 Compare March 24, 2022 16:13

tenderlove merged commit 8008fb7 into ruby:master Mar 24, 2022

kddnewton deleted the constant-invalidation branch March 24, 2022 16:15

nobu mentioned this pull request Mar 25, 2022

Revert "Finer-grained inline constant cache invalidation" #5715

Merged

fallwith mentioned this pull request Apr 4, 2022

Ruby 3.2 support - Report on the new VM's :constant_cache stat instead of the old :global_constant_state newrelic/newrelic-ruby-agent#1059

Closed

intrip mentioned this pull request Jan 11, 2023

Ruby 3.2.0 gauge error for the RubyVM.stat API changes DataDog/dd-trace-rb#2534

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Finer-grained inline constant cache invalidation #5433

Finer-grained inline constant cache invalidation #5433

kddnewton commented Jan 12, 2022 •

edited by XrXr

Loading

maximecb Jan 12, 2022

maximecb commented Jan 12, 2022

nobu Jan 13, 2022

shyouhei Jan 19, 2022

kddnewton Jan 25, 2022

shyouhei Jan 26, 2022

maximecb Jan 26, 2022

michalmalecki left a comment

michalmalecki Mar 8, 2022

kddnewton Mar 17, 2022

michalmalecki Mar 17, 2022

Finer-grained inline constant cache invalidation #5433

Finer-grained inline constant cache invalidation #5433

Conversation

kddnewton commented Jan 12, 2022 • edited by XrXr Loading

Current behavior

Proposed behavior

Benchmarks

Choose a reason for hiding this comment

maximecb commented Jan 12, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michalmalecki left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kddnewton commented Jan 12, 2022 •

edited by XrXr

Loading