Signature cache for overloads for faster compilation

[ehsantn]

This adds a signature cache to overloads, to return the output signature quickly if the same input type set is already compiled. This is especially useful for the inlined overload cases where there is no other cache in the code path (dispatcher has a cache to avoid recompilation). My benchmark shows over 30% faster compilation time (hard to simplify and share the benchmark unfortunately).

[esc]

@ehsantn thanks for submitting this. I have added it to the queue for review. Incidentally, there seem to be some typing related failures on public CI.

[ehsantn]

Any ideas on how to test this? Seems tricky since the effect is lower compilation time and the internal data structures are not exposed.

[sklam]

I think we can test by counting the number of compilation triggered for an overload using the event recording added in #6626

[stuartarchibald]

Also have to wonder how this will impact target overloads? I think this is assuming there's a 1:1 map between a signature and compiled impl?

[ehsantn]

I think we can test by counting the number of compilation triggered for an overload using the event recording added in #6626

This is a good idea. I'll give it a try when I get a chance.

[ehsantn]

Also have to wonder how this will impact target overloads? I think this is assuming there's a 1:1 map between a signature and compiled impl?

Yes, this is assuming a 1:1 map between signature and implementation, but it's flexible. For example, it doesn't save the signature for the overloads that change the signature. I don't quite know how the target overloads will look like, but target could be part of the cache key or we can just turn this off for those overloads.

[ehsantn]

@sklam I just tried adding a test using the new events feature. However, the numba:compile event is not triggered unless if actual compilation is necessary. The numba:compiler_lock even shows some difference, but doesn't seem reliable for testing (a small change somewhere else can make the test fail). We could add new events, but seems too much to me just for testing this feature. I think manual testing may be enough for this PR.

[ehsantn]

I'm not able to reproduce the CI error locally. Any thoughts?

[stuartarchibald]

I'm not able to reproduce the CI error locally. Any thoughts?

It'll probably be something very specific about the test slice and the order the tests are run on that CI combination.

[github-actions]

This pull request is marked as stale as it has had no activity in the past 3 months. Please respond to this comment if you're still interested in working on this. Many thanks!

[dlee992]

I am interested in continuing this PR. Will test and re-push if available.