-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use of opaque types causes an extreme slow down in dialyzer #7835
Comments
Hi! Can you cherry-pick the top of https://github.com/jhogberg/otp/tree/john/dialyzer/investigate-extreme-opaque-slowdown and see if it helps? |
I have a custom version of Dialyzer which is optimised for the codebase I usually work with. It has a lot of changes, so the impact in the OSS version might be different, but I found memoising opaques from records in their own ETS table beside records in codeserver helped a lot, since this operation is performed potentially quite a few times: -spec lookup_mod_record_opaques(module(), codeserver()) -> [erl_types:erl_type()].
lookup_mod_record_opaques(Mod, #codeserver{opaques = OpaqueDict}=CS) when is_atom(Mod) ->
case ets:lookup_element(OpaqueDict, Mod, 2, not_memoised_yet) of
not_memoised_yet ->
Records = lookup_mod_records(Mod, CS),
Opaques = erl_types:t_opaque_from_records(Records),
ets_map_store(Mod, Opaques, OpaqueDict),
Opaques;
Memoised ->
Memoised
end. I do hope to make a PR with the full change at some point, but I thought I'd share this snippet now since it may be relevant. |
@hazardfn have you had a chance to look at that branch? |
I gave it a quick look before having to move onto other things, it didn't seem to have a massive impact on the timing, but certainly some, I wrote down that it did have a reasonable impact on the CPU usage, it stopped my M1 from constantly being thermally throttled during the process. I have been meaning to swing back and actually collect some timing data for you but just haven't had the chance at work yet 😞. I'll try to find a moment tomorrow, doing it from my machine I need to collect a whole new set of baselines for you than the ones above, as they were done in a GitHub runner. |
That'd be great, that branch is still expected to be horrifically slow but if we can confirm that it helps it shouldn't take too long to bring |
Just as a short update, the work we're doing with nominal types (erlang/eep#60) will fix this problem. It'll most likely make it into OTP 28. |
Describe the bug
We have a fairly large Elixir project that has ~500 declared opaque types. With these opaque types in the project the dialyzer run time can be upwards of 2 hours, this slow down also seems to affect the creation of the PLTs. A coworker has created a table that compares our dialyzer run times:
The Opaque Removed run times were collected by replacing all of our
@opaque
types with@type
NOTE: Our CI Pipeline has a timeout of 2 hours across the 2 steps hence the timeout in the first value.
To Reproduce
I'm unsure exactly how noticeable the slowdown is per
@opaque
type, however doing a run with a number of@opaque
types and comparing once all of those are changed to@type
should highlight the difference. This time appears to grow as more opaque types are added.Expected behavior
While I understand the additional checks that may be required for an opaque type will extend runtime, I wouldn't have expected the difference to be so drastic. Perhaps there is room for optimization here.
Affected versions
Tested only with 25.3.2.7
Additional context
N/A
The text was updated successfully, but these errors were encountered: