How to interpret Mypy internal profiling?

[Ryang20718]

Hi, I know many of the hidden flags are for internal uses, but I found it extremely helpful when understanding where mypy is spending time. Mainly trying to understand how the mypy team profiles mypy as we've been trying to understand where mypy is spending most of the time in order to better optimize developer experience, however mypy has been taking 10s+ per file to lint.

Our mypy version is 0.960. Main ask is wondering how to interpret results of timing stats of these hidden flags?

I was wondering if these times generated are done in parallel or sequential?
Flag: --timing-stats

common.file_a 140
.....
common.file_b 143

--dump-build-stats

Wondering how to interpret these times for each individual line. Are they supposed to add up sequentially or are they done in parallel?

cache_commit_time:      4.76837158203125e-07
data_json_load_time:    0.2347877025604248
deserialize_time:       0.20813846588134766
files_parsed:           321
find_module_calls:      11137
find_module_time:       0.5428452491760254
fm_cache_size:          3728
fresh_metas:            1045
fresh_trees:            1045
graph_load_time:        1.7382361888885498
graph_size:             1366
load_meta_from_dict_time:0.0034983158111572266
load_meta_load_time:    0.07038664817810059
load_meta_time:         0.07388496398925781
load_tree_time:         0.4732956886291504
metastore_read_time:    0.09546065330505371
modules_parsed:         321
nodes_left:             0
parse_time:             0.717149019241333
process_fresh_time:     0.5325548648834229
sccs_left:              0
stubs_found:            161
stubs_parsed:           0
validate_meta_time:     0.010566473007202148
validate_stat_time:     0.008595943450927734

[AlexWaygood]

we've been trying to understand where mypy is spending most of the time in order to better optimize developer experience, however mypy has been taking 10s+ per file to lint.

Our mypy version is 0.960.

It's tangential to the question you're asking here, but if mypy performance is important to you, I would suggest upgrading to the latest mypy (1.3). Mypy 0.960 is a very old version of mypy, and many optimisations have been made in recent versions.

[Ryang20718]

@AlexWaygood

We've tried upgrading, but the perf degradations have limited our advances.
mypy 1.3.0 brings this file check to 1m 50s whilst 0.960 takes 10s.

[AlexWaygood]

@AlexWaygood

We've tried upgrading, but the perf degradations have limited our advances. mypy 1.3.0 brings this file check to 1m 50s whilst 0.960 takes 10s.

I see, I'm sorry to hear that.

[hauntsaninja]

--timing-stats is sequential. If your code is open source, I can take a look. mypy 1.4 will also have some nice perf improvements, but if you're seeing a 12x slowdown there's probably something more specific happening.

[Ryang20718]

Really appreciate the help, but unfortunately private repo 😢

based on currentl profiling, if a file takes 8-10s to lint, 5 is spent on following file imports (that's on us due to a lot of imports) However, I'm not sure what the rest of the time is spent on. Is --dump-build-stats parallel or sequential as well?

[hauntsaninja]

Probably sequential? There's also --line-checking-stats