Defer importing ens._normalization to speed up web3 import by 27% #3285
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What was wrong?
As of version 7.0.0-beta.2 the
import web3
command takes 375 ms to run*, which is quite long for a library import operation. This affects the usability of command-line tools built around web3.py, where import times are critical.How was it fixed?
By deferring the import of
ens._normalization
until the first call of theens.utils.normalize_name
function, the execution time ofimport web3
is reduced to 275 ms* i.e. by 27%.Todo:
Details
Profiling* showed that the
ens._normalization
module performs two lengthy operations at import time:ens/spec/normalization_spec.json
that is 3 Mbytes in size takes 7% of the total web3 import time_extract_valid_codepoints()
function that performs transformations on the loaded data takes 16% of the total web3 import timeThese are very costly operations to perform at every single import, with the sole purpose of initializing the
ens.utils.normalize_name
function.*: Measurements were taken with
timeit
andpyinstrument
on an AMD Ryzen 7 machine with M.2 SSD using Python 3.10.12:and