Skip to content

v0.0.56, lazy imports

Choose a tag to compare

@tamnd tamnd released this 06 Sep 02:02
· 4 commits to main since this release
v0.0.56
f605113

One lesson in this one, R05 on lazy imports, plus a fix to R03 that the browser probe turned up along the way.

PEP 810 landed in 3.15 and added one soft keyword to the import statement. lazy import json binds the name straight away and leaves the finding, reading and running of the module until the first time something reads that name back. R05 weighs what an unused import costs, shows the five line LazyLoader dance you had to write before 3.15, and disassembles four import statements to show that a lazy import compiles to the same IMPORT_NAME opcode as a plain one with the mode carried in two spare bits of the argument.

From there it works in a temporary directory on sys.path, so every module it looks at is one the cell above wrote. The placeholder gets inspected without being resolved, resolved by a single bare name read, and watched escaping into another variable where it resolves on the read rather than on the copy. Six spellings of the keyword get compiled to see which the symbol table refuses. The behaviour gets turned on without the keyword, once with a filter and once with a __lazy_modules__ list. A module raises during resolution and the two part traceback gets read.

Three Tier 1 recordings back the measured parts. Deferring is worth about three quarters of a run on a file that imports twelve modules and uses one. The other two show something I did not expect: waking a placeholder takes the interpreter wide import lock rather than the per module name lock an ordinary import takes, and holds it for the whole resolution. On a free threaded build, four threads importing four different modules the ordinary way keep 3.60 cores busy, and four threads waking four different deferred imports keep 0.98. That is filed as issue #184.

The R03 fix is unrelated to lazy imports. The Emscripten filesystem keeps whole second timestamps, so a directory mtime does not always move when a file is written into it and FileFinder keeps serving a stale listing, which made one cell fail at random in the browser. It now drops the finder out of sys.path_importer_cache and says why.

Also here: six diagrams, three glossary terms, rows in both READMEs, and refreshed citations, claims and probe recordings.

The repo is now at 76 lessons, 229 glossary terms, 601 claims and 37 Tier 1 experiments, all of it green under just check and all 76 lessons running end to end on Pyodide.

Merged: #183.