v0.0.55, frozen modules

R04, the fourth of the nine runtime lessons, and the one about the modules that live inside the binary. R03 finished on a fact it did not explain: import os never opens os.py. This is why.

The problem is a real chicken and egg. The import system is written in Python, in Lib/importlib/_bootstrap.py, so there is no way to import it. CPython cuts the loop by compiling that file during its own build, marshalling the code object, and writing the bytes into the binary as a C array. The lesson proves it rather than asserting it. The compiled module body of _frozen_importlib contains zero IMPORT_NAME opcodes, which is exactly what makes it loadable with nothing running, while _frozen_importlib_external, loaded second and by then with an import system to use, contains eight. init_importlib hands sys and _imp in as arguments, which is why _bootstrap.py never writes import sys.

Ten cells, and each one watches rather than describes. The thirty three frozen names split into the three arrays frozen.c keeps them in, which the flag treats differently: three for the import system that no setting can remove, nineteen for what a bare startup needs, eleven hello world modules for the test suite. A frozen code object pulled out of the binary and weighed. os asked five questions about where it came from. The flag turned off inside the running interpreter, so you can watch os move from FrozenImporter to SourceFileLoader and back without leaving the notebook. A startup run twice under -v, counting the files the second run reads that the first one does not.

The facts worth keeping. A frozen module knows perfectly well which file it would have opened: find_spec works the path out from sys._stdlib_dir and parks it on the spec as loader_state, and the loader copies it onto __file__, so the spec says frozen and __file__ says a real path and both are true. That is the whole reason a traceback through frozen code is readable, because linecache sees a filename starting with <frozen and reads __file__ out of the globals instead. The cost is not where you would guess: both paths end in the same marshal.loads over the same bytes, so what freezing removes is the finder search and the file read in front of it, and since every file involved is an already compiled .pyc, it is not saving compilation either.

Two Tier 1 recordings put a number on it, and the interesting part is that the two builds disagree about the default. initconfig.c turns frozen modules off under Py_DEBUG, so the program was rewritten to ask for on and off explicitly on every child and report the default as a measured fact rather than an assumption. Freezing gives back 14.2 percent of a release startup and 9.9 percent of a debug one. Both measurements alternate the two cases round by round, because running one case forty times and then the other measures the page cache.

Four new glossary terms: frozen module, import bootstrap, loader state and module alias. Six diagrams and twenty one citations. Both READMEs updated. GLOSSARY.md is 226 terms and CLAIMS.md is 589 claims across 75 lessons.

Nine of the ten cells run end to end in a browser. The tenth needs a second process, so it prints a line saying so.