M07. The collector nobody calls

GLOSSARY.md

@@ -8,7 +8,7 @@ This file is generated from `pyxray/src/pyxray/glossary.py`. Edit that and run `
 
 ## Index
 
-[ASDL](#asdl) | [Argument Clinic](#argument-clinic) | [C stack](#c-stack) | [C3 linearization](#c3-linearization) | [DISABLE](#disable) | [EXTENDED_ARG](#extended_arg) | [GC pre header](#gc-pre-header) | [JIT](#jit) | [MRO](#mro) | [PEG parser](#peg-parser) | [PYTHONMALLOC](#pythonmalloc) | [PyVarObject](#pyvarobject) | [Pyodide](#pyodide) | [WebAssembly](#webassembly) | [abstract interpreter](#abstract-interpreter) | [abstract syntax tree](#abstract-syntax-tree) | [adaptive counter](#adaptive-counter) | [adaptive instruction](#adaptive-instruction) | [allocator domain](#allocator-domain) | [arena](#arena) | [assembler](#assembler) | [backtrace](#backtrace) | [basic block](#basic-block) | [biased reference counting](#biased-reference-counting) | [binding](#binding) | [block](#block) | [blurb](#blurb) | [borrowed reference](#borrowed-reference) | [bound method](#bound-method) | [bytecode](#bytecode) | [cached hash](#cached-hash) | [calling convention](#calling-convention) | [cases generator](#cases-generator) | [cell](#cell) | [class cell](#class-cell) | [closure](#closure) | [code generation](#code-generation) | [code object](#code-object) | [code point](#code-point) | [code unit](#code-unit) | [coding cookie](#coding-cookie) | [cold block](#cold-block) | [compact dict](#compact-dict) | [compact int](#compact-int) | [compact string](#compact-string) | [computed goto](#computed-goto) | [configure](#configure) | [constant folding](#constant-folding) | [control flow graph](#control-flow-graph) | [copy and patch](#copy-and-patch) | [cycle collector](#cycle-collector) | [data descriptor](#data-descriptor) | [data stack](#data-stack) | [deallocation](#deallocation) | [debug build](#debug-build) | [debug hooks](#debug-hooks) | [deferred reference counting](#deferred-reference-counting) | [deoptimization](#deoptimization) | [descriptor](#descriptor) | [devguide](#devguide) | [digit array](#digit-array) | [dispatch](#dispatch) | [dispatch table](#dispatch-table) | [environment changed](#environment-changed) | [eval loop](#eval-loop) | [evaluation order](#evaluation-order) | [exact type check](#exact-type-check) | [exception table](#exception-table) | [executor](#executor) | [f string](#f-string) | [finalized bit](#finalized-bit) | [finalizer](#finalizer) | [frame](#frame) | [frame object](#frame-object) | [free list](#free-list) | [free threaded build](#free-threaded-build) | [free variable](#free-variable) | [gdb](#gdb) | [generated file](#generated-file) | [generation](#generation) | [grammar](#grammar) | [guard](#guard) | [header file](#header-file) | [heap type](#heap-type) | [immortal object](#immortal-object) | [immortalization](#immortalization) | [indent and dedent](#indent-and-dedent) | [inline cache](#inline-cache) | [inline values](#inline-values) | [instance dictionary](#instance-dictionary) | [instruction](#instruction) | [instruction DSL](#instruction-dsl) | [instruction pointer](#instruction-pointer) | [instrumented instruction](#instrumented-instruction) | [interning](#interning) | [left recursion](#left-recursion) | [line table](#line-table) | [magic number](#magic-number) | [marshal](#marshal) | [metaclass](#metaclass) | [micro operation](#micro-operation) | [mimalloc](#mimalloc) | [mimalloc heap](#mimalloc-heap) | [monitoring events](#monitoring-events) | [new reference](#new-reference) | [object](#object) | [object header](#object-header) | [obmalloc](#obmalloc) | [oparg](#oparg) | [opcode](#opcode) | [over allocation](#over-allocation) | [ownership](#ownership) | [parser generator](#parser-generator) | [pdb](#pdb) | [pointer](#pointer) | [pool](#pool) | [pool header](#pool-header) | [probe sequence](#probe-sequence) | [product type](#product-type) | [profile guided optimization](#profile-guided-optimization) | [pseudo instruction](#pseudo-instruction) | [pyc file](#pyc-file) | [pyconfig](#pyconfig) | [quickening](#quickening) | [reentrancy](#reentrancy) | [reference count](#reference-count) | [reference cycle](#reference-cycle) | [reference leak](#reference-leak) | [regen](#regen) | [regrtest](#regrtest) | [replacement field](#replacement-field) | [resource](#resource) | [resurrection](#resurrection) | [scope](#scope) | [segment](#segment) | [segmentation fault](#segmentation-fault) | [short circuiting](#short-circuiting) | [side exit](#side-exit) | [single character cache](#single-character-cache) | [size class](#size-class) | [slot](#slot) | [slot wrapper](#slot-wrapper) | [small int cache](#small-int-cache) | [small integer cache](#small-integer-cache) | [soft keyword](#soft-keyword) | [specialization](#specialization) | [specialization family](#specialization-family) | [split table](#split-table) | [stack depth](#stack-depth) | [stack effect](#stack-effect) | [stack reference](#stack-reference) | [static type](#static-type) | [stencil](#stencil) | [stolen reference](#stolen-reference) | [string kind](#string-kind) | [struct](#struct) | [sum type](#sum-type) | [symbol table](#symbol-table) | [symbol table pass](#symbol-table-pass) | [t string](#t-string) | [tagged integer](#tagged-integer) | [tagged pointer](#tagged-pointer) | [tail call](#tail-call) | [test case](#test-case) | [tier one](#tier-one) | [tier two](#tier-two) | [token](#token) | [tokenizer](#tokenizer) | [tool id](#tool-id) | [trace](#trace) | [trace function](#trace-function) | [traceback](#traceback) | [type object](#type-object) | [underflow](#underflow) | [unwinding](#unwinding) | [value stack](#value-stack) | [varint](#varint) | [watcher](#watcher) | [weak reference](#weak-reference) | [weakref callback](#weakref-callback) | [weakref offset](#weakref-offset) | [zero cost exceptions](#zero-cost-exceptions)
+[ASDL](#asdl) | [Argument Clinic](#argument-clinic) | [C stack](#c-stack) | [C3 linearization](#c3-linearization) | [DISABLE](#disable) | [EXTENDED_ARG](#extended_arg) | [GC pre header](#gc-pre-header) | [JIT](#jit) | [MRO](#mro) | [PEG parser](#peg-parser) | [PYTHONMALLOC](#pythonmalloc) | [PyVarObject](#pyvarobject) | [Pyodide](#pyodide) | [WebAssembly](#webassembly) | [abstract interpreter](#abstract-interpreter) | [abstract syntax tree](#abstract-syntax-tree) | [adaptive counter](#adaptive-counter) | [adaptive instruction](#adaptive-instruction) | [allocator domain](#allocator-domain) | [arena](#arena) | [assembler](#assembler) | [backtrace](#backtrace) | [basic block](#basic-block) | [biased reference counting](#biased-reference-counting) | [binding](#binding) | [block](#block) | [blurb](#blurb) | [borrowed reference](#borrowed-reference) | [bound method](#bound-method) | [bytecode](#bytecode) | [cached hash](#cached-hash) | [calling convention](#calling-convention) | [cases generator](#cases-generator) | [cell](#cell) | [class cell](#class-cell) | [closure](#closure) | [code generation](#code-generation) | [code object](#code-object) | [code point](#code-point) | [code unit](#code-unit) | [coding cookie](#coding-cookie) | [cold block](#cold-block) | [collection threshold](#collection-threshold) | [compact dict](#compact-dict) | [compact int](#compact-int) | [compact string](#compact-string) | [computed goto](#computed-goto) | [configure](#configure) | [constant folding](#constant-folding) | [control flow graph](#control-flow-graph) | [copy and patch](#copy-and-patch) | [cycle collector](#cycle-collector) | [data descriptor](#data-descriptor) | [data stack](#data-stack) | [deallocation](#deallocation) | [debug build](#debug-build) | [debug hooks](#debug-hooks) | [deferred reference counting](#deferred-reference-counting) | [deoptimization](#deoptimization) | [descriptor](#descriptor) | [devguide](#devguide) | [digit array](#digit-array) | [dispatch](#dispatch) | [dispatch table](#dispatch-table) | [environment changed](#environment-changed) | [eval loop](#eval-loop) | [evaluation order](#evaluation-order) | [exact type check](#exact-type-check) | [exception table](#exception-table) | [executor](#executor) | [f string](#f-string) | [finalized bit](#finalized-bit) | [finalizer](#finalizer) | [frame](#frame) | [frame object](#frame-object) | [free list](#free-list) | [free threaded build](#free-threaded-build) | [free variable](#free-variable) | [gdb](#gdb) | [generated file](#generated-file) | [generation](#generation) | [grammar](#grammar) | [guard](#guard) | [header file](#header-file) | [heap type](#heap-type) | [immortal object](#immortal-object) | [immortalization](#immortalization) | [indent and dedent](#indent-and-dedent) | [inline cache](#inline-cache) | [inline values](#inline-values) | [instance dictionary](#instance-dictionary) | [instruction](#instruction) | [instruction DSL](#instruction-dsl) | [instruction pointer](#instruction-pointer) | [instrumented instruction](#instrumented-instruction) | [interning](#interning) | [left recursion](#left-recursion) | [line table](#line-table) | [magic number](#magic-number) | [marshal](#marshal) | [metaclass](#metaclass) | [micro operation](#micro-operation) | [mimalloc](#mimalloc) | [mimalloc heap](#mimalloc-heap) | [monitoring events](#monitoring-events) | [new reference](#new-reference) | [object](#object) | [object header](#object-header) | [obmalloc](#obmalloc) | [oparg](#oparg) | [opcode](#opcode) | [over allocation](#over-allocation) | [ownership](#ownership) | [parser generator](#parser-generator) | [pdb](#pdb) | [permanent generation](#permanent-generation) | [pointer](#pointer) | [pool](#pool) | [pool header](#pool-header) | [probe sequence](#probe-sequence) | [product type](#product-type) | [profile guided optimization](#profile-guided-optimization) | [pseudo instruction](#pseudo-instruction) | [pyc file](#pyc-file) | [pyconfig](#pyconfig) | [quickening](#quickening) | [reentrancy](#reentrancy) | [reference count](#reference-count) | [reference cycle](#reference-cycle) | [reference leak](#reference-leak) | [regen](#regen) | [regrtest](#regrtest) | [replacement field](#replacement-field) | [resource](#resource) | [resurrection](#resurrection) | [scope](#scope) | [segment](#segment) | [segmentation fault](#segmentation-fault) | [short circuiting](#short-circuiting) | [side exit](#side-exit) | [single character cache](#single-character-cache) | [size class](#size-class) | [slot](#slot) | [slot wrapper](#slot-wrapper) | [small int cache](#small-int-cache) | [small integer cache](#small-integer-cache) | [soft keyword](#soft-keyword) | [specialization](#specialization) | [specialization family](#specialization-family) | [split table](#split-table) | [stack depth](#stack-depth) | [stack effect](#stack-effect) | [stack reference](#stack-reference) | [static type](#static-type) | [stencil](#stencil) | [stolen reference](#stolen-reference) | [string kind](#string-kind) | [struct](#struct) | [sum type](#sum-type) | [symbol table](#symbol-table) | [symbol table pass](#symbol-table-pass) | [t string](#t-string) | [tagged integer](#tagged-integer) | [tagged pointer](#tagged-pointer) | [tail call](#tail-call) | [test case](#test-case) | [tier one](#tier-one) | [tier two](#tier-two) | [token](#token) | [tokenizer](#tokenizer) | [tool id](#tool-id) | [trace](#trace) | [trace function](#trace-function) | [traceback](#traceback) | [type object](#type-object) | [underflow](#underflow) | [unwinding](#unwinding) | [value stack](#value-stack) | [varint](#varint) | [watcher](#watcher) | [weak reference](#weak-reference) | [weakref callback](#weakref-callback) | [weakref offset](#weakref-offset) | [zero cost exceptions](#zero-cost-exceptions)
 
 ## Reading the source
 
@@ -1206,6 +1206,22 @@ The object records which thread created it. That thread adds and subtracts in a
 
 Also written `ob_ref_local`, `ob_ref_shared`, `ob_tid`. First met in M06. See also [free threaded build](#free-threaded-build), [reference count](#reference-count), [object header](#object-header). In the source: [`Include/refcount.h:105-117@v3.15.0rc1#_Py_REFCNT`](https://github.com/python/cpython/blob/v3.15.0rc1/Include/refcount.h#L105-L117).
 
+### permanent generation
+
+**A fourth list the collector keeps and never looks at.**
+
+`gc.freeze()` moves everything currently tracked into it, and nothing comes back out until `gc.unfreeze()`. It exists for servers that load their data and then fork worker processes: the collector writing to an object's header is enough to make the operating system copy that page into every child, so a process that stops walking its old data stops copying it.
+
+Also written `gc.freeze`, `gc.get_freeze_count`. First met in M07. See also [generation](#generation), [cycle collector](#cycle-collector). In the source: [`Python/gc.c:1735-1743@v3.15.0rc1#_PyGC_Freeze`](https://github.com/python/cpython/blob/v3.15.0rc1/Python/gc.c#L1735-L1743).
+
+### collection threshold
+
+**How high a generation's counter has to get before the collector runs.**
+
+The default is 2000 for the youngest and 10 for the two above it. The youngest counts tracked objects that are alive right now, so it goes up when you make one and down when one is freed. The other two count collections of the generation below, so ten passes over the young list is what earns one pass over the middle.
+
+Also written `gc.get_threshold`, `gc.get_count`. First met in M07. See also [generation](#generation), [cycle collector](#cycle-collector). In the source: [`Include/internal/pycore_interp_structs.h:271-278@v3.15.0rc1#GC_GENERATION_INIT`](https://github.com/python/cpython/blob/v3.15.0rc1/Include/internal/pycore_interp_structs.h#L271-L278).
+
 ### immortalization
 
 **Switching an object that is already alive over to the immortal count.**

README.md

@@ -148,6 +148,7 @@ A lesson can also end with a boss fight, which is a problem the text does not so
 | M04 | [Who owns what](lessons/m04-who-owns-what/m04.ipynb) | The number every object carries and the protocol for who is allowed to change it, why sys.getrefcount always answers one high and how to read the honest number out of memory instead, the eight different ways of holding an object that are all worth exactly one, owned references against borrowed ones in both the C API and the bytecode, destruction happening inside the instruction that dropped the count rather than later, why a destructor never sees a half updated container, and the two objects holding each other whose counts stop at one | M7 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/m04-who-owns-what/m04.ipynb) |
 | M05 | [The ones that never die](lessons/m05-the-ones-that-never-die/m05.ipynb) | The objects whose reference count nothing ever changes, why the interpreter has them at all, and how to find every one of them from Python. Reading the immortal marker out of the header and checking it against sys._is_immortal, the whole roster from None and the small integers to the 256 one character strings and every static type, the rule that separates a static type from a heap type in builtins, a binary search that finds the small integer range moving from 256 to 1024 between 3.14 and 3.15, the forty odd kilobytes the process never gives back in exchange, why interned and immortal are two different bits, and why sys.intern is the closest thing to a way in and still does not do it | M7 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/m05-the-ones-that-never-die/m05.ipynb) |
 | M06 | [Two counts, one object](lessons/m06-two-counts-one-object/m06.ipynb) | What has to change about reference counting before two threads can run Python at the same time, and the three cheaper answers the free threaded build reaches for before it pays for an atomic. Playing the lost update out by hand, reading the count and the type pointer out of an object header, the deferred marker at PY_SSIZE_T_MAX over 8 and the rule that gives it to top level functions and classes and modules but not to nested functions, the count split into a plain local half for the owning thread and an atomic shared half for everybody else, the two flag bits hiding in the bottom of the shared field, and why the header growing by sixteen bytes costs a list and a dict nothing at all | M7 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/m06-two-counts-one-object/m06.ipynb) |
+| M07 | [The collector nobody calls](lessons/m07-the-collector-nobody-calls/m07.ipynb) | The cycle collector running thousands of times in a program that never mentions it, what makes it start, and why the most expensive pass is the one CPython works hardest to skip. The counter that goes up when a tracked object is made and down when one is freed, the bit set at two thousand that the interpreter checks between bytecode instructions, the three lists and the fact that only the first of them counts objects, following one dictionary as it gets promoted from generation 0 to 2, the same cycle being freed by one pass and invisible to another purely because of age, the quarter rule that turns hundreds of full passes into a handful, the collector untracking one layer of nested tuples per pass, and gc.freeze for servers that fork | M7 | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tamnd/cpython-internals/blob/main/lessons/m07-the-collector-nobody-calls/m07.ipynb) |
 
 More are landing in order. [lessons/README.md](lessons/README.md) explains how one is put together and how to run them locally.