optree v0.20.0

optree v0.20.0

Added

• Preserve dict insertion order on unflatten for the Python tree_flatten_one_level path mirroring the C++ path by @XuehaiPan in #280.
• Add built-in frozendict support for Python 3.15+ (PEP 814), with PyTreeKind.FROZENDICT, optree.treespec_frozendict(), optree.treespec.frozendict(), and optree.typing.FrozenDict by @XuehaiPan in #274.
• Migrate to Python_FIND_ABI for CMake's FindPython by @XuehaiPan in #292.
• Add Python 3.15 and Python 3.15t support by @XuehaiPan in #293.

Changed

• Update minimal version of typing-extensions to 4.10.0 for typing_extensions.TypeIs by @XuehaiPan in #285.

Fixed

• Define Py_GIL_DISABLED for free-threaded debug builds on Windows when building the C extension to work around an upstream CMake FindPython bug by @XuehaiPan in #285.
• Correct rare-case bugs from adversarial audit across deadlocks, caching, serialization, memory safety, and Python frontend by @XuehaiPan in #290.

Details of fixed rare-case bugs

• Fix a deadlock when registering or unregistering a pytree node concurrently with flattening, caused by releasing the GIL while holding the registry lock by @XuehaiPan in #290.
• Fix a deadlock when a duplicate registration or an unregistration of an unregistered type formats its error message, which runs the type's __repr__ while the registry lock is held and wedges any thread that is flattening by @XuehaiPan in #290.
• Fix registering an already-registered collections.namedtuple subclass or PyStructSequence emitting the override warning even though the registration is rejected, which raised UserWarning instead of ValueError under warnings-as-errors by @XuehaiPan in #290.
• Fix a deadlock when unregistering a pytree node drops the last reference to its flatten or unflatten function, so a __del__ or weakref callback re-entered the registry while its lock was held by @XuehaiPan in #290.
• Fix a deadlock while flattening on free-threading builds, caused by acquiring the non-recursive dictionary insertion order lock twice by @XuehaiPan in #290.
• Fix a spurious SystemError from optree._C.get_registry_size() when a concurrent registration slipped between its two reads by @XuehaiPan in #290.
• Fix optree.tree_iter() hanging uninterruptibly when the is_leaf predicate or a custom flatten function advances the same iterator, now reported as a RuntimeError by @XuehaiPan in #290.
• Fix the type caches handing an interpreter a value owned by another one, by keying them on the interpreter in addition to the type address by @XuehaiPan in #290.
• Fix the type caches retaining entries for a finalized interpreter, which leaked immortal keys and could be inherited by a fresh interpreter reusing the same ID by @XuehaiPan in #290.
• Fix a deadlock in the PyStructSequence field cache, caused by re-acquiring the GIL while still holding the cache lock by @XuehaiPan in #290.
• Fix the type caches publishing an entry that a failed cleanup registration would orphan, leaving a value owned by an interpreter with no callback able to evict it, whether the publish preceded that registration or raced another thread still performing it by @XuehaiPan in #290.
• Fix optree.structseq_fields() and the struct sequence accessors reporting the trailing hidden field names for unnamed sequence slots, by mapping each field to the slot its member offset names; the pure Python fallback, which cannot read those offsets, recovers the positions from a probe instance instead of assuming the unnamed slots trail the named ones by @XuehaiPan in #290.
• Fix the treespec repr printing the raw unnamed-field marker as if it were a keyword argument, now rendered as <unnamed@N> by @XuehaiPan in #290.
• Fix StructSequenceEntry reporting an unnamed sequence slot as field='unnamed field', which reads as a real field name, now reported by its index by @XuehaiPan in #290.
• Fix PyTreeSpec.__setstate__() accepting a malformed state, which could read out of bounds or abort the interpreter when the treespec was later used by @XuehaiPan in #290.
• Fix PyTreeSpec.__getstate__() returning the treespec's internal mutable containers, so mutating the pickled state corrupted an otherwise immutable treespec by @XuehaiPan in #290.
• Fix PyTreeSpec.__setstate__() borrowing the key lists from the state it was given rather than copying them, so mutating that state afterwards silently reordered the restored treespec by @XuehaiPan in #290.
• Fix pickling a PyTreeSpec with protocol 0 or 1 aborting the interpreter, by reducing through copyreg.__newobj__ by @XuehaiPan in #290.
• Fix PyTreeSpec.compose(), PyTreeSpec.broadcast_to_common_suffix(), treespec_transform(), and treespec_from_collection() silently rebinding a custom node to a different registration when an empty namespace adopted a non-empty one by @XuehaiPan in #290.
• Fix PyTreeSpec.broadcast_to_common_suffix() sorting the argument treespec's dictionary keys in place while building its key-mismatch error message, corrupting a treespec the caller still holds by @XuehaiPan in #290.
• Fix the dictionary key order being lost when a treespec built under the global namespace is promoted to an insertion-ordered namespace by @XuehaiPan in #290.
• Fix treespec_from_collection() on a leaf reporting an escalated UserWarning as a confusing SystemError, by checking the return value of PyErr_WarnEx() by @XuehaiPan in #290.
• Fix treespec_from_collection() keeping the caller's namespace on a leaf or None root, which made otherwise identical treespecs compare unequal by @XuehaiPan in #290.
• Fix treespec_is_prefix() and treespec_is_suffix() comparing against a stale subtree when a dictionary node's keys had been reordered by @XuehaiPan in #290.
• Fix the tree iterator never reporting or clearing its is_leaf predicate to the garbage collector, leaking any reference cycle that passes through it by @XuehaiPan in #290.
• Fix a list shrunk by an is_leaf predicate part way through flattening being read out of bounds on Python versions before 3.13.0a4, now raising IndexError by @XuehaiPan in #290.
• Fix a reference cycle passing through a registered custom type not being collectable once the registry no longer holds the registration, by reporting the registration's members to the garbage collector when a treespec's own nodes hold every reference to it; a cycle spanning several treespecs remains uncollectable by @XuehaiPan in #290.
• Fix a deeply nested treespec overflowing the native stack and crashing in treespec_paths(), treespec_accessors(), and PyTreeSpec.broadcast_to_common_suffix() instead of raising RecursionError by @XuehaiPan in #290.
• Fix DataclassEntry and AttrsEntry resolving an integer entry against every declared field rather than the children the registration emitted, which returned the wrong attribute for a class holding a metadata or non-init field, now read from the record register_node() leaves on the class itself by @XuehaiPan in #290.
• Fix optree.dataclasses.register_node() and optree.integrations.attrs.register_node() leaving a class in a half-registered state that could never be registered again: the registry entry and the field marker are now committed together, and a failure of either rolls the other back by @XuehaiPan in #290.
• Fix optree.dataclasses.register_node() silently dropping InitVar pseudo-fields, which are neither children nor metadata and cannot round-trip, now rejected with a pointer to the generic API by @XuehaiPan in #290.
• Fix tree_broadcast_common() and broadcast_common() applying the caller's is_leaf predicate to an internal sentinel tree, which could raise from the predicate or collapse a filled subtree and under-replicate by @XuehaiPan in #290.
• Fix PyTreeEntry equality and hashing comparing the bytecode of __call__() and codify() rather than the methods themselves, so two entry classes that happened to share an implementation compared equal by @XuehaiPan in #290.
• Fix PyTreeAccessor overriding __eq__() while inheriting tuple.__ne__(), so comparing one with a plain tuple of equal entries reported False for both == and != by @XuehaiPan in #290.
• Fix GetAttrEntry.codify() emitting invalid attribute access for a field name that is not an identifier, now rendered as a getattr() call by @XuehaiPan in #290.
• Fix prefix_errors() raising AssertionError instead of reporting an error for a custom node whose per-instance entries differ while its metadata does not, a pair broadcast_prefix() accepts by @XuehaiPan in #290.
• Fix prefix_errors() raising TypeError while formatting a dictionary key mismatch when the keys have different types, now ordered with total_order_sorted() by @XuehaiPan in #290.
• Fix tree_transpose_map(), tree_transpose_map_with_path(), tree_transpose_map_with_accessor(), and tree_partition() rejecting a leafless outer structure when an explicit inner_treespec leaves nothing to infer by @XuehaiPan in #290.
• Fix the tree_broadcast_map() family flattening a custom node twice when called with a single input tree, which broke a one-shot flatten function and contradicted the documented equivalence with tree_map() by @XuehaiPan in #290.
• Fix treespec_entry() and treespec_child() annotating their index as int while the runtime accepts any SupportsInt or SupportsIndex object by @XuehaiPan in #290.
• Fix optree.utils.total_order_sorted() mistaking a TypeError raised by the caller's key callback for a comparison failure and silently returning the sequence unsorted, and calling the callback twice per element by @XuehaiPan in #290.
• Fix a dictionary whose keys cannot be compared being flattened in a partially sorted order instead of insertion order, when a comparison raised part way through the sort by @XuehaiPan in #290.
• Fix tree_broadcast_common() documenting that its results share one structure and broadcast_common() documenting that it returns two pytrees rather than two lists of leaves by @XuehaiPan in #290.

Full Changelog: v0.19.1...v0.20.0