Skip to content

Conversation

@andrewleech
Copy link
Contributor

Summary

Adds enum support (Enum, IntEnum, Flag, IntFlag, StrEnum, auto, unique) via micropython-lib submodule and implements minimal metaclass features needed to support it and python-statemachine.

Built on dpgeorge's b31c1de metaclass branch from 2020. Reviewed Jos Verlinde's PR #18362 (PEP 3115/487 metaclass support), then focused on the subset needed for enum and python-statemachine without implementing full PEP 487 init_subclass.

Metaclass Features

Implemented as optional ROM-level features with separate config flags:

Feature Config Flag Bytes Default Level Status
__init__ invocation MICROPY_PY_METACLASS_INIT +136 CORE Included
Operator support MICROPY_PY_METACLASS_OPS +240 EXTRA Included
Property/method lookup MICROPY_PY_METACLASS_PROPERTIES +88 EXTRA Included
__prepare__ (PEP 3115) MICROPY_PY_METACLASS_PREPARE +84 FULL Included
__init_subclass__ (PEP 487) - - - Not included
Metaclass conflict detection - - - Not included

Total C overhead: 540 bytes when all features enabled (FULL level).

The init feature enables python-statemachine's class registration pattern. Properties enable accessing .events and .states on the class. Operators enable len(EnumClass) and member in EnumClass. prepare enables enum's auto() value generation.

Enum Features

Complete implementation via micropython-lib submodule, based on PEP 435 (basic enums) and PEP 663 (Flag additions):

  • Enum - base enumeration with member management
  • IntEnum - integer-valued with arithmetic (duck-typed, not true int subclass)
  • Flag - bitwise flags with |, &, ^, ~ operators
  • IntFlag - integer-compatible flags
  • StrEnum - string-valued (Python 3.11+)
  • auto() - automatic value assignment
  • @unique - duplicate value prevention

Frozen as bytecode: ~5,428 bytes.

Modular structure with lazy loading:

  • core.py - Enum, IntEnum, EnumMeta (~1.5KB frozen)
  • flags.py - Flag, IntFlag (~500 bytes frozen, loaded on demand)
  • extras.py - StrEnum, auto, unique (~450 bytes frozen, loaded on demand)

Total implementation: 540 bytes C + 5,428 bytes Python = 5,968 bytes (1.6% increase on STM32 PYBV10).

CPython Compatibility

Tested against CPython 3.13's official enum test suite:

  • 448 tests run
  • 445 passed (99.3%)
  • 3 failed (Flag combination membership edge case, now fixed)

Works:

  • All class-based enum definitions
  • auto() value generation
  • Explicit and mixed values
  • Iteration, lookup, comparison, repr
  • Flag bitwise operations
  • @unique decorator
  • Type mixins (int, str, float, date)
  • Pickling/unpickling
  • members, dir(), introspection
  • Thread-safe enum creation

Not implemented:

  • Functional API (Enum('Name', 'A B C')) - use class syntax instead
  • missing(), ignore(), generate_next_value() hooks
  • Boundary modes (STRICT, CONFORM, EJECT, KEEP)

Known limitation: IntEnum members fail isinstance(member, int) check but all operations work correctly. Documented in tests/cpydiff/types_enum_isinstance.py.

STM32 Size Measurements (PYBV10)

Individual feature costs:

Configuration Size (bytes) Overhead % Change
Baseline (no features) 368,108 - -
+ METACLASS_INIT 368,244 +136 +0.037%
+ METACLASS_OPS 368,348 +240 +0.065%
+ METACLASS_PROPERTIES 368,196 +88 +0.024%
+ METACLASS_PREPARE 368,192 +84 +0.023%

Cumulative by ROM level:

ROM Level Features Size (bytes) Overhead % Change
MINIMAL None 368,108 baseline 0.000%
CORE INIT 368,244 +136 +0.037%
EXTRA INIT+OPS+PROPERTIES 368,572 +464 +0.126%
FULL All metaclass 368,648 +540 +0.147%

With enum module frozen:

Configuration Size (bytes) Overhead
EXTRA + enum frozen 374,016 +5,444 bytes from EXTRA baseline
FULL + enum frozen 374,076 +5,428 bytes from FULL baseline

Note: cumulative cost (540 bytes) is less than sum of individual features (548 bytes) due to code sharing.

Testing

Unix port coverage variant:

  • 1053 tests passed
  • 22 tests skipped
  • 2 pre-existing failures unrelated to these changes (extmod/select_poll_fd.py, misc/sys_settrace_features.py)

Tests added:

  • tests/basics/enum_auto.py - auto() value generation
  • tests/basics/enum_flag.py - Flag and IntFlag operations
  • tests/basics/enum_strenum.py - StrEnum functionality
  • tests/basics/class_metaclass_init.py - metaclass init
  • tests/basics/class_metaclass_prepare.py - prepare support
  • tests/basics/class_metaclass_property.py - property lookup
  • tests/cpydiff/types_enum_isinstance.py - documents IntEnum limitation

Tests fixed:

  • tests/basics/metaclass.py - now passes with new .exp file (was failing before due to missing init support)

Implementation Details

Core C files modified:

  • py/mpconfig.h - ROM level config flags
  • py/objtype.c - metaclass new/call/init support, type checking fixes, DEBUG_printf removal
  • py/modbuiltins.c - prepare integration in build_class
  • py/objobject.c - object.new custom metaclass support
  • py/vm.c - LOAD_ATTR fast path fix for type objects

Enum module:

  • lib/micropython-lib - submodule updated to include enum package
  • ports/unix/variants/standard/manifest.py - require("enum")
  • ports/unix/variants/coverage/manifest.py - require("enum")

The enum implementation lives in micropython-lib (separate repository) and is included via submodule reference. Both Unix variants (standard and coverage) freeze the enum package into their builds.

dpgeorge and others added 8 commits November 14, 2025 12:57
Signed-off-by: Damien George <damien@micropython.org>
Adds four new configuration flags to control metaclass functionality:
- MICROPY_PY_METACLASS_INIT: Enable metaclass __init__ invocation
- MICROPY_PY_METACLASS_OPS: Enable metaclass operator overloading
- MICROPY_PY_METACLASS_PROPERTIES: Enable metaclass properties/methods
- MICROPY_PY_METACLASS_PREPARE: Enable __prepare__ method (PEP 3115)

These flags allow ports to balance functionality against code size,
enabling enum support and python-statemachine compatibility.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Fixes type checking throughout the codebase to support custom metaclasses:
- py/vm: Fix LOAD_ATTR fast path to avoid treating type objects as instances
- py/objobject: Update object.__new__ to accept custom metaclasses
- py/objtype: Replace assert() checks with mp_obj_is_subclass_fast()

Adds metaclass method support:
- type_make_new now looks up custom __new__ through inheritance chain
- type_call checks for custom __call__ on metaclass
- Enables metaclass customization of class creation and instantiation

Removes DEBUG_printf statements for cleaner production code.

This enables enum.Enum and python-statemachine compatibility.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Implements PEP 3115 by calling __prepare__ before executing class body.
The __prepare__ method allows metaclasses to return a custom namespace
dict (e.g., OrderedDict) for tracking member insertion order.

Execution order in __build_class__:
1. Determine metaclass
2. Call __prepare__(name, bases) if it exists
3. Execute class body in returned namespace
4. Call metaclass(name, bases, namespace) to create class

This is required for enum.auto() to generate sequential values based
on definition order. Guarded by MICROPY_PY_METACLASS_PREPARE.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Metaclass tests:
- class_metaclass_init.py: Test __init__ invocation on metaclass
- class_metaclass_prepare.py: Test __prepare__ method (PEP 3115)
- class_metaclass_property.py: Test properties and methods on metaclasses

Enum tests:
- enum_auto.py: Test auto() value generation
- enum_flag.py: Test Flag and IntFlag bitwise operations
- enum_strenum.py: Test StrEnum string-valued enums

CPython difference:
- types_enum_isinstance.py: Document isinstance() behavior difference

These tests validate the metaclass features added in previous commits
and provide compatibility verification with micropython-lib enum package.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Adds enum package from micropython-lib to unix port variants:
- coverage variant: For testing metaclass and enum features
- standard variant: For general enum availability

The enum package provides Enum, IntEnum, Flag, IntFlag, StrEnum,
and auto() compatible with CPython's enum module.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
Updates micropython-lib submodule to include enum package with fixes:
- Fix __import__() to use positional args (MicroPython compatibility)
- Fix IntFlag bitwise operations to use _value_ attribute directly
- Add Flag, IntFlag, StrEnum, auto(), and unique() support

These changes enable full enum functionality with the metaclass features
added in previous commits.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
The metaclass.py test now produces output because metaclass __init__
invocation is now implemented and functional. Generate the expected
output file based on the correct behavior.

Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
@github-actions
Copy link

github-actions bot commented Nov 14, 2025

Code size report:

Reference:  esp32/boards: Add Silicognition ManT1S board definition. [27544a2]
Comparison: tests/basics: Add skip detection for metaclass and enum tests. [merge of 48bae9a]
  mpy-cross: +1768 +0.466% [incl +64(data)]
   bare-arm:  +616 +1.090% 
minimal x86: +1726 +0.917% [incl +64(data)]
   unix x64: +16824 +1.960% standard[incl +2368(data)]
      stm32: +1344 +0.340% PYBV10
     mimxrt: +1480 +0.393% TEENSY40
        rp2: +1376 +0.149% RPI_PICO_W
       samd: +1540 +0.566% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32: +1525 +0.335% VIRT_RV32

@codecov
Copy link

codecov bot commented Nov 14, 2025

Codecov Report

❌ Patch coverage is 81.11111% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.24%. Comparing base (2762fe6) to head (48bae9a).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
py/objtype.c 77.48% 34 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #18416      +/-   ##
==========================================
- Coverage   98.38%   98.24%   -0.15%     
==========================================
  Files         171      171              
  Lines       22294    22447     +153     
==========================================
+ Hits        21933    22052     +119     
- Misses        361      395      +34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Adds feature detection to skip tests when required features are not
available (e.g. in minimal/standard build variants).

- class_metaclass_init.py: Skip if MICROPY_PY_METACLASS_INIT disabled
- class_metaclass_property.py: Skip if MICROPY_PY_METACLASS_PROPERTIES disabled
- class_metaclass_prepare.py: Already had skip for __prepare__
- enum_auto.py: Add enum import check before __prepare__ check
- enum_flag.py: Skip if enum module not available
- enum_strenum.py: Skip if enum module not available
- metaclass.py: Skip if MICROPY_PY_METACLASS_INIT disabled

This fixes CI failures on build variants where these features are
disabled by config.

Signed-off-by: Andrew Leech <andrew@alelec.net>
Signed-off-by: Andrew Leech <andrew.leech@planetinnovation.com.au>
@Josverl
Copy link
Contributor

Josverl commented Nov 14, 2025

Thanks Andrew,
On first read I much prefer this PR over my attempt due to lower code size and proof-of-pudding by the Enum and validation of that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants