[WIP] PEP 3115 metaclass and PEP 487 __init_subclass__

[Josverl]

This PR aims to adds optional support for two Python features to MicroPython.

• PEP 3115: Metaclass support with metaclass= keyword syntax
• PEP 487: __init_subclass__ automatic hook with full **kwargs support

The goal of this PR is to discuss if either or both of the PEPs are worth the trade-off of features versus firmware size.

Note : Based on the code size report there are some unguarded features that impact code size that will need to be addressed

These would enable:

1. Modern Python class features for MicroPython with minimal overhead
2. Three configuration options for different resource constraints
3. Full CPython compatibility when enabled (PEP 3115 + PEP 487)
4. Zero impact when disabled - all features behind config guards
5. Comprehensive test coverage with graceful feature detection

• Enables typing module - Supports @dataclass_transform decorator
• Enables dataclasses - Foundation for future dataclass implementation
• Enables generic types - User-defined generics via metaclasses
• Industry standard patterns - Plugins, registries now
• Minimal cost - Only 0.6% size increase for full modern class features

All features are intended to be optional, guarded by configuration flags, and designed for minimal code size impact while maintaining near full CPython compatibility.

Modified Files

1. py/mpconfig.h

   • Added MICROPY_METACLASS, MICROPY_METACLASS_PREPARE, MICROPY_INIT_SUBCLASS flags

2. py/modbuiltins.c

   • Modified __build_class__ to accept keyword arguments
   • Implemented metaclass resolution algorithm (PEP 3115)
   • Added optional __prepare__ method support
   • Extract and filter kwargs for metaclass and __init_subclass__

3. py/objtype.c

   • Modified mp_obj_new_type to accept metaclass and kw_args parameters
   • Added __init_subclass__ invocation logic (PEP 487)
   • Added base class validation for custom metaclasses

4. py/vm.c

   • Updated LOAD_ATTR fast path to handle type objects with custom metaclasses

5. py/runtime.c

   • Fixed error message generation for custom metaclasses

Code Size Impact (Unix standard variant)

Comprehensive measurements across all configuration permutations:

Configuration	METACLASS	PREPARE	INIT_SUBCLASS	text (bytes)	Total (bytes)	Increase	% Increase
Baseline	0	0	0	177,717	193,741	-	-
METACLASS only	1	0	0	178,341	194,365	+624	+0.32%
METACLASS + PREPARE	1	1	0	178,493	194,517	+776	+0.40%
INIT_SUBCLASS only	0	0	1	178,477	194,501	+760	+0.39%
METACLASS + INIT_SUBCLASS ⭐	1	0	1	178,909	194,933	+1,192	+0.62%
All features	1	1	1	179,093	195,117	+1,376	+0.71%

⭐ Recommended configuration for dataclass_transform support

Features by Configuration

Details

Configuration 1: METACLASS Only (+672 bytes)

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (0)

Features:

• ✅ class C(metaclass=Meta): syntax
• ✅ Metaclass inheritance from base classes
• ✅ Metaclass conflict detection and resolution
• ✅ Metaclass __init__ customization
• ✅ Multi-level metaclass inheritance
• ❌ No __prepare__ method
• ❌ No __init_subclass__ support

Use Case: Applications that need custom metaclasses but not dataclass patterns.

Configuration 2: METACLASS + PREPARE (800 bytes)

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (0)

Additional Features:

• ✅ __prepare__ method support for custom namespace initialization

Use Case: Full PEP 3115 compliance (rare requirement in embedded systems).

Configuration 3: INIT_SUBCLASS Only (728 bytes)

#define MICROPY_METACLASS (0)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (1)

Features:

• ✅ Automatic __init_subclass__ invocation
• ✅ Full **kwargs support (enables dataclass_transform)
• ✅ Implicit classmethod behavior
• ✅ First-base-only invocation per PEP 487
• ✅ Multiple inheritance with super() chaining
• ❌ No metaclass= keyword syntax

Use Case: Applications needing dataclass_transform but not explicit metaclasses. BEST SIZE/FUNCTIONALITY RATIO for dataclasses.

Configuration 4: METACLASS + INIT_SUBCLASS (+1,152 bytes) ⭐

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (0)
#define MICROPY_INIT_SUBCLASS (1)

Features:

• ✅ All features from Config 1 and Config 3 combined
• ✅ Full dataclass_transform support
• ✅ Complete typing module patterns
• ❌ No __prepare__ (rarely needed)

Use Case: RECOMMENDED - Full modern Python class features for dataclasses and type systems.

Configuration 5: All Features (+1,248 bytes)

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (1)

Features:

• ✅ Complete PEP 3115 + PEP 487 implementation
• ✅ Full CPython compatibility

Use Case: Maximum compatibility, when code size is less critical.

Trade-Off Analysis

Size contributions

Sizes measured on unix standard build using mpbuild :

Config	Size	features	diff	diff	diff	total
standard	683564	0 0 0				-
1	684236	1 0 0	672			672
2	684364	1 1 0	672	128		800
3	684292	0 0 1			728	728
4	684716	1 0 1	424		728	1152
5	684812	1 1 1	672	128	448	1248

Size Savings Options

Option	Configuration	Bytes Saved	Trade-Off
Skip prepare	(1,0,1) vs (1,1,1)	-128	Loses rarely-used feature, no practical impact
Skip METACLASS	(0,0,1) vs (1,0,1)	-424	Loses metaclass= syntax, keeps dataclass support
Skip INIT_SUBCLASS	(1,0,0) vs (1,0,1)	-480	Loses dataclass_transform, keeps metaclasses

Alternatives:

Several additional optimization strategies were explored:

1. Remove kwargs support from init_subclass

   • Saves: ~200-250 bytes
   • BREAKS: dataclass_transform pattern (primary use case)
   • Verdict: Not viable

2. Simplify metaclass resolution

   • Saves: ~80-100 bytes
   • BREAKS: Proper conflict detection, multiple inheritance
   • Verdict: Compromises correctness

3. Stack-based allocation

   • INCREASES size by +160 bytes due to conditional logic overhead
   • Verdict: Counter-productive

4. Additional granular flags

   • Adding more config flags for sub-features increases overhead
   • Shared code paths mean splitting would duplicate code
   • Verdict: Makes codebase more complex without real savings

Code Sharing Benefits

The current implementation benefits from significant code sharing:

• Keyword extraction logic shared between metaclass and init_subclass
• Metaclass resolution reuses type checking infrastructure
• Memory management uses existing m_new/m_del inline functions
• Combined features cost less than sum of parts

Configuration Recommendations

Details

Embedded/Resource-Constrained (+728 bytes)

Configuration 3: INIT_SUBCLASS Only

#define MICROPY_METACLASS (0)
#define MICROPY_INIT_SUBCLASS (1)

Rationale:

• Enables modern dataclass patterns with minimal overhead

• Best size-to-functionality ratio

• Most embedded applications don't need explicit metaclass syntax

• User Defined Generics not possible without METACLASS.

General Purpose (+1,152 bytes)

Configuration 4: METACLASS + INIT_SUBCLASS ⭐

#define MICROPY_METACLASS (1)
#define MICROPY_INIT_SUBCLASS (1)

Rationale:

• Complete modern Python class features
• Supports typing module, dataclasses, and custom metaclasses
• Only 0.6% size increase
• Recommended default for ports with >256KB flash

Maximum Compatibility (+1,248 bytes)

Configuration 5: All Features

#define MICROPY_METACLASS (1)
#define MICROPY_METACLASS_PREPARE (1)
#define MICROPY_INIT_SUBCLASS (1)

Rationale:

• Full PEP 3115 + PEP 487 compliance
• For desktop/high-resource targets
• Ensures maximum CPython code compatibility

Minimal (Baseline, +0 bytes)

Configuration: All Disabled (Default)

#define MICROPY_METACLASS (0)
#define MICROPY_INIT_SUBCLASS (0)

Rationale:

• Ultra-constrained targets (<128KB flash)
• Simple applications without advanced class features
• Default to maintain backward compatibility

Usage Examples

Details

Example 1: Metaclass (Requires MICROPY_METACLASS=1)

class Meta(type):
    def __init__(cls, name, bases, dct):
        cls.auto_added = 'by metaclass'
        cls.attr_count = len(dct)

class Base(metaclass=Meta):
    x = 1
    y = 2

class Derived(Base):  # Inherits Meta as metaclass
    z = 3

print(Base.auto_added)      # 'by metaclass'
print(Base.attr_count)      # 2
print(Derived.auto_added)   # 'by metaclass'
print(Derived.attr_count)   # 1
print(type(Derived).__name__)  # 'Meta'

Example 2: dataclass_transform Pattern (Requires MICROPY_INIT_SUBCLASS=1)

class ModelBase:
    def __init_subclass__(cls, *, init=True, frozen=False, eq=True, order=True):
        # Note: No @classmethod decorator needed (implicit per PEP 487)
        cls._config = {
            'init': init,
            'frozen': frozen,
            'eq': eq,
            'order': order
        }
        # Generate methods based on config...
        if init:
            cls._generate_init()
        if eq:
            cls._generate_eq()

class CustomerModel(
    ModelBase,
    init=False,
    frozen=True,
    eq=False,
    order=False,
):
    id: int
    name: str

print(CustomerModel._config)
# {'init': False, 'frozen': True, 'eq': False, 'order': False}

Example 3: Plugin Registry (Requires MICROPY_INIT_SUBCLASS=1)

class PluginBase:
    _plugins = []
    
    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        PluginBase._plugins.append(cls)

class MyPlugin(PluginBase):
    pass

class AnotherPlugin(PluginBase):
    pass

print(len(PluginBase._plugins))  # 2
print(MyPlugin in PluginBase._plugins)  # True

Example 4: Multiple Inheritance with Chaining (Requires MICROPY_INIT_SUBCLASS=1)

class BaseA:
    def __init_subclass__(cls, **kwargs):
        cls.from_a = True
        super().__init_subclass__(**kwargs)  # Chain to next base

class BaseB:
    def __init_subclass__(cls, **kwargs):
        cls.from_b = True
        super().__init_subclass__(**kwargs)

class Multi(BaseA, BaseB):
    pass

# Per PEP 487: Only first base's __init_subclass__ is called
# BaseA chains to BaseB via super()
print(hasattr(Multi, 'from_a'))  # True
print(hasattr(Multi, 'from_b'))  # True (via chaining)

Testing

• tests/basics/class_metaclass.py - 6 comprehensive metaclass tests
• tests/basics/class_init_subclass.py - 7 init_subclass tests with kwargs
• tests/cpydiff/core_class_metaclass.py - Documents __prepare__ limitation

All tests gracefully skip when features are disabled.

Compatibility

• Backward compatible - No breaking changes, features disabled by default
• Tested across configurations - All feature combinations verified
• Port-specific configuration - Each port can choose appropriate level
• Standards compliant - Follows PEP 3115 and PEP 487 specifications

Recommended Adoption Strategy

1. Phase 1: Enable for Unix and larger ports (METACLASS + INIT_SUBCLASS)
2. Phase 2: Enable INIT_SUBCLASS only for medium-resource ports
3. Phase 3: Keep disabled for minimal/constrained ports
4. Documentation: Update port-specific docs with configuration guidance

Technical Notes

PEP 487 Compliance Notes

• __init_subclass__ is implicitly a classmethod (no decorator needed)
• Only first base's __init_subclass__ is called (per MRO)
• Chaining requires explicit super().__init_subclass__(**kwargs) call
• Receives all class kwargs except metaclass=

PEP 3115 Compliance Notes

• metaclass= keyword syntax fully supported
• Proper metaclass resolution with conflict detection
• Multi-level metaclass inheritance works correctly
• __prepare__ method optional (controlled by separate flag)

Known Limitations (by design)

1. When MICROPY_METACLASS_PREPARE=0:

   • __prepare__ method not called
   • Documented in tests/cpydiff/core_class_metaclass.py
   • Rarely needed in embedded applications

2. Decorator behavior:

   • Explicit @classmethod on __init_subclass__ not supported
   • Documented in tests/cpydiff/core_class_initsubclass_classmethod.py
   • Not needed per PEP 487 (implicit classmethod)

Performance Impact

• Runtime: Negligible - only affects class creation (not instance operations)
• Memory: No additional RAM usage beyond class structures themselves

Performance Impact Analysis

Runtime performance impact of PEP 3115 (METACLASS) and PEP 487 (INIT_SUBCLASS) features.

Test Platform

• Port: Unix (standard variant)
• Platform: Linux x86_64
• Compiler: GCC with -O2 optimization
• Test Method: Average of 3 runs per configuration

Performance Results

Details

Test	Baseline	METACLASS Only	INIT_SUBCLASS Only	Both Enabled
class_creation	693 µs	770 µs (+11.1%)	758 µs (+9.4%)	707 µs (+2.0%)
inheritance	602 µs	639 µs (+6.1%)	642 µs (+6.6%)	638 µs (+6.0%)
method_calls	620 µs	581 µs (-6.3%)	578 µs (-6.8%)	584 µs (-5.8%)
attr_access	351 µs	328 µs (-6.6%)	329 µs (-6.3%)	323 µs (-8.0%)

All measurements in microseconds (µs). Percentages show runtime change vs baseline.

Analysis

Class Creation Overhead

• METACLASS only: +11.1% overhead when creating classes
• INIT_SUBCLASS only: +9.4% overhead when creating classes
• Both enabled: +2.0% overhead (shared code paths optimize combined case)

The overhead is expected as the implementation must check for metaclass resolution and invoke __init_subclass__ hooks during class creation.

Inheritance Performance

• Consistent ~6% overhead across all configurations
• Expected due to additional metaclass resolution checks when creating derived classes
• Overhead is consistent and predictable

Method Calls & Attribute Access

• Improvement of 5-8% in both method calls and attribute access
• Counterintuitive result likely due to:
  • Better code layout/alignment after adding guarded code
  • Compiler optimization differences
  • Cache effects from slightly different memory layout

Important: These improvements are within measurement noise and should not be considered a performance benefit. The key finding is that method calls and attribute access show no significant performance degradation.

Summary

Performance Impact

Negligible runtime overhead for typical operations

• Method calls: No degradation (measurement noise)
• Attribute access: No degradation (measurement noise)
• Class creation: 2-11% slower (only during class definition time)
• Inheritance: ~6% slower (only during class definition time)

Key Findings

1. Hot path unchanged: Method invocation and attribute access (the most common operations) show no performance regression
2. Cold path overhead acceptable: Class creation overhead only matters during module import/initialization
3. Combined features optimized: Enabling both features together has less overhead than individual features due to shared code paths

Recommendations

• For production code: Performance impact is minimal since class creation happens at module import time
• For dynamic class generation: 2-11% overhead on class creation is acceptable for the additional functionality gained
• Overall assessment: The runtime performance impact is negligible for typical MicroPython applications

Performance Test Descriptions

• class_creation: Creating 500 simple classes with methods (measures class construction overhead)
• inheritance: Creating 500 derived classes (measures inheritance + metaclass resolution overhead)
• method_calls: 5000 method invocations (measures hot path performance)
• attr_access: 5000 attribute accesses (measures attribute lookup performance)

[github-actions]

Code size report:

Reference:  tests/serial_test.py: Allow up to 2 seconds between bytes. [2762fe6]
Comparison: py/objtype: Fixup METACLASS guards in type_make_new(). [merge of cf3e30c]
  mpy-cross:  +208 +0.055% 
   bare-arm:  +124 +0.219% 
minimal x86:  +246 +0.131% 
   unix x64:  +216 +0.025% standard
      stm32:  +120 +0.030% PYBV10
     mimxrt:  +120 +0.032% TEENSY40
        rp2:  +136 +0.015% RPI_PICO_W
       samd:  +120 +0.044% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:  +142 +0.031% VIRT_RV32

[codecov]

Codecov Report

❌ Patch coverage is 86.36364% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.35%. Comparing base (2762fe6) to head (cf3e30c).

Files with missing lines	Patch %	Lines
py/objtype.c	84.21%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #18362      +/-   ##
==========================================
- Coverage   98.38%   98.35%   -0.03%     
==========================================
  Files         171      171              
  Lines       22294    22320      +26     
==========================================
+ Hits        21933    21953      +20     
- Misses        361      367       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Josverl]

@andrewleech , I need to add some tests for .__prepare__ , and I now notice an issue with class creation.
You seem to be on top of that and to avoid duplicating efforts or merge conflicts I'll wait for now.
I found (actually @jepler did ) that the metaclass tests need to be moved out of the tests/basics folder for CI to be happy.