Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

py: Add support for native C classes to call Python methods of child instance #5660

Closed

Conversation

dpgeorge
Copy link
Member

This PR is inspired by https://forum.micropython.org/viewtopic.php?f=3&t=7662 and the native module example at https://github.com/jamesbowman/py-bteve/tree/2407c82f852937bdf430cb12469e2b59ab588b3e/mpy, and also CircuitPython's fix adafruit#2550 (see adafruit@81c3bc4 and adafruit@f6a635b)

The aim is to allow Python classes to subclass native C classes (eg list) and for the C base class to then be able to call methods of the Python child instance. Eg:

class A:
    def foo(self):
        self.bar() # call inherited method
class B(A):
    def bar(self):
        print('B.bar')
b = B()
b.foo() # should eventually call b.bar()

except that we want to implement class A in native C code.

Currently this is not possible to do because native classes get passed their native instance, even if the top-level object is a derived user instance (eg as b is above).

One fix is to not extract out the native instance of derived user instances when invoking C methods, but instead pass the top-level instance to the C method. The problem with this is that all C methods must now be aware that they could be passed a user instance and must handle it accordingly.

The PR here does it a different way, by adding a new type flag that is set only on C classes that can handle a user instance coming in as the "self" argument to their C method. The advantage of this approach is that classes like tuple/list/dict that don't ever call methods of their inherited objects don't need to have this flag set and so don't need to change. Only C classes that want to support this feature can enable the flag (there will be overhead in code size and execution time for the C methods to test and extract the native instance).

This PR also exposes the new API function to dynamic native modules, and provides a new example of a dynamic native module that uses this feature (see original forum post linked above).

@tannewt FYI

Such a type is then expected to use the following to obtain a pointer to
itself from the incoming argument:

    mp_native_t *self = mp_obj_cast_to_native_base_unchecked(self)
@dpgeorge dpgeorge added the py-core Relates to py/ directory in source label Feb 18, 2020
@tannewt
Copy link
Sponsor

tannewt commented Feb 18, 2020

Thank you for introducing a way to support this!

My preference would be to always give native methods the instance. Yes, it will require more changes up front but in the long term MicroPython's internals will be more uniform. Having a separate flag on a type places logic around what a native function uses outside and separate from the function itself. By always passing in the instance, it is always explicit what portion of an object is used by a native function.

@dpgeorge
Copy link
Member Author

I agree with @tannewt 's point above, that it would be nice for self to always be passed as the subclass, and for all methods to then unpack it to the native base (if needed). But that is a huge change and requires adding a lot of unpacking code to all builtin methods (eg all methods of str, float, etc).

I'll close this PR due to inactivity/lack of interest.

@dpgeorge dpgeorge closed this Nov 17, 2021
@dpgeorge dpgeorge deleted the py-native-type-can-handle-subclass branch November 17, 2021 02:35
tannewt pushed a commit to tannewt/circuitpython that referenced this pull request Dec 7, 2021
espressif: busio.SPI: Use SPI_DMA_CH_AUTO
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
py-core Relates to py/ directory in source
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants