Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mpy-tool.py qstr linking fails with @native on a member method #4829

Closed
jimmo opened this issue Jun 4, 2019 · 6 comments

Comments

@jimmo
Copy link
Contributor

commented Jun 4, 2019

A simple way to repro is to add @micropython.native to any of the methods in dht.py (e.g. DHT22.temperature) and build for PYBV11.

I think in read_raw_code, off = read_uint(f, qstr_win) shouldn't be passing qstr_win.

But then this uncovered a different problem with _link_qstr. Depending on which other .mpy files were being passed to mpy-tool. If the last .mpy to be processed was non-native (i.e. native_arch == 0), then it hits the assert 0 if there are any kind != 0 qstrs. Alternatively, if the last .mpy was the one with native code (e.g. dht.mpy, native_arch=5), it looks like the branch for ARM can't work (is_obj and i don't exist).

@dpgeorge

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

Thanks for the report, I can confirm it.

The main problem is that most config values should be per-mpy file, not global (as they are currently treated).

@dpgeorge dpgeorge added the bug label Jun 4, 2019

@dpgeorge

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

Fixed by faf3d3e

@dpgeorge dpgeorge closed this Jun 4, 2019

dpgeorge added a commit that referenced this issue Jun 4, 2019
tools/mpy-tool.py: Fix linking qstrs in native code, and multiple files.
Fixes errors in the tool when 1) linking qstrs in native ARM-M code; 2)
freezing multiple files some of which use native code and some which don't.

Fixes issue #4829.
@hacksterous

This comment has been minimized.

Copy link

commented Jun 5, 2019

Build now goes through, but I get a runtime error when accessing my function:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: function missing required positional argument #31

I see the these warnings during build:

build-BLACK_F407VE/frozen_mpy.c:20725:83: warning: excess elements in array initializer
     0x00, 0x31, 0xfb, 0x6e, 0x98, 0x47, 0x05, 0x90, 0x00, 0xf0, 0x00, 0xb8, 0x0b, 0x98, 0x00, 0x28,
                                                                                   ^~~~
build-BLACK_F407VE/frozen_mpy.c:20725:83: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20725:89: warning: excess elements in array initializer
     0x00, 0x31, 0xfb, 0x6e, 0x98, 0x47, 0x05, 0x90, 0x00, 0xf0, 0x00, 0xb8, 0x0b, 0x98, 0x00, 0x28,
                                                                                         ^~~~
build-BLACK_F407VE/frozen_mpy.c:20725:89: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20725:95: warning: excess elements in array initializer
     0x00, 0x31, 0xfb, 0x6e, 0x98, 0x47, 0x05, 0x90, 0x00, 0xf0, 0x00, 0xb8, 0x0b, 0x98, 0x00, 0x28,
                                                                                               ^~~~
build-BLACK_F407VE/frozen_mpy.c:20725:95: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:5: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
     ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:5: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:11: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
           ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:11: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:17: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                 ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:17: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:23: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                       ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:23: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:29: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                             ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:29: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:35: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                   ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:35: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:41: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                         ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:41: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:47: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                               ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:47: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:53: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                     ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:53: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:59: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                           ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:59: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:65: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                                 ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:65: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:71: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                                       ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:71: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:77: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                                             ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:77: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:83: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                                                   ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:83: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:89: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                                                         ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:89: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20726:95: warning: excess elements in array initializer
     0x00, 0xf0, 0x05, 0x80, 0x7b, 0x69, 0x98, 0x47, 0xd7, 0xf8, 0x84, 0x30, 0x98, 0x47, 0x05, 0x98,
                                                                                               ^~~~
build-BLACK_F407VE/frozen_mpy.c:20726:95: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20727:5: warning: excess elements in array initializer
     0x20, 0xb0, 0xfe, 0xbd,
     ^~~~
build-BLACK_F407VE/frozen_mpy.c:20727:5: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20727:11: warning: excess elements in array initializer
     0x20, 0xb0, 0xfe, 0xbd,
           ^~~~
build-BLACK_F407VE/frozen_mpy.c:20727:11: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20727:17: warning: excess elements in array initializer
     0x20, 0xb0, 0xfe, 0xbd,
                 ^~~~
build-BLACK_F407VE/frozen_mpy.c:20727:17: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20727:23: warning: excess elements in array initializer
     0x20, 0xb0, 0xfe, 0xbd,
                       ^~~~
build-BLACK_F407VE/frozen_mpy.c:20727:23: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:5: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
     ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:5: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:11: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
           ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:11: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:17: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
                 ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:17: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:23: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
                       ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:23: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:29: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
                             ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:29: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:35: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
                                   ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:35: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:41: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
                                         ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:41: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20728:47: warning: excess elements in array initializer
     0x80, 0x14, 0x00, 0x30, 0x02, 0x00, 0x00, 0x05,
                                               ^~~~
build-BLACK_F407VE/frozen_mpy.c:20728:47: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20729:5: warning: excess elements in array initializer
     MP_QSTR___truediv__ & 0xff, MP_QSTR___truediv__ >> 8,
     ^~~~~~~~~~~~~~~~~~~
build-BLACK_F407VE/frozen_mpy.c:20729:5: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20729:33: warning: excess elements in array initializer
     MP_QSTR___truediv__ & 0xff, MP_QSTR___truediv__ >> 8,
                                 ^~~~~~~~~~~~~~~~~~~
build-BLACK_F407VE/frozen_mpy.c:20729:33: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20730:5: warning: excess elements in array initializer
     MP_QSTR_mpap_dot_py & 0xff, MP_QSTR_mpap_dot_py >> 8,
     ^~~~~~~~~~~~~~~~~~~
build-BLACK_F407VE/frozen_mpy.c:20730:5: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20730:33: warning: excess elements in array initializer
     MP_QSTR_mpap_dot_py & 0xff, MP_QSTR_mpap_dot_py >> 8,
                                 ^~~~~~~~~~~~~~~~~~~
build-BLACK_F407VE/frozen_mpy.c:20730:33: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
build-BLACK_F407VE/frozen_mpy.c:20731:5: warning: excess elements in array initializer
     0xff,
     ^~~~
build-BLACK_F407VE/frozen_mpy.c:20731:5: note: (near initialization for 'fun_data_mpap__lt_module_gt__mpap___truediv__')
@dpgeorge

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

@hacksterous can you share (link to) the frozen files that lead to this error, or at least provide a subset that still have the error?

@dpgeorge dpgeorge reopened this Jun 5, 2019

@hacksterous

This comment has been minimized.

Copy link

commented Jun 5, 2019

Attached the bad mpy.
mpap-bad.zip

Here is the link to the original Python.

I have one method using the native decorator.
Sorry, I could not get a smaller subset to show the warnings.

@dpgeorge

This comment has been minimized.

Copy link
Member

commented Jun 11, 2019

Thanks, I was able to reproduce the error by freezing mpap.py with a native decorator on the __truediv__ method. Should be fixed now by 9d3031c

@dpgeorge dpgeorge closed this Jun 11, 2019

dpgeorge added a commit that referenced this issue Jun 11, 2019
tools/mpy-tool.py: Fix linking of qstr objects in native ARM Thumb code.
Previously, when linking qstr objects in native code for ARM Thumb, the
index into the machine code was being incremented by 4, not 8.  It should
be 8 to account for the size of the two machine instructions movw and movt.
This patch makes sure the index into the machine code is incremented by the
correct amount for all variations of qstr linking.

See issue #4829.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.