Problems enabling LTO build on nrf

[simark]

I am trying to understand a bit better how string compression is supposed to work. In particular, I have the feeling that in my nrf-based build, the raw strings are stored in the flash as well as the compressed strings.

For example, if I build like this:

$ pwd
.../circuitpython-dev/ports/nrf
$ make BOARD=pca10059
...

And look for this particular string: https://github.com/adafruit/circuitpython/blob/master/py/lexer.c#L345

$ strings build-pca10059/firmware.bin | grep unicode
unicode name escapes

In de5a9d7, you (@tannewt) said that this happens if you don't have LTO enabled, which completely makes sense. However, when I try to enable LTO (by adding -flto to both CFLAGS and LDFLAGS in the Makefile), I get an executable that doesn't really look valid (on top of taking a really long time to try to link!):

$ make BOARD=pca10059
...
LINK build-pca10059/firmware.elf
   text	   data	    bss	    dec	    hex	filename
      0	      0	   2048	   2048	    800	build-pca10059/firmware.elf
arm-none-eabi-objcopy -O binary build-pca10059/firmware.elf build-pca10059/firmware.bin
arm-none-eabi-objcopy -O ihex build-pca10059/firmware.elf build-pca10059/firmware.hex
Create firmware.uf2
../../tools/uf2/utils/uf2conv.py -f 0xADA52840 -c -o "build-pca10059/firmware.uf2" build-pca10059/firmware.hex
Traceback (most recent call last):
  File "../../tools/uf2/utils/uf2conv.py", line 279, in <module>
    main()
  File "../../tools/uf2/utils/uf2conv.py", line 260, in main
    (ext, len(outbuf), appstartaddr))
TypeError: %x format: an integer is required, not NoneType
make: *** [Makefile:274: uf2] Error 1

Is this expected, anything I'm doing wrong?

These are the compiler/linker I'm using:

$ arm-none-eabi-gcc --version
arm-none-eabi-gcc (Arch Repository) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ arm-none-eabi-ld --version
GNU ld (GNU Binutils) 2.31.1
Copyright (C) 2018 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

[tannewt]

Ya, it looks like lto isn't enabled for nrf. (We may not have noticed since we are less space constrained.) That means the English form will be compiled in along side the compressed translation. There is a stub translate method that is generated with a huge mapping from english to the translation that ideally gets optimized away.

That zero length binary can happen when enabling LTO. To fix, you add the used attribute to key methods like handlers which then call everything else and keep it around.

[dhalbert]

Also, we haven't tried gcc 8.2.0 yet. We're on 7.3.1. But that is probably not the problem.

[simark]

Ya, it looks like lto isn't enabled for nrf. (We may not have noticed since we are less space constrained.) That means the English form will be compiled in along side the compressed translation. There is a stub translate method that is generated with a huge mapping from english to the translation that ideally gets optimized away.

Indeed, translate.o is on top of the list for .text size.

That zero length binary can happen when enabling LTO. To fix, you add the used attribute to key methods like handlers which then call everything else and keep it around.

Sorry, which handlers are you referring to?

[tannewt]

Handlers like Reset_Handler (calls main eventually) and HardFault_Handler that are defined by ARM.

[dhalbert]

Take a look in ports/atmel-samd

[simark]

Thanks, giving it a try.

[simark]

I added __attribute__((used)) to the various handlers in device/nrf52/startup_nrf52840.c, it didn't seem to change anything.

[dhalbert]

try handler in supervisor/port.c

[simark]

Still nothing. I find it a bit strange that despite marking some functions as used (and even main, in main.c, is marked as used), I still get 0 bytes of .text.

[tannewt]

I'm not sure what's left. Are your changes available somewhere to try? Are you trying to recover the flash space that the english strings take up?

[simark]

The commands I mentioned earlier are with the master branch of the current repo (I am using the pca10059 board as an example), so you should be able to reproduce easily.

Yes, I am trying to free up flash space, as we are currently running of space.