makefiles: FEATURE_OPTIONAL picolib defaults picolib even if that is not installed -> examples/gnrc_minimal: fail building for cortex without picolib

[kfessel]

Description

Building gnrc_minimal for nucleo-f767zi using gcc uses FEATURES_OPTIONAL += picolibc and fails
(similar result for sam21-xpr)

maybe related to: comments in #12305
RIOT-OS/riotdocker#109
RIOT-OS/riotdocker#116

@bergzand and @benpicco might want to have a look at this seems like RIOT-OS/riotdocker#116 suggests an update

Steps to reproduce the issue

cd RIOT/examples/gnrc_minimal
QUIET=0 BOARD=nucleo-f767zi make

Expected results

successful build

Actual results

picolib compilation fails:

...
make[3]: Leaving directory '_RIOT_/sys/net/network_layer/ipv6/hdr'
"make" -C _RIOT_/sys/picolibc_syscalls_default
make[3]: Entering directory '_RIOT_/sys/picolibc_syscalls_default'
mkdir -p _RIOT_/examples/gnrc_test/bin/nucleo-f767zi/picolibc_syscalls_default/
arm-none-eabi-gcc \
        -DRIOT_FILE_RELATIVE=\"sys/picolibc_syscalls_default/syscalls.c\" \
        -DRIOT_FILE_NOPATH=\"syscalls.c\" \
        -DLOG_LEVEL=LOG_NONE   -DCONFIG_GNRC_NETIF_IPV6_ADDRS_NUMOF=2 -DGNRC_NETIF_IPV6_GROUPS_NUMOF=2 -DCONFIG_GNRC_IPV6_NIB_NUMOF=1 -DCONFIG_GNRC_IPV6_NIB_OFFL_NUMOF=1  -Werror -DCPU_FAM_STM32F7 -DSTM32F767xx -DCPU_LINE_STM32F767xx -DSTM32_FLASHSIZE=2097152U -D__SYSTEM_STM32F7XX_H -mno-thumb-interwork -mcpu=cortex-m7 -mlittle-endian -mthumb -mfloat-abi=hard -mfpu=fpv5-sp-d16 -ffunction-sections -fdata-sections -fno-builtin -fshort-enums -ggdb -g3 -Os -DCPU_MODEL_STM32F767ZI -DCPU_CORE_CORTEX_M7 -DRIOT_APPLICATION=\"gnrc_minimal\" -DBOARD_NUCLEO_F767ZI=\"nucleo-f767zi\" -DRIOT_BOARD=BOARD_NUCLEO_F767ZI -DCPU_STM32=\"stm32\" -DRIOT_CPU=CPU_STM32 -DMCU_STM32=\"stm32\" -DRIOT_MCU=MCU_STM32 -std=c99 -fno-common -ffunction-sections -fdata-sections -Wall -Wextra -Wmissing-include-dirs -DNDEBUG -fno-delete-null-pointer-checks -fdiagnostics-color -Wstrict-prototypes -Wold-style-definition -gz -Wformat=2 -Wformat-overflow -Wformat-truncation -include '_RIOT_/examples/gnrc_test/bin/nucleo-f767zi/riotbuild/riotbuild.h'  -I_RIOT_/core/include -I_RIOT_/drivers/include -I_RIOT_/sys/include -I_RIOT_/boards/nucleo-f767zi/include -I_RIOT_/boards/common/nucleo/include -I_RIOT_/boards/common/stm32/include -I_RIOT_/boards/common/nucleo144/include -I_RIOT_/cpu/stm32/include -I_RIOT_/cpu/stm32/include/vendor/cmsis/f7/Include -I_RIOT_/cpu/stm32/include/clk -I_RIOT_/cpu/cortexm_common/include -I_RIOT_/cpu/cortexm_common/include/vendor -I_RIOT_/sys/libc/include -I_RIOT_/sys/net/link_layer/eui_provider/include -MQ '_RIOT_/examples/gnrc_test/bin/nucleo-f767zi/picolibc_syscalls_default/syscalls.o' -MD -MP -c -o _RIOT_/examples/gnrc_test/bin/nucleo-f767zi/picolibc_syscalls_default/syscalls.o _RIOT_/sys/picolibc_syscalls_default/syscalls.c
_RIOT_/sys/picolibc_syscalls_default/syscalls.c:236:5: error: implicit declaration of function 'FDEV_SETUP_STREAM' [-Werror=implicit-function-declaration]
    FDEV_SETUP_STREAM(picolibc_put, picolibc_get, picolibc_flush, _FDEV_SETUP_RW);
    ^~~~~~~~~~~~~~~~~
_RIOT_/sys/picolibc_syscalls_default/syscalls.c:236:67: error: '_FDEV_SETUP_RW' undeclared here (not in a function)
    FDEV_SETUP_STREAM(picolibc_put, picolibc_get, picolibc_flush, _FDEV_SETUP_RW);
                                                                ^~~~~~~~~~~~~~

Versions

Operating system: Linux
Build environment: GCC
RIOT: current master

Operating System Environment
----------------------------
        Operating System: "Ubuntu" "19.10 (Eoan Ermine)"
                Kernel: Linux 5.3.0-64-generic x86_64 x86_64
            System shell: /usr/bin/dash (probably dash)
            make's shell: /usr/bin/dash (probably dash)

Installed compiler toolchains
-----------------------------
            native gcc: gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
        arm-none-eabi-gcc: arm-none-eabi-gcc (15:7-2018-q2-6) 7.3.1 20180622 (release) [ARM/embedded-7-branch revision 261907]
                avr-gcc: missing
        mips-mti-elf-gcc: missing
        msp430-elf-gcc: missing
    riscv-none-elf-gcc: missing
riscv64-unknown-elf-gcc: missing
    riscv-none-embed-gcc: missing
    xtensa-esp32-elf-gcc: missing
xtensa-esp8266-elf-gcc: missing
                    clang: clang version 9.0.0-2 (tags/RELEASE_900/final)

Installed compiler libs
-----------------------
    arm-none-eabi-newlib: "3.1.0"
    mips-mti-elf-newlib: missing
        msp430-elf-newlib: missing
    riscv-none-elf-newlib: missing
riscv64-unknown-elf-newlib: missing
riscv-none-embed-newlib: missing
xtensa-esp32-elf-newlib: missing
xtensa-esp8266-elf-newlib: missing
                avr-libc: missing (missing)

Installed development tools
---------------------------
                ccache: missing
                    cmake: cmake version 3.13.4
                cppcheck: missing
                doxygen: 1.8.13
                    git: git version 2.20.1
                    make: GNU Make 4.2.1
                openocd: Open On-Chip Debugger 0.10.0+dev-01068-g7816f62f-dirty (2020-02-25-19:11)
                python: Python 2.7.17
                python2: Python 2.7.17
                python3: Python 3.7.5
                flake8: error: /usr/bin/python3: No module named flake8
            coccinelle: missing

[benpicco]

Do you have picolibc installed? The Debian package only has minimal dependencies and provides the latest version.

[kfessel]

No, even though it should compile i think.
Maybe without using picolibc if the upstream version does not work and the Debian Version is not installed (might need another detection mechanism)

[benpicco]

The FEATURE_PROVIDED does not check if picolibc is installed, it checks if it's implemented for that architecture.
So picolibc will be used even if you don't have it installed - and then the build fails.

(I don't know how we could detect if picolibc is installed)

[kfessel]

there is a detector in RIOT/makefiles/libc/picolibc.mk line 3 i think it would be possible to use a similar line could be applied for FEATURE_PROVIDED in cpu/cortexm_common/Makefile.features

ifeq ($(shell arm-none-eabi-gcc -specs=picolibc.specs -E - 2>/dev/null >/dev/null </dev/null ; echo $$?),0)
    FEATURES_PROVIDED += picolibc
endif

[benpicco]

Now I wonder how we could do that in Kconfig
@leandrolanzieri can Kconfig evaluate shell output?

[leandrolanzieri]

Now I wonder how we could do that in Kconfig
@leandrolanzieri can Kconfig evaluate shell output?

Yes, see the shell macro function here.

[kfessel]

it may also be possible to filter the FEATURES_PROVIDED list in RIOT/makefiles/libc/picolibc.mk

[nmeum]

I am running into the same problem with BOARD=hifive1 since 2692957.

[kfessel]

seems like I should make this a PR

[kfessel]

I stepped into a problem: The decision which libc to use is done by the dependency resoulution in L:388 of RIOT/Makefile.include include $(RIOTMAKE)/dependency_resolution.inc.mk but at this point no toolchain is assingned which is done in L:417 include $(RIOTMAKE)/toolchain/$(TOOLCHAIN).inc.mk the detection mechanism

ifneq (,$(filter picolibc,$(USEMODULE)))
  # Test if picolibc.specs is available
  ifneq ($(shell $(LINK) -specs=picolibc.specs -E - 2>/dev/null >/dev/null </dev/null ; echo $$?),0)
... //filter-out picolibc

relies on knowing the tool chain $(LINK) therfor this decision has to be delayed

[fjmolinas]

I can reproduce the issue, but I think it's just another bug caught by #15973. Rebase on that PR, both libc would not get selected.

In master I get:

The following features may conflict: newlib picolibc
Rationale: Only one standard C library can be used

EXPECT undesired behaviour!

With #15973 I get nothing and it compiles fine

BUILD_IN_DOCKER=1 BOARD=nucleo-f767zi make
Launching build container using image "riot/riotbuild:latest".
docker run --rm --tty --user $(id -u) -v '/usr/share/zoneinfo/Europe/Paris:/etc/localtime:ro' -v '/home/francisco/workspace/RIOT:/data/riotbuild/riotbase:delegated' -e 'RIOTBASE=/data/riotbuild/riotbase' -e 'CCACHE_BASEDIR=/data/riotbuild/riotbase' -e 'BUILD_DIR=/data/riotbuild/riotbase/build' -e 'RIOTPROJECT=/data/riotbuild/riotbase' -e 'RIOTCPU=/data/riotbuild/riotbase/cpu' -e 'RIOTBOARD=/data/riotbuild/riotbase/boards' -e 'RIOTMAKE=/data/riotbuild/riotbase/makefiles'      -e 'BOARD=nucleo-f767zi'  -w '/data/riotbuild/riotbase/examples/gnrc_minimal/' 'riot/riotbuild:latest' make      
Unable to find image 'riot/riotbuild:latest' locally
latest: Pulling from riot/riotbuild
d519e2592276: Pull complete 
d22d2dfcfa9c: Pull complete 
b3afe92c540b: Pull complete 
0ce36d6675b3: Pull complete 
74e61de93f7d: Pull complete 
a83a6c7cada3: Pull complete 
6e93685fbe68: Pull complete 
4382b173682a: Pull complete 
33f17dc473d1: Pull complete 
39a1debfd8db: Pull complete 
2bb60dc247d8: Pull complete 
b6a10c02bb3f: Pull complete 
26f6ffe1e165: Pull complete 
3b1725da4752: Pull complete 
51dc08fbcb09: Pull complete 
d7ea0fe5fad2: Pull complete 
019eca447d68: Pull complete 
f9c9dd4a1b20: Pull complete 
2c52544a6c29: Pull complete 
32a303d76fd6: Pull complete 
e82de04a2956: Pull complete 
10c09f0b54d7: Pull complete 
0d212ce65789: Pull complete 
93ef51007a8a: Pull complete 
8e3bb6ca9b77: Pull complete 
b5d3f73471e4: Pull complete 
964e405b0cb3: Pull complete 
4faacc02582b: Pull complete 
8dd29f8d325f: Pull complete 
Digest: sha256:0673aacb9dfd9b0a00f7695b60dbbc0ce48437e9fc354904fde92611dea9fde2
Status: Downloaded newer image for riot/riotbuild:latest
Building application "gnrc_minimal" for "nucleo-f767zi" with MCU "stm32".

"make" -C /data/riotbuild/riotbase/boards/nucleo-f767zi
Remapping MOSI of SPI_DEV(0) from PA7 to PB5 to solve pin conflict. (PA7 is also connected to the RMII_DV of the Ethernet Phy.)
"make" -C /data/riotbuild/riotbase/boards/common/nucleo
"make" -C /data/riotbuild/riotbase/core
"make" -C /data/riotbuild/riotbase/cpu/stm32
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common
"make" -C /data/riotbuild/riotbase/cpu/cortexm_common/periph
"make" -C /data/riotbuild/riotbase/cpu/stm32/periph
"make" -C /data/riotbuild/riotbase/cpu/stm32/stmclk
"make" -C /data/riotbuild/riotbase/cpu/stm32/vectors
"make" -C /data/riotbuild/riotbase/drivers
"make" -C /data/riotbuild/riotbase/drivers/netdev
"make" -C /data/riotbuild/riotbase/drivers/periph_common
"make" -C /data/riotbuild/riotbase/sys
"make" -C /data/riotbuild/riotbase/sys/auto_init
"make" -C /data/riotbuild/riotbase/sys/div
"make" -C /data/riotbuild/riotbase/sys/evtimer
"make" -C /data/riotbuild/riotbase/sys/fmt
"make" -C /data/riotbuild/riotbase/sys/iolist
"make" -C /data/riotbuild/riotbase/sys/luid
"make" -C /data/riotbuild/riotbase/sys/malloc_thread_safe
"make" -C /data/riotbuild/riotbase/sys/net/crosslayer/inet_csum
"make" -C /data/riotbuild/riotbase/sys/net/gnrc
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netapi
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/ethernet
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/hdr
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netif/init_devs
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/netreg
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/icmpv6/echo
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6/hdr
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ipv6/nib
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/network_layer/ndp
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/pkt
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktbuf
"make" -C /data/riotbuild/riotbase/sys/net/gnrc/pktbuf_static
"make" -C /data/riotbuild/riotbase/sys/net/link_layer/eui_provider
"make" -C /data/riotbuild/riotbase/sys/net/link_layer/l2util
"make" -C /data/riotbuild/riotbase/sys/net/netif
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/icmpv6
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/ipv6/addr
"make" -C /data/riotbuild/riotbase/sys/net/network_layer/ipv6/hdr
"make" -C /data/riotbuild/riotbase/sys/picolibc_syscalls_default
"make" -C /data/riotbuild/riotbase/sys/pm_layered
"make" -C /data/riotbuild/riotbase/sys/random
"make" -C /data/riotbuild/riotbase/sys/stdio_uart
"make" -C /data/riotbuild/riotbase/sys/xtimer
   text	   data	    bss	    dec	    hex	filename
  25872	     36	   7328	  33236	   81d4	/data/riotbuild/riotbase/examples/gnrc_minimal/bin/nucleo-f767zi/gnrc_minimal.elf

[fjmolinas]

@kfessel can you test with #15973?

[kfessel]

@fjmolinas: i just did: In #15973 riot defaults to picolibc -> if it is missing the linking fails for picolibc_syscalls_default
with my PR (#15993 rebased on #15973) the missing picolibc is detected and exiting newlibc is used.

my rebased PR: https://github.com/kfessel/RIOT/tree/p-feature-precheck-test << I do not know how to describe that better

I don't use the docker build thing.

[aabadie]

#16008 is merged so can this be closed @kfessel ?

[kfessel]

i think something like #15993 is still missing (check for picolibc on the build system) but if such a feature is unlikely to be merged you may close this with a comment stating that (fail loud is enough for me)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

[maribu]

Let's close this. We don't check e.g. for GCC or newlib being installed either and rather refer to BUILD_IN_DOCKER=1 for anyone not being an expert in setting up toolchains anyway.