libraries.libc.newlib test fails

[stephanosio]

Describe the bug
tests/lib/mem_alloc/libraries.libc.newlib test fails when actually building with the newlib.

NEWLIB_LIBC_NANO config defaults to y (when building with the GNU ARM Embedded toolchain) and prj_newlib.conf does not set NEWLIB_LIBC_NANO=n, so the libraries.libc.newlib test was linking with the newlib nano variant (libc_nano.a) instead of the normal variant (libc.a), meaning libraries.libc.newlib test was never actually tested using the non-nano newlib until now (this in itself needs to be fixed).

zephyr/lib/libc/Kconfig

Lines 43 to 51 in 240d519

	if "$(ZEPHYR_TOOLCHAIN_VARIANT)" = "gnuarmemb"
	config NEWLIB_LIBC_NANO
	bool "Build with newlib-nano C library"
	default y
	help
	Build with newlib-nano library, for small embedded apps.
	The newlib-nano library for ARM embedded processors is a part of the
	GNU Tools for ARM Embedded Processors.
	endif

To Reproduce
Steps to reproduce the behavior:

1. Add CONFIG_NEWLIB_LIBC_NANO=n to tests/lib/mem_alloc/prj_newlib.conf.
2. cd tests/lib/mem_alloc
3. mkdir build; cd build
4. cmake -DBOARD=mps2_an385 -DCONF_FILE=prj_newlib.conf ..
5. make run

Screenshots or console output

Running test suite test_c_lib_dynamic_memalloc
===================================================================
starting test - test_malloc
PASS - test_malloc
===================================================================
starting test - test_free
PASS - test_free
===================================================================
starting test - test_calloc

    Assertion failed at ../src/main.c:70: test_calloc: ((cptr) is NULL)
calloc failed, errno: 0
FAIL - test_calloc
===================================================================
starting test - test_realloc
PASS - test_realloc
===================================================================
starting test - test_reallocarray
SKIP - test_reallocarray
===================================================================
starting test - test_memalloc_all

    Assertion failed at ../src/main.c:168: test_memalloc_all: ((clc_ptr) is NULL)
calloc failed, errno: 0
FAIL - test_memalloc_all
===================================================================
starting test - test_memalloc_max
PASS - test_memalloc_max
===================================================================
Test suite test_c_lib_dynamic_memalloc failed.
===================================================================
PROJECT EXECUTION FAILED

Environment (please complete the following information):

• OS: Ubuntu 18.04
• Toolchain: GNU ARM Embedded 8-2019-q3-update
• Commit SHA: 240d519

Additional context

• This issue is not specific to qemu or mps2_an385 board. It also fails in the same manner using a physical sam_e70_xplained board.
• This test does not fail on other qemu boards with MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT=n (e.g. qemu_cortex_m3) because NEWLIB_LIBC_ALIGNED_HEAP_SIZE gets ignored.
  

  zephyr/lib/libc/Kconfig

  Lines 53 to 56 in 39f2281

  	config NEWLIB_LIBC_ALIGNED_HEAP_SIZE
  	int "Newlib aligned heap size"
  	depends on MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT
  	depends on USERSPACE

• This issue is not specific to the GNU ARM Embedded toolchain. The same failure occurs when when testing with a newlib that is locally built without --enable-newlib-nano-malloc.

This issue might have something to do with what is being done here:

zephyr/lib/libc/newlib/libc-hooks.c

Lines 23 to 30 in 240d519

	#if CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE
	K_APPMEM_PARTITION_DEFINE(z_malloc_partition);
	#define MALLOC_BSS K_APP_BMEM(z_malloc_partition)
	/* Compiler will throw an error if the provided value isn't a power of two */
	MALLOC_BSS static unsigned char __aligned(CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE)
	heap_base[CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE];
	#define MAX_HEAP_SIZE CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE

[stephanosio]

cc @andrewboie @andyross @ioannisg

[stephanosio]

Note that this issue will be relevant to the Zephyr SDK as well in the near future, as zephyrproject-rtos/sdk-ng#153 builds the normal newlib (libc.a) without --enable-newlib-nano-malloc.

--enable-newlib-nano-malloc is only specified for the nano variant (libc_nano.a).

[stephanosio]

The problem is that the newlib full malloc implementation (mallocr.c) requests a relatively large 4096-byte memory chunk through _sbrk, which exceeds the CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE of 512-byte for this test.

After increasing the heap size, this test passes.

Nano malloc

*** Booting Zephyr OS build v2.2.0-rc1-82-gcd936779c77e  ***
Running test suite test_c_lib_dynamic_memalloc
===================================================================
starting test - test_malloc
_sbrk: count = 0
_sbrk: count = 48
PASS - test_malloc
===================================================================
starting test - test_free
PASS - test_free
===================================================================
starting test - test_calloc
_sbrk: count = 208
PASS - test_calloc
===================================================================
starting test - test_realloc
PASS - test_realloc
===================================================================
starting test - test_reallocarray
SKIP - test_reallocarray
===================================================================
starting test - test_memalloc_all
PASS - test_memalloc_all
===================================================================
starting test - test_memalloc_max
PASS - test_memalloc_max
===================================================================
Test suite test_c_lib_dynamic_memalloc succeeded
===================================================================
PROJECT EXECUTION SUCCESSFUL
QEMU: Terminated

Full malloc

*** Booting Zephyr OS build v2.2.0-rc1-82-gcd936779c77e  ***
Running test suite test_c_lib_dynamic_memalloc
===================================================================
starting test - test_malloc
_sbrk: count = 64
_sbrk: count = 1472
PASS - test_malloc
===================================================================
starting test - test_free
PASS - test_free
===================================================================
starting test - test_calloc
_sbrk: count = 4096

    Assertion failed at ../src/main.c:70: test_calloc: ((cptr) is NULL)
calloc failed, errno: 0
FAIL - test_calloc
===================================================================
starting test - test_realloc
PASS - test_realloc
===================================================================
starting test - test_reallocarray
SKIP - test_reallocarray
===================================================================
starting test - test_memalloc_all
_sbrk: count = 4096

    Assertion failed at ../src/main.c:168: test_memalloc_all: ((clc_ptr) is NULL)
calloc failed, errno: 0
FAIL - test_memalloc_all
===================================================================
starting test - test_memalloc_max
PASS - test_memalloc_max
===================================================================
Test suite test_c_lib_dynamic_memalloc failed.
===================================================================
PROJECT EXECUTION FAILED