ARM mbed: Compilation of project with long file paths not possible

[maxgerhardt]

What kind of issue is this?

• PlatformIO Core.
  If you’ve found a bug, please provide an information below.

---

Configuration

Operating system: Windows 10 x64

PlatformIO Version (platformio --version): 4.0.1b3

Description of problem

When compiling the sample project given below, compilation fails as soon as the first source file of an external library or the main project is compiled. The occurring error is

arm-none-eabi-g++: error: CreateProcess: No such file or directory

Which might hint at file path length problems (see MarlinFirmware/Marlin#7967).

Steps to Reproduce

1. Import the sample project given below
2. Before compiling, execute git config --system core.longpaths true, otherwise th git clone will fail due to overlong filepaths
3. Compile on Windows 10
4. Observer error message

Actual Results

Processing nucleo_l152re (platform: ststm32; board: nucleo_l152re; framework: mbed)
CONFIGURATION: https://docs.platformio.org/page/boards/ststm32/nucleo_l152re.html
PLATFORM: ST STM32 5.5.0 > ST Nucleo L152RE
HARDWARE: STM32L152RET6 32MHz, 80KB RAM, 512KB Flash
DEBUG: Current (stlink) On-board (stlink) External (blackmagic, jlink)
PACKAGES: toolchain-gccarmnoneeabi 1.70201.0 (7.2.1), framework-mbed 5.51204.190701 (5.12.4)
Collecting mbed sources...
LDF: Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 1 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <ArduinoCore-nRF528x-mbedos> #a4b622b [git+https://github.com/arduino/ArduinoCore-nRF528x-mbedos.git] (C:\Users\Maxi\Documents\stackoverflow_testing\.pio\libdeps\nucleo_l152re\ArduinoCore-nRF528x-mbedos)

arm-none-eabi-g++ -o .pio\build\nucleo_l152re\src\main_esp.o -c -std=gnu++98 -fno-rtti [...] @"C:\Users\Maxi\Documents\stackoverflow_testing\.pio\build\nucleo_l152re\longcmd-7828fe635133cd1a11dd36e80ff553da" src\main_esp.cpp

arm-none-eabi-g++: error: CreateProcess: No such file or directory

Expected Results

Compilation success.

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:nucleo_l152re]
platform = ststm32
board = nucleo_l152re
framework = mbed
build_flags = -D PIO_FRAMEWORK_MBED_RTOS_PRESENT
lib_deps = https://github.com/arduino/ArduinoCore-nRF528x-mbedos.git

Source file to reproduce issue:

// Arduino code
#include "Arduino.h"

void setup() {
    pinMode(LED_BUILTIN, OUTPUT);
    Serial.begin(115200);
    Serial.println("Welcome to Arduino on Mbed OS");
}

void loop() {
    digitalWrite(LED_BUILTIN, HIGH);
    Serial.println("LED is now on!");
    delay(1000);
    digitalWrite(LED_BUILTIN, LOW);
    Serial.println("LED is now off!");
    delay(1000);
}

// Mbed OS code
int main() {
    setup();
    while (1) loop();
}

[valeros]

I don't think we can properly overcome this limitation as the input size of WINAPI function CreateProcess is limited to 32767 characters. As a workaround, you can install packages somewhere in the root directory of your drive (e.g C:\) by using packages_dir option.

[maxgerhardt]

The problem still persists when the library installed in C:\PLIB\ though :(.

[platformio]
default_envs = nucleo_l152re
libdeps_dir = C:\PLIB\

[env:nucleo_l152re]
platform = ststm32
board = nucleo_l152re
framework = mbed
build_flags = -D PIO_FRAMEWORK_MBED_RTOS_PRESENT
lib_deps = https://github.com/arduino/ArduinoCore-nRF528x-mbedos.git

|-- <ArduinoCore-nRF528x-mbedos> #a4b622b [git+https://github.com/arduino/ArduinoCore-nRF528x-mbedos.git] (C:\PLIB\nucleo_l152re\ArduinoCore-nRF528x-mbedos)
...
arm-none-eabi-g++ -o .pio\build\nucleo_l152re\lib493\ArduinoCore-nRF528x-mbedos\cores\arduino\Interrupts.o [..] @"C:\Users\Maxi\Documents\stackoverflow_testing\.pio\build\nucleo_l152re\longcmd-5b09eca3e2f9c5ab662062cc89abf51b" C:\PLIB\nucleo_l152re\ArduinoCore-nRF528x-mbedos\cores\arduino\Interrupts.cpp

arm-none-eabi-g++: error: CreateProcess: No such file or directory

Is it the execution of the above arm-none-eabi-g++ command that fails? I cut it short for here but this doesn't seem like 32K of text to me in the console. Are we sure this is the cause of the problem? Are you e.g. able to reproduce that?

[maxgerhardt]

The command for the main_esp.cpp is 1645 characters long and references a longcmd of a size of 245724 characters. Basically the library causes every possible include path for all each of its subfolders to be included, leading to an enourmous amount of includes. Is that correct behaviour?

-I"C:/PLIB/nucleo_l152re/ArduinoCore-nRF528x-mbedos/cores/arduino/mbed/targets/TARGET_Analog_Devices/TARGET_ADUCM302X/TARGET_ADUCM3029" 
[..]
-I"C:/PLIB/nucleo_l152re/ArduinoCore-nRF528x-mbedos/cores/arduino/mbed/targets/TARGET_WIZNET/TARGET_W7500x/W7500x_Peripheral_Library" 

[valeros]

There are plenty of includes in the framework itself so you need to move it to the root folder as well. Are you sure that ArduinoCore-nRF528x-mbedos can be used as a library without proper configuration? It seems that it's a full-fledged Arduino core.

I cut it short for here but this doesn't seem like 32K of text to me in the console. Are we sure this is the cause of the problem? Are you e.g. able to reproduce that?

I've bumped into similar problems before, all characters you pass to the compiler in form of raw arguments or in a special file are eventually transferred to the CreateProcess function and when the total length of these symbols is greater than 32KB it fails somewhere in toolchain core.

[maxgerhardt]

Are you sure that ArduinoCore-nRF528x-mbedos can be used as a library without proper configuration?

It's an mbed-os library which implements the Arduino HAL using mbed-os, see http://blog.janjongboom.com/2019/08/01/arduino-mbed.html

I've bumped into similar problems before, all characters you pass to the compiler in form of raw arguments or in a special file are eventually transferred to the CreateProcess function and when the total length of these symbols is greater than 32KB it fails somewhere in toolchain core.

If that is indeed the case, then the library must be modified to stop spamming 245KB in include paths. I'll look into this.

[maxgerhardt]

@valeros Another example of this problem has popped up in https://community.platformio.org/t/avr-g-error-createprocess-no-such-file-or-directory/12046/13?u=maxgerhardt, where compilation of a PlatformIO provided example project fails (https://github.com/platformio/platform-nordicnrf52/tree/master/examples/mbed-ble-thermometer).

I am able to reproduce that error

arm-none-eabi-g++: error: CreateProcess: No such file or directory
work-mbed@5.51401.200110\components\802.15.4_RF\stm-s2lp-rf-driver\source\NanostackRfPhys2lp.cpp
arm-none-eabi-g++ -o .pio\build\delta_dfbm_nq620\FrameworkMbed\components\802.15.4_RF\stm-s2lp-rf-driver\source\at24mac_s2lp.o -c -std=gnu++14 -fno-rtti -Wvla -c -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fmessage-length=0 -fno-exceptions -ffunction-sections -fdata-sections -funsigned-char -MMD -fno-delete-null-pointer-checks -fomit-frame-pointer -Os -g -DMBED_TRAP_ERRORS_ENABLED=1 -mcpu=cortex-m4 -mthumb -mfpu=fpv4-sp-d16 -mfloat-abi=softfp -DMBED_ROM_START=0x0 -DMBED_ROM_SIZE=0x80000 -DMBED_RAM_START=0x20000000 -DMBED_RAM_SIZE=0x10000 -include mbed_config.h -DPLATFORMIO=40202 -DARDUINO_GENERIC -DPIO_FRAMEWORK_MBED_RTOS_PRESENT -DARM_MATH_CM4 -DBOARD_PCA10040 -DCMSIS_VECTAB_VIRTUAL -DCMSIS_VECTAB_VIRTUAL_HEADER_FILE=\"cmsis_nvic.h\" -DCOMPONENT_NSPE=1 -DCOMPONENT_PSA_SRV_EMUL=1 -DCOMPONENT_PSA_SRV_IMPL=1 -DCONFIG_GPIO_AS_PINRESET -DDEVICE_ANALOGIN=1 -DDEVICE_FLASH=1 -DDEVICE_I2C=1 -DDEVICE_I2C_ASYNCH=1 -DDEVICE_INTERRUPTIN=1 -DDEVICE_LPTICKER=1 -DDEVICE_PORTIN=1 -DDEVICE_PORTINOUT=1 -DDEVICE_PORTOUT=1 -DDEVICE_PWMOUT=1 -DDEVICE_SERIAL=1 -DDEVICE_SERIAL_ASYNCH=1 -DDEVICE_SERIAL_FC=1 -DDEVICE_SLEEP=1 -DDEVICE_SPI=1 -DDEVICE_SPI_ASYNCH=1 -DDEVICE_SYSTICK_CLK_OFF_DURING_SLEEP=1 -DDEVICE_TRNG=1 -DDEVICE_USTICKER=1 -DFEATURE_BLE=1 -DMBED_MPU_CUSTOM -DMBED_TICKLESS -DNRF52 -DNRF52_PAN_12 -DNRF52_PAN_15 -DNRF52_PAN_20 -DNRF52_PAN_30 -DNRF52_PAN_31 -DNRF52_PAN_36 -DNRF52_PAN_51 -DNRF52_PAN_53 -DNRF52_PAN_54 -DNRF52_PAN_55 -DNRF52_PAN_58 -DNRF52_PAN_62 -DNRF52_PAN_63 -DNRF52_PAN_64 -DSWI_DISABLE0 -DTARGET_CORDIO -DTARGET_CORDIO_LL -DTARGET_CORTEX -DTARGET_CORTEX_M -DTARGET_DELTA_DFBM_NQ620 -DTARGET_FF_ARDUINO -DTARGET_LIKE_CORTEX_M4 -DTARGET_LIKE_MBED -DTARGET_M4 -DTARGET_MCU_NRF52832 -DTARGET_NAME=DELTA_DFBM_NQ620 -DTARGET_NORDIC -DTARGET_NORDIC_CORDIO -DTARGET_NRF52 -DTARGET_NRF52832 -DTARGET_NRF5x -DTARGET_RELEASE -DTARGET_RTOS_M4_M7 -DTARGET_SDK_15_0 -DTARGET_SOFTDEVICE_NONE -DTOOLCHAIN_GCC -DTOOLCHAIN_GCC_ARM -D__CMSIS_RTOS -D__CORTEX_M4 -D__FPU_PRESENT=1 -D__MBED_CMSIS_RTOS_CM -D__MBED__=1 @"C:\Users\Maxi\mbed-ble-thermometer\.pio\build\delta_dfbm_nq620\longcmd-4973fc6c3c0d3f729a439ffc951d0e04" C:\Users\Maxi\.platformio\packages\frameworarm-none-eabi-g++: error: CreateProcess: No such file or directory

And again the long command has 32720 bytes

ls -l "C:\Users\Maxi\mbed-ble-thermometer\.pio\build\delta_dfbm_nq620\longcmd-4973fc6c3c0d3f729a439ffc951d0e04"
-rw-rw-rw-  1 Maxi 0 32720 2020-02-25 15:54 C:\Users\Maxi\mbed-ble-thermometer\.pio\build\delta_dfbm_nq620\longcmd-4973fc6c3c0d3f729a439ffc951d0e04

However the longcommand is also full of include directories for components not used in the project or thohse who would not appear during a real mbed-os build. It seems the generating code just recursively adds all folder paths to the list, while only excluding some? This code would need a rework if not even PIO's example projects can be compiled..

[prostoAlexandr]

I faced the same problem trying to compile blink example from mbed repository. Are there any known ways to fix that issue?

[valeros]

@prostoAlexandr It's a known issue, try to use packages_dir option so the include paths will be shorter, for example:

[platformio]
packages_dir = C:\\pio

[env:nucleo_l152re]
platform = ststm32
board = nucleo_l152re
framework = mbed

[prostoAlexandr]

@valeros It actually worked for me! Thanks