Compile error with 55.03.34 in combination with custom_sdkconfig

[qniens]

Platform

ESP32

IDE / Tooling

pioarduino

What happened?

Project will not compile with version 55.03.34 in combination with custom_sdkconfig in platformio.ini
It doesn't matter which settings are set. When in Hybrid mode it will give the following linking error.
Deleting the .platformio folder and reinstalling all tools doesn't make a difference.

Stack Trace

Linking .pio\build\test\firmware.elf
C:/Users/quint/.platformio/packages/toolchain-xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/14.2.0/../../../../xtensa-esp-elf/bin/ld.exe: .pio/build/test/lib66f/Network/NetworkClient.cpp.o:(.literal._ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EED5Ev[_ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EED5Ev]+0x0): undefined reference to `__atomic_fetch_add_4'
C:/Users/quint/.platformio/packages/toolchain-xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/14.2.0/../../../../xtensa-esp-elf/bin/ld.exe: .pio/build/test/lib66f/Network/NetworkClient.cpp.o: in function `__gnu_cxx::__exchange_and_add(int volatile*, int)':
C:/Users/quint/.platformio/packages/toolchain-xtensa-esp-elf/xtensa-esp-elf/include/c++/14.2.0/ext/atomicity.h:66:(.text._ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EED2Ev[_ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EED5Ev]+0xe): undefined reference to `__atomic_fetch_add_4'
C:/Users/quint/.platformio/packages/toolchain-xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/14.2.0/../../../../xtensa-esp-elf/bin/ld.exe: C:/Users/quint/.platformio/packages/toolchain-xtensa-esp-elf/xtensa-esp-elf/include/c++/14.2.0/ext/atomicity.h:66:(.text._ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EED2Ev[_ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EED5Ev]+0x26): undefined reference to `__atomic_fetch_add_4'
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\test\firmware.elf] Error 1

Minimal Reproductible Example (MRE)

[main.cpp]

#include <Arduino.h>
#include <AsyncTCP.h>
#include <ESPAsyncWebServer.h>

AsyncWebServer asyncServer(8080);

void setup() {
}

void loop() {
}

platformio.ini

[env:test]
platform =  https://github.com/pioarduino/platform-espressif32/releases/download/55.03.34/platform-espressif32.zip
framework = arduino
board = esp32doit-devkit-v1
monitor_speed = 115200

lib_deps =
  ESP32Async/AsyncTCP
  ESP32Async/ESPAsyncWebServer

custom_component_remove =
    espressif/esp_hosted
    espressif/esp_wifi_remote
    espressif/esp-dsp
    espressif/esp32-camera
    espressif/libsodium
    espressif/esp-modbus
    espressif/qrcode
    espressif/esp_insights
    espressif/esp_diag_data_store
    espressif/esp_diagnostics
    espressif/esp_rainmaker
    espressif/rmaker_common

custom_sdkconfig  = 
    CONFIG_BT_ENABLED=n

I confirm that:

• I have read the documentation.
• I have searched for similar discussions.
• I have searched for similar issues.
• I have looked at the examples.
• I have upgraded to the lasted version of ESPAsyncWebServer (and AsyncTCP for ESP32).

[qniens]

I'm not sure if I need to post this here, or in https://github.com/pioarduino/platform-espressif32

[me-no-dev]

AFAIK pioarduino uses newer/different toolchain than what we use in Arduino (and what is used in the same IDF version). Not sure if that is not a trigger of a sort

[mathieucarbou]

@qniens : just for my knowledge: did you compare the compilation output table for flash size / IRAM / DRAN between using hybrid compilation with your settings and using -03 and LTO compilation with normal ardunio ? there is a big difference ?

[qniens]

No, I haven't. But I will send you the info once it is running again

[Jason2866]

AFAIK pioarduino uses newer/different toolchain than what we use in Arduino (and what is used in the same IDF version). Not sure if that is not a trigger of a sort

pioarduino uses the same toolchains as ArduinoIDE core 3.3.4 (Tasmota framework does use newer gcc).

[mathieucarbou]

I tried to switch back to 3.3.3 pioarduino platform and in my project I see these same errors as in 3.3.4 and also as described above. So I don't know...

[qniens]

FYI, I have opened an issue on the pioarduino git.
pioarduino/platform-espressif32#335

[Jason2866]

@qniens @mathieucarbou
please add to your platformio.ini config:

build_unflags = -mdisable-hardware-atomics

The linker error should be gone.
@me-no-dev Do you have an idea what has changed in IDF which makes this neccessary. The flag is set in pioarduino-build.py

    CCFLAGS=[
        "-Os",
        "-mlongcalls",
        "-ffunction-sections",
        "-fdata-sections",
        "-Wno-error=unused-function",
        "-Wno-error=unused-variable",
        "-Wno-error=unused-but-set-variable",
        "-Wno-error=deprecated-declarations",
        "-Wno-error=extra",
        "-Wno-unused-parameter",
        "-Wno-sign-compare",
        "-Wno-enum-conversion",
        "-gdwarf-4",
        "-ggdb",
        "-mdisable-hardware-atomics",
        "-freorder-blocks",
        "-Wwrite-strings",
        "-fstack-protector",
        "-fstrict-volatile-bitfields",
        "-fno-jump-tables",
        "-fno-tree-switch-conversion",
        "\"-DCHIP_ADDRESS_RESOLVE_IMPL_INCLUDE_HEADER=<lib/address_resolve/AddressResolve_DefaultImpl.h>\"",
        "-MMD"
    ],

It does work for "normal" compile with precompiled Arduino Libs. Honestly currently i dont understand what is the reason failing with HybridCompile since there is the sdkconfig used stored in Arduino framework. Only the sdkconfig which is set in custom_sdkconfig is changed.

[me-no-dev]

No idea what could have changed. I think the toolchain got updated a couple of versions ago, but do not quote me on that

[mathieucarbou]

@Jason2866 : thanks, it works, indeed!

@qniens : FYI, I was able to do the comparison and I got the same results. it is not worth using hybrid compilation in your case with these flags since there is no gain.

My (quite big) app with hybrid compilation (with the flags above)

┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Memory Type/Section ┃ Used [bytes] ┃ Used [%] ┃ Remain [bytes] ┃ Total [bytes] ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Flash Code          │      1444168 │          │                │               │
│    .text            │      1444168 │          │                │               │
│ Flash Data          │       868848 │          │                │               │
│    .rodata          │       868592 │          │                │               │
│    .appdesc         │          256 │          │                │               │
│ IRAM                │        81047 │    61.83 │          50025 │        131072 │
│    .text            │        80019 │    61.05 │                │               │
│    .vectors         │         1028 │     0.78 │                │               │
│ DRAM                │        71880 │    39.77 │         108856 │        180736 │
│    .bss             │        48064 │    26.59 │                │               │
│    .data            │        23816 │    13.18 │                │               │
└─────────────────────┴──────────────┴──────────┴────────────────┴───────────────┘

My (quite big) app without hybrid compilation and with LTO:

┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Memory Type/Section ┃ Used [bytes] ┃ Used [%] ┃ Remain [bytes] ┃ Total [bytes] ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ Flash Code          │      1365616 │          │                │               │
│    .text            │      1365616 │          │                │               │
│ Flash Data          │       814460 │          │                │               │
│    .rodata          │       814204 │          │                │               │
│    .appdesc         │          256 │          │                │               │
│ IRAM                │       104075 │     79.4 │          26997 │        131072 │
│    .text            │       103047 │    78.62 │                │               │
│    .vectors         │         1028 │     0.78 │                │               │
│ DRAM                │        74064 │    59.45 │          50516 │        124580 │
│    .bss             │        48096 │    38.61 │                │               │
│    .data            │        25968 │    20.84 │                │               │
└─────────────────────┴──────────────┴──────────┴────────────────┴───────────────┘

I really don't explain the flash size increase though... I would have expected the opposite. But for the RAM, that's good!

[qniens]

@Jason2866 Yes that works! Thanks

[Jason2866]

Hah, found the reason. Arduino Libs are compiled with PSRAM support. This enables automatically CONFIG_STDATOMIC_S32C1I_SPIRAM_WORKAROUND=y which switches on -mdisable-hardware-atomics
The example uses a board without PSRAM and support is disabled. So there is CONFIG_STDATOMIC_S32C1I_SPIRAM_WORKAROUNDoption not needed AND the -mdisable-hardware-atomics is not needed and not activated.
Since pioarduino uses the pioarduinio-build.py for the Arduino final step the option -mdisable-hardware-atomics is wrong here!!
Checked this by using a board with PSRAM. HybridCompile succeeded succesfully.
Need to find a way to remove the -mdisable-hardware-atomics setting for Arduino in HybridCompile mode when a board is used without PSRAM

[qniens]

@mathieucarbou I’d like to use hybrid mode to improve Ethernet performance, particularly for UDP.

[Jason2866]

Fixed. Platform rereleased. Delete .platformio/platforms and .platformio/.cache to force a download of the fixed updated package

[mathieucarbou]

Confirmed - works fine without -mdisable-hardware-atomics

[mathieucarbou]

BTW - I don't know if you also did more than that, but it was failing before with LTO, now it works ERR talked too soon... Removing the pio/build folder and recompiling hows a lot of errors: "plugin needed to handle lto object"

Users/mat/.platformio/packages/toolchain-xtensa-esp-elf/bin/../lib/gcc/xtensa-esp-elf/14.2.0/../../../../xtensa-esp-elf/bin/ld: .pio/build/pro-esp32/lib70d/U8g2/clib/u8x8_d_t6963.c.o: plugin needed to handle lto object

[Jason2866]

LTO is not working, since IDF code can not compiled with enabled LTO. One possibility is to enable LTO after the Arduino Libs have been compiled. The Arduino code part can be compiled with LTO. Maybe i will add a feature to do this automatically when LTO is enabled.

[qniens]

Once again, thank you all for the quick support, much appreciated.