Difference between running from "arduino.exe" and command line "arduino-builder.exe"

[alorbach]

Hi,

I am building an esp32 project within the Arduino IDE on Windows for ESP32.
I am using the exact same command line parameters that the Arduino IDE puts into "arduino-builder.exe" manually in a batch script. However the outcome of the compiled binary is different.

This is the command line generated by Arudino IDE:

"C:\Program Files (x86)\Arduino\arduino-builder"
 -compile
 -logger=machine
 -hardware "C:\Program Files (x86)\Arduino\hardware"
 -hardware C:\Users\USERNAME\AppData\Local\Arduino15\packages
 -tools "C:\Program Files (x86)\Arduino\tools-builder"
 -tools "C:\Program Files (x86)\Arduino\hardware\tools\avr"
 -tools C:\Users\USERNAME\AppData\Local\Arduino15\packages
 -built-in-libraries "C:\Program Files (x86)\Arduino\libraries"
 -libraries D:\!cvsroot\Arduino\libraries
 -fqbn=esp32:esp32:esp32:PSRAM=enabled,PartitionScheme=min_spiffs,CPUFreq=240,FlashMode=qio,FlashFreq=80,FlashSize=4M,UploadSpeed=921600,DebugLevel=none
 -vid-pid=10C4_EA60
 -ide-version=10813
 -build-path C:\Users\AL8775~1.ADI\AppData\Local\Temp\arduino_build_512285
 -warnings=none
 -build-cache C:\Users\AL8775~1.ADI\AppData\Local\Temp\arduino_cache_713031
 -prefs=build.warn_data_percentage=75
 -prefs=runtime.tools.mkspiffs.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\mkspiffs\0.2.3
 -prefs=runtime.tools.mkspiffs-0.2.3.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\mkspiffs\0.2.3
 -prefs=runtime.tools.xtensa-esp32-elf-gcc.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0
 -prefs=runtime.tools.xtensa-esp32-elf-gcc-1.22.0-80-g6c4433a-5.2.0.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0
 -prefs=runtime.tools.esptool_py.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\2.6.1
 -prefs=runtime.tools.esptool_py-2.6.1.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\2.6.1
 -verbose C:\project\Controller.ino

This is the command line used in my batch script

C:\Program Files (x86)\Arduino\arduino-builder.exe
 -compile
 -logger=machine
 -hardware "C:\Program Files (x86)\Arduino\hardware"
 -hardware "C:\Users\USERNAME\AppData\Local\Arduino15\packages"
 -tools "C:\Program Files (x86)\Arduino\tools-builder"
 -tools "C:\Program Files (x86)\Arduino\hardware\tools\avr"
 -tools "C:\Users\USERNAME\AppData\Local\Arduino15\packages"
 -built-in-libraries "C:\Program Files (x86)\Arduino\libraries"
 -libraries "D:\!cvsroot\Arduino\libraries"
 -fqbn=esp32:esp32:esp32:PSRAM=enabled,PartitionScheme=min_spiffs,CPUFreq=240,FlashMode=qio,FlashFreq=80,FlashSize=4M,UploadSpeed=921600,DebugLevel=none
 -vid-pid=10C4_EA60
 -ide-version=10813
 -build-path "C:\temp\_build\m5camx"
 -warnings=none
 -build-cache "C:\temp\_cache\m5camx"
 -prefs=build.warn_data_percentage=75
 -prefs=runtime.tools.mkspiffs.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\mkspiffs\0.2.3
 -prefs=runtime.tools.mkspiffs-0.2.3.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\mkspiffs\0.2.3
 -prefs=runtime.tools.xtensa-esp32-elf-gcc.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0
 -prefs=runtime.tools.xtensa-esp32-elf-gcc-1.22.0-80-g6c4433a-5.2.0.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\xtensa-esp32-elf-gcc\1.22.0-80-g6c4433a-5.2.0
 -prefs=runtime.tools.esptool_py.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\2.6.1
 -prefs=runtime.tools.esptool_py-2.6.1.path=C:\Users\USERNAME\AppData\Local\Arduino15\packages\esp32\tools\esptool_py\2.6.1
 -verbose "C:\project\Controller.ino"

However the result is different, when I compile it using my batch script, important preprocessor variables are missing which results in a build missing PSRAM support for example.

And I don't understand why this is not working, I am using the exact same command line parameters as Arduino IDE is using.
Yet the result is that the

[alranel]

You should really consider using arduino-cli instead. It's much more powerful and actively maintained :)

[alorbach]

I played arround with arduino-cli but it is impossible to get my sketch running there.
arduino-cli is not finding esp32 internal libraries. However after playing arround for 3-4 hours, I got back to arduino-builder and out of the sudden, it creates a correct build now ... o.O
I don't have to understand this ...

[ubidefeo]

@alorbach
if it works in Arduino Builder it must work in arduino-cli

[alorbach]

at the moment it does not, it appears to be a problem if you have more than 1 location for third party libraries which I have.