XIP and builtin intrisincs

[borjatelladotecnalia]

Hi

I'm working on an STM32-based platform that runs a WebAssembly (WASM) application performing compute-intensive tasks involving double-precision and floating-point operations. Due to limited RAM and FLASH space, I aim to execute the WASM module directly from FLASH using Execute In Place (XIP).

Note:
I suspect that the --xip flag leads to "nan" and "integer overflows" when double values are very big or small. The same wasm module compiled without --xip in principle runs smoothly. Makes sense?

Toolchain

• Compiler: Clang (compiling C source files to WASM)
• AOT Compiler: wamrc (WebAssembly Micro Runtime Ahead-of-Time compiler)

Objective

Enable XIP to reduce memory footprint while maintaining full support for floating-point and double-precision computations.

Initial Approach

I compiled the WASM module using the following command:

./wamrc --xip --size-level=3 \
        --target=thumbv8m.base \
        --cpu=cortex-m33 \
        --target-abi=eabi \
        -o myfile_v8m.aot ./myfile.wasm

This enabled XIP as expected. However, at runtime, some double values were replaced by NaN, leading to incorrect calculations and integer overflows.
Following what is decribed in "https://github.com/bytecodealliance/wasm-micro-runtime/blob/main/doc/xip.md" I changed the compilation script to:

./wamrc --xip --size-level=3 \
        --target=thumbv8m.base \
        --cpu=cortex-m33 \
        --target-abi=eabi \
        --enable-builtin-intrinsics=fp.common \
        -o myfile_v8m.aot ./myfile.wasm

With this configuration, the WASM module fails to load and outputs the following error:

AOT module load failed: cannot apply relocation to text section for aot file generated with "--enable-indirect-mode" flag

Am I missing something?

[lum1n0us]

@no1wudi any comments?

[borjatelladotecnalia]

Just some additional information, my clang compilation looks like this:

project(wasm-app C ASM)


if (NOT DEFINED WASI_SDK_DIR)
    set (WASI_SDK_DIR               "~/wa-release/wasi-sdk")
endif ()

set (CMAKE_C_FLAGS                  "-O3 -fPIC")
set (CMAKE_C_COMPILER_TARGET        "wasm32")
set (CMAKE_C_COMPILER               "${WASI_SDK_DIR}/bin/clang")


include_directories(../wasm-src)
# Some other include folders here...


set (CMAKE_EXE_LINKER_FLAGS
     "-nostdlib -z stack-size=4096\
     -Wl,--no-entry \
     -Wl,--export=__main_argc_argv  \
     -Wl,--export=__heap_base \
     -Wl,--export=__data_end \
     -Wl,--allow-undefined "
)

# Som other things done here.... 

# Finally Custom command to compile wams into aot and binary dump to a .h file.
add_custom_command(
    TARGET my_file.wasm
    POST_BUILD
    COMMAND ../do_aot.sh
    COMMENT "Running post-build step"
)

[borjatelladotecnalia]

This is the reduced use case where I'm struggling with -xip flag. The code below has the following outcome
I guess that is something related to memory aligment done with xip flag that impacts in floats ...

1.- Compilation including -xip:

Float Value is 0.000000
Float Values is 0.000000
Values is 4
Values is 4

2.- Compilation just removing -xip flag from wamrc compilation

Float Value is 35.500000
Float Values is 6400.000000
Values is 4
Values is 4

Do I miss somethig during compilation?

// my_flk.h
typedef struct
{
    float   sampling_freq;
    float   cutoff_freq_3b;
    float   cutoff_freq_3a;
    uint8_t  decimation_factor_3b_1; // after cascade filter 1
    uint8_t  decimation_factor_3b_2; // after cascade filter 2
    uint8_t  decimation_factor_4; 
    
} wasm_flicker_filter_variables_t;

typedef struct
{
    /* 8-bit data */
    uint8_t decimation_factor_3b;
    uint8_t decimation_sample_3b_1;
    uint8_t decimation_sample_3b_2;
    uint8_t decimation_output;
   
} hs_flicker_t;

void my_hs_flicker_init(hs_flicker_t *_this, wasm_flicker_filter_variables_t init_filter_vars);

// my_flk.c
#include <stdio.h>

void my_hs_flicker_init(hs_flicker_t *_this,   wasm_flicker_filter_variables_t init_filter_vars_local)
{
    printf ("Float Value is %f\n",(float) init_filter_vars_local.cutoff_freq_3b);
    printf ("Float Values is %f\n",(float) init_filter_vars_local.sampling_freq);
    printf ("Values is %d\n", init_filter_vars_local.decimation_factor_3b_1);
    printf ("Values is %d\n", init_filter_vars_local.decimation_factor_3b_2);
}

// main.c

int main(int argc, char **argv)
{

    wasm_flicker_filter_variables_t wasm_init_filter_vars;
    wasm_init_filter_vars.sampling_freq          = 50 * 128;
    wasm_init_filter_vars.cutoff_freq_3b         = 35.5;
    wasm_init_filter_vars.decimation_factor_3b_1 = 4;
    wasm_init_filter_vars.decimation_factor_3b_2 = 4;
    wasm_init_filter_vars.cutoff_freq_3a         = 0.05;
    wasm_init_filter_vars.decimation_factor_4    = 8;

    my_hs_flicker_init(&ws_mtr_flicker_v1, wasm_init_filter_vars);
    
    return 0;
}

[no1wudi]

@borjatelladotecnalia Is the FPU enabled in the Cortex-M33 chip you are using?

[borjatelladotecnalia]

@no1wudi yes, in principle it is enabled. These "float" operations only fail when -xip is included during the wamrc compilation.
Don't hesitate asking for any test that you think I could do to put some light on this issue.
I don't think the issue is directly related to float values themselves, but rather to how they are stored in structs or in flash memory (XIP). I might be missing something related to alignment or stack?

[borjatelladotecnalia]

@no1wudi , @lum1n0us , I was wondering if it is possible to disassembly the .aot file generated by the wamrc compiler, is it?

[no1wudi]

@borjatelladotecnalia Yes, you can use the --format=llvmir-opt to get llvm ir or use --format=object to get the ELF based object file, then you can use objdump to disassembly it.

Could you show the llvm IR file or disassembly file for this issue?

[borjatelladotecnalia]

@no1wudi , yes I can provide the disassembly, where do you want me to paste it? Just here as another "comment" entry.

I simplified my test case to:

int main(int argc, char **argv)
{

    printf("!! WAMR app Running-1 !!\n");

    volatile double va = 435;
    volatile double vb = 13.5;
    volatile double vc = 0.0;
    vc = va / vb ;
    printf ("Numerator   value is %f\n", va);
    printf ("Denominator value is %f\n", vb);
    printf ("Division    value is %f\n", vc);
}

My stm32u has single precision FPU. My tests:

Test	Clang Compilaton flags	Wamrc Flags	Printed Output
01	-O0 -fPIC	--xip --opt-level=0 --size-level=3 --target=thumbv8m.base --cpu=cortex-m33 --target-abi=eabi	Numerator=435.000 , Denominator=13.5000, Division=0.0310 (?? just the inverse!!)
02	-O0 -fPIC	--xip --size-level=3 --target=thumbv8m.base --cpu=cortex-m33 --target-abi=eabi	Numerator=435.000 , Denominator=13.5000, Division=nan
03	-O3 -fPIC	--xip --opt-level=0 --size-level=3 --target=thumbv8m.base --cpu=cortex-m33 --target-abi=eabi	Numerator=435.000 , Denominator=13.5000, Division=nan
04	-O3 -fPIC	--size-level=3 --target=thumbv8m.base --cpu=cortex-m33 --target-abi=eabi	Numerator=435.000 , Denominator=13.5000, Division=32.2222

Clang Link flags in all cases:

set (CMAKE_EXE_LINKER_FLAGS
     "-nostdlib -z stack-size=8192 \
     -Wl,--no-entry \
     -Wl,--export=__main_argc_argv  \
     -Wl,--export=__heap_base \
     -Wl,--export=__data_end \
     -Wl,--allow-undefined "
)

[srberard]

I think you need to set --target-abi to eabihf to enable hardware floating point when running the AOT compile. Also, you probably don't want to specify --enable-builtin-intrinsics.

[borjatelladotecnalia]

@srberard , Thanks!!.
That solved my issue. It is working for with eabihf and --xip flag. For us --xip is critical, we run a small platform (2MB Flash, ~700KB RAM).

[srberard]

Excellent, glad that worked.