Support for: SIMD

[GiorgosXou]

Just learned about SIMD from a comment about pangrams in stackoverflow. ESP32 I think supports SIMD ... and possibly many others, so why not?

[StuartIanNaylor]

Esp32-S3 supports SIMD as it has vector instructions, not so sure about the plain old esp32 though.
https://github.com/espressif/esp-dl
The benchmarks are for the esp-dl that have been optimised with vector instructions.
https://docs.espressif.com/projects/esp-dsp/en/latest/esp32/esp-dsp-benchmarks.html

I think cadence holds some IP as the layers and functions avail are limited such as no LSTM or Leaky Relu...

Would be great to have a full SIMD support on this microcontroller.

[GiorgosXou]

Interesting, seems as if only ESP-32 S3 supports SIMD instructions and right now I only have the C3. Thankfully though, I can test\play around with the ansi versions of the functions. Thanks for bringing this to my attention, I'll see what I can do.

[StuartIanNaylor]

Likely will work the same but with an S3 up to x10 faster that they got with some of there benches.
I am not sure why with tflite4micro why recurrent layers are missing and likely because I haven't a clue at that level.
GRU & LSTM are sadly missed as quite a few small models use them but the S3 is quite a boost with ML and not much more https://www.lilygo.cc/products/t7-s3

[GiorgosXou]

@StuartIanNaylor new version, I just added a basic SIMD optimization, improving feedforward for ESP32-S3 aproximatelly from O(n^3) to O(n^2).

[GiorgosXou]

here's some references and a minimal example for anyone confushed by the limited documentation:

A basic example of code using "dsps_add_f32"

#include <esp_dsp.h>
#define VECTOR_SIZE 16


void printv(float * v){
  Serial.print("Vector: ");
  for (int i = 0; i < VECTOR_SIZE; i++) {
    Serial.print(v[i]);
    Serial.print(" ");
  }  
  Serial.println();
}


void setup() {
  Serial.begin(115200);

  float a[VECTOR_SIZE];
  float b[VECTOR_SIZE];
  float result[VECTOR_SIZE];

  // Initialize the arrays with some data
  for (int i = 0; i < VECTOR_SIZE; i++) {
    a[i] = i;
    b[i] = VECTOR_SIZE - i;
  }
  printv(a);
  printv(b);
  
  dsps_add_f32(a, b, result, VECTOR_SIZE, 1, 1, 1); // Works just fine on any ESP-32 because a macro translates it to dsps_add_f32_ansi if it doesn't support SIMD instructions
  //dsps_add_f32_ae32(a, b, result, VECTOR_SIZE, 1, 1, 1); // Compiles only for ESP-32-S3 at the momment
  printv(result);
}

void loop() {}

dsps_add_f32 defined as ansi or ae32 based on support

[GiorgosXou]

Partially done (at least for ESPs):

NeuralNetworks/src/NeuralNetwork.h

Lines 1964 to 1967 in 5cd31c9

	#elif !defined(REDUCE_RAM_WEIGHTS_LVL2)
	dsps_dotprod_f32(inputs, weights[i], &outputs[i], _numberOfInputs); // https://github.com/GiorgosXou/NeuralNetworks/discussions/16#discussioncomment-7479256
	#else
	dsps_dotprod_f32(inputs, &me->weights[me->i_j], &outputs[i], _numberOfInputs);