Repository navigation
ESP-DL Release v3.3.14
ESP-DL v3.3.14 continues the lightweight inference framework for ESP chips. Since v3.2.0 it adds mixed-precision quantization, faster operators on ESP32-S3 and ESP32-P4, new vision and audio models, and support for ESP32-S31 and ESP32 C series.
Full changelog: v3.2.0...master
Performance
Conv, Gemm, MatMul, Resize, Reduce, LUT, Transpose, and Pooling have been reworked, including cache tiling, INT16 pointwise weight reuse, and PIE kernels for ESP32-S3, and ESP32-P4.
On representative models such as MobileNetV2 and YOLO11:
- ESP32-S3: more than 3× faster
- ESP32-P4: more than 1.9× faster
Models can also drop unused operators and Conv kernels at compile time, which reduces binary size.
Quantization
- w8a16 mixed quantization is fully supported from v3.3.11, with ESP32-S3 and ESP32-P4 kernels. 8-bit, 16-bit, and mixed precision can be combined in one model.
- Per-channel Conv and Gemm on ESP32-P4. This needs ESP-PPQ ≥ 1.2.10 and ESP-DL ≥ 3.3.1. Existing per-tensor models still run.
- TrainedQuantizedThreshold (TQT) calibration, plus an INT16 LUT pipeline.
- AutoQuant searches quantization strategies and parameters automatically.
- espdl-quantize skill iterates on esp-ppq settings to recover accuracy with less manual tuning.
New models
| Model | What it does | Targets |
|---|---|---|
| YOLO26n | NMS-free COCO detection. | ESP32-S3, ESP32-P4 |
| YOLO11n-Seg | COCO instance segmentation | ESP32-S3, ESP32-S31, ESP32-P4 |
| PP-OCRv6 | On-device OCR for Chinese, English, and 46 Latin-script languages | ESP32-P4 |
| Speaker verification | Voiceprint verification, 16-bit PCM input, on-device embedding database | ESP32-S31, ESP32-P4 |
| Hand detection / gesture recognition | Hand detection and 8+ gesture classes | ESP32-S3, ESP32-S31, ESP32-P4 |
| Person ReID | Person re-identification | ESP32-S3, ESP32-S31, ESP32-P4 |
| Motion detection | Frame-to-frame motion change | ESP32 series |
| ESPDet-Pico face | espdet-pico human face detection | ESP32-S3, ESP32-S31, ESP32-P4 |
MobileNetV2, YOLO11n, YOLO11n-pose, face, hand, cat, and dog models were refreshed, and latency numbers were updated.
Runtime and platform
- StreamingCache for streaming models, including inserting the cache table by hand.
- ESP32-S31 is supported, including the precision fix. ESP32 C series Conv is accelerated. ESP-IDF v6.0 is supported.
- Dual-core workers are reused across runs.
reset()is available on models and modules. Models can take multiple input shapes. .espdlfiles can be opened in Netron. FlatBuffers loading and the PDL3 model package were updated.- Default allocations use
MALLOC_CAP_DL_DEFAULT. - Image preprocessing adds grayscale, YUV422, and faster color conversion on ESP32-P4.
- Audio adds int16 and int32 fbank, int16 and int32 FFT/RFFT, and a shared FFT table.
- espdl-operator, esp32s3-pie-simd, and esp32p4-pie-simd skills help an agent implement operators and rewrite them with PIE instructions.
Operators
| Operator | int8 | int16 | w8a16 | float32 | Restrictions |
|---|---|---|---|---|---|
| Add(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D |
| AveragePool(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support 1d/2d, don't support dilation |
| Clip(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Concat(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Conv(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | ✖ | Support 1d/2d conv, groups only support 1 or input_channels |
| ConvTranspose(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | Support ConvTranspose by InsertZeros + Conv |
| DepthToSpace(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Div(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D |
| Elu(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Equal(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | |
| Exp(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Flatten(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Gather(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Gemm(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | ✖ | |
| GlobalAveragePool(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support 1d/2d |
| Greater(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | |
| GreaterOrEqual(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | |
| GRU(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| HardSigmoid(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| HardSwish(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| LayerNormalization(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Supports normalization over specified axis with scale and bias |
| LeakyRelu(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Less(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | |
| LessOrEqual(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | |
| Log(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| LogSoftmax(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Dtype of output is float32 |
| LpNormalization(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| LSTM(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| MatMul(ESP-DL)(ONNX) | ✔ | ✔ | ✔ | ✖ | Support up to 4D. w8a16 applies to static input1 only. |
| MaxPool(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | Support 1d/2d, don't support dilation |
| Mod(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D, multidirectional broadcasting, fmod=1 (C-style). Per ONNX spec fmod=0 is integer-only. |
| Mul(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D |
| Neg(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Pad(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Do not support wrap mode |
| Pow(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D, multidirectional broadcasting |
| PRelu(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceL1(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceL2(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceLogSum(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceLogSumExp(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceMax(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceMean(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceMin(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceProd(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D |
| ReduceSum(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| ReduceSumSquare(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Relu(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Reshape(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Resize(ESP-DL)(ONNX) | ✔ | ✖ | ✖ | ✖ | support 1d/2d nearest/linear/bilinear, don't support roi and antialias |
| ReverseSequence(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✖ | |
| RMSNormalization(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Supports normalization over specified axis with scale weight |
| ScatterND(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Supports reduction operations: none, add, mul, max, min |
| Sigmoid(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Slice(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Softmax(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Dtype of output is float32 |
| SpaceToDepth(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Split(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Sqrt(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Squeeze(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Sub(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | Support up to 4D |
| Swish(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Tanh(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Transpose(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ | |
| Unsqueeze(ESP-DL)(ONNX) | ✔ | ✔ | ✖ | ✔ |
Bug fixes
- AveragePool no longer loops forever on float inference.
- Dual-core mode no longer double-frees workers.
- Memory planning works when only internal RAM is available.
- In-place ops on multi-branch graphs, and an incorrect internal address, are fixed.
- ESP32-S31 precision mismatch and an ESP32-S3 int16 FFT overflow are fixed.
- ESP-IDF v6.0 builds use the PSA Crypto API instead of the removed mbedtls AES headers.