esp-hpl

High performance logging drop-in for ESP-IDF. The component swaps ESP-IDF's vprintf handler for a tiny ring-buffer-backed logger that ships log lines to a background task. The calling task hardly blocks, even when many logs are emitted in quick succession.

Motivation

Traditional ESP-IDF logging writes each line straight to UART (real or JTAG-emulated). That stream is slow compared to typical workloads, so emitting logs while sending CAN frames, tracking allocations, or chasing timing bugs can add significant latency—and sometimes even mask problems because the timing pressure eases when logs slow everything down. HPL keeps logging from getting in the way: it queues formatted lines into a large buffer (PSRAM when available, otherwise IRAM) almost instantly and a dedicated background task drains the buffer to the UART at its own pace. Your tasks spend far less time blocked on I/O, and you still get counters to see if backpressure caused drops.

Features

• PSRAM-backed ring buffer (1 MiB default) with IRAM fallback (16 KiB).
• Drop counter and total line counter to detect pressure.
• Minimal integration: install with a single call, remove cleanly.
• Compatibility shim header (HighPerformanceLogger.hpp) for existing ESPenlaub-based code.

Version

Current component version: 1.1.0 (see CHANGELOG.md for details).

Installation

Add the component to your project's idf_component.yml:

dependencies:
  mickeyl/esp-hpl:
    git: "https://github.com/mickeyl/esp-hpl.git"

That's it. Menuconfig options are available under ESP HPL if you want to tune buffer sizes or require PSRAM (disable IRAM fallback).

Usage

#include <esp_hpl.hpp>
#include <esp_log.h>

extern "C" void app_main(void) {
    // Install the high performance logger (replaces ESP-IDF's vprintf handler).
    esp_hpl::HighPerformanceLogger::init();

    ESP_LOGI("EXAMPLE", "Logging stays responsive even under load");

    // ... run your application ...

    // Optionally disable and restore the default logger.
    esp_hpl::HighPerformanceLogger::setEnabled(false);
}

C projects

esp_hpl can be used from C-only projects through the thin wrapper:

#include <esp_hpl_c.h>

void app_main(void) {
    esp_hpl_init(); // installs the logger
    // your code...
}

API overview

• init() – installs the logger and starts the flusher task.
• setEnabled(bool) / isEnabled() – toggles usage; disabling restores the previous ESP-IDF logger.
• shutdown() – stops the flusher task, frees buffers, and restores the original vprintf handler.
• usingPSRAM() – true if the buffer lives in PSRAM.
• droppedCount() / totalLinesLogged() – counters for backpressure diagnostics.

Configuration knobs

Menuconfig options (via idf.py menuconfig):

• CONFIG_ESP_HPL_PSRAM_BUFFER_SIZE (default 1048576)
• CONFIG_ESP_HPL_IRAM_BUFFER_SIZE (default 16384)
• CONFIG_ESP_HPL_ALLOW_IRAM_FALLBACK (default y)

Compile-time macros remain supported and override defaults when defined:

The following macros can be defined at compile time (for example through CFLAGS/CXXFLAGS or idf.py build -D) to adjust buffer sizing:

• ESP_HPL_LOG_LINE_MAXLEN (default 256)
• ESP_HPL_PSRAM_BUFFER_SIZE (default 1*1024*1024)
• ESP_HPL_IRAM_BUFFER_SIZE (default 16*1024)
• ESP_HPL_ALLOW_IRAM_FALLBACK (default 1)

Example project

See examples/basic_logger for a minimal ESP-IDF application that installs the logger, emits a burst of logs, and prints the drop counters.

Build it like any other ESP-IDF example:

cd examples/basic_logger
idf.py set-target esp32
idf.py build flash monitor

CMakeLists.txt in the example already registers the component from this repository through EXTRA_COMPONENT_DIRS.

Notes

• Because the logger replaces ESP-IDF's vprintf, install it before the first log output for consistent behavior.
• If you disable the logger (setEnabled(false)), logging falls back to the previous handler immediately.
• When PSRAM is unavailable, the IRAM buffer is intentionally small; watch droppedCount() under load and adapt buffer sizes if needed.
• If IRAM fallback is disabled and PSRAM allocation fails, the logger aborts after logging PSRAM allocation failed and IRAM fallback disabled!.

License

MIT. See LICENSE.