Added a few more ESP32-ism files used by ESPAsyncWebServer by MitchBradley · Pull Request #17 · pschatzmann/Arduino-Emulator

MitchBradley · 2026-03-31T02:37:00Z

(Sorry about the ifdef screwup with my last PR)

ESPAsyncWebServer needs a few extra things that are present in the ESP32 Arduino framework, namely

libb64 - It is possible to get this from linux and macos host packages, but the ESP Arduino version of libb64 has a few tweaks that make it very cumbersome to use the standard libb64 - plus you have to make sure to install the host packages and include the libs on the linker. It is far easier to just include libb64/ here.
md5 - similarly, you can get it from openssl, but it is similarly cumbersome, and quite easy to just drop MD5Builder here
cbuf - yet another (small) ESP-ism that ESPAsyncTCP relies on

I think (hope) that this is the last set of changes related to this.

Copilot

Pull request overview

This PR adds several ESP32/ESP8266-compatibility source files into ArduinoCore-Linux/cores/arduino to satisfy dependencies required by ESPAsyncWebServer / ESPAsyncTCP (base64, MD5, and a circular buffer utility).

Changes:

Added an MD5Builder implementation backed by BearSSL MD5.
Vendored libb64 (base64 encode/decode) sources and license metadata.
Added cbuf circular buffer implementation used by async TCP/webserver stacks.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
ArduinoCore-Linux/cores/arduino/MD5Builder.h	Declares MD5Builder API used by higher-level networking/web libs
ArduinoCore-Linux/cores/arduino/MD5Builder.cpp	Implements MD5 hashing and stream ingestion
ArduinoCore-Linux/cores/arduino/libb64/LICENSE	Adds libb64 licensing text
ArduinoCore-Linux/cores/arduino/libb64/AUTHORS	Adds libb64 author attribution
ArduinoCore-Linux/cores/arduino/libb64/cencode.h	Base64 encoder public header
ArduinoCore-Linux/cores/arduino/libb64/cencode.c	Base64 encoder implementation
ArduinoCore-Linux/cores/arduino/libb64/cdecode.h	Base64 decoder public header
ArduinoCore-Linux/cores/arduino/libb64/cdecode.c	Base64 decoder implementation
ArduinoCore-Linux/cores/arduino/cbuf.h	Declares circular buffer class used by async networking code
ArduinoCore-Linux/cores/arduino/cbuf.cpp	Implements circular buffer operations (read/write/resize)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ArduinoCore-Linux/cores/arduino/MD5Builder.cpp

Copilot · 2026-03-31T02:41:40Z

ArduinoCore-Linux/cores/arduino/MD5Builder.cpp

+bool MD5Builder::addStream(Stream &stream, const size_t maxLen) {
+    const int buf_size = 512;
+    int maxLengthLeft = maxLen;
+


In addStream(), maxLen is a size_t but it is immediately narrowed into int maxLengthLeft = maxLen;. On 64-bit hosts or large inputs this can overflow/truncate and cause incorrect loop behavior. Use size_t for the remaining length (and adjust comparisons/casts around stream.available() / readBytes) to avoid narrowing.

Fixed; maxLengthLeft is now size_t

Copilot · 2026-03-31T02:41:41Z

ArduinoCore-Linux/cores/arduino/MD5Builder.h

+#pragma once
+
+#include <api/String.h>
+#include <Stream.h>
+#include <bearssl/bearssl_hash.h>
+
+class MD5Builder {
+private:
+    br_md5_context _ctx;
+    uint8_t _buf[16];
+public:
+    void begin(void);
+    void add(const uint8_t * data, const uint16_t len);
+    void add(const char * data) {
+        add((const uint8_t*)data, strlen(data));
+    }


MD5Builder.h uses strlen() in inline methods but does not include <string.h>/<cstring>. This makes the header non-self-contained and can fail to compile depending on include order. Please include the proper header (and/or avoid calling strlen in the header).

Added "include "

ArduinoCore-Linux/cores/arduino/cbuf.cpp

Copilot · 2026-03-31T02:41:41Z

ArduinoCore-Linux/cores/arduino/cbuf.cpp

+    newSize += 1;
+    // not lose any data
+    // if data can be lost use remove or flush before resize
+    if((newSize < bytes_available) || (newSize == _size)) {


In cbuf::resize(), after newSize += 1 (to account for the sentinel), the check if ((newSize < bytes_available) ...) is too weak. When newSize == bytes_available the code will set _end = newbuf + bytes_available, which equals _bufend and later writes will dereference _end out of bounds. The condition should ensure the internal buffer size is strictly greater than the number of stored bytes (e.g., newSize <= bytes_available should reject), accounting for the sentinel slot.

Suggested change

if((newSize < bytes_available) || (newSize == _size)) {

if((newSize <= bytes_available) || (newSize == _size)) {

ArduinoCore-Linux/cores/arduino/cbuf.cpp

MitchBradley · 2026-03-31T02:53:17Z

Yuck, the bearssl dependency. The ESP32 version of MD5Builder depends on the ESP32's ROM implementation of md5. This version that I included is cribbed from earlephilhower's ArduinoPico framework. Looks like I have some more work to do to clean this up.

* Eliminated MD5Builder's dependency on BearSSL by copying publicly-available md5 code directly into MD5Builder and adjusting it to use class variables instead of a separate context structure. * Fixed the well-known off-by-one size bug in the cbuf code that was copied from the ESP32 Arduino code. The sizes returned by all methods now reflect that actual data capacity of the buffer, without counting the sentinel byte that should be hidden. I asked for multiple AI reviews of the new code and also verified that it is compatible with the one use of cbuf in ESPAsyncWebServer

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ArduinoCore-Linux/cores/arduino/cbuf.cpp

ArduinoCore-Linux/cores/arduino/MD5Builder.cpp

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ArduinoCore-Linux/cores/arduino/libb64/cencode.c

ArduinoCore-Linux/cores/arduino/libb64/cdecode.c

ArduinoCore-Linux/cores/arduino/cbuf.h

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

ArduinoCore-Linux/cores/arduino/cbuf.cpp

ArduinoCore-Linux/cores/arduino/libb64/cdecode.c

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

MitchBradley · 2026-04-01T22:56:47Z

I think this is ready, after several rounds of CoPilot review.

pschatzmann · 2026-04-02T06:41:20Z

Did you test this with the ESPAsyncServer ?

I suggest that you provide a simple example in the examples directory...

MitchBradley · 2026-04-02T19:01:58Z

Yes, I tested with ESPAsyncWebServer. I am making https://github.com/bdring/FluidNC truly portable, including running it under Linux, MacOS, and Windows. (The use cases for hosted builds include demonstration, rapid development, and integration testing). At present, it is working on ESP32 (the original target), ESP32-S3, RP2040, RP2350, Linux, MacOS, and (partially) Windows, with networking support across the board. It originally used synchronous networking, but we recently switched to ESPAsyncWebServer for performance and (hopefully) reliability reasons.

Making it work on Linux and MacOS required changes to Arduino-Emulator, ESPAsyncWebServer (mostly the addition of "|| defined (HOST}" clauses, and the creation of a Posix version of the AsyncTCP package in addition to the preexisting ESP and RP AsyncTCP packages.

Deploying this is tedious exercise since I have to PR against existing packages in addition to publishing a new one, on top of the changes to the application itself. I am trying to do the PRs in stages so I can focus.

I can create an example, but the example will not work until the other packages are ready - and the creation thereof would delay the PR activity on those packages. What do you think is the best way forward?

Added a few more ESP32-ism files used by ESPAsyncWebServer

e403072

Copilot AI review requested due to automatic review settings March 31, 2026 02:37

Copilot started reviewing on behalf of MitchBradley March 31, 2026 02:37 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

MitchBradley requested a review from Copilot April 1, 2026 21:42

Copilot started reviewing on behalf of MitchBradley April 1, 2026 21:42 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

ArduinoCore-Linux/cores/arduino/cbuf.cpp Show resolved Hide resolved

ArduinoCore-Linux/cores/arduino/cbuf.cpp Outdated Show resolved Hide resolved

ArduinoCore-Linux/cores/arduino/MD5Builder.cpp Outdated Show resolved Hide resolved

MitchBradley added 2 commits April 1, 2026 12:03

This time for sure on cbuf

8103815

Corrected comment about digest input size

5210bf0

MitchBradley requested a review from Copilot April 1, 2026 22:08

Copilot started reviewing on behalf of MitchBradley April 1, 2026 22:09 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

MitchBradley and others added 2 commits April 1, 2026 15:18

Update ArduinoCore-Linux/cores/arduino/libb64/cencode.c

905a6c9

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update ArduinoCore-Linux/cores/arduino/libb64/cdecode.c

f92d828

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

MitchBradley requested a review from Copilot April 1, 2026 22:26

Copilot started reviewing on behalf of MitchBradley April 1, 2026 22:27 View session

Copilot AI reviewed Apr 1, 2026

View reviewed changes

MitchBradley and others added 2 commits April 1, 2026 15:36

Update ArduinoCore-Linux/cores/arduino/cbuf.cpp

c36c93f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update ArduinoCore-Linux/cores/arduino/libb64/cdecode.c

8321cc6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

pschatzmann merged commit 9c7eb76 into pschatzmann:main Apr 2, 2026
1 check passed

	if((newSize < bytes_available) \|\| (newSize == _size)) {
	if((newSize <= bytes_available) \|\| (newSize == _size)) {

Conversation

MitchBradley commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

MitchBradley Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

MitchBradley Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

MitchBradley Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MitchBradley commented Mar 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MitchBradley commented Apr 1, 2026

Uh oh!

pschatzmann commented Apr 2, 2026

Uh oh!

MitchBradley commented Apr 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants