WebUIBuildPipeline

WebUI Build Pipeline

How a Raft project's optional Web UI is built, gzipped and packaged into the device's filesystem image so that RaftWebServer can serve it from /.

Raft does not prescribe a Web UI framework — any tool that emits a dist/ folder of static assets from npm run build will work. The integration is done by scripts/GenWebUI.py, wired into the build by RaftProject.cmake.

The project's TestWebUI and most demo apps use parcel.js because it needs almost no configuration, but Vite, Webpack, esbuild or rollup are equally fine.

How it fits into the build

features.cmake          systypes/<sys>/                       build/
   set(UI_SOURCE_PATH …) ─►  WebUI/                              ├─ BuildWebUI/   (parcel output, gzipped)
                              ├─ package.json                    └─ FSImage/      (combined static + Web UI)
                              ├─ src/                                  └─ flashed as the `fs` partition
                              └─ (parcel config, deps)

For each SysType the build does:

1. Validate UI_SOURCE_PATH. RaftProject.cmake resolves it relative to the SysType folder and fails the build with a clear message if the directory or its package.json is missing.
2. Run GenWebUI.py. The script invokes npm install (skippable with --npminstall), then npm run build, in the source folder.
3. Pick up the build output. All .html, .js, .css files (and optionally .map files with --incmap) from the dist/ folder are copied into build/<sys>/BuildWebUI/. By default each file is gzipped on the way over so the on-device server can serve Content-Encoding: gzip directly.
4. Merge into the filesystem image. CopyFSAndWebUI copies the user's static FS image (FS_IMAGE_PATH) over to build/<sys>/FSImage/, then overlays the gzipped Web UI assets on top.
5. Build the FS partition image. RaftGenFSImage.cmake runs littlefs_create_partition_image (or SPIFFS) with FLASH_IN_PROJECT, so idf.py flash flashes both the firmware and the FS partition.

The relevant CMake variables a SysType's features.cmake controls are:

Variable	Purpose	Example
UI_SOURCE_PATH	Path (relative to the SysType folder) to the Web UI project — the directory that contains package.json.	"../Common/WebUI"
FS_IMAGE_PATH	Path to extra static files merged into the FS image (data, certs, configs).	"../Common/FSImage"
FS_TYPE	"littlefs" (default) or "spiffs".	"littlefs"

Omit UI_SOURCE_PATH entirely to skip the Web UI build and ship just the static FS image.

SysType layout

A typical Web UI-enabled SysType looks like:

TestWebUI/
├── CMakeLists.txt              ← thin shim, includes RaftProject.cmake
├── components/                 ← C++ application code
└── systypes/
    ├── Common/
    │   ├── FSImage/            ← non-WebUI static assets (fonts, JSON, ...)
    │   ├── WebUI/              ← UI_SOURCE_PATH points here
    │   │   ├── package.json
    │   │   ├── src/
    │   │   │   ├── index.html
    │   │   │   ├── index.tsx
    │   │   │   └── ...
    │   │   └── (parcel/vite config)
    │   └── features.cmake      ← shared by all variants
    └── RaftI2CBox/
        ├── partitions.csv
        ├── sdkconfig.defaults
        ├── SysTypes.json       ← runtime SysMod config
        └── features.cmake      ← variant-specific overrides

features.cmake in a variant typically does little more than set(SYSTYPE "Common")-style sharing; both files are concatenated by Raft's SysType machinery.

What GenWebUI.py does

The script is small and intentionally framework-agnostic. The interesting steps:

1. Empties the destination folder (build/<sys>/BuildWebUI/) so stale assets don't survive into the FS image.
2. npm install (unless --npminstall is passed). Honours your project's package-lock.json.
3. npm run build. The script does not care which bundler runs as long as it produces a flat dist/ folder. The folder name is --distFolder (default dist).
4. For every output file with extension .html / .js / .css (and optionally .map):
   • If gzip is enabled (default), gzip-compress to <filename>.gz in the destination.
   • Otherwise move/rename verbatim.

CMake passes options via WEB_UI_GEN_FLAGS, e.g. --incmap for source maps in development builds.

How the device serves the assets

RaftWebServer mounts the FS partition and serves files from it. When a file ending in .gz exists alongside the requested name (e.g. index.html.gz for /index.html), the server returns the compressed bytes with Content-Encoding: gzip. Modern browsers handle the decompression transparently. Result: the on-device flash usage is roughly halved with no runtime CPU cost on the ESP32.

The FS partition is sized in partitions.csv — see Partitions and Flash Layout. A WebUI built from a typical React+MUI starter compresses to ~150–250 KB; allocate 0x80000 (512 KB) or more for the fs partition unless you have measured otherwise.

Choosing a bundler

Any bundler that emits a flat dist/ works. The conventions are:

• A single-page app: one index.html plus hashed .js and .css bundles.
• No directory nesting in the output (or, if you must nest, accept that GenWebUI.py only walks the top level — extend the script if needed).
• Inline images / fonts as data URLs or convert them via optimizeDeps so they do not need to be served as additional files.

Parcel is the lowest-configuration option (npm i -D parcel; npx parcel build src/index.html). Vite produces smaller bundles with similar effort. Webpack works but typically needs an explicit config for hashed output names. The example WebUIs in RaftI2C/TestWebUI and the Raft Web App repository use parcel.

Development workflow

The on-device build is gzipped and intended for production. While developing the UI:

cd systypes/Common/WebUI
npm install
npm run start          # parcel dev server, hot reload

Point the dev server at a real (or simulated) device's REST API by adding a proxy or by enabling CORS on the device's RaftWebServer (enableCORS: 1 in its config). When happy, build the firmware target normally — idf.py build will pick up the latest source files (the CMake target depends on ${UI_SOURCE_PATH}/src/*) and rebuild the bundle.

If only the Web UI changed, idf.py app-flash fs-flash reflashes just the affected partitions and is much faster than a full flash.

Skipping the Web UI

For headless devices, comment out set(UI_SOURCE_PATH ...) in features.cmake. The build still produces an FS image from FS_IMAGE_PATH and flashes it; the WebUI step is replaced by an empty no-op target (add_custom_target(WebUI ALL COMMENT "No WebUI defined")).

Troubleshooting

Symptom	Cause and fix
WebUI Build Error … directory does not exist	UI_SOURCE_PATH is set but resolves to nothing under the SysType folder. Either create the directory or comment out the set(UI_SOURCE_PATH ...) line.
package.json missing error	The directory exists but is empty / mis-pointed. Confirm it actually contains the Web UI project root.
npm install repeats every build	Expected on a clean checkout. Use --npminstall (passed by setting WEB_UI_GEN_FLAGS=--npminstall in your project) to skip after the first run.
Browser gets 404 for assets	The asset extension is not in the include set (.html / .js / .css / optionally .map). Either rename or extend GenWebUI.py's extensions_to_include list for that file type.
FS partition full	The Web UI is too big for the fs partition. Increase the partition size in partitions.csv or trim the bundle. See Partitions and Flash Layout.

See also

• Web Server SysMod — serves the assets at runtime, including transparent gzip handling.
• Partitions and Flash Layout — sizing the fs partition.
• SysTypes and Top-level SysType Config — where features.cmake and SysTypes.json live.
• scripts/GenWebUI.py — the script itself; small and easy to extend.