feats: trace visualizer, docs

examples/camera_web_server/README.md

@@ -0,0 +1,56 @@
+# Grove Vision AI V2 Web Camera Example
+
+![Tracking Example](img/example_0.gif)
+
+Run AI models, streaming, tracking and visualizing the results using [Grove Vision AI V2](https://www.seeedstudio.com/Grove-Vision-AI-V2-Kit-p-5852.html) on a web page, an expanded rework of [ESP32 CameraWebServer](https://github.com/espressif/arduino-esp32/tree/master/libraries/ESP32/examples/Camera/CameraWebServer).
+
+## Prerequisites
+
+Software:
+
+- [Arduino IDE](https://www.arduino.cc/en/software)
+- [Seeed_Arduino_SSCMA](https://github.com/seeed-studio/Seeed_Arduino_SSCMA)
+- [ArduinoJson](https://arduinojson.org/v7/how-to/install-arduinojson/)
+- [ArduinoEigen](https://www.arduino.cc/reference/en/libraries/eigen/)
+
+Hardware:
+
+- [Grove Vision AI V2](https://www.seeedstudio.com/Grove-Vision-AI-V2-Kit-p-5852.html)
+- Espressif Arduino capble MCU board that supports Wi-Fi, e.g. [XIAO (ESP32)](https://www.seeedstudio.com/XIAO-ESP32S3-p-5627.html)
+- Router
+
+## Getting Started
+
+1. Install the required libraries in Arduino IDE.
+2. Open the example in Arduino IDE: `File` -> `Examples` -> `Seeed_Arduino_SSCMA` -> `camera_web_server`.
+3. Select the correct board and port.
+4. Edit the `ssid` and `password` in the `camera_web_server.ino` sketch to match your Wi-Fi network.
+5. Upload the sketch to the board, and open the Serial Monitor, when connected to the Wi-Fi network, the IP address of the board will be printed, connect to it using a web browser.
+
+## Features
+
+### HTTP API
+
+- `/` - Main page with a video stream from the camera
+- `/stream` - Stream from the camera (stream server runs on port 8080)
+    - `/stream/frame` - camera fream example
+    - `/stream/result` - invoke/sample raw results in JSON format
+- `/command?base64=` - Send a base64 encoded AT command to the board
+
+### URL only streaming
+
+You can use the URL `http://<IP_ADDRESS>/stream/frame` to view the camera stream on browsers or other APPS that support MJPEG streaming without any additinal process.
+
+![Streaming Example](img/example_1.gif)
+
+### AI inference results visualization
+
+The web camera page supports visualizing bounding boexes, poses, classes and keypoints, and for each trackable objects (bounding boxes), we put a unique ID on its top left corner.
+
+![Pose Example](img/example_2.gif)
+
+### Userfriendly error handling
+
+The web camera page will notify the user if there is an issue during streaming and the potential cause of the issue, e.g. the camera cable is lossen, or the device lost connection to the Wi-Fi network.
+
+![Error handling Example](img/example_3.gif)

examples/camera_web_server/web/index.html

@@ -913,6 +913,14 @@
                 syncIou(iouSlider.value);
             };
 
+            const traceLimit = 15;
+            let   traceMap = {};
+            let   traceStamp = 0;
+
+            const clearTraceMap = () => {
+                traceMap = {};
+            };
+
             const stopStream = async () => {
                 disable(saveButton);
                 disable(scoreSlider);
@@ -925,6 +933,8 @@
                 }
                 isStreaming = false;
 
+                clearTraceMap();
+
                 await fetch((() => {
                     const cmd = 'BREAK';
                     const encodedCmd = btoa(cmd);
@@ -948,6 +958,50 @@
                 enable(differedSwitch);
             };
 
+            const removeStaledTraces = () => {
+                for (const id in traceMap) {
+                    if (traceMap[id]['stamp'] != traceStamp) {
+                        delete traceMap[id];
+                    }
+                }
+                traceStamp += 1;
+            };
+
+            const plotTrace = async(id, pt_x, pt_y, color) => {
+                if (id == -1) {
+                    return;
+                }
+                try {
+                    if (traceMap.hasOwnProperty(id)) {
+                        const trace = traceMap[id];
+                        trace['pts'].push([pt_x, pt_y]);
+                        while (trace['pts'].length > traceLimit) {
+                            trace['pts'].shift();
+                        }
+                        trace['stamp'] = traceStamp;
+                    } else {
+                        traceMap[id] = {
+                            'stamp': traceStamp,
+                            'pts': [[pt_x, pt_y]]
+                        };
+                    }
+
+                    // draw trace line between every 2 points
+                    streamPreviewCtx.strokeStyle = color;
+                    streamPreviewCtx.lineWidth = 1.5;
+                    streamPreviewCtx.beginPath();
+                    for (let i = 1; i < traceMap[id]['pts'].length; i += 1) {
+                        const pt = traceMap[id]['pts'][i];
+                        const pt_prev = traceMap[id]['pts'][i - 1];
+                        streamPreviewCtx.moveTo(pt_prev[0], pt_prev[1]);
+                        streamPreviewCtx.lineTo(pt[0], pt[1]);
+                    }
+                    streamPreviewCtx.stroke();
+                } catch (e) {
+                    console.error('Failed to plot trace:', e);
+                }
+            };
+
             const visualizeBoundingBoxes = (boxes) => {
                 let classesMap = null;
                 if (modelInfo && modelInfo.hasOwnProperty('classes')) {
@@ -982,7 +1036,11 @@
                     streamPreviewCtx.font = 'bold 12px arial';
                     streamPreviewCtx.fillStyle = '#ffffff';
                     streamPreviewCtx.fillText(`${tag}: ${score}`, x - w / 2 + 5, y - h / 2 - 2);
+
+                    plotTrace(id, x, y, color);
                 }
+
+                removeStaledTraces();
             };
 
             const visualizeClasses = (classes) => {
@@ -1488,7 +1546,7 @@
             };
 
             const streamPreviewReadableStream = async (url, { signal }) => {
-                const response = await fetch(url, { signal });
+                const response = await fetch(url, { cache: 'no-cache', signal:signal });
                 const reader = response.body.getReader();
                 const decoder = new TextDecoder();
                 let buffer = '';