Add high-speed binary file upload to Notehub

examples/binary-mode/upload/README.md

@@ -0,0 +1,85 @@
+# Binary Upload Example
+
+Upload a binary file from a Notecard to a remote server via Notehub, using the note-python SDK's chunked upload mechanism.
+
+This example includes two scripts:
+
+- **`binary_upload_example.py`** — Runs on the host (e.g. Raspberry Pi) connected to a Notecard. Reads `blues_logo.png`, chunks it through the Notecard's binary buffer, and sends it to Notehub via `web.post`.
+- **`receive_binary.py`** — A minimal HTTP server that receives the binary data routed from Notehub and saves it to disk.
+
+## Prerequisites
+
+- A [Blues Notecard](https://blues.com/products/notecard/) connected via USB serial
+- A [Notehub](https://notehub.io) account and project
+- Python 3.7+
+- `pyserial` and `note-python` installed (`pip install pyserial note-python`)
+- [ngrok](https://ngrok.com/) (or another tunnel) to expose the receive server publicly
+
+## Setup
+
+### 1. Start the receive server
+
+On the machine where you want to receive files:
+
+```bash
+python3 receive_binary.py
+```
+
+This starts an HTTP server on port 8080 (pass a different port as an argument if needed). Files are saved to the current directory.
+
+### 2. Expose the server with ngrok
+
+In a separate terminal:
+
+```bash
+ngrok http 8080
+```
+
+Copy the HTTPS forwarding URL (e.g. `https://abc123.ngrok.io`).
+
+### 3. Create a Notehub proxy route
+
+In [Notehub](https://notehub.io), go to your project's **Routes** and create a new **General HTTP/HTTPS** route:
+
+- **Route alias**: `upload`
+- **URL**: your ngrok HTTPS URL
+
+### 4. Configure and run the upload script
+
+Edit `binary_upload_example.py` and set:
+
+- **`PRODUCT_UID`** — your Notehub product UID (e.g. `com.your-company:your-project`)
+- **`ROUTE_ALIAS`** — the route alias from step 3 (default: `upload`)
+- **Serial port** — update the `serial.Serial(...)` path to match your Notecard's port
+
+Then run:
+
+```bash
+python3 binary_upload_example.py
+```
+
+The script will:
+
+1. Connect the Notecard to Notehub and wait for a connection
+2. Read `blues_logo.png` (~222 KB)
+3. Upload it in 64 KB chunks, printing progress after each chunk
+4. Print a summary with total bytes, duration, and throughput
+
+### 5. Check the output
+
+The receive server prints a line for each file received:
+
+```
+Listening on port 8080. Saving files to: /your/current/directory
+Press Ctrl+C to stop.
+
+[14:23:01] Received 222,511 bytes -> blues_logo.png
+```
+
+## Chunk size tuning
+
+The `MAX_CHUNK_SIZE` constant in `binary_upload_example.py` controls how large each chunk is. The Notecard's binary buffer can hold ~250 KB, but large single-chunk uploads over cellular can time out. The default of 64 KB is a good balance between throughput and reliability. Lower it to 32 KB if you experience timeouts on slow connections.
+
+## File naming
+
+Files are named using the Notecard's `label` field (sent as the `X-Notecard-Label` HTTP header by Notehub). If no label is present, the server generates a timestamped filename with an extension inferred from the file's magic bytes (`.png`, `.jpg`, `.pdf`, `.bin`, etc.).

examples/binary-mode/upload/binary_upload_example.py

@@ -0,0 +1,90 @@
+"""note-python binary upload example.
+
+This example uploads binary data to a Notehub proxy route using the
+high-speed chunked upload mechanism. The data is staged through the
+Notecard's binary buffer and sent to Notehub via web.post.
+
+Before running this example:
+1. Create a Proxy Route in your Notehub project (e.g. pointing to
+   https://httpbin.org/post or your own endpoint).
+2. Set PRODUCT_UID below to your Notehub product UID.
+3. Set ROUTE_ALIAS to the alias of your proxy route.
+
+Targets Raspberry Pi and other Linux systems.
+"""
+import os
+import sys
+import time
+
+sys.path.insert(0, os.path.abspath(
+                os.path.join(os.path.dirname(__file__), '..', '..')))
+
+import serial  # noqa: E402
+
+import notecard  # noqa: E402
+from notecard import hub  # noqa: E402
+from notecard.upload import upload  # noqa: E402
+
+
+PRODUCT_UID = 'com.your-company:your-product'
+ROUTE_ALIAS = 'upload'
+# Keep chunks small enough to reliably transfer over cellular.
+# The Notecard buffer can hold ~250KB, but pushing that much data
+# in a single web.post over cellular often times out.
+MAX_CHUNK_SIZE = 65536  # 64 KB
+
+
+def on_progress(info):
+    """Print upload progress after each chunk."""
+    print(f'  Chunk {info["chunk"]}/{info["total_chunks"]} '
+          f'- {info["percent_complete"]:.1f}% '
+          f'- {info["avg_bytes_per_sec"]:.0f} B/s '
+          f'- ETA {info["eta_secs"]:.1f}s')
+
+
+def run_example():
+    """Connect to Notecard and upload binary data to Notehub."""
+    port = serial.Serial('/dev/ttyUSB0', 115200)
+    card = notecard.OpenSerial(port, debug=True)
+
+    # Connect the Notecard to Notehub.
+    hub.set(card, product=PRODUCT_UID, mode='continuous')
+
+    # Wait for the Notecard to connect to Notehub.
+    print('Waiting for Notehub connection...')
+    while True:
+        rsp = hub.status(card)
+        connected = rsp.get('connected', False)
+        status_msg = rsp.get('status', '')
+        if connected:
+            print('Connected to Notehub.')
+            break
+        print(f'  Not yet connected: {status_msg}')
+        time.sleep(2)
+
+    # Read the image file to upload.
+    image_path = os.path.join(os.path.dirname(__file__), 'blues_logo.png')
+    with open(image_path, 'rb') as f:
+        data = f.read()
+
+    print(f'Uploading {image_path} ({len(data)} bytes) '
+          f'to route "{ROUTE_ALIAS}"...')
+
+    result = upload(
+        card,
+        data,
+        route=ROUTE_ALIAS,
+        label='blues_logo.png',
+        content_type='image/png',
+        max_chunk_size=MAX_CHUNK_SIZE,
+        progress_cb=on_progress,
+    )
+
+    print(f'Upload complete: {result["bytes_uploaded"]} bytes '
+          f'in {result["chunks"]} chunks, '
+          f'{result["duration_secs"]:.1f}s '
+          f'({result["bytes_per_sec"]:.0f} B/s)')
+
+
+if __name__ == '__main__':
+    run_example()

examples/binary-mode/upload/receive_binary.py

@@ -0,0 +1,144 @@
+#!/usr/bin/env python3
+"""Minimal HTTP server that receives binary files routed from Notehub.
+
+Receives binary files via a General HTTP/HTTPS route and saves them to
+the current directory.
+
+Usage:
+    python3 receive_binary.py [port]
+
+    Default port: 8080
+
+Setup:
+    1. Run this script (optionally with ngrok to expose it publicly):
+           python3 receive_binary.py
+           ngrok http 8080
+
+    2. In Notehub, create a General HTTP/HTTPS route pointing to this
+       server's URL (or your ngrok URL).
+
+    3. Send a binary file from your Notecard:
+           {"req": "web.post", "route": "<your-route>", "binary": true, ...}
+
+    Files are saved to the current directory with a name derived from the
+    Notecard's "label" field, or falling back to a timestamped filename
+    with an extension inferred from the file's magic bytes.
+"""
+
+import os
+import sys
+import time
+from http.server import BaseHTTPRequestHandler, HTTPServer
+
+# Magic bytes used to infer file extensions
+MAGIC_SIGNATURES = [
+    (b"\x89PNG", "png"),
+    (b"\xff\xd8\xff", "jpg"),
+    (b"%PDF", "pdf"),
+    (b"GIF8", "gif"),
+    (b"PK\x03\x04", "zip"),
+    (b"\x1f\x8b", "gz"),
+]
+
+DEFAULT_PORT = 8080
+
+
+def decode_chunked(data: bytes) -> bytes:
+    """Decode HTTP chunked transfer encoding."""
+    output = b""
+    pos = 0
+    while pos < len(data):
+        end = data.find(b"\r\n", pos)
+        if end == -1:
+            break
+        size_str = data[pos:end].decode(errors="ignore").strip()
+        if not size_str:
+            break
+        try:
+            size = int(size_str, 16)
+        except ValueError:
+            break
+        if size == 0:
+            break
+        pos = end + 2
+        output += data[pos:pos + size]
+        pos += size + 2
+    return output
+
+
+def infer_extension(data: bytes) -> str:
+    """Infer file extension from magic bytes."""
+    for magic, ext in MAGIC_SIGNATURES:
+        if data[: len(magic)] == magic:
+            return ext
+    return "bin"
+
+
+def make_filename(label: str, data: bytes) -> str:
+    """Return the label if provided, or a timestamped filename."""
+    if label:
+        return label
+    ext = infer_extension(data)
+    timestamp = time.strftime("%Y%m%d_%H%M%S")
+    return f"received_{timestamp}.{ext}"
+
+
+class BinaryReceiveHandler(BaseHTTPRequestHandler):
+    """Handle incoming binary POST requests from Notehub."""
+
+    def do_POST(self):
+        """Receive a binary file and save it to disk."""
+        # Read body, handling both Content-Length and chunked encoding
+        content_length = self.headers.get("Content-Length")
+        transfer_encoding = self.headers.get("Transfer-Encoding", "")
+
+        if content_length:
+            raw = self.rfile.read(int(content_length))
+        else:
+            raw = self.rfile.read()
+
+        if "chunked" in transfer_encoding.lower():
+            body = decode_chunked(raw)
+        else:
+            body = raw
+
+        if not body:
+            self._respond(400, "Empty body")
+            return
+
+        # Notehub sets X-Notecard-Label to the note's label field
+        label = self.headers.get("X-Notecard-Label", "").strip()
+        filename = make_filename(label, body)
+
+        filepath = os.path.join(os.getcwd(), filename)
+        with open(filepath, "wb") as f:
+            f.write(body)
+
+        print(f"[{time.strftime('%H:%M:%S')}] Received {len(body):,} bytes -> {filename}")
+        self._respond(200, "OK")
+
+    def _respond(self, code: int, message: str):
+        self.send_response(code)
+        self.send_header("Content-Type", "text/plain")
+        self.end_headers()
+        self.wfile.write(message.encode())
+
+    def log_message(self, format, *args):
+        """Suppress default request logging."""
+        pass
+
+
+def main():
+    """Start the binary receive server."""
+    port = int(sys.argv[1]) if len(sys.argv) > 1 else DEFAULT_PORT
+    server = HTTPServer(("", port), BinaryReceiveHandler)
+    print(f"Listening on port {port}. Saving files to: {os.getcwd()}")
+    print("Press Ctrl+C to stop.\n")
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        print("\nStopped.")
+
+
+if __name__ == "__main__":
+    main()