Skip to content

DEW Report #2 ‐ 2026‐06‐26

Jump to bottom
Pierre Raybaut edited this page Jun 26, 2026 · 2 revisions

Report #2 - Date: 2026-06-26

Scope of the report

This is the second progress report on the DataLab Experimental Web Interface (DEW) project, funded by the NLnet Foundation as part of the NGI0 Commons Fund. It continues from Report #1 and covers the work delivered in the DataLab-Web repository between 2026-05-10 (the cut-off of Report #1, commit 537b60f) and the date of this report.

This report covers Task 1 exclusively, and concludes that Task 1 is now fully delivered.

The following table collects the main project links related to this report:

Item URL
Repository https://github.com/DataLab-Platform/web
Live demo (GH Pages) https://datalab-platform.com/web/
Sigima (engine) https://github.com/DataLab-Platform/Sigima
DataLab-Kernel https://github.com/DataLab-Platform/DataLab-Kernel

Executive Summary

Since Report #1, DataLab-Web matured from a prototype (v0.1.0) into a production-ready, localized, and highly responsive application spanning seven tagged releases (v0.2.0 → v0.6.2), with 218 commits merged in the period. The two open milestones from Report #1 — 1h (performance and interruptible processing) and 1i (internationalisation framework and French translation) — are fully delivered. Therefore, all nine Task 1 milestones (1a–1i) are successfully complete.

Core application additions beyond the milestones include a command palette, a quick-access toolbar, a non-modal docked ROI editor, spatial multi-image overlays, image profile extraction, over 70 scientific colormaps, a detailed processing-error traceback dialog, bundled example plugins, user-guided tours, an in-app release notes viewer, an interactive AI Assistant panel, and a TypeScript Remote SDK.

Milestone Status for Report #2

All Task 1 milestones (1a–1g) reported as complete in Report #1 remain complete. This section documents the closure of the remaining two milestones.

✅ 1h. Performance optimisation and interruptible processing

This milestone is now complete. Key architectural deliverables have been validated and integrated:

  1. Off-Thread Execution (Worker Default). To prevent heavy computations from blocking the user interface, the entire Pyodide runtime (Python kernel and hierarchical object model) has been ported to run in a Dedicated Web Worker behind a typed RuntimeApi façade (defined in src/runtime/kernelWorker.ts and src/runtime/WorkerRuntimeProxy.ts), with data sent zero-copy using transferables. Based on comprehensive regression testing, the worker execution mode has been promoted to the default (configured in src/runtime/runtimeMode.ts), with ?runtime=main kept as an instant rescue parameter.
  2. On-Disk Spill Storage (Lifting the Memory Ceiling). Because the web is capped by a ~2 GB heap in 32-bit WebAssembly (wasm32), we leveraged the Origin Private File System (OPFS) to implement an on-disk storage engine (coded in src/storage/OpfsSyncObjectStore.ts). When activated, it spills heavy arrays from memory onto the disk synchronously, keeping the in-memory working set bounded by disk quotas rather than heap ceilings.
  3. Interruptible Processing. Long-running batch executions (1-to-1 or 2-to-1 operations on multiple selected objects) run iteratively on the UI-side, allowing the user to press Cancel or Esc to safely abort the loop at the next object boundary while retaining already-produced outputs. Single un-chunkable calculations inside Pyodide represent an explicitly accepted limit: interrupting a single in-process run requires a SharedArrayBuffer (which breaks static hosting by forcing cross-origin headers), while spawning a concurrent compute worker exhausts browser memory constraints leading to out-of-memory errors on large arrays. The batch-level loop cancellation represents a secure, static-friendly, and memory-safe implementation.
  4. Busy Indicator for Single Long Processings. A single-object processing that takes longer than ~1.5 s now shows a non-cancellable "Computing…" progress dialog with an indeterminate animated bar, replacing the previous behaviour of only greying the UI. This gives clear feedback for operations such as a moving median on a large image without requiring a second runtime or SharedArrayBuffer.
  5. Optimised and Lazy Bundling. Production bundle size was reduced from ~2.05 MB to ~414 kB gzipped through deliberate code-splitting of Plotly, CodeMirror, and React, plus lazy loading of parameter editors and colormap tables.

Validation: Covered by Jest/Vitest unit tests (such as tests/ts/components/ProgressDialog.test.tsx), specialized Python benchmarks, and complete Playwright end-to-end regression suites running fully in Worker mode.

✅ 1i. Internationalisation framework and French translation

This milestone is complete (introduced in v0.4.0) and spans both development surfaces:

  • React UI surface: Every UI string is wrapped in t(), using automatic catalog extraction (npm run i18n:extract) and validation checks via CI (npm run i18n:check) to prevent empty keys.
  • Python/Pyodide surface: Core computation and standard parameter dialog outputs are translated by loading gettext .mo catalog resources from Pyodide's virtual filesystem (Sigima/guidata) matching the active UI language.

The application automatically checks browser settings, falls back to English, allows standard user selection, or supports URL override via ?lang=. French is fully implemented and tested as the reference regional language.

Quality and testing

DataLab-Web's test pyramid was run and verified fully green in Worker default mode:

  • Vitest + RTL: Executes TypeScript unit and component tests.
  • Playwright: Validates chromium end-to-end user paths.
  • pytest on Pyodide: Verifies the headless Python computation core.
  • Pre-commit check integrations (Ruff, ESLint, Prettier) ensure style compliance.

Code References for Report #2

Key documents:

Next steps after Report #2

With Task 1 fully delivered, future opportunities include:

  1. Transitioning to 64-bit WebAssembly (wasm64) once stable to natively lift the 4 GB limit.
  2. Developing the Snapshot Sync Server to allow saving/sharing immutable workspaces.
  3. Conducting a comprehensive WCAG-focused accessibility and security audit.

Overall progress summary

Status legend: ✅ done · 🟡 partial · 🚧 in progress · ⬜ not started.

Task Sub-task Status
1. Web Frontend Prototype ✅
1a. Architecture consolidation and decision record ✅
1b. Browser runtime and Python ↔ JS bridge ✅
1c. Core UI: signal and image panels, object tree, menus, dialogs, ROI editor ✅
1d. Macros and notebooks subsystem ✅
1e. Plugin system compatible with the desktop Qt plugin API ✅
1f. I/O subsystem: browser-side HDF5 access, text import wizard, file save/load ✅
1g. Static deployment for sub-path hosting (e.g. GitHub Pages) ✅
1h. Performance optimisation and interruptible processing ✅
1i. Internationalisation framework and French translation ✅

Task 1 (Web Frontend Prototype) is fully delivered. Snapshot synchronisation and WCAG/security work are outside the current MoU scope and are not covered by this report.

Clone this wiki locally