Skip to content

v3.0.0-rc.1

Pre-release
Pre-release

Choose a tag to compare

@erinaldidb erinaldidb released this 05 Mar 21:43
· 321 commits to main since this release
475d2ff

Pixels 3.0.0 Release Candidate 1

Highlights

Pixels 3.0.0 is a major release that introduces DICOMweb Gateway, Lakebase integration, Variant Data Type support, a full DICOM Redactor, Genie Space automation, and significant performance and security improvements across the board.


What's New

DICOMweb Gateway (Production-Ready)

  • Full DICOMweb compliant architecture with split dicom_web (viewer) + dicom_web_gateway (QIDO/WADO/STOW) Databricks Apps
  • QIDO-RS: Query studies, series, and instances with full DICOM UID hierarchy support
  • WADO-RS: Retrieve DICOM instances and frames with progressive frame streaming, multiframe support, and gzip compression
  • STOW-RS: Store DICOM files with streaming upload, background processing jobs, and concurrency control
  • HTTP/2 support via Hypercorn for improved throughput
  • Built-in benchmark dashboard and metrics dashboard for monitoring gateway performance
  • BOT (Basic Offset Table) cache with priority-based prefetching algorithm for fast frame retrieval
  • LRU prefetcher cache with configurable cache sizes (up to 4GB)
  • Cloud-direct upload path for optimized STOW performance
  • Token passthrough, credential caching, and refreshable token pool for robust auth

Lakebase Integration

  • Lakebase Autoscale support for tier-2 file path caching and frame indexing
  • Unity Catalog-aligned database and schema mapping with Reverse ETL Sync compatibility
  • Row-Level Security (RLS) sync with Unity Catalog groups and rules
  • Configurable connection pool sizes with refreshable token pools
  • SQL injection prevention using sql.Identifier parameterization
  • Dedicated SQL scripts for schema creation, DICOM frames tables, metrics, and RLS policies

Variant Data Type Support

  • Full compatibility with Databricks Variant Data Type for DICOM metadata storage
  • Variant migration script for upgrading existing catalogs
  • Updated OHIF viewer and MONAILabel clients for variant-compatible data handling
  • NaN record handling and binary image cleanup for JSON conversion

DICOM Redactor

  • New Redactor class for PHI redaction in DICOM metadata and pixel data
  • LLM-powered metadata redaction with configurable prompts
  • Bbox-based pixel redaction without OpenCV dependency (uses plain arrays)
  • Support for single and multi-layer JPEG2000 lossy/lossless compression
  • Redaction job API with notebook-driven task execution
  • Comprehensive unit tests for the redactor pipeline

Genie Space Automation

  • Programmatic Genie Space creation via API
  • Vector Search integration with DICOM tags index
  • Automated setup notebook for Genie Spaces

OHIF Viewer Upgrade

  • Upgraded to OHIF v3.11.1 (from 3.9.x) — major viewer upgrade
  • Added microscopy viewer support
  • Better multiframe DICOM handling with HTJ2K encoding
  • OOM fix for HTJ2K DICOMs in browser
  • OIDC compatibility for DICOMweb integration
  • MONAI Label button restored in viewer
  • Volume Rendering preset images added

MONAILabel & Model Serving

  • Upgraded to MONAI 1.5.2 and MONAILabel latest
  • Upgraded to PYDICOM 3.0.1
  • Migrated models to generate .dcm segmentation files (DICOM SEG)
  • Fix on writing compressed DCM SEG files
  • GPU loading and request handling configuration
  • Custom NibabelWriter for improved segmentation output
  • DBR version upgraded to 17.3.x-scala2.13

Databricks Asset Bundles (DABs)

  • Full DABs support for demo deployment (dabs/dbdemos/)
  • Split resources into independent YAML files (dashboard, ingest, viewer app, Unity Catalog)
  • Notebook-driven DAB installation (99-run-install.ipynb)

VLM Integration

  • Vision-Language Model integration for DICOM image analysis
  • Configurable prompts with temperature and max_tokens settings
  • Prompt handler class for structured VLM interactions

Performance Improvements

  • Async WADO-RS with dedicated I/O pool and increased parallelism
  • Streaming responses for metadata and frame retrieval
  • SQL connection pooling with batched inserts
  • Pre-cached instance paths on startup
  • One-shot indexer for uncompressed multiframe DICOMs
  • fsspec-based remote zip extraction for huge files
  • Configurable sequential/concurrent upload modes
  • Memory limits for queue uploads

Security Fixes

  • MODERATE-02: Internal error details no longer leaked in HTTP responses
  • MODERATE-04: Gateway request counters made thread-safe
  • MODERATE-05: assert replaced with proper validation in production code
  • MODERATE-07: Potential PHI removed from debug logs
  • MODERATE-12: Credential stampede guard logic bug fixed
  • LOW-01: Inconsistent databricks-sdk version pinning resolved
  • LOW-03: Token prefix cache key collision risk mitigated
  • LOW-05: File handle leaks in middleware fixed
  • LOW-07: Unmounted dicomweb_perf_compare function fixed
  • LOW-08: Duplicate GZipMiddleware import removed
  • SQL injection prevention in Lakebase queries
  • Volume existence check logic corrected

Infrastructure & CI

  • New CI workflows: safe PR checks, manual lint/test runner, manual AWS integration tests
  • Separated requirements.txt and requirements-ai.txt for ML/non-ML use cases
  • Updated Makefile with venv support
  • Notebooks migrated to .ipynb format (01-dcm-demo, workflows)
  • Added DatabricksFile class for Volume file validation and extraction
  • pylibjpeg-libjpeg made optional (instructions in README for codec setup)

Breaking Changes

  • OHIF v3.11.1 includes breaking changes from v3.9.x — custom extensions may need updates
  • Lakebase now uses PIXELS schema instead of public
  • Deep parameter defaults to False in catalog operations
  • init_tables removed from Catalog.init() — use fixtures instead
  • Workflow scripts migrated from .py to .ipynb

Dependencies

  • databricks-sdk updated and version-pinned consistently
  • psycopg2 upgrade for Lakebase connectivity
  • python-multipart added for STOW-RS support
  • pydicom 3.0.1
  • MONAI 1.5.2
  • Removed cv2 as hard dependency (optional for frame handling)

Full Changelog: 2.2.0...3.0.0-rc.1