Integrate open-source 3D/spherical image dataset libraries #41
Description
Summary
Investigate integrating 3D, spherical, and potentially 4D image datasets into the web pattern generator as stimulus templates or built-in examples. May work best as a standalone companion application rather than embedded in the main tool.
Candidate Datasets
| Dataset | Type | Images/Scenes | Key Features | Format | License |
|---|---|---|---|---|---|
| SYNS (Southampton-York) | Spherical | ~100 scenes | LiDAR range + HDR imagery, panoramic stereo pairs | Co-registered HDR + LiDAR | Research |
| Laval Outdoor HDR | Spherical | 205 panoramas | Full-HDR (22 f-stops) outdoor | Equirectangular, multi-exposure | Research |
| Poly Haven | Spherical | 500+ nature HDRIs | 360° HDR panoramas, time/weather metadata | EXR, HDR | CC0 (Public Domain) |
| UPenn Natural Image Database | Planar | ~4000 scenes | Calibrated Nikon D70, baboon habitat | Luminance, RGB, LMS | Research |
| 3D60 | Spherical | Multi-modal stereo | RGB, Depth maps, Normal maps | PNG, EXR | Research |
| Leader360V | Spherical Video | 10K+ videos | 360° video with instance segmentation (198 categories) | ERP 2048x1024, 30fps | Research |
| WildScenes | LiDAR + 2D | 9,306 images + 12,148 submaps | Semantic annotations in 2D and 3D | LiDAR + images, 6-DoF | Research |
Technical Considerations
- Need to handle standard formats: equirectangular projection, HDR/EXR, LiDAR point clouds
- Conversion pipeline from spherical images to arena pixel patterns
- Resolution mapping (high-res source → low-res LED arena)
- May need downsampling and quantization (to GS2/GS16 levels)
Approach
This may work best as a standalone companion application rather than embedded in the main Pattern Editor:
- Upload spherical image → preview on 3D arena model → extract arena-resolution pattern
- Could use existing arena-geometry.js for coordinate mapping
- Consider both static frames and video/temporal sequences
Dependencies
- Should follow completion of observer perspective features (maDisplayTools Update icon generator status to 'In Development' with v0.3 #15, webDisplayTools observer issue)
- Builds on existing spherical geometry implementation (arena-geometry.js)
Priority
Future enhancement / exploratory