Instanced geometry LOD: mesh LOD chains and billboard imposters

[brendancol]

Context

Follow-up to #74 (terrain LOD landed in #78). Roadmap Phase 2 item (see #57). Terrain tiles now render at distance-appropriate resolution via TerrainLODManager, but instanced geometry from place_mesh() still renders full-detail meshes at all distances. A cell tower 5 km away gets the same triangle count as one 50 m from camera.

Goal

Per-instance LOD selection so distant objects render with simplified meshes or billboard imposters, reducing GPU memory and BVH traversal cost for large scenes.

Current State

• place_mesh() creates IAS instance transforms all pointing to a single shared GAS (full-detail mesh)
• lod.py already has simplify_mesh() (quadric decimation via trimesh) and build_lod_chain() that produce [(verts, indices), ...] per LOD level — but nothing consumes them yet
• _MeshChunkManager in engine.py handles spatial loading/unloading of chunks but all at one resolution
• Baked meshes store (v, idx, base_z) tuples per geometry group

Design

Mesh LOD Levels

Level	Description	Use case
LOD 0	Original mesh	Close-up, full detail
LOD 1	~50% triangles (quadric decimation)	Mid-range
LOD 2	~10% triangles or convex hull	Far
LOD 3	Billboard imposter (textured quad)	Very far / below ~16px screen size

LOD Chain Generation

• On place_mesh(), build the LOD chain via build_lod_chain() and register a separate GAS per level
• GAS IDs: {geometry_id}_lod{level} (e.g. tower_lod0, tower_lod1)
• LOD 0 GAS is the current full-detail mesh — no change for close objects
• Store the chain on the geometry layer so GeometryLayerManager knows about all levels

Billboard Imposters (LOD 3)

• Pre-render each mesh from 8 azimuth × 2 elevation angles into a texture atlas (offline, via existing render() path with orthographic camera)
• At runtime, select the closest viewing angle and render an alpha-tested textured quad
• Imposters use a separate SBT hit group with texture lookup in closest-hit
• Transition: when screen-space projected bounding sphere drops below ~16px

Per-Instance LOD Selection

• Each frame (or on camera movement threshold, same pattern as TerrainLODManager._update_threshold), compute distance from camera to each instance group centroid
• Map distance → LOD level using compute_lod_level() from lod.py
• Swap IAS instance transform to reference the appropriate LOD GAS
• Batch IAS rebuild — don't rebuild per-instance, collect all changes and rebuild once
• Hysteresis band (~10% of threshold distance) to prevent LOD flickering at boundaries

Integration with Chunk Manager

• _MeshChunkManager cache key extends to (chunk_id, lod_level)
• On LOD change, swap cached mesh data; on cache miss, build from LOD chain
• Z re-snapping (see MEMORY.md) must use the correct terrain resolution for the active terrain LOD tile underneath

Implementation Plan

1. LOD chain storage — extend geometry layer metadata to hold [(verts, indices)] per mesh source
2. Multi-GAS registration — build and register one GAS per LOD level per mesh, {id}_lod{N} naming
3. Distance-based LOD assignment — per-instance distance check, LOD level lookup, IAS instance swap
4. Hysteresis — prevent rapid LOD toggling at boundary distances
5. Billboard generation — offline pre-render pass, texture atlas packing, alpha-test hit group in kernel.cu
6. Chunk manager LOD awareness — extend cache key, LOD-aware load/unload
7. Viewer integration — toggle key (Shift+?), HUD stats showing instance LOD distribution

Scope Boundaries

• Billboard imposters are a stretch goal — mesh LOD levels (steps 1–4) come first
• No continuous LOD (CLOD) — discrete levels are sufficient
• No per-frame IAS rebuild; batch on camera movement threshold
• LOD chain is generated in-memory from the source mesh, no disk caching

Key Files

• rtxpy/lod.py — simplify_mesh(), build_lod_chain(), compute_lod_level()
• rtxpy/viewer/terrain_lod.py — TerrainLODManager (pattern to follow for instance LOD)
• rtxpy/engine.py — _MeshChunkManager, _rebuild_at_resolution(), baked mesh handling
• rtxpy/accessor.py — place_mesh() API
• rtxpy/rtx.py — GAS/IAS management, add_geometry(), remove_geometry()
• cuda/kernel.cu — closest-hit programs (new hit group needed for billboard alpha test)

References

• Parent issue: Level of detail: terrain LOD, instanced LOD, and hybrid cluster GAS #74
• Roadmap: Project roadmap: architecture, rendering, simulation, platform, digital twin #57 (Phase 2 — Level of Detail)
• Terrain LOD PR: Add distance-based terrain LOD system #78