Fixed
- Inputs whose dense data is spread across a very large extent (e.g. a few
small dense regions scattered across a whole country) no longer OOM during
the build step. The counting grid has a fixed resolution, so over a huge
extent a single grid cell can cover kilometres of ground and collect far
more than the chunk target — the planner then emitted one enormous chunk per
such cell, and the build loaded it whole into memory. The chunk planner now
sub-splits an over-target finest-grid cell into deeper octree sub-chunks
(routed during distribute via a deeper descent), and the build step gained a
spill path that sub-divides any chunk whose actual point distribution still
exceeds the per-chunk memory budget. The build and merge steps now also bound
how many chunks they hold in memory at once — concurrency scales with the
memory budget (and cores) instead of running one chunk per core regardless of
budget, so peak build memory tracks--memory-limitfrom ~1 GB up to large
multi-core hosts. Together these guarantee bounded build memory regardless of
how points concentrate, letting large, geographically sparse datasets convert
within a small--memory-limit. - Such inputs no longer produce a wildly over-fragmented octree (millions of
tiny, nearly-empty deep nodes). When the build had to split a region deep for
memory reasons, those deep voxels became permanent output nodes even where the
data was sparse, bloating the node count ~100× and, in turn, the writer's
memory. The bottom-up build now collapses any subtree whose points fit a
single leaf into one node, so output node depth tracks point density rather
than the build's memory-driven split depth. The writer's per-node bookkeeping
was also reworked to keep a single key+count list instead of several parallel
copies, so it scales to large node counts without holding gigabytes of index. - The writer no longer OOMs when producing very large outputs. It used to
encode a whole memory-budget-sized batch of nodes before compressing, holding
the entire batch's encoded bytes (which could be ~half the budget) resident
alongside the compressor's working set. It now streams a fixed-size window of
points at a time — encode in parallel, compress in order, free, advance — so
the writer's peak memory is a small constant independent of the output size or
--memory-limit.
Full Changelog: v0.10.1...v0.11.0