From a full data-transport audit at b0f8780.
Today: _ingest_xy canonicalizes and runs the fused zone_maps_pair scan; then, if not is_sorted(x), line (python/xy/marks.py:801-810), area (~:887-891), and error_band (~:963-966) argsort + gather into new columns that are ingested again (second full zone scan each), while the original unsorted canonical columns stay in the ColumnStore for the figure's lifetime — referenced by nothing, but counted in memory_report.
Cost: ~2x ingest CPU and ~2x canonical memory for the common unsorted-line case.
Proposed fix: canonicalize via columns._canonicalize first and sort before the single ingest; or replace the store entries after sorting (checkpoint/rollback machinery already exists in the store).
From a full data-transport audit at b0f8780.
Today:
_ingest_xycanonicalizes and runs the fusedzone_maps_pairscan; then, ifnot is_sorted(x), line (python/xy/marks.py:801-810), area (~:887-891), and error_band (~:963-966) argsort + gather into new columns that are ingested again (second full zone scan each), while the original unsorted canonical columns stay in the ColumnStore for the figure's lifetime — referenced by nothing, but counted inmemory_report.Cost: ~2x ingest CPU and ~2x canonical memory for the common unsorted-line case.
Proposed fix: canonicalize via
columns._canonicalizefirst and sort before the single ingest; or replace the store entries after sorting (checkpoint/rollback machinery already exists in the store).