Sort line inputs before canonical ingest

[Alek99]

Root cause

Line-like marks first canonicalized and committed unsorted x/y columns (including fused zone-map scans), then detected unsorted x, gathered stable-sorted copies, and committed those copies as new columns. The original canonical columns remained resident in the Figure's ColumnStore even though no trace referenced them. Area/error-band repeated the pattern across three columns.

That doubled long-lived canonical memory and repeated zone-map work for the common unsorted-input case.

Changes

• add a ColumnStore path that:
  • canonicalizes every parallel input first
  • validates all lengths before mutating the store
  • derives one stable x order
  • gathers every parallel column with that order
  • preserves each canonical kind and increments honest copy accounting
  • commits only the final sorted columns
• retain the fused zone-map scan for the first x/y pair
• reuse one gathered array when parallel inputs share the same live storage
• route line, non-scalar area, and error-band through the sorted single-ingest path
• keep scalar area baselines constant without an unnecessary sort gather
• preserve transaction rollback and mark-specific length errors
• add a CodSpeed row and document the canonical-before-commit contract

Measured impact

100k random rows, 15 warmed construction repetitions:

mark	main median	branch median	main residency	branch residency
line	7.344 ms	6.870 ms	4 columns / 3.2 MB	2 columns / 1.6 MB
area	7.145 ms	6.607 ms	6 columns / 4.8 MB	3 columns / 2.4 MB
error band	7.059 ms	6.585 ms	6 columns / 4.8 MB	3 columns / 2.4 MB

Canonical residency is cut exactly in half. A separate 1M-row reversed-line probe improved from 2.753 ms to 2.204 ms (about 20%).

Verification

• broad non-browser suite: 2164 passed, 94 skipped, 1 deselected
• relevant line/area/error-band/component/animation/Arrow suites: 222 passed
• new focused storage/sort/kind/rollback regressions: 11 passed
• independent Figure suite: 151 passed, 1 environment-only Chromium deselection
• targeted component metadata tests: 2 passed
• new CodSpeed row: passed
• stable duplicate-x order, datetime/NaT kind and wire metadata, categorical order, non-finite zone maps, identity dedup, length validation, and ingest-copy accounting are explicitly covered
• zone-map instrumentation proves only the final sorted line pair is scanned
• changed-file ty: passed
• Ruff and format checks: passed
• full pre-commit (including docs codespell): passed
• git diff --check: passed

The one known deselection is the current-main minified-bundle marker mismatch already fixed by #177. The Chromium full-path check passed separately outside the restricted sandbox.

Closes #170

[greptile-apps]

Greptile Summary

This PR sorts line-like inputs before committing their canonical columns. The main changes are:

• Adds one atomic sorted-ingest path for parallel columns.
• Routes line, non-scalar area, and error-band inputs through that path.
• Preserves stable ordering, column kinds, rollback, and copy accounting.
• Adds focused tests, documentation, and a CodSpeed benchmark.

Confidence Score: 5/5

This looks safe to merge.

No blocking issues found in the changed code.

T-Rex Logs

What T-Rex did

• The focused ingest test suite ran and all 11 focused cases passed in 0.42 seconds, including the three line-like residency parameterizations.
• A broader regression attempt was executed for the 251-test figure/component set; it finished with an environment-level exit code of 137, and there were no pytest assertion failures emitted.

_{Ran code and verified through T-Rex}

Important Files Changed

Filename	Overview
python/xy/columns.py	Adds structured length validation and stable sorting before parallel columns are committed.
python/xy/_figure.py	Adds a Figure adapter that preserves mark-specific errors and store rollback.
python/xy/marks.py	Moves line, array-baseline area, and error-band construction to the sorted-ingest path.
tests/test_figure.py	Covers ordering, kinds, non-finite values, deduplication, copy accounting, validation, and rollback.
tests/test_components.py	Checks stable sorting and category metadata for component-resolved line data.
benchmarks/test_codspeed_kernels.py	Adds an unsorted-line benchmark with canonical residency assertions.

_{Reviews (1): Last reviewed commit: "Sort line inputs before canonical ingest" | Re-trigger Greptile}

[codspeed-hq]

Merging this PR will improve performance by 13.5%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 96 untouched benchmarks
🆕 1 new benchmark
⏩ 1 skipped benchmark¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	test_first_payload_line_unsorted_x	27 ms	23.8 ms	+13.5%
🆕	test_unsorted_line_ingest_medium	N/A	21.2 ms	N/A

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing agent/canonicalize-sorted-ingest (56a6065) with main (2a7d898)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩