Skip to content

Sequentially scan Parquet files, process each row group in parallel#266

Merged
brawer merged 1 commit into
mainfrom
scan-row-groups
May 26, 2026
Merged

Sequentially scan Parquet files, process each row group in parallel#266
brawer merged 1 commit into
mainfrom
scan-row-groups

Conversation

@brawer
Copy link
Copy Markdown
Collaborator

@brawer brawer commented May 26, 2026

After this change, it takes 45 seconds on a MacBook Air 2026 with Apple M5 chip to find the best-scoring candidate in OpenStreetMap for every place in AllThePlaces. This time does not include the format conversion from OSM Protocol Buffers and AllThePlaces GeoJSON files to Parquet.

After this change, it takes 45 seconds on a MacBook Air 2026 with
Apple M5 chip to find the best-scoring candidate in OpenStreetMap for
every place in AllThePlaces. This time does not include the format
conversion from OSM Protocol Buffers and AllThePlaces GeoJSON files
to Parquet.
@coveralls
Copy link
Copy Markdown

Coverage Report for CI Build 26437041694

Coverage increased (+0.3%) to 94.19%

Details

  • Coverage increased (+0.3%) from the base build.
  • Patch coverage: 6 uncovered changes across 1 file (77 of 83 lines covered, 92.77%).
  • 1 coverage regression across 1 file.

Uncovered Changes

File Changed Covered %
src/diff_places.rs 59 53 89.83%
Total (3 files) 83 77 92.77%

Coverage Regressions

1 previously-covered line in 1 file lost coverage.

File Lines Losing Coverage Coverage
src/places/place_index.rs 1 99.25%

Coverage Stats

Coverage Status
Relevant Lines: 3477
Covered Lines: 3275
Line Coverage: 94.19%
Coverage Strength: 230.26 hits per line

💛 - Coveralls

@brawer brawer merged commit e193abf into main May 26, 2026
5 checks passed
@brawer brawer deleted the scan-row-groups branch May 26, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants