Skip to content

Conversation

@alamb
Copy link

@alamb alamb commented Nov 24, 2025

Update DataFusion results for the DataFusion 51 release (TODO add blog URL here when published)

I followed the directions in

Changes

  • Update some readme contents to remove outdated contents
  • Fix lukewarm-code tagging (the scripts in this repository run datafusion-cli from scratch each time, so there are no caches maintained from query to query) - Tag runs as lukewarm #692 (comment)
  • Add scripts to convert from csv --> json result format

Variants:

  • DataFusion parquet
  • DataFusion parquet-partitioned

Note I did not include datafusion with vortex
(TBD ping SpiralDB)

Results included

  • c6a.4xlarge
  • c6a.2xlarge
  • c6a.xlarge
  • c6a.large

Not sure

  • c8g.4xlarge
  • t3a.small

@CLAassistant
Copy link

CLAassistant commented Nov 24, 2025

CLA assistant check
All committers have signed the CLA.

@alamb alamb force-pushed the alamb/update_datafusion branch from 0c9ff10 to f9c2654 Compare November 24, 2025 15:17
@rschu1ze rschu1ze self-assigned this Nov 24, 2025
@rschu1ze
Copy link
Member

@alamb Please ping me when this PR is ready for review - thanks.

@alamb
Copy link
Author

alamb commented Nov 24, 2025

Thank you @rschu1ze -- I am still doing some performance analysis (you can see details here if you care apache/datafusion#18909). I will let you know when it is ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update ClickBench benchmarks with DataFusion 51.0.0

3 participants