Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PERF] [Delta Lake] Add IO multithreading arg to daft.read_delta_lake(). #2029

Merged
merged 1 commit into from
Mar 21, 2024

Conversation

clarkzinzow
Copy link
Contributor

Add expert IO multithreading arg to daft.read_delta_lake(), allowing the user to explicitly enable or disable IO multithreading.

@clarkzinzow clarkzinzow changed the title [Delta Lake] Add IO multithreading arg to daft.read_delta_lake(). [PERF] [Delta Lake] Add IO multithreading arg to daft.read_delta_lake(). Mar 21, 2024
Copy link

codecov bot commented Mar 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.69%. Comparing base (c2db062) to head (57ad6c7).
Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2029      +/-   ##
==========================================
- Coverage   84.70%   84.69%   -0.01%     
==========================================
  Files          62       62              
  Lines        6768     6771       +3     
==========================================
+ Hits         5733     5735       +2     
- Misses       1035     1036       +1     
Files Coverage Δ
daft/io/_delta_lake.py 89.36% <100.00%> (ø)

... and 2 files with indirect coverage changes

@clarkzinzow clarkzinzow merged commit 79d6510 into main Mar 21, 2024
34 of 35 checks passed
@clarkzinzow clarkzinzow deleted the clark/delta-lake-multithreaded-io branch March 21, 2024 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant