Skip to content

Commit

Permalink
test eager vs lazy in CI
Browse files Browse the repository at this point in the history
  • Loading branch information
ritchie46 committed Nov 18, 2021
1 parent 37e1807 commit 0a57531
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/benchmark.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ jobs:
cd tests/db-benchmark
Rscript -e 'install.packages("data.table", repos="https://Rdatatable.github.io/data.table")'
Rscript groupby-datagen.R 1e7 1e2 5 0
echo "LAZY vs EAGER tests"
python lazy_vs_eager.py
echo "ON STRINGS"
python main.py on_strings
echo "ON CATEGORICALS"
Expand Down
9 changes: 9 additions & 0 deletions py-polars/tests/db-benchmark/lazy_vs_eager.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import polars as pl

path = "G1_1e7_1e2_5_0.csv"
predicate = pl.col("v2") < 5

shape_eager = pl.read_csv(path).filter(predicate).shape

shape_lazy = (pl.scan_csv(path).filter(predicate)).collect().shape
assert shape_lazy == shape_eager

0 comments on commit 0a57531

Please sign in to comment.