You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With 20 million rows, pandas is quite slow for reading in data and manipulating it. After a quick assessment of modin and vaex, vaex seems like an easy to use and fast solution. modin was a bit slow for my use case. dask is another option, but based on benchmarks posted online, it also seems like it won't lead to much speed up over raw pandas (though lazy evaluation would probably lead to less swapping).
The text was updated successfully, but these errors were encountered:
With 20 million rows, pandas is quite slow for reading in data and manipulating it. After a quick assessment of
modin
andvaex
,vaex
seems like an easy to use and fast solution.modin
was a bit slow for my use case.dask
is another option, but based on benchmarks posted online, it also seems like it won't lead to much speed up over raw pandas (though lazy evaluation would probably lead to less swapping).The text was updated successfully, but these errors were encountered: