You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
By specifying a condition on one or more columns of the stored table, data can be read using far less memory than a full read combined with a selection of rows. Related to issue #15 and issue #16: data can be read using a stream object and selection can be done on chunks of data, rather than the complete data set. Restrictions:
Condition cannot contain aggregate statements that depend on the whole set, e.g. median(ColA) / sum(ColA).
Size of result is not known in advance, so a binding of smaller result sets is required (like data.table's rbindlist). This will have an effect on performance.
The text was updated successfully, but these errors were encountered:
On the other hand, because we read in separate chunks anyway, a conditional read feature is well suited for a multi-threaded implementation, provided we can implement the conditional statements in C++.
By specifying a condition on one or more columns of the stored table, data can be read using far less memory than a full read combined with a selection of rows. Related to issue #15 and issue #16: data can be read using a stream object and selection can be done on chunks of data, rather than the complete data set. Restrictions:
median(ColA) / sum(ColA)
.data.table
'srbindlist
). This will have an effect on performance.The text was updated successfully, but these errors were encountered: