-
Notifications
You must be signed in to change notification settings - Fork 203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SNAP-2366] row buffer fault-in, forced rollover, merge small batches #1046
base: master
Are you sure you want to change the base?
Commits on May 30, 2018
-
Sumedh Wale committed
May 30, 2018 Configuration menu - View commit details
-
Copy full SHA for 2972c4d - Browse repository at this point
Copy the full SHA 2972c4dView commit details -
Trilok Khairnar committed
May 30, 2018 Configuration menu - View commit details
-
Copy full SHA for ff3cca0 - Browse repository at this point
Copy the full SHA ff3cca0View commit details
Commits on May 31, 2018
-
[SNAP-2366] row buffer fault-in, forced rollover, merge small batches
- add check for the two cases in table stats service: - "large enough" row buffer (currently "large enough" is anything more than maxDeltaRows/8) that has not seen any updates/deletes since the last check; in this case schedule a task to force rollover the row buffer in column table - also check if a bucket of column table has multiple small batches (non-transactional check); if so then submit a task to merge those after checking for transactional snapshot; merge is done by locally created ColumnTableScan->ColumnInsertExec plan where the scan uses an iterator only on the small batches - added a ColumnFormatStatsIterator that can take a bunch of stats rows and create an iterator over just those (like required for batch merge) - added new scan metrics for disk reads: a) disk rows from row buffer, b) partial column batches on disk, c) full column batches on disk - extended SQLMetrics types with a new SPLIT_SUM_METRIC that allows displaying multiple metrics against a common name; ColumnTableScan now uses this to combine some metrics else it becomes too large in display (especially for the newly added disk read metrics) - use hive-metadata (ExternalTableMetaData) to get number of rows instead of getting from row buffer table (that is subject to changes in future)
Sumedh Wale committedMay 31, 2018 Configuration menu - View commit details
-
Copy full SHA for 608bf3d - Browse repository at this point
Copy the full SHA 608bf3dView commit details -
Sumedh Wale committed
May 31, 2018 Configuration menu - View commit details
-
Copy full SHA for cd21c4d - Browse repository at this point
Copy the full SHA cd21c4dView commit details
Commits on Jun 4, 2018
-
Some optimizations and fixed few issues
- fixed disk metrics collection added previously; set the metric correctly for both row buffer iterator (ResultSetTraversal) and ColumnFormatIterator - added a metric for remote batch fetch - fixed multiple ColumnTableScans causing split metrics to add up into one ColumnTableScan; now use a unique ID for split metrics for each ColumnTableScan instance - fix an NPE in SnappyTableStatsProviderService while filling up result map from members since CHM cannot hold null values - use a common entry map in ColumnFormatIterator disk iteration instead of creating separate for every column batch - added implementation of PURGE_CODEGEN_CACHES as StoreCallbacksImpl.clearCodegenCaches - limit to one task per table for background rolloverRowBuffer and mergeSmallBatches tasks - replaced a few usage of Map.put with justPut for koloboke maps
Sumedh Wale committedJun 4, 2018 Configuration menu - View commit details
-
Copy full SHA for 00f6cda - Browse repository at this point
Copy the full SHA 00f6cdaView commit details -
Sumedh Wale committed
Jun 4, 2018 Configuration menu - View commit details
-
Copy full SHA for a1dadd9 - Browse repository at this point
Copy the full SHA a1dadd9View commit details
Commits on Jun 5, 2018
-
Fixing precheckin failure in SNAP-2365 and and adding similar fix for…
… replicated table
Trilok Khairnar committedJun 5, 2018 Configuration menu - View commit details
-
Copy full SHA for cf1eb84 - Browse repository at this point
Copy the full SHA cf1eb84View commit details
Commits on Jun 6, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Conflicts: core/src/main/scala/io/snappydata/SnappyTableStatsProviderService.scala core/src/main/scala/io/snappydata/TableStatsProviderService.scala store
Sumedh Wale committedJun 6, 2018 Configuration menu - View commit details
-
Copy full SHA for 215c6e3 - Browse repository at this point
Copy the full SHA 215c6e3View commit details -
Merge remote-tracking branch 'origin/SNAP-2365' into SNAP-2366
Conflicts: cluster/src/test/scala/io/snappydata/benchmark/TPCHColumnPartitionedTable.scala
Sumedh Wale committedJun 6, 2018 Configuration menu - View commit details
-
Copy full SHA for 7f69db0 - Browse repository at this point
Copy the full SHA 7f69db0View commit details
Commits on Jun 8, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Conflicts: core/src/main/scala/org/apache/spark/sql/store/CompressionCodecId.scala
Sumedh Wale committedJun 8, 2018 Configuration menu - View commit details
-
Copy full SHA for 9595f20 - Browse repository at this point
Copy the full SHA 9595f20View commit details
Commits on Jun 28, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedJun 28, 2018 Configuration menu - View commit details
-
Copy full SHA for 88045eb - Browse repository at this point
Copy the full SHA 88045ebView commit details -
coarse container locking to fix rollover/merge when running in parall…
…el with update/delete
Sumedh Wale committedJun 28, 2018 Configuration menu - View commit details
-
Copy full SHA for c858273 - Browse repository at this point
Copy the full SHA c858273View commit details -
Sumedh Wale committed
Jun 28, 2018 Configuration menu - View commit details
-
Copy full SHA for 0ea9eb9 - Browse repository at this point
Copy the full SHA 0ea9eb9View commit details
Commits on Jul 10, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedJul 10, 2018 Configuration menu - View commit details
-
Copy full SHA for dbdbcb7 - Browse repository at this point
Copy the full SHA dbdbcb7View commit details -
Sumedh Wale committed
Jul 10, 2018 Configuration menu - View commit details
-
Copy full SHA for c824cd6 - Browse repository at this point
Copy the full SHA c824cd6View commit details
Commits on Aug 2, 2018
-
Sumedh Wale committed
Aug 2, 2018 Configuration menu - View commit details
-
Copy full SHA for bf26063 - Browse repository at this point
Copy the full SHA bf26063View commit details -
Sumedh Wale committed
Aug 2, 2018 Configuration menu - View commit details
-
Copy full SHA for f423108 - Browse repository at this point
Copy the full SHA f423108View commit details -
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedAug 2, 2018 Configuration menu - View commit details
-
Copy full SHA for 59588c0 - Browse repository at this point
Copy the full SHA 59588c0View commit details
Commits on Aug 3, 2018
-
fixing putInto which clears context prematurely
added an OperationContext in SnappySession that can be used to persist context across multiple plan executions (e.g. caching for putInto, then actual execution)
Sumedh Wale committedAug 3, 2018 Configuration menu - View commit details
-
Copy full SHA for 6ec25d0 - Browse repository at this point
Copy the full SHA 6ec25d0View commit details -
Sumedh Wale committed
Aug 3, 2018 Configuration menu - View commit details
-
Copy full SHA for 4905afc - Browse repository at this point
Copy the full SHA 4905afcView commit details -
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedAug 3, 2018 Configuration menu - View commit details
-
Copy full SHA for fa3f639 - Browse repository at this point
Copy the full SHA fa3f639View commit details -
Sumedh Wale committed
Aug 3, 2018 Configuration menu - View commit details
-
Copy full SHA for 778bb4b - Browse repository at this point
Copy the full SHA 778bb4bView commit details
Commits on Aug 6, 2018
-
Sumedh Wale committed
Aug 6, 2018 Configuration menu - View commit details
-
Copy full SHA for 647d243 - Browse repository at this point
Copy the full SHA 647d243View commit details
Commits on Aug 10, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedAug 10, 2018 Configuration menu - View commit details
-
Copy full SHA for 86f23d9 - Browse repository at this point
Copy the full SHA 86f23d9View commit details -
fix build issues after master merge
Sumedh Wale committedAug 10, 2018 Configuration menu - View commit details
-
Copy full SHA for 94c45d6 - Browse repository at this point
Copy the full SHA 94c45d6View commit details
Commits on Aug 31, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedAug 31, 2018 Configuration menu - View commit details
-
Copy full SHA for 4fb08bf - Browse repository at this point
Copy the full SHA 4fb08bfView commit details -
Sumedh Wale committed
Aug 31, 2018 Configuration menu - View commit details
-
Copy full SHA for e7960a9 - Browse repository at this point
Copy the full SHA e7960a9View commit details
Commits on Nov 2, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedNov 2, 2018 Configuration menu - View commit details
-
Copy full SHA for f2be757 - Browse repository at this point
Copy the full SHA f2be757View commit details -
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedNov 2, 2018 Configuration menu - View commit details
-
Copy full SHA for 872bd8b - Browse repository at this point
Copy the full SHA 872bd8bView commit details -
Sumedh Wale committed
Nov 2, 2018 Configuration menu - View commit details
-
Copy full SHA for ce7fac9 - Browse repository at this point
Copy the full SHA ce7fac9View commit details
Commits on Dec 22, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedDec 22, 2018 Configuration menu - View commit details
-
Copy full SHA for 08a6956 - Browse repository at this point
Copy the full SHA 08a6956View commit details
Commits on Dec 29, 2018
-
Merge remote-tracking branch 'origin/master' into SNAP-2366
Sumedh Wale committedDec 29, 2018 Configuration menu - View commit details
-
Copy full SHA for 772c4af - Browse repository at this point
Copy the full SHA 772c4afView commit details