Multi-level Anti-caching Memory Stats

[apavlo]

Right now H-Store's memory stats infrastructure through the @Statistics MEMORYsysproc call only supports information for two tiers. It reports the amount of data (in tuples, blocks, and bytes) that is stored in main memory and out in the anti-cache storage (see MemoryStats). With our new multi-level anti-cache infrastructure from @mikelj, we want to be able to extract out memory stats for each individual level. Right now this is currently not possible because the MemoryStatsschema is fixed and it will break a lot of stuff if we try to make it have a dynamic set of fields based on the current anti-cache configuration.

What I propose we add is a new @Statistics target called ANTICACHEMEMORY (see SysProcSelector. This sysproc invocation will return the stats for all the different anti-cache levels across the entire cluster (separated per partition). This is going to be tricky because there is a bunch of stuff that we're going to have to change to make this work:

1. The first thing we need to do is add support in the EE for storing these new stats per level. Again, because we are going to have dynamic levels, we can't reuse the TableStats infrastructure. Instead, we could implement a new StatsSource called AntiCacheStats for the AntiCacheEvictionManager that keeps track of this information. Each row in the AntiCacheStats internal table would correspond to a single level at that partition for a particular table. It should include the same information as (TableStats.
2. The next step is to be able to have the EE return this information when asked. You will need to hook up the new AntiCacheStats object to the VoltDBEngine's stats manager. First you need to create a new StatisticsSelectorType(C++) and SysProcSelector (Java) enum entry. Make sure that you give the C++ one the same value as the Java one. Then in VoltDBEngine::rebuildTableCollections() you will want to register the AntiCacheStats handle for each table. Note that you have to create a special id # for the entry because we currently can't directly support multiple StatsSources per table. See computeIndexStatsId as an example of how we get around this problem for indexes. You'll want to create your own computeAntiCacheStatsId function in VoltDBEngine.h that does the same thing but have yours bitshift the tableId by 8 bits instead of 16.
3. Now the last step in the EE is to return the AntiCacheStats when the Java layer asks for it. This is easy to do because we already have support for it with TableStats and IndexStats. Just add the new support in the switch statement in VoltDBEngine::getStats() for the STATISTICS_SELECTOR_TYPE_ANTICACHEMEMORYand use computeAntiCacheStatsId to convert the given tableIds into the proper ids for the AntiCacheStats objects.
4. Now with the EE taken care of, we need to extend the Java code to be able to retrieve this new information and return it when @Statistics is invoked with ANTICACHEMEMORY. Currently the way the memory stats stuff works is that we periodically go down into the EE and collect the stats information and cache it up in the Java layer in MemoryStats. I don't think it will slow us down if we do the same thing here. So you'll want to make a new AntiCacheMemoryStats object that is based on the existing MemoryStats. Then in PartitionExecutor create a new updateAntiCacheMemoryStats() method that is similar to PartitionExecutor::updateMemoryStats() that invokes ExecutionEngine::getStats() with your new SysProcSelector.ANTICACHEMEMORY.
5. Modify PartitionExecutor::tick() so that it calls your new PartitionExecutor::updateAntiCacheMemoryStats() method right after it calls updateMemoryStats(). Be sure to add a check to see whether anti-caching is enabled and whether there is at least one table marked as evictable, since we don't want to be doing this unless the system is actually using anti-caching.
6. Modify HStoreSite::initStatSources() to register the Java layer AntiCacheMemoryStats object with the system.
7. Modify Statistics to be able to support SysProcSelector.ANTICACHEMEMORY. First add a "data" and "aggregate" entry to SysProcFragmentId (search for PF_anticacheProfiler* and just copy that example). Then add these in the static block in Statistics that calls addStatsFragments(). This will automatically setup the ability to do the scatter-and-gather for the collection of data across multiple partitions. Finally, add your new PF_anticacheMemoryData and PF_anticacheMemoryAggregator entries to the switch statement in Statistics::executePlanFragment() (again, follow the same example as PF_anticacheProfiler*).
8. Lastly, add new test cases to check whether the sysproc call @Statistics ANTICACHEMEMORY is working correctly in TestAntiCacheSuite.