-
Notifications
You must be signed in to change notification settings - Fork 4
CAPI RowCache Wiki
This wiki provides detailed information about CAPI-RowCache.
CAPI-RowCache supports multiple CAPI-Flash cards, so that you can distribute requests across the multiple cards and can get better aggregated throughput. Each CAPI-Flash card exposes multiple (usually, two) device files, e.g. /dev/sg0 and /dev/sg1. Depending on the model, either each of the device files corresponds to a different flash address space, or all of the device files point at the same flash address space. You must first find out how your CAPI-Flash cards are configured and what the size of each flash address space (a.k.a. LUN) is. It is beyond the scope of this wiki page, and please consult the manual or the system integrator.
No matter how many CAPI-Flash cards you have or how you want to use the multiple device files, CAPI-RowCache allows flexible configurations on how to use them. Here is the format of the com.ibm.capiflash.cassandra.cache.devices property:
-Dcom.ibm.capiflash.cassandra.cache.devices=<DEVICE>:<OFFSET>:<SIZE>[:<DEVICE>:...][,<DEVICE>:<OFFSET>:<SIZE>[:<DEVICE>:...],...]
<DEVICE> is a device file name for CAPI-Flash, such as /dev/sg0. <OFFSET> is the start address (in 4-KB blocks) of CAPI-RowCache in the device file. <SIZE> is the size in Giga bytes to be used for CAPI-RowCache in the device file. Make sure that <OFFSET> plus <SIZE> (more precisely, <OFFSET> * 4 * 1024 + <SIZE> * 1024 * 1024 * 1024) does not exceed the size of the device file. The device files concatenated by : are supposed to share the same flash address space. CAPI-RowCache will distribute requests among the device files in a round-robin manner for better throughput. The device files separated by , are supposed to point at different flash address spaces. CAPI-RowCache will split its cache across the device files, based on their specified sizes, and will distribute requests according to a hash function.
-Dcom.ibm.capiflash.cassandra.cache.devices=/dev/sg0:0:512
This uses the first 512 GB in /dev/sg0.
-Dcom.ibm.capiflash.cassandra.cache.devices=/dev/sg0:0:400,/dev/sg1:8388608:300
This uses the first 400 GB in /dev/sg0 and 300 GB starting from 8388608 (= 32 GB) in /dev/sg1. This assumes that /dev/sg0 and /dev/sg1 point at two different flash address spaces.
-Dcom.ibm.capiflash.cassandra.cache.devices=/dev/sg6:0:750:/dev/sg7
This uses the first 750 GB in /dev/sg6 and /dev/sg7 that are configured to share the same flash address space.
-Dcom.ibm.capiflash.cassandra.cache.devices=/dev/sg0:33554432:350:/dev/sg1,/dev/sg6:0:350:/dev/sg7
This uses 350 GB starting from 33554432 (= 128 GB) in /dev/sg0 and /dev/sg1 and the first 350 GB in /dev/sg6 and /dev/sg7.
For better load balancing, you should use the same sizes in the device files that do not share a flash address space, unless other restrictions prevent it. In the previous examples, the fourth example evenly splits 700 GB between the two flash address spaces (350 GB + 350 GB), which is good, while the second example does not (400 GB + 300 GB), which is bad. The starting offsets do not matter for performance.
CAPI-RowCache currently does not cache rows whose size is 4 KB or larger each.