0XBAD - broadcasty distributed shared memory key/value cache
License
jackdoe/0XBAD
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
* 0XBAD broadcasty distributed shared memory key/value cache the whole thing is horribly simple, using FNV32 hashing to build a table (without collision detection - so if 2 keys collide they are overwritten every time) [box A] - my $v = $cache->find($key) || $cache->store_and_broadcast($key,slow_subroutine(),0,12345); [box B] - daemon receives the broadcast message; adds it into the local cache [box C] - daemon receives the broadcast message; adds it into the local cache [box D] - udp is lost in this case, when a request for $key hits [box D] it will do the same $cache->store_and_broadcast() and rebroadcast the item. * install ruby: gem install xbad perl: cd perl && perl Makefile.PL && make && make test && make install * usage my $CACHE = BAD->new(0xbeef,16,128) creates new shared memory segment with id 0xBEEF the segment size is 2^16 * 128 bytes, and it can hold at most 2^16 items my $value = $CACHE->find($key) - looks up my $value = $CACHE->store($key,$x_value,$expire_after); my $value = $CACHE->store_and_broadcast($key,$x_value,$expire_after,$broadcast_port); if $expire_after is 0, the item never expires ruby: b = XBAD.new(0xBAD,16,128) b.store(key,value,expire_after) value = b.find(key) * broadcasting store_and_broadcast sends udp broadcast to the specified port * fixed size every item in the cache is with size sp->item_size (specified on pool creation) and the shared memory segment is created when you specify exactly how many items between 2^16 and 2^32 you want to have in the pool. my $b = BAD->new(0xBEEF,17,128); # will create 2^17 items with size 128 bytes this is very ineficient, but for caching small-ish items it works fine. * speed perl: (key: aaaaaaaaaaaaaaaabbbbbbbbbbbbbbb value 'a' x 100) native hash store: 4644160.73/s 0xBAD store: 3062807.05/s memcached store: 24514.01/s native hash get: 3476691.62/s 0xBAD get: 2153601.64/s memcached get: 25216.77/s * native hash - perl's $hash{$key} = $value and $value = $hash{$key} ruby: hash set 3173242.4 i/s - 15841482 in 4.993171s xbad set 3760115.4 i/s - 18829228 in 5.008079s hash get 4008048.8 i/s - 20056464 in 5.004359s xbad get 1844151.9 i/s - 8363250 in 5.010949s hash mis 4262632.4 i/s - 21317955 in 5.001340s xbad mis 4441230.5 i/s - 22199103 in 4.999691s memcache set 13164.1 i/s - 66689 in 5.070756s memcache get 14114.6 i/s - 70788 in 5.026147s memcache cset 12767.4 i/s - 64501 in 5.076763s memcache cget 13474.4 i/s - 68202 in 5.082600s hash clone set 437421.6 i/s - 2196828 in 5.031107s hash clone get 446848.6 i/s - 2230452 in 4.996158s xbad set-encoded 794074.4 i/s - 3953088 in 5.006689s xbad get-encoded 372012.9 i/s - 1855488 in 5.035192s xbad set-encoded-sn 558399.0 i/s - 2798724 in 5.023483s xbad get-encoded-sn 372813.0 i/s - 1858272 in 5.028326s * hash- is ruby's native hash hash[key] = value and value = hash[key] * hash clone is hash[key] = value.clone and value = hash[key].clone xbad - encoded is Sereal (https://github.com/Sereal/Sereal) encoded xbad encoded-sn is Sereal encoded and snappy compressed * locking everything is protected by simple gcc atomic builtins worst case is somebody locks an item and goes away (interrupted by signal or something)
About
0XBAD - broadcasty distributed shared memory key/value cache
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published