Skip to content
This repository has been archived by the owner on Aug 10, 2019. It is now read-only.

Any advices of compaction & compression integrate with Anna's actor ? #55

Closed
leftjs opened this issue Jan 22, 2019 · 0 comments
Closed

Comments

@leftjs
Copy link

leftjs commented Jan 22, 2019

Hi, riselab's researchers:

I have seen your kv write function, it's just a normal file i/o.

Anna can get high throughput but can't get similar compression rate like cassandra or scylla.

if I want to implement some LSM features in an Anna's actor thread, features as follows:

  1. memtables and WAL for cache write
  2. a background thread, that can check memtable size, and flush to ssfile when memtable up to special size.
  3. a background compaction checking thread, that can trigger a compaction action when the count of ssfiles until special size.
  4. a background compaction thread, that can compact many kvs at trigger time.

Like above, I need to use multi-thread in an Anna's actor threads, coordination in multi-threads will reduce Anna's actor throughput. Do you have any ideas for doing this?

What's more, I need really column storage to save a series of kvs, then I can use some algos to compress my data, such as simple8b, zigzag, snappy and so on. How can I achieve this in Anna?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants