Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stream buffer flush threshold #43

Closed
derekjn opened this issue Jan 2, 2014 · 1 comment
Closed

Add stream buffer flush threshold #43

derekjn opened this issue Jan 2, 2014 · 1 comment
Assignees
Labels

Comments

@derekjn
Copy link
Contributor

derekjn commented Jan 2, 2014

Let's go with a global (per-node) stream buffer size for now. Stream buffers should not be blindly flushed to disk by the background writer process. Instead, they should be flushed when evicted via standard LRU invalidation (when no other buffers are available to LRU flush).

Loop through all table buffers when evicting before looking at any stream buffers. When determining which buffer to flush to make room for a new one, stream buffers should look at evicting other stream buffers, and table buffers should look at other table buffers. If none can be found for a given buffer type, look at the other buffer type's buffers.

@derekjn
Copy link
Contributor Author

derekjn commented Jan 9, 2014

Stream buffers should only be flushed under two conditions:

  1. Stream buffer pool size exceeds its limit. In this case, we should flush a bunch of them to disk.
  2. Process termination.

@derekjn derekjn closed this as completed Jan 10, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant