Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit max number of elements in a queue #4

Closed
iefserge opened this issue Nov 18, 2014 · 4 comments
Closed

Limit max number of elements in a queue #4

iefserge opened this issue Nov 18, 2014 · 4 comments

Comments

@iefserge
Copy link
Contributor

Thank you for this great library!

I was wondering if it's possible to limit a maximum number of elements in a queue (possibly per producer, at compile time)? In some cases it's necessary to prevent unlimited queue growth. I know I can override memory allocation callbacks but maybe there is a simpler way.

@cameron314
Copy link
Owner

Hmm, interesting. There's no nice way of doing that right now (sorry!). I'll have to think about it.

In the meantime, here's a (hackish) workaround:
First, use only explicit producers (because they never relinquish their blocks).
Then, before unleashing your threads, enqueue and dequeue exactly the per-producer maximum number of items (use a multiple of the block size) from each producer. This will pre-fill the producers' block lists internally. Then, if you use only try_enqueue with those pre-allocated producer tokens, the operation will fail if there's not enough room in the producer, achieving the per-producer maximum size guarantee, albeit with significant effort and limitations.

(You can set the initial size of the queue (constructor argument) to the sum total of the producer sizes to reduce dynamic allocation in this workaround.)

@iefserge
Copy link
Contributor Author

@cameron314 Thanks, I'll try that.

@cameron314 cameron314 self-assigned this Nov 21, 2014
cameron314 added a commit that referenced this issue Dec 3, 2014
…arenthese around arithmetic for it (completes issue #4)
@cameron314
Copy link
Owner

I've added a MAX_SUBQUEUE_SIZE trait. It rounds up to the next block size boundary, and is checked at the block level (so as not to introduce any extra overhead in the fast path); if an enqueue operation would cause a block to be allocated that could cause the size of that producer's sub-queue to exceed the rounded maximum, the enqueue operation will gracefully fail (this includes bulk enqueue operations).

Cheers!

@iefserge
Copy link
Contributor Author

iefserge commented Dec 4, 2014

Thank you, very cool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants