New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CB to pass to no-cache mode when csub-cache has a dangerous memory size #2780

Open
fgalan opened this Issue Dec 20, 2016 · 5 comments

Comments

Projects
None yet
2 participants
@fgalan
Copy link
Member

fgalan commented Dec 20, 2016

It has been tested that when the number of subscriptions in the csubs cache is very large (300000 csubs), they it can exhaust the system memory an the CB crashes. This issue doesn't happen when -noCache is active.

CB should implement a protection mechanism, so if the csub cache size is very large (a CLI parameter will define the limit) it passes to nocache mode until the number of csub number (and cache memory size) decreases undel the limit.

Note that 300000 subscriptions is 3 orders of magnitude greater than current uses cases. Thus, the issue is not very probable in a real case (it is rather a corner case) but a protection measure could be developed anyway.

@kzangeli kzangeli changed the title CB passes to nocache mode when csub has a dangeround memory size CB to pass to no-cache mode when csub-cache has a dangerous memory size Jan 13, 2017

@fgalan

This comment has been minimized.

Copy link
Member Author

fgalan commented Sep 25, 2018

Joining issue #1634 (now closed), which is about this same thing:

Current csubs cache stores all subscriptions, so it is a full mirror of DB subscriptions. However, when the number of csubs becomes large (e.g. 1,000,0000 subscriptions) it have a severe footprint on memory (it could be even crash CB).

This issues is about implementing a best approach, e.g. LRU policy (or other https://en.wikipedia.org/wiki/Cache_algorithms). Details have to be elaborated.

@fgalan

This comment has been minimized.

Copy link
Member Author

fgalan commented Sep 25, 2018

Documentation at section https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/user/known_limitations.md#subscription-cache-limitation should be updated to remove the reference to this issue, once completed.

@kzangeli

This comment has been minimized.

Copy link
Member

kzangeli commented Sep 25, 2018

Instead of hardcoding a 'critical' number of issues, I'd propose to implement a very simple algorithm, based on the available RAM in the host, to calculate the number of issues for when to disable the sub-cache

@fgalan

This comment has been minimized.

Copy link
Member Author

fgalan commented Sep 25, 2018

Both could make sense, depending on the use case. Ideally, something like:

-csubCacheLimit 100000

or

-csubCacheLimit adaptative

would be the best.

@kzangeli

This comment has been minimized.

Copy link
Member

kzangeli commented Sep 25, 2018

Sure.
I'd set default value to 'adaptative' (a value of 0).
If -csubCacheLimit is used on command line, it overrides the default calculated value

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment