Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Lua scripts for redis key counting #312

Closed
MartinKolarik opened this issue Mar 28, 2023 · 1 comment · Fixed by #342
Closed

Use Lua scripts for redis key counting #312

MartinKolarik opened this issue Mar 28, 2023 · 1 comment · Fixed by #342
Assignees

Comments

@MartinKolarik
Copy link
Member

Right now, in updateMeasurementCount(), we iterate over all measurements using a default scan size, which is ten records. That means about 7k calls just to count the keys. We could use a larger batch size as in store cleanup() but since we don't need the keys at all, just their count, it's still a lot of unnecessary IO. We should replace this with a small lua script, similar to this that just returns the number of matching keys. The redis client has built-in support for script handling: https://github.com/redis/node-redis#lua-scripts

@alexey-yarmosh
Copy link
Member

As I see we may also benefit from using lua scripts in places with a chain of calls where next call is based on the data of the previous one:

And in places with multiple parallel redis operations:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants