Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite _move helper in Lua (to add Redis 7 support) #284

Merged
merged 5 commits into from
Jun 16, 2023
Merged

Rewrite _move helper in Lua (to add Redis 7 support) #284

merged 5 commits into from
Jun 16, 2023

Conversation

nsaje
Copy link
Contributor

@nsaje nsaje commented Jun 15, 2023

closes #268

In order to not need the execute_pipeline script that doesn't work on Redis >= 6.2.7 (and also on earlier versions with security patches backported, like on AWS ElastiCache), I rewrote the _move() helper in Lua directly.

Added the ability to include existing scripts as local functions when loading a script from file in order to facilitate code reuse.

Any suggestions how to improve the Lua code? For example Redis doesn't know how to pass bools and nulls to Lua so I had to convert them to strings, any better options?

elseif mode == "min" then
zadd_update_min({ key }, { score, member })
elseif mode == "max" then
zadd_update_max({ key }, { score, member })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like we could just call zadd with gt?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though lt and gt are not supported by Redis 4.0, so we should think whether we can drop support here.

(nx and xx are supported though)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replying just once, applies to all 😄

two reasons I decided against it:

  • NX and XX were added in v3.0.2, LT & GT were added in 6.2.0. we're currently running TT on 3.2 so we can't use LT&GT
  • I wanted to keep the implementation as close as possible to the original, just rewritten to Lua. That's because this change is risky as is, I don't want to do optimizations and refactors in the same step as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an example for the 2nd point - the native implementation of NX, XX, GT could be subtly different from what we do in our scripts. It could slip through and work most of the time but cause bugs or corruption down the line.

Copy link
Contributor

@neob91-close neob91-close left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks correct in general, except the keys should be passed as the KEYS input.

@nsaje nsaje requested a review from neob91-close June 16, 2023 10:06
tasktiger/task.py Outdated Show resolved Hide resolved
tasktiger/redis_scripts.py Outdated Show resolved Hide resolved
tasktiger/lua/move_task.lua Show resolved Hide resolved
@nsaje nsaje requested a review from thomasst June 16, 2023 11:10
@nsaje nsaje merged commit b1019e6 into master Jun 16, 2023
40 checks passed
@nsaje nsaje deleted the redis7 branch June 16, 2023 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Redis >= 6.2.7 (Lua readonly global tables)
3 participants