By default prevent cross slot operations in functions and scripts with # #10615
Conversation
madolson
changed the title
madolson
added
state:major-decision
|
PR for docs: https://github.com/redis/redis-doc/pull/1893/files |
Adds documentation for a new lua flag, allow-cross-slot-keys which is implemented here: redis/redis#10615 Co-authored-by: Oran Agra <oran@redislabs.com>
|
Out of curiosity, who uses cross-slot keys in scripts? If keys are in different shards, the script doesn't work anyway. Is it only in single-shard clusters (does anyone use that?) or do some users make sure that certain slots reside on the same shard? |
|
@zuiderkwast The only "legitimate" cases I am aware of are people who will do a bunch of incremental scan work in a script. Since we don't really expose a cursor local to a slot, you might need to do cross slot operations since the scan will return multiple items from the same bucket (so to be atomic you might want to process them together). The script would then return the cursor back, so the caller could send it again. Since they are never doing real cross slot commands, since each individual command will only ever operate on same slot keys, and the data is definitely in the given node, since they were returned from scan, it's pretty safe. There might be others, but since I know of at least one I didn't feel comfortable introducing a backwards breaking change. I'm sure there are plenty of other esoteric use cases where people know what they are doing (carefully constructing the slot space so data is together) so that it works. I don't think this is too common, and even if it was we would want to disallow that. I think we probably want to have some flag like "allowed-script-flags" or attach it to the ACLs to help enable better control there though. |
|
Thanks! With CSCAN (or SLOTSCAN) we can obsolete this use case. 👍 [Edit] Maybe not completely. It might still be better to scan multiple slots in one script to avoid too many round trips. |
…h # (redis#10615) Adds the `allow-cross-slot-keys` flag to Eval scripts and Functions to allow scripts to access keys from multiple slots. The default behavior is now that they are not allowed to do that (unlike before). This is a breaking change for 7.0 release candidates (to be part of 7.0.0), but not for previous redis releases since EVAL without shebang isn't doing this check. Note that the check is done on both the keys declared by the EVAL / FCALL command arguments, and also the ones used by the script when making a `redis.call`. A note about the implementation, there seems to have been some confusion about allowing access to non local keys. I thought I missed something in our wider conversation, but Redis scripts do block access to non-local keys. So the issue was just about cross slots being accessed.
Adds the
allow-cross-slot-keysflag to Eval scripts and Functions to allow scripts to access keys from multiple slots.The default behavior is now that they are not allowed to do that (unlike before).
This is a breaking change for 7.0 release candidates (to be part of 7.0.0), but not for previous redis releases since EVAL without shebang isn't doing this check.
Note that the check is done on both the keys declared by the EVAL / FCALL command arguments, and also the ones used by the script when making a
redis.call.A note about the implementation, there seems to have been some confusion about allowing access to non local keys. I thought I missed something in our wider conversation, but Redis scripts do block access to non-local keys. So the issue was just about cross slots being accessed.
to do:
Resolves, #10503.