Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(streams): fixing xreadgroup's behavior to be compatible with Redis #1908

Merged
merged 6 commits into from Sep 24, 2023

Conversation

theyueli
Copy link
Contributor

@theyueli theyueli commented Sep 21, 2023

fixes #1860

Therefore, now:

127.0.0.1:6379> flushdb
OK
127.0.0.1:6379> "XREADGROUP" "GROUP" group consumer COUNT 10 BLOCK 5000 STREAMS stream >
(error) NOGROUP No such key 'stream' or consumer group 'group' in XREADGROUP with GROUP option

(used to block for 5 seconds..)

and

127.0.0.1:6379> flushdb
OK
127.0.0.1:6379> "XGROUP" "CREATE" stream group "0" MKSTREAM
OK
127.0.0.1:6379> keys *
1) "stream"
127.0.0.1:6379> "XREADGROUP" "GROUP" group consumer COUNT 10 BLOCK 5000 STREAMS stream >
(nil)
(5.02s)

(used to return immediately without blocking)

@theyueli theyueli added the bug Something isn't working label Sep 21, 2023
@theyueli
Copy link
Contributor Author

hi @Abhra303 @andydunstall , i saw you implemented the code that I modified... could you please help check if that makes sense? Thanks!

// return NoGroupOrKey error.
// We are simply mimicking Redis' error message here...
// However, we could actually report more precise error message...
(*cntx)->SendError(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I didn't want to send an error here because, in case of multiple keys given, we may want to receive entries from existing groups (excluding the non-existent group). Wdyt?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we do not send error here, I'm afraid that may lead to different behavior than Redis (as it returns error right away if any key or group doesn't exist)? did I miss anything?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you are right.. i just verified this behavior with Redis, will make a change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Abhra303 after trying more example with Redis, I found Redis also returns no group error as long as there is one group that can't be found for xreadgroup. I tested on Redis with the following unit test (modified from your previous test):

  resp = Run({"xreadgroup", "group", "group", "alice", "streams", "mystream", "foo", ">", ">"});
  // returns no group error as "group" was not created for mystream.
  EXPECT_THAT(resp, ArgType(RespExpr::ERROR));

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@theyueli, Abrha is mostly working on control plane stuff. Therefore, I prefer that Dragonfly's PRs will be approved by core-team members. @kostasrim , @chakaz , @adiholden are good candidates depending on day of week and holidays :)

You can still consult with Abrhadeep about the changes, of course.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@theyueli, Abrha is mostly working on control plane stuff. Therefore, I prefer that Dragonfly's PRs will be approved by core-team members. @kostasrim , @chakaz , @adiholden are good candidates depending on day of week and holidays :)

You can still consult with Abrhadeep about the changes, of course.

sounds good! just tagged the core team members as reviewers.

src/server/stream_family.cc Outdated Show resolved Hide resolved
src/server/stream_family.cc Outdated Show resolved Hide resolved
@theyueli theyueli changed the title bug (streams): fixing xreadgroup's behavior to be compatible with Redis' xreadgroup bug(streams): fixing xreadgroup's behavior to be compatible with Redis' xreadgroup Sep 21, 2023
@theyueli theyueli marked this pull request as draft September 21, 2023 05:12
@theyueli theyueli marked this pull request as ready for review September 22, 2023 07:54
@theyueli theyueli changed the title bug(streams): fixing xreadgroup's behavior to be compatible with Redis' xreadgroup bug(streams): fixing xreadgroup's behavior to be compatible with Redis Sep 22, 2023
src/server/stream_family.cc Outdated Show resolved Hide resolved
src/server/stream_family.cc Outdated Show resolved Hide resolved
src/server/stream_family.cc Outdated Show resolved Hide resolved
// However, we could actually report more precise error message...
string error_msg =
NoGroupOrKey(stream, opts->group_name, " in XREADGROUP with GROUP option");
cntx->transaction->Conclude();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making a general comment here (no need for action on your side on this PR).

I really do not like that we conclude transactions manually, it should be covered in some RAII class such that OnScopeExit it concludes itself. Maybe a good solution for this pattern is the Abseil variant called Cleanup. And yes I know we conclude before we Send and error but honestly, it won't affect the performance if we literally conclude after we report the error...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really do not like that we conclude transactions manually, it should be covered in some RAII class such that OnScopeExit it concludes itself.

I was asking @romange about doing RAII for this just a moment... I found out I need to conclude transaction so that unit test won't crash..

@Abhra303
Copy link
Contributor

LGTM

@theyueli theyueli merged commit bbcfee1 into dragonflydb:main Sep 24, 2023
7 checks passed
@theyueli theyueli self-assigned this Sep 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

XREADGROUP issues: DF consumes from non-existing stream and does not block on newly created stream
5 participants