Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle Reader Group compatibility between Pravega versions < 0.9 and 0.9+ #5693

Closed
pbelgundi opened this issue Feb 9, 2021 · 0 comments · Fixed by #5720
Closed

Handle Reader Group compatibility between Pravega versions < 0.9 and 0.9+ #5693

pbelgundi opened this issue Feb 9, 2021 · 0 comments · Fixed by #5720
Assignees
Labels
area/client kind/bug Correctness issue

Comments

@pbelgundi
Copy link
Contributor

pbelgundi commented Feb 9, 2021

Problem description
Reader Groups created with older client library (< 0.9.x) do not push metadata(ReaderGroupConfig) to controller.
These also do not have the additional fields of ReaderGroupId, generation and updateConfig flag which were added to ReaderGroupConfig in version 0.9.
During a rolling upgrade, if some readers are on older client version and some on the new (0.9+) this can lead to a problem where values of the new fields are incorrectly overwritten by the readers using older client.
So there are 2 problems to be solved here:

  1. During the upgrade of Pravega Client from a version < 0.9 to version 0.9+, Reader Groups should be able to operate normally(do reads/writes) without any failures or DUDL.
  2. Post upgrade (once all readers are on new Client Library) users should be able to start using the CBR feature
    in Pravega by changing Reader Group Configuration as appropriate.

Problem location
Client

Suggestions for an improvement
2 ways to solve the combability problem:

  1. Pravega Client should have the old & new methods for RG Create/Update/Delete APIs and as long as the values of new fields in RGConfig are at default, it should continue using the old APIs. Once all clients have been updated to the new version, users should invoke a "migrateRG" API to move to the new code base by invoking controller.createReaderGroup with ReaderGroupConfig for this existing Reader Group. From this point Client code will start using the new APIs for create/update/delete Reader Group.
    Downside of this approach is that we continue using the old APIs till user explicitly invokes migrate()

  2. Every client that has upgraded can start using the new API for RG Create/Update/Delete that makes controller calls on each of these operations to update the ReaderGroup metadata on Controller.
    While the upgrade is in progress ( few clients are still on the older version) if a resetReaderGroup is done by an older client, it will only update the State Synchronizer and will make State Synchronizer inconsistent with the RG Metadata on Controller.
    If we can guarantee that during the upgrade( as long as even 1 client is on the older version) any reset/delete calls are not made on the ReaderGroup, this approach should work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/client kind/bug Correctness issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant