Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make MemberMap.SINGLETON_MEMBER_LIST_VERSION value configurable #12528

Closed
lukasherman opened this issue Mar 7, 2018 · 6 comments
Closed

Make MemberMap.SINGLETON_MEMBER_LIST_VERSION value configurable #12528

lukasherman opened this issue Mar 7, 2018 · 6 comments

Comments

@lukasherman
Copy link
Contributor

@lukasherman lukasherman commented Mar 7, 2018

FlakeIdGenerator relies on nodeId, which is currently automatically assigned. It would be helpful to set up a different base member ID for certain use cases, so that flake IDs generated are always unique, even when generating nodes are not participating in the same cluster.

@metanet metanet self-assigned this Mar 7, 2018
@metanet metanet added this to the 3.10 milestone Mar 7, 2018
@viliam-durina
Copy link
Contributor

@viliam-durina viliam-durina commented Mar 8, 2018

I suppose you are referring to the use case mentioned here:

The scenario is A/B type deployment, where old application cluster version A is still running and new separate application cluster version B goes from standby to running mode. For a limited period of time there is possibility for A and B clusters to generate duplicate IDs.

There's one solution available out of the box right now: FlakeIdGeneratorConfig.setIdOffset(). If you can cap the time during which A & B will both generate IDs, you configure the ID offset for cluster B as follows:

idOffset = DAYS.toMillis(parallelRunDays) << 22;

22 is the number of bits reserved for nodeId and sequence in the ID. If you underestimate the time, you can't change it later.

@lukasherman
Copy link
Contributor Author

@lukasherman lukasherman commented Mar 8, 2018

It would effectively modify the timestamp part of the generated ID, right? We are using timestamp part to identify datastore partition where data should be persisted, at least on yy-mm-dd level. This is great FlakeId feature.
FlakeIdGeneratorConfig seems like a great place to put nodeIdOffset.

@viliam-durina
Copy link
Contributor

@viliam-durina viliam-durina commented Mar 9, 2018

Yes, it modifies the timestamp part of the ID.

@pveentjer
Copy link
Member

@pveentjer pveentjer commented Mar 28, 2018

Does this need to be resolved in the 3.10 time window? It seems that it an improvement of an existing feature. So unless we need to break API's, I'm suggesting to move this task to 3.11 (or 3.10 maintenance if it is low risk and high value).

@metanet
Copy link
Contributor

@metanet metanet commented Mar 28, 2018

We handled this request with a small addition into the flake id configuration without breaking any existing API.
See the pr for more details: #12706

@lukasherman please see the FlakeIdGeneratorConfig for the new node id offset field.

Regards,

@metanet metanet closed this Mar 28, 2018
@lukasherman
Copy link
Contributor Author

@lukasherman lukasherman commented Mar 28, 2018

Thanks, it looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.