-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Thin arbiter quorum for 2-way replication #352
Comments
For me it is absolutely not clear how this can deprecate 2 way replications- we use just 2 machines in our cluster with it. |
@pranithk What are the implication when upgrading a cluster, with this approach? Today, the upgrade process, is to upgrade one of the replica bricks and then wait for healing to catch up before upgrading the other brick as in here. So with the thin arbiter, does this change at all? Or, how does the upgrade procedure change (including upgrading the arbiter in future releases)? |
@slesru sorry for the delay in response, was on vacation for a while.
|
@ShyamsundarR upgrade procedure is same. At least I am not able to see any conflicting cases to prevent it at the moment. |
Some discussions I had with team offline, which I am recording here for wider audience. Challenges of management layer latency issuesOne of the main goal of thin arbiter is to have 3rd node in remote/cloud setup which can have higher latency. But our management layer would become very slow with this, if we want to manage the thin-arbiter through
Now, the A clear document with all tested steps to revert a bad brick to good brick in the thin arbiter.This is very important in this design as there are possibilities where just because of one (or few) file pending heal, a whole brick, which may contain millions of files may be marked as bad. In that case, we need a mechanism to override the code-path by resetting the relevant flags. Feedback welcome @pranithk @itisravi @karthik-us @aspandey |
A patch https://review.gluster.org/19545 has been posted that references this issue. |
A patch https://review.gluster.org/19835 has been posted that references this issue. |
I found a race bug in the earlier design where split-brains can happen. I just had a discussion with @aspandey @itisravi @karthik-us about the problem and solution. Ashish will be updating the design doc with the relevant changes. This is going to be fixed as a separate bug after the initial implementation. @aspandey Please add a comment once you update the design document. |
I have updated the design document as per the discussion on issue raised because of race between shd up call and transaction. |
@amarts @ShyamsundarR There was no consensus in the document which brought doc/spec approved flags about the format of the document last I checked. What do you guys think we should do to get the spec/doc approved flags considering that the google-doc already has the documentation of the work that is done? Should we send one more patch to glusterfs-specs? |
A patch https://review.gluster.org/19940 has been posted that references this issue. |
Flags provided after revisiting the document and the CLI format discussions in glusterd2 issues (referenced above). |
Updates #352 Change-Id: I3d8caa6479dc8e48bec62a09b056971bb061f0cf Signed-off-by: Ashish Pandey <aspandey@redhat.com>
1. Create thin arbiter index file during mount. 2. Set pending marker in thin arbiter id file in case of failure. Change-Id: I269eb8d069f0323f1fc616175e5e5eb7b91d5f82 updates: #352 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Updates #352 Change-Id: I1bbb3c652ba33cec6aa37f3700370674077fb17d Signed-off-by: karthik-us <ksubrahm@redhat.com>
@pranithk |
@hsafe For the case you explain, it will be fine.
|
@pranithk |
Updates gluster#352 Change-Id: I3d8caa6479dc8e48bec62a09b056971bb061f0cf Signed-off-by: Ashish Pandey <aspandey@redhat.com>
1. Create thin arbiter index file during mount. 2. Set pending marker in thin arbiter id file in case of failure. Change-Id: I269eb8d069f0323f1fc616175e5e5eb7b91d5f82 updates: gluster#352 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Updates gluster#352 Change-Id: I1bbb3c652ba33cec6aa37f3700370674077fb17d Signed-off-by: karthik-us <ksubrahm@redhat.com>
Discussion on thin arbiter volume - #352 (comment) Main idea of having this rpm package is to deploy thin-arbiter without glusterd and other commands on a node, and all we need on that tie-breaker node is to run a single glusterfs command. Also note that, no other glusterfs installation needs thin-arbiter.so. Make sure RPM contains sample vol file, which can work by default, and a script to configure that volfile, along with translator image. Change-Id: Ibace758373d8a991b6a19b2ecc60c93b2f8fc489 updates: bz#1674389 Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Discussion on thin arbiter volume - #352 (comment) Main idea of having this rpm package is to deploy thin-arbiter without glusterd and other commands on a node, and all we need on that tie-breaker node is to run a single glusterfs command. Also note that, no other glusterfs installation needs thin-arbiter.so. Make sure RPM contains sample vol file, which can work by default, and a script to configure that volfile, along with translator image. Change-Id: Ibace758373d8a991b6a19b2ecc60c93b2f8fc489 updates: bz#1672818 Signed-off-by: Amar Tumballi <amarts@redhat.com> Signed-off-by: Ashish Pandey <aspandey@redhat.com> (cherry picked from commit ca9bef7)
Thin-arbiter design document details the design which introduces new kind of quorum in 2-way replication in AFR which adds quorum at a brick level. One of the use cases this can be used in is stretch cluster scenario. Users can also use it if they find it fits their use case better than arbiter. Once this becomes stable, we should start looking at deprecating 2-way replication IMHO.
Please add your comments to the document itself, so that the complete discussion is in the document.
The text was updated successfully, but these errors were encountered: