Support volfile generation for Thin Arbiter volume type

[amarts]

More on what is thin arbiter @ gluster/glusterfs#352

What support is needed by glusterd2

Support in volume set

User would give the details of tie-breaker for the whole cluster (this is current MVP), there may be an RFE for per volume tie-breaker. That is future.

glustercli volume set all global.tie-breaker <IP>:<port>://<brick> (or similar).

support in volfile generation

glustercli volume create dr-ta-vol replica 2 <brick0> <brick1> <brick2> <brick3> --type thin-arbiter

This would generate the volfile of just 4 bricks, and a client volfile which looks like below

volume thin-arbiter-client
   type protocol/client
   option remote-host <tie-breaker-node IP>
   option remote-port 24007 # optional (should match what is provided in tie-breaker setup)
end-volume

volume cp-0 # brick0
  type protocol/client
end-volume 

volume cp-1 # brick1
  type protocol/client
end-volume 

volume replicate-0
  type cluster/replicate
  subvolumes cp-0 cp-1 thin-arbiter-client # Notice the order, they want to keep tie-breaker as last child 
end-volume

volume cp-2 # brick2
  type protocol/client
end-volume 

volume cp-3 # brick3
  type protocol/client
end-volume 

volume replicate-1
  type cluster/replicate
  subvolumes cp-2 cp-3 thin-arbiter-client # Notice the order, they want to keep tie-breaker as last child 
end-volume

.....
....

For the initial version the requirement is this. This issue would be enhanced if there are more requirements.

@aspandey @itisravi @karthik-us @pranithk

[aravindavk]

Sent patch to support this feature in Volfile. #673

Assumptions:

• Thin Arbiter Brick will be set as Volume option glustercli volume set <volname> cluster.replicate.thin-arbiter <host:brick>
• Volume option name is thin-arbiter in cluster/replicate

To test the Volfile generation,

• Add "thin-arbiter" option to cluster/replicate options table
• Create Replica 2 Volume using CLI(Glusterd2 CLI)
• Set Volume option using above volume set command.

Generated Client Volfile will have Thin arbiter details in it.

Let us know if any changes required in the behavior.

[prashanthpai]

*cc @aspandey

[amarts]

Noticed that patch does handle one thing more than what is described:

Thin Arbiter Brick will be set as Volume option glustercli volume set cluster.replicate.thin-arbiter host:brick

It handles host:brick<:port> ie, a way to give different port for the tie-breaker.

other comment is about name. I vote for 'tie-breaker' as the name of client protocol.

[itisravi]

Thin Arbiter Brick will be set as Volume option glustercli volume set cluster.replicate.thin-arbiter host:brick
Volume option name is thin-arbiter in cluster/replicate

@aravindavk I have used thin-arbiter as a string option in https://review.gluster.org/#/c/19835/7/xlators/cluster/afr/src/afr.c@1120

I saw that running glustercli volume set <volname> afr.thin-arbiter <host>:/bricks/thin-arb-brick> --advanced did not set the volume option in afr.

[aravindavk]

I saw that running glustercli volume set afr.thin-arbiter :/bricks/thin-arb-brick> --advanced did not set the volume option in afr.

Any errors?

Glusterd2 loads all the list of options to validate from *.so files. Is the above mentioned xlator option available in the installed glusterfs?

[itisravi]

@aravindavk It will be good to have a single command to create the thin arbiter volume instead of the 2 step process of {creating replica 2 + doing a volume set }.

So something like'gluster volume create <volname> replica 2 thin-arbiter <hosts: data bricks> <host:thin arbiter brick> would be good.

[aravindavk]

@aravindavk It will be good to have a single command to create the thin arbiter volume instead of the 2 step process of {creating replica 2 + doing a volume set }.

In Glusterd2, Option can be set during Volume Create itself.

So something like'gluster volume create replica 2 thin-arbiter <hosts: data bricks> <host:thin arbiter brick> would be good.

Above format adds confusion if we have more than one sub volume. I suggest the following format

glustercli volume create <volname> --replica 2 --thin-arbiter <host:thin-arbiter-brick> <hosts: data brick1> <hosts: data brick2>...

[aravindavk]

I see two .so files for replicate, afr.so and replicate.so. Option is set to "afr.thin-arbiter" but in volfile option is used as type cluster/replicate. Please suggest which one to be considered

[itisravi]

For all afr options, we don't use either. The way it works currently is gluster v set <volname> cluster.<option name> <value> , for example gluster vol set testvol cluster.consistent-metadata on

[aravindavk]

Volfile support and CLI support is available.