Command line arg equivalent for libgfapi

[poornimag]

Currently, there is no api in gfapi, where we can set some options (like the command line argument for Fuse mount / brick processes). We have come across the necessity in multiple circumstances (thin client, xlator options etc.)

One way would be to accept variable number of arguments in the glfs_new().
eg: glfs_new ("volname", "--thin-client", "--process-name=smbd client1");

Other options would be to provide an api to set command line options:
eg: glfs_set_options (fs, "key", "value");
OR glfs_set_options (fs, "key", "value"...);
But, this means the options can be set only after glfs_new, which cannot be used for thin client and may be other use cases. So i suggest that we got with glfs_new() taking variable number of arguments.

Thoughts @nixpanic @soumyakoduri @anoopcs9 @thotz ?

[soumyakoduri]

+1. We definitely need a way to configure these options via gfapi as well.

Taking reference from cephfs mount options, can we instead have gfapi.conf where we can list in all these options and their values. We can then define a new API (maybe "glfs_parse_options") to parse these options and set appropriate/default values before calling glfs_new_* or other routines.

This way application users get more flexibility in terms of configuration.

[aravindavk]

Another solution is preparing options struct before the actual call. In the below example, function new_opts returns options struct with default values. If any change required respective set option function can be called. Actual function call can receive this options struct instead of variable number of arguments.

#include <stdio.h>                                                                                                                                                                           
                                                                                                                                                                                             
struct Options{                                                                                                                                                                              
    int opt1;                                                                                                                                                                                
    int opt2;                                                                                                                                                                                
};                                                                                                                                                                                           
                                                                                                                                                                                             
struct Options new_opts()                                                                                                                                                                    
{ 
    /* Initialize with default values */                                                                                                                                                                                           
    struct Options opts = {opt1: 0, opt2: 1};                                                                                                                                                
    return opts;                                                                                                                                                                             
}                                                                                                                                                                                            
                                                                                                                                                                                             
void set_opt1(struct Options *opt, int val) {                                                                                                                                                
    opt->opt1 = val;                                                                                                                                                                         
}                                                                                                                                                                                            
                                                                                                                                                                                             
void set_opt2(struct Options *opt, int val){                                                                                                                                                 
    opt->opt2 = val;                                                                                                                                                                         
}                                                                                                                                                                                            
                                                                                                                                                                                             
int main()                                                                                                                                                                                   
{                                                                                                                                                                                            
    struct Options opts = new_opts();                                                                                                                                                        
    set_opt1(&opts, 2);                                                                                                                                                                      
    /* Actual call with options as parameter, Ex: glfs_new("volume", opts); */                                                                                                                                                                                  
    printf("%d %d\n", opts.opt1, opts.opt2);                                                                                                                                                 
    return 0;                                                                                                                                                                                
}

[poornimag]

@soumyakoduri Thats good, that way its not hard coded while programming, and can be chnaged runtime, every 5 min or so the conf file can be read to reload config. But what do we do if there are multiple instances of gfapi on the system, do we have one .conf(one section per volume) for all the gfapi instances? If thats the case, its almost same as reconfigurable gfapi-volume-options?

[poornimag]

@aravindavk This can be used, what if there are many mount options like in fuse. Wouldn't the number of calls be so many? Not that all the options will be set at once.

[aravindavk]

This can be used, what if there are many mount options like in fuse. Wouldn't the number of calls be so many? Not that all the options will be set at once.

True. Good defaults will help minimize calling multiple commands. This is similar to builder pattern followed in many object oriented programming languages.

[soumyakoduri]

@poornimag For multiple glfs instances, one way I could think of is that users can have multiple gfapi.conf files as per their requirement and should be able to associate those gfapi<>.conf with the different shares

For eg., considering we create two sample gfapi.conf

gfapi_1.conf
{ thin_client = true;
log_level = warning;
}

gfapi_2.conf
{ thin_client = false; }

Now we can associate shares with those options on the application as below,

share_name = /vol1/dir1;
server_name = server1;
gfapi_conf_path = gfapi_1.conf

share_name = /vol2;
server_name = server1;
gfapi_conf_path = gfapi_1.conf

share_name = /vol1/dir3;
server_name = server2;
gfapi_conf_path = gfapi_2.conf

So the only extra configuration option applications (Samba/NFS-Ganesha) need to add is gfapi_conf_path. If this approach seems okay, we can then look into how reload can be managed as not all options can be changed dynamically once gfapi instance is created.

[nixpanic]

This looks a little related to #263, which intends to introduce an API to discover and set any xlator options in a more useful way than glfs_set_xlator_option(). All(most all) mount options that fuse accepts are converted to xlator options, any options that are different should get exposed as an option for some xlator (maybe a global one, maybe api.so).

For a configuration file, that is definitely useful. NFS offers /etc/nfsmount.conf with a similar functionality. It is completely transparent to the mount -t nfs ... command, but defaults can be overwritten when not passed on the cmdline. This is something not only gfapi would benefit from, it would also be great to have for fuse mounts (not all environments allow passing mount options, looking at you OpenShift).

I would not want to have a function to pass a configfile for mount options. That would require all applications to at least call a new glfs_parse_options() function, making this feature less useful. Any gfapi application should automatically have the settings in the configfile applied.

[poornimag]

@soumyakoduri You mean configurable conf path options, how will it be for self heal daemon, qemu etc.? have a default locations where it is defined?
@nixpanic I missed that issue while searching. With config file, i have one doubt, if different instances of gfapi on the same system requires different conf options?

[nixpanic]

@poornimag yes, I am not sure how different instances of gfapi would use different config files. I see this as a fallback approach, and not something that should be used in normal circumstances.

By default the application should configure the client-side options with the functionality from #263, if an application does not support (new?) options, the configfile approach can be used temporarily until the application is extended. For this, I think a configuration for all gfapi instances is acceptable.

We could think of having an environment variable for selecting an alternative configuration file though. And, as it probably should be an option somewhere, #263 would expose the path as well.

[poornimag]

I thought this issue is a duplicate of #263. In #263, it specifies that the new xlator_set api will be called after glfs_new() before glfs_init(), which is not acceptable for certain options like thin-client and other mount options. We need to be able to specify either in glfs_new() or, before. For this there are 4 approaches specified:

1. .conf file
2. glfs_new() taking variable number of arguments
3. glfs_setopt1() APIs for every option.
4. API that allows setting any xlator option as mentioned in Configuration/mount options for gfapi #263
   Hybrid approach of 1 and 4/2, is redundant. If conf files and good defaults can serve the purpose why add other APIs?

Also, there could be scenarios, where one glfs instance requires different conf options than the others. Eg: glfs_instances in smbd process, may require different options than self-heal glfs instances or uss instances etc.

[soumyakoduri]

Extending on top of what you both have mentioned, by default we could use etc/gluster/gfapi/mount.conf config location (stored in fs->mount_conf_path).

In case we need to configure different options for a specific gfapi instance, we could introduce new API - glfs_new(volname, gfapi_conf_path) which shall override default location for that instance.

[amarts]

I was thinking of keeping #303 as the mechanism to handle most of these options even on the client stack (ie, fuse mount / gfapi etc).

One way to extend it to handle one specific client itself is by controlling the 'getspec()' in glusterd-handshake.c to provide a specific file to that particular client. Can be done by appending IP address to the volfile path, and getspec looking for that particular file when a client connects.

[aravindavk]

Extending on top of what you both have mentioned, by default we could use etc/gluster/gfapi/mount.conf config location (stored in fs->mount_conf_path).

In case we need to configure different options for a specific gfapi instance, we could introduce new API - glfs_new(volname, gfapi_conf_path) which shall override default location for that instance.

+1

This approach looks good, similar to global /etc/bashrc and $HOME/.bashrc

[stale]

Thank you for your contributions.
Noticed that this issue is not having any activity in last ~6 months! We are marking this issue as stale because it has not had recent activity.
It will be closed in 2 weeks if no one responds with a comment here.

[sunnyku]

Restoring this as lots of work has been done/ discussed here its not a good choice to close this now but to work upon.

[stale]

Thank you for your contributions.
Noticed that this issue is not having any activity in last ~6 months! We are marking this issue as stale because it has not had recent activity.
It will be closed in 2 weeks if no one responds with a comment here.

[stale]

Closing this issue as there was no update since my last update on issue. If this is an issue which is still valid, feel free to open it.