How to hvfs.sh
Clone this wiki locally
Format of pomegranate configuration files
site: every instance of Pomegranate component is a site. It includes mds server, mdsl server, r2 server, clients and amc clients.
There are four configuration files in 'conf/' directory: 'hvfs.conf', 'mds.conf', 'mdsl.conf', and 'ut.conf'.
File 'hvfs.conf' describes the cluster of pomegranate servers and clients. Each valid line contains a site entry. The format is:
type:ip_address:tcp_port:logical_id # for example mds:127.0.0.1:8210:0 # means that site type is mds, ip address is 127.0.0.1, tcp port is 8210, and logical id is 0
Notes: 1. Valid value for type is mds/mdsl/r2/client/amc. 2. This configuration file will be broadcast to all the nodes. Thus, do not use local reference, i.e. 127.0.0.1, unless all the sites are on the same machine. 3. Clients have to be described in the config file. At this moment, we do not support automatically detect client address.
File 'mds.conf' describes the environment variables of MDS server. The default value for MDS server in the config file is OK to run in your machine. If you know why you have to change them, feel free to change them. Otherwise, do not change them. There is a complete table about the supported environment variables at Table: MDS Environment Variables.
[root@glnode09 hvfs]# cat conf/mds.conf # Pomegranate file system Metadata Server config file # File system id # # 0: used by the client.ut for unit test, you can use this id for fs access # 1: used by key/value store fsid=0 # Mode can be 0 or 1. 0 means using the config in source code; 1 means using # the config in conf/hvfs.conf. If you do not know the impact, do NOT change # it. mode=1 # Memlimit is by default disabled. If it is enabled, you have to provide the # true memory limit in bytes by setting 'hvfs_mds_memlimit'. hvfs_mds_opt_memlimit=0 # Memory limit in bytes hvfs_mds_memlimit=2g # Transaction group commit interval. Note that it may be changed dynamically if hvfs_mds_dati is enabled (default behaver). hvfs_mds_txg_interval=5 #hvfs_mds_dati=0 # Memory only mode disabled. # If it is enabled, all the dirty ITBs are not writen back to MDSL. hvfs_mds_opt_memonly=0 # Ignore this value type=0 # How many ITBs are preallocated at startup? cache=0 # Interval to commit dirty entries in bitmap cache (default 5s). #hvfs_mds_bitmap_cache_interval=5 # Interval to emit profiling info (default 5s). #hvfs_mds_profiling_thread_interval=5 # Interval to do heartbeat (default 60s). #hvfs_mds_hb_interval=10 # How many threads used by service pool? #hvfs_mds_spool_threads=8
File 'mdsl.conf' describes the environment variables of MDSL server. The default value for MDSL server in the config file is OK to run in your machine. If you know why you have to change them, feel free to change them. Otherwise, do not change them. There is a complete table about the supported environment variables at Table: MDSL Environment Variables.
# Pomegranate file system Metadata Storage Server config file # Mode can be 0 or 1. 0 means using the config in source code; 1 means using # the config in conf/hvfs.conf. If you do not know the impact, do NOT change # it. mode=1 # Plot mode for gnuplot, otherwise use the human mode hvfs_mdsl_prof_plot=1 # Drop all the write-backs (default disabled) hvfs_mdsl_opt_write_drop=0 # Set the default file chunk size for ITB file (default 64MB) hvfs_mdsl_itb_file_chunk=67108864 # Set the default file chunk size for data file (default 64MB) hvfs_mdsl_data_file_chunk=67108864
File 'ut.conf' describes the unit test configurations.
# Pomegranate file system Unit test config file # mode can be 0 or 1. 0 means using the config in source code; 1 means using # the config in conf/hvfs.conf. If you do not know the impact, do NOT change # it. mode=1 # op can be 0/1/2/3/4/5/100/200 # op=0 means do CREATE operations # op=1 means do LOOKUP operations # op=2 means do UNLINK operations # op=3 means do CREATE_DIR operations # op=4 means do WRITE_DATA operations # op=5 means do READ_DATA operations # op=100 means do CREATE, LOOKUP, UNLINK sequentially # op=200 means do CREATE, LOOKUP, WRITE_DATA, READ_DATA, UNLINK sequentially op=100 # thread number thread=5 # how many files do you want to test? entry=200000 # how many clients do you want to start? # if nr is -1, it means to start all the clients in conf/hvfs.conf file. nr=-1
How to use bin/hvfs.sh?
File 'bin/hvfs.sh" is a tool to start Pomegranate servers locally or remotely based on the configuration in file 'conf/hvfs.conf'. You have to install ssh firstly if you do not have ssh in your box.
If you have made the cluster SSH authorized, you can start mds/mdsl server easily as following:
$ bin/hvfs.sh start $ # use 'bin/hvfs.sh stat' to check if all the servers are started
Otherwise, you have to use the expect script to start servers:
$ USERNAME=your_username PASSWD=your_passwd bin/hvfs/sh start
Note that, the USERNAME and PASSWD will be feed to each ssh connection. Thus, all the machines should have the same username and password.
If you want to stop the cluster, do as following:
$ bin/hvfs.sh stop
$ USERNAME=your_username PASSWD=your_passwd bin/hvfs.sh stop
If you want to kill the cluster, do as following:
$ bin/hvfs.sh kill
$ USERNAME=your_username PASSWD=your_passwd bin/hvfs.sh kill
There are some more commands in the help menu of hvfs.sh. Just type 'bin/hvfs.sh', you can get the full help menu.