Skip to content

DocDeploy

Jump to bottom
anjanakar edited this page Oct 11, 2017 · 15 revisions

Deploying SLASH2

This document describes process of deploying services for a SLASH2 file system deployment. Refer to DocOverview for a primer on SLASH2 deployment basics.

MDS

This service requires a ZFS pool (zpool) to store the system metadata (file system namespace, file attributes, etc.). In addition, storage for the operations journal is needed for the MDS daemon to track on-going operations persistently.

Starting slashd

Command-line flags, runtime files, environment variables, etc. are all described in slashd(8). A wrapper script is provided in the distribution (slashd.sh) that monitors and restarts slashd upon failure and compiles and sends bug reports to the SLASH2 development team when problems arise in operation.

Modify the file pfl/utils/daemon/pfl_daemon.cfg and add configuration sections for the deployment.

Finally, launch the MDS:

mds# slashd.sh

Controlling live operation of slashd

slmctl(8) serves as the interface to the running slashd(8) process, providing administrators with capability to query parameters and control behavior. A simple "peer connections" query is a good way to test general service availability:

mds$ slmctl -sc
resource                                     host type     flags stvrs txcr #ref
================================================================================
XWFS
  xwfs_psc_ios7_0                  sense7.psc.edu serial   -OM-- 19149    8    0
  xwfs_tacc_0                       129.114.32.90 serial   -OM-- 19150    8    0
  xwfs_tacc_1                       129.114.32.91 serial   -OM-- 19150    8    0
  xwfs_tacc_2                       129.114.32.92 serial   -OM-- 19150    8    0
  xwfs_tacc_3                       129.114.32.93 serial   --M-- 19150    8    0
clients
		       firehose2.psc.teragrid.org          -O---     0    8    2

The above command lists clients, I/O servers, and other MDS nodes connected to this MDS and also demonstrates the basic health of the MDS.

zfs and zpool

Once the MDS is operational, the zfs(8) and zpool(8) commands may be used for any necessary pool administration:

mds# zfs get compressratio
NAME            PROPERTY       VALUE  SOURCE
xwfs_test       compressratio  3.28x  -
xwfs_test@test  compressratio  3.04x  -

mds# zpool iostat
	      capacity     operations    bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
xwfs_test   1001M   148G      3      0  7.69K  7.23K

mds# zpool scrub xwfs_test
mds# zfs send xwfs_test@test > /local/xwfs_test@test.zstream

The SLASH2 I/O service (sliod)

Starting sliod

Prior to starting sliod(8) on an I/O server, make sure the following items are available:

  • the SLASH2 deployment configuration file, usually /local/$DEPLOYMENT.s2/slcfg
  • the SLASH2 deployment daemon list file, usually /usr/local/pfl_daemon.cfg
  • the SLASH2 deployment secret communication key, usually /local/$DEPLOYMENT.s2/var/authbuf.key, which is automatically generated by the MDS or may be done so manually by slkeymgt(8)
  • the storage that will be contributed to the SLASH2 deployment. sliod can serve any POSIX file system to the SLASH2 network as long as the resource has been formatted with slmkfs(8).

Launch the I/O service:

io# sliod.sh

Controlling live operation of sliod

Similar to slmctl(8), sliod(8) has its own control interface program: slictl(8). A similar basic sliod daemon health and connection status inquiry may be performed:

io$ slictl -sc
resource                                    host type      flags stvrs txcr #ref
================================================================================
XWFS
  xwfs_mds                         citron.psc.edu mds      -O--P 19111    8    0
  xwfs_tacc_0                       129.114.32.90 serial   -----     0    0    0
  xwfs_tacc_1                       129.114.32.91 serial   -----     0    0    0
  xwfs_tacc_2                       129.114.32.92 serial   -----     0    0    0
  xwfs_tacc_3                       129.114.32.93 serial   -----     0    0    0

The SLASH2 Client Service mount_slash

Starting mount_slash

Similiar to the sliod(8) prerequisites, prior to starting mount_slash(8) on a client machine, make sure the following items are available:

  • the SLASH2 deployment configuration file, usually /local/$DEPLOYMENT.s2/slcfg
  • the SLASH2 deployment daemon list file, usually /usr/local/pfl_daemon.cfg
  • the SLASH2 deployment secret communication key, usually /local/$DEPLOYMENT.s2/var/authbuf.key
  • the attachment mount point (e.g. mkdir /s2)

Launch the client service:

client# mount_slash.sh

Controlling live operation of mount_slash

Like slmctl(8) and slictl(8), an analogous command msctl(8) is provided to control live operation of mount_slash(8). Again, a connection status request can be used to check connectivity to peers and basic client service health:

client$ msctl -sc
resource                                    host type      flags stvrs txcr #ref
================================================================================
XWFS
  xwfs_mds                         citron.psc.edu mds      -O--- 19111    8  601
  xwfs_psc_ios7_0                  sense7.psc.edu serial   ----- 19149    8    0
  xwfs_tacc_0                       129.114.32.90 serial   -O--- 19150    8    0
  xwfs_tacc_1                       129.114.32.91 serial   -O--- 19150    8    0
  xwfs_tacc_2                       129.114.32.92 serial   -O--- 19150    8    0
  xwfs_tacc_3                       129.114.32.93 serial   -O--- 19150    8    0

6.3. SLASH2 Software Updates

To update an existing slash2 tree, cd to the directory (/local/src/p for example), then run

        client$ git pull
        client$ make build
        client$ sudo make install

And ensure you have any deployment configuration cloned locally as well. This example uses the faux deployment:

$ git clone ssh://local-git-server/g/inf.faux inf/faux

New binaries should now be installed on each machine. The final procedure is to restart the services on each machine:

ios# pkill sliod                   # for IOS nodes
mds# pkill slashd                  # for MDS nodes
cli# pkill mount_slash             # for clients

PSC

SLASH2

Funded in part by:
NARA NSF

Clone this wiki locally