Skip to content

Commit

Permalink
CLI/Config.py: per-user clush.conf changes
Browse files Browse the repository at this point in the history
Use the new following user clush.conf locations:
* $XDG_CONFIG_HOME/clustershell/clush.conf
* $HOME/.local/etc/clustershell/clush.conf

$HOME/.clush.conf is kept as 1.6 compat for now.

Closes #111.

Change-Id: Ie887655e588d09edb41fb621732a4364c512521c
  • Loading branch information
thiell committed Aug 31, 2015
1 parent d08235c commit 6860608
Show file tree
Hide file tree
Showing 6 changed files with 130 additions and 56 deletions.
6 changes: 6 additions & 0 deletions ChangeLog
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
2015-08-28 S. Thiell <sthiell@stanford.edu>

* CLI/Config.py: better per-user clush.conf support. clush now also checks
for $XDG_CONFIG_HOME/clustershell/clush.conf and
$HOME/.local/etc/clustershell/clush.conf (ticket #111).

2015-08-27 S. Thiell <sthiell@stanford.edu>

* CLI/Nodeset.py: add --list-all / -L to list groups from all group
Expand Down
66 changes: 26 additions & 40 deletions doc/man/man5/clush.conf.5
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.\" Man page generated from reStructeredText.
.\" Man page generated from reStructuredText.
.
.TH CLUSH.CONF 5 "2015-03-23" "1.7" "ClusterShell User Manual"
.TH CLUSH.CONF 5 "2015-08-27" "1.7" "ClusterShell User Manual"
.SH NAME
clush.conf \- Configuration file for clush
.
Expand Down Expand Up @@ -32,19 +32,18 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
..
.SH DESCRIPTION
.sp
\fCclush\fP(1) obtains configuration options from the following sources in the
\fBclush\fP(1) obtains configuration options from the following sources in the
following order:
.INDENT 0.0
.INDENT 3.5
.INDENT 0.0
.IP 1. 3
.
command\-line options
.IP 2. 3
.
user configuration file (\fI~/.clush.conf\fP)
user configuration file (\fI$XDG_CONFIG_HOME/clustershell/clush.conf\fP)
.IP 3. 3
.
local pip user installation (\fI$HOME/.local/etc/clustershell/clush.conf\fP)
.IP 4. 3
system\-wide configuration file (\fI/etc/clustershell/clush.conf\fP)
.UNINDENT
.UNINDENT
Expand All @@ -57,93 +56,76 @@ of one main section:
.INDENT 0.0
.TP
.B Main
.
Program options definition
.UNINDENT
.SS [Main]
.sp
Configuration parameters of the \fCMain\fP section are described below.
Configuration parameters of the \fBMain\fP section are described below.
.INDENT 0.0
.TP
.B fanout
.
Size of the sliding window of ssh connectors.
.TP
.B connect_timeout
.
Timeout in seconds to allow a connection to establish. This parameter is
passed to ssh. If set to \fI0\fP, no timeout occurs.
.TP
.B command_timeout
.
Timeout in seconds to allow a command to complete since the connection has
been established. This parameter is passed to ssh. In addition, the
ClusterShell library ensures that any commands complete in less than
( connect_timeout + command_timeout ). If set to \fI0\fP, no timeout occurs.
.TP
.B color
.
Whether to use ANSI colors to surround node or nodeset prefix/header with
escape sequences to display them in color on the terminal. Valid arguments
are \fCnever\fP, \fCalways\fP or \fCauto\fP (which use color if standard
are \fBnever\fP, \fBalways\fP or \fBauto\fP (which use color if standard
output/error refer to a terminal). Colors are set to [34m (blue foreground
text) for stdout and [31m (red foreground text) for stderr, and cannot be
modified.
.TP
.B fd_max
.
Maximum number of open file descriptors permitted per clush process (soft
resource limit for open files). This limit can never exceed the system
(hard) limit. The \fIfd_max\fP (soft) and system (hard) limits should be high
enough to run \fCclush\fP, although their values depend on your \fIfanout\fP value.
enough to run \fBclush\fP, although their values depend on your \fIfanout\fP value.
.TP
.B history_size
.
Set the maximum number of history entries saved in the GNU readline history
list. Negative values imply unlimited history file size.
.TP
.B node_count
.
Should \fCclush\fP display additional (node count) information in buffer
Should \fBclush\fP display additional (node count) information in buffer
header? (\fIyes\fP/\fIno\fP)
.TP
.B verbosity
.
Set the verbosity level: \fI0\fP (quiet), \fI1\fP (default), \fI2\fP (verbose) or more
(debug).
.TP
.B ssh_user
.
Set the ssh user to use for remote connection (default is to not specify).
.TP
.B ssh_path
.
Set the ssh binary path to use for remote connection (default is \fIssh\fP).
.TP
.B ssh_options
.
Set additional options to pass to the underlying ssh command.
.TP
.B scp_path
.
Set the scp binary path to use for remote copy (default is \fIscp\fP).
.TP
.B scp_options
.
Set additional options to pass to the underlying scp command. If not
specified, ssh_options are used instead.
.TP
.B rsh_path
.
Set the rsh binary path to use for remote connection (default is
\fIrsh\fP). You could easily use mrsh or krsh by simply changing this value.
.TP
.B rcp_path
.
Same a rsh_path for rcp command. (Default is \fIrcp\fP)
.TP
.B rsh_options
.
Set additional options to pass to the underlying rsh/rcp command.
.UNINDENT
.SH EXAMPLES
Expand All @@ -165,29 +147,33 @@ node_count: yes
.SH FILES
.INDENT 0.0
.TP
.B \fI~/.clush.conf\fP
.sp
This is the per\-user configuration file.
.TP
.B \fI/etc/clustershell/clush.conf\fP
.sp
System\-wide configuration file.
System\-wide clush configuration file.
.TP
.B \fI$XDG_CONFIG_HOME/clustershell/clush.conf\fP
User configuration file for clush. If $XDG_CONFIG_HOME is not defined,
\fI$HOME/.config/clustershell/clush.conf\fP is used instead.
.TP
.B \fI$HOME/.local/etc/clustershell/clush.conf\fP
Local user configuration file for clush (default installation for pip \-\-user)
.TP
.B \fI~/.clush.conf\fP
Deprecated per\-user clush configuration file.
.UNINDENT
.SH HISTORY
.sp
As of ClusterShell version 1.3, the \fCExternal\fP section has been removed
from \fIclush.conf\fP. External commands whose outputs were used by \fCclush\fP
As of ClusterShell version 1.3, the \fBExternal\fP section has been removed
from \fIclush.conf\fP\&. External commands whose outputs were used by \fBclush\fP
(\-a, \-g, \-X) are now handled by the library itself and defined in
\fCgroups.conf\fP(5).
\fBgroups.conf\fP(5).
.SH SEE ALSO
.sp
\fCclush\fP(1), \fCnodeset\fP(1), \fCgroups.conf\fP(5)
\fBclush\fP(1), \fBnodeset\fP(1), \fBgroups.conf\fP(5)
.sp
\fI\%http://clustershell.sourceforge.net/\fP
.SH AUTHOR
Stephane Thiell, CEA DAM <stephane.thiell@cea.fr>
Stephane Thiell, <sthiell@stanford.edu>
.SH COPYRIGHT
CeCILL-C V1
.\" Generated by docutils manpage writer.
.\"
.
16 changes: 14 additions & 2 deletions doc/sphinx/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,20 @@ Configuration
clush
-----

The configuration file */etc/clustershell/clush.conf* defines default values
for several *clush* tool parameters.
The following configuration file defines system-wide default values for
several *clush* tool parameters::

/etc/clustershell/clush.conf

*clush* settings might then be overridden per user if one of the following
files is found, in priority order::

$XDG_CONFIG_HOME/clustershell/clush.conf
$HOME/.config/clustershell/clush.conf (if $XDG_CONFIG_HOME not defined)
$HOME/.local/etc/clustershell/clush.conf
$HOME/.clush.conf (deprecated, for 1.6 compatibility only)

The following table describes available *clush* config file settings.

+-----------------+----------------------------------------------------+
| Key | Value |
Expand Down
25 changes: 16 additions & 9 deletions doc/txt/clush.conf.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
Configuration file for `clush`
------------------------------

:Author: Stephane Thiell, CEA DAM <stephane.thiell@cea.fr>
:Date: 2015-03-23
:Author: Stephane Thiell, <sthiell@stanford.edu>
:Date: 2015-08-27
:Copyright: CeCILL-C V1
:Version: 1.7
:Manual section: 5
Expand All @@ -21,8 +21,9 @@ DESCRIPTION
following order:

1. command-line options
2. user configuration file (*~/.clush.conf*)
3. system-wide configuration file (*/etc/clustershell/clush.conf*)
2. user configuration file (*$XDG_CONFIG_HOME/clustershell/clush.conf*)
3. local pip user installation (*$HOME/.local/etc/clustershell/clush.conf*)
4. system-wide configuration file (*/etc/clustershell/clush.conf*)

For each parameter, the first obtained value will be used.

Expand Down Expand Up @@ -109,11 +110,18 @@ Simple configuration file.
FILES
=====

*~/.clush.conf*
This is the per-user configuration file.

*/etc/clustershell/clush.conf*
System-wide configuration file.
System-wide clush configuration file.

*$XDG_CONFIG_HOME/clustershell/clush.conf*
User configuration file for clush. If $XDG_CONFIG_HOME is not defined,
*$HOME/.config/clustershell/clush.conf* is used instead.

*$HOME/.local/etc/clustershell/clush.conf*
Local user configuration file for clush (default installation for pip --user)

*~/.clush.conf*
Deprecated per-user clush configuration file.


HISTORY
Expand All @@ -131,4 +139,3 @@ SEE ALSO
``clush``\(1), ``nodeset``\(1), ``groups.conf``\(5)

http://clustershell.sourceforge.net/

10 changes: 9 additions & 1 deletion lib/ClusterShell/CLI/Config.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@

import ConfigParser
import os
from os.path import expanduser

from ClusterShell.CLI.Display import VERB_QUIET, VERB_STD, \
VERB_VERB, VERB_DEBUG, THREE_CHOICES
Expand Down Expand Up @@ -78,7 +79,14 @@ def __init__(self, options, filename=None):
files = [filename]
else:
files = ['/etc/clustershell/clush.conf',
os.path.expanduser('~/.clush.conf')]
# deprecated user config, kept in 1.x for 1.6 compat
expanduser('~/.clush.conf'),
# default pip --user config file
expanduser('~/.local/etc/clustershell/clush.conf'),
# per-user clush.conf config top override
os.path.join(os.environ.get('XDG_CONFIG_HOME',
expanduser('~/.config')),
'clustershell', 'clush.conf')]
self.read(files)

# Apply command line overrides
Expand Down
63 changes: 59 additions & 4 deletions tests/CLIConfigTest.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
#!/usr/bin/env python
# ClusterShell.CLI.Config test suite
# Written by S. Thiell 2010-09-19
# Written by S. Thiell


"""Unit test for CLI.Config"""

import resource
import os.path
import shutil
import sys
import tempfile
import unittest

sys.path.insert(0, '../lib')

from TLib import make_temp_dir

from ClusterShell.CLI.Clush import set_fdlimit
from ClusterShell.CLI.Config import ClushConfig, ClushConfigError
Expand Down Expand Up @@ -293,7 +296,59 @@ def testClushConfigWithInstalledConfig(self):
config = ClushConfig(options)
self.assert_(config != None)

def testClushConfigUserOverride(self):
"""test CLI.Config.ClushConfig (XDG_CONFIG_HOME user config)"""

if __name__ == '__main__':
suites = [unittest.TestLoader().loadTestsFromTestCase(CLIClushConfigTest)]
unittest.TextTestRunner(verbosity=2).run(unittest.TestSuite(suites))
# XXX Test should be improved when CLUSTERSHELL_CONFIG is available
# Improvement: override CLUSTERSHELL_CONFIG and set a sys clush config
# then verify that user config overrides CLUSTERSHELL_CONFIG as
# expected...
# For now, it has been tested manually. This test only really only
# ensures that user config is taken into account.

xdg_config_home_save = os.environ.get('XDG_CONFIG_HOME')

# Create fake XDG_CONFIG_HOME
dname = make_temp_dir()
try:
os.environ['XDG_CONFIG_HOME'] = dname

# create $XDG_CONFIG_HOME/clustershell/clush.conf
usercfgdir = os.path.join(dname, 'clustershell')
os.mkdir(usercfgdir)
cfgfile = open(os.path.join(usercfgdir, 'clush.conf'), 'w')
cfgfile.write("""
[Main]
fanout: 42
connect_timeout: 14
command_timeout: 0
history_size: 100
color: never
verbosity: 2
ssh_user: trump
ssh_path: ~/bin/ssh
ssh_options: -oSomeDummyUserOption=yes
""")

cfgfile.flush()
parser = OptionParser("dummy")
parser.install_display_options(verbose_options=True)
parser.install_connector_options()
options, _ = parser.parse_args([])
config = ClushConfig(options) # filename=None to use defaults!
self.assertEqual(config.color, WHENCOLOR_CHOICES[0])
self.assertEqual(config.verbosity, VERB_VERB) # takes biggest
self.assertEqual(config.fanout, 42)
self.assertEqual(config.connect_timeout, 14)
self.assertEqual(config.command_timeout, 0)
self.assertEqual(config.ssh_user, 'trump')
self.assertEqual(config.ssh_path, '~/bin/ssh')
self.assertEqual(config.ssh_options, '-oSomeDummyUserOption=yes')
cfgfile.close()

finally:
if xdg_config_home_save:
os.environ['XDG_CONFIG_HOME'] = xdg_config_home_save
else:
del os.environ['XDG_CONFIG_HOME']
shutil.rmtree(dname, ignore_errors=True)

0 comments on commit 6860608

Please sign in to comment.