CLI/Config.py: per-user clush.conf changes

Use the new following user clush.conf locations:
* $XDG_CONFIG_HOME/clustershell/clush.conf
* $HOME/.local/etc/clustershell/clush.conf

$HOME/.clush.conf is kept as 1.6 compat for now.

Closes #111.

Change-Id: Ie887655e588d09edb41fb621732a4364c512521c

ChangeLog

@@ -1,3 +1,9 @@
+2015-08-28 S. Thiell <sthiell@stanford.edu>
+
+	* CLI/Config.py: better per-user clush.conf support. clush now also checks
+	for $XDG_CONFIG_HOME/clustershell/clush.conf and
+	$HOME/.local/etc/clustershell/clush.conf (ticket #111).
+
 2015-08-27 S. Thiell <sthiell@stanford.edu>
 
 	* CLI/Nodeset.py: add --list-all / -L to list groups from all group

doc/man/man5/clush.conf.5

@@ -1,6 +1,6 @@
-.\" Man page generated from reStructeredText.
+.\" Man page generated from reStructuredText.
 .
-.TH CLUSH.CONF 5 "2015-03-23" "1.7" "ClusterShell User Manual"
+.TH CLUSH.CONF 5 "2015-08-27" "1.7" "ClusterShell User Manual"
 .SH NAME
 clush.conf \- Configuration file for clush
 .
@@ -32,19 +32,18 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 ..
 .SH DESCRIPTION
 .sp
-\fCclush\fP(1) obtains configuration options from the following sources in the
+\fBclush\fP(1) obtains configuration options from the following sources in the
 following order:
 .INDENT 0.0
 .INDENT 3.5
 .INDENT 0.0
 .IP 1. 3
-.
 command\-line options
 .IP 2. 3
-.
-user configuration file (\fI~/.clush.conf\fP)
+user configuration file (\fI$XDG_CONFIG_HOME/clustershell/clush.conf\fP)
 .IP 3. 3
-.
+local pip user installation (\fI$HOME/.local/etc/clustershell/clush.conf\fP)
+.IP 4. 3
 system\-wide configuration file (\fI/etc/clustershell/clush.conf\fP)
 .UNINDENT
 .UNINDENT
@@ -57,93 +56,76 @@ of one main section:
 .INDENT 0.0
 .TP
 .B Main
-.
 Program options definition
 .UNINDENT
 .SS [Main]
 .sp
-Configuration parameters of the \fCMain\fP section are described below.
+Configuration parameters of the \fBMain\fP section are described below.
 .INDENT 0.0
 .TP
 .B fanout
-.
 Size of the sliding window of ssh connectors.
 .TP
 .B connect_timeout
-.
 Timeout in seconds to allow a connection to establish. This parameter is
 passed to ssh. If set to \fI0\fP, no timeout occurs.
 .TP
 .B command_timeout
-.
 Timeout in seconds to allow a command to complete since the connection has
 been established. This parameter is passed to ssh.  In addition, the
 ClusterShell library ensures that any commands complete in less than
 ( connect_timeout + command_timeout ). If set to \fI0\fP, no timeout occurs.
 .TP
 .B color
-.
 Whether to use ANSI colors to surround node or nodeset prefix/header with
 escape sequences to display them in color on the terminal. Valid arguments
-are \fCnever\fP, \fCalways\fP or \fCauto\fP (which use color if standard
+are \fBnever\fP, \fBalways\fP or \fBauto\fP (which use color if standard
 output/error refer to a terminal). Colors are set to [34m (blue foreground
 text) for stdout and [31m (red foreground text) for stderr, and cannot be
 modified.
 .TP
 .B fd_max
-.
 Maximum number of open file descriptors permitted per clush process (soft
 resource limit for open files). This limit can never exceed the system
 (hard) limit. The \fIfd_max\fP (soft) and system (hard) limits should be high
-enough to run \fCclush\fP, although their values depend on your \fIfanout\fP value.
+enough to run \fBclush\fP, although their values depend on your \fIfanout\fP value.
 .TP
 .B history_size
-.
 Set the maximum number of history entries saved in the GNU readline history
 list. Negative values imply unlimited history file size.
 .TP
 .B node_count
-.
-Should \fCclush\fP display additional (node count) information in buffer
+Should \fBclush\fP display additional (node count) information in buffer
 header? (\fIyes\fP/\fIno\fP)
 .TP
 .B verbosity
-.
 Set the verbosity level: \fI0\fP (quiet), \fI1\fP (default), \fI2\fP (verbose) or more
 (debug).
 .TP
 .B ssh_user
-.
 Set the ssh user to use for remote connection (default is to not specify).
 .TP
 .B ssh_path
-.
 Set the ssh binary path to use for remote connection (default is \fIssh\fP).
 .TP
 .B ssh_options
-.
 Set additional options to pass to the underlying ssh command.
 .TP
 .B scp_path
-.
 Set the scp binary path to use for remote copy (default is \fIscp\fP).
 .TP
 .B scp_options
-.
 Set additional options to pass to the underlying scp command. If not
 specified, ssh_options are used instead.
 .TP
 .B rsh_path
-.
 Set the rsh binary path to use for remote connection (default is
 \fIrsh\fP). You could easily use mrsh or krsh by simply changing this value.
 .TP
 .B rcp_path
-.
 Same a rsh_path for rcp command. (Default is \fIrcp\fP)
 .TP
 .B rsh_options
-.
 Set additional options to pass to the underlying rsh/rcp command.
 .UNINDENT
 .SH EXAMPLES
@@ -165,29 +147,33 @@ node_count: yes
 .SH FILES
 .INDENT 0.0
 .TP
-.B \fI~/.clush.conf\fP
-.sp
-This is the per\-user configuration file.
-.TP
 .B \fI/etc/clustershell/clush.conf\fP
-.sp
-System\-wide configuration file.
+System\-wide clush configuration file.
+.TP
+.B \fI$XDG_CONFIG_HOME/clustershell/clush.conf\fP
+User configuration file for clush. If $XDG_CONFIG_HOME is not defined,
+\fI$HOME/.config/clustershell/clush.conf\fP is used instead.
+.TP
+.B \fI$HOME/.local/etc/clustershell/clush.conf\fP
+Local user configuration file for clush (default installation for pip \-\-user)
+.TP
+.B \fI~/.clush.conf\fP
+Deprecated per\-user clush configuration file.
 .UNINDENT
 .SH HISTORY
 .sp
-As of ClusterShell version 1.3, the \fCExternal\fP section has been removed
-from \fIclush.conf\fP.  External commands whose outputs were used by \fCclush\fP
+As of ClusterShell version 1.3, the \fBExternal\fP section has been removed
+from \fIclush.conf\fP\&.  External commands whose outputs were used by \fBclush\fP
 (\-a, \-g, \-X) are now handled by the library itself and defined in
-\fCgroups.conf\fP(5).
+\fBgroups.conf\fP(5).
 .SH SEE ALSO
 .sp
-\fCclush\fP(1), \fCnodeset\fP(1), \fCgroups.conf\fP(5)
+\fBclush\fP(1), \fBnodeset\fP(1), \fBgroups.conf\fP(5)
 .sp
 \fI\%http://clustershell.sourceforge.net/\fP
 .SH AUTHOR
-Stephane Thiell, CEA DAM  <stephane.thiell@cea.fr>
+Stephane Thiell, <sthiell@stanford.edu>
 .SH COPYRIGHT
 CeCILL-C V1
 .\" Generated by docutils manpage writer.
-.\" 
 .

doc/sphinx/config.rst

@@ -6,8 +6,20 @@ Configuration
 clush
 -----
 
-The configuration file */etc/clustershell/clush.conf* defines default values
-for several *clush* tool parameters.
+The following configuration file defines system-wide default values for
+several *clush* tool parameters::
+
+    /etc/clustershell/clush.conf
+
+*clush* settings might then be overridden per user if one of the following
+files is found, in priority order::
+
+    $XDG_CONFIG_HOME/clustershell/clush.conf
+    $HOME/.config/clustershell/clush.conf (if $XDG_CONFIG_HOME not defined)
+    $HOME/.local/etc/clustershell/clush.conf
+    $HOME/.clush.conf (deprecated, for 1.6 compatibility only)
+
+The following table describes available *clush* config file settings.
 
 +-----------------+----------------------------------------------------+
 | Key             | Value                                              |

doc/txt/clush.conf.txt

@@ -6,8 +6,8 @@
 Configuration file for `clush`
 ------------------------------
 
-:Author: Stephane Thiell, CEA DAM  <stephane.thiell@cea.fr>
-:Date:   2015-03-23
+:Author: Stephane Thiell, <sthiell@stanford.edu>
+:Date:   2015-08-27
 :Copyright: CeCILL-C V1
 :Version: 1.7
 :Manual section: 5
@@ -21,8 +21,9 @@ DESCRIPTION
 following order:
 
  1. command-line options
- 2. user configuration file (*~/.clush.conf*)
- 3. system-wide configuration file (*/etc/clustershell/clush.conf*)
+ 2. user configuration file (*$XDG_CONFIG_HOME/clustershell/clush.conf*)
+ 3. local pip user installation (*$HOME/.local/etc/clustershell/clush.conf*)
+ 4. system-wide configuration file (*/etc/clustershell/clush.conf*)
 
 For each parameter, the first obtained value will be used.
 
@@ -109,11 +110,18 @@ Simple configuration file.
 FILES
 =====
 
-*~/.clush.conf*
-  This is the per-user configuration file.
-
 */etc/clustershell/clush.conf*
-  System-wide configuration file.
+  System-wide clush configuration file.
+
+*$XDG_CONFIG_HOME/clustershell/clush.conf*
+  User configuration file for clush. If $XDG_CONFIG_HOME is not defined,
+  *$HOME/.config/clustershell/clush.conf* is used instead.
+
+*$HOME/.local/etc/clustershell/clush.conf*
+  Local user configuration file for clush (default installation for pip --user)
+
+*~/.clush.conf*
+  Deprecated per-user clush configuration file.
 
 
 HISTORY
@@ -131,4 +139,3 @@ SEE ALSO
 ``clush``\(1), ``nodeset``\(1), ``groups.conf``\(5)
 
 http://clustershell.sourceforge.net/
-

lib/ClusterShell/CLI/Config.py

@@ -37,6 +37,7 @@
 
 import ConfigParser
 import os
+from os.path import expanduser
 
 from ClusterShell.CLI.Display import VERB_QUIET, VERB_STD, \
     VERB_VERB, VERB_DEBUG, THREE_CHOICES
@@ -78,7 +79,14 @@ def __init__(self, options, filename=None):
             files = [filename]
         else:
             files = ['/etc/clustershell/clush.conf',
-                     os.path.expanduser('~/.clush.conf')]
+                     # deprecated user config, kept in 1.x for 1.6 compat
+                     expanduser('~/.clush.conf'),
+                     # default pip --user config file
+                     expanduser('~/.local/etc/clustershell/clush.conf'),
+                     # per-user clush.conf config top override
+                     os.path.join(os.environ.get('XDG_CONFIG_HOME',
+                                                 expanduser('~/.config')),
+                                  'clustershell', 'clush.conf')]
         self.read(files)
 
         # Apply command line overrides

tests/CLIConfigTest.py

@@ -1,17 +1,20 @@
 #!/usr/bin/env python
 # ClusterShell.CLI.Config test suite
-# Written by S. Thiell 2010-09-19
+# Written by S. Thiell
 
 
 """Unit test for CLI.Config"""
 
 import resource
+import os.path
+import shutil
 import sys
 import tempfile
 import unittest
 
 sys.path.insert(0, '../lib')
 
+from TLib import make_temp_dir
 
 from ClusterShell.CLI.Clush import set_fdlimit
 from ClusterShell.CLI.Config import ClushConfig, ClushConfigError
@@ -293,7 +296,59 @@ def testClushConfigWithInstalledConfig(self):
         config = ClushConfig(options)
         self.assert_(config != None)
 
+    def testClushConfigUserOverride(self):
+        """test CLI.Config.ClushConfig (XDG_CONFIG_HOME user config)"""
 
-if __name__ == '__main__':
-    suites = [unittest.TestLoader().loadTestsFromTestCase(CLIClushConfigTest)]
-    unittest.TextTestRunner(verbosity=2).run(unittest.TestSuite(suites))
+        # XXX Test should be improved when CLUSTERSHELL_CONFIG is available
+        # Improvement: override CLUSTERSHELL_CONFIG and set a sys clush config
+        # then verify that user config overrides CLUSTERSHELL_CONFIG as
+        # expected...
+        # For now, it has been tested manually. This test only really only
+        # ensures that user config is taken into account.
+
+        xdg_config_home_save = os.environ.get('XDG_CONFIG_HOME')
+
+        # Create fake XDG_CONFIG_HOME
+        dname = make_temp_dir()
+        try:
+            os.environ['XDG_CONFIG_HOME'] = dname
+
+            # create $XDG_CONFIG_HOME/clustershell/clush.conf
+            usercfgdir = os.path.join(dname, 'clustershell')
+            os.mkdir(usercfgdir)
+            cfgfile = open(os.path.join(usercfgdir, 'clush.conf'), 'w')
+            cfgfile.write("""
+[Main]
+fanout: 42
+connect_timeout: 14
+command_timeout: 0
+history_size: 100
+color: never
+verbosity: 2
+ssh_user: trump
+ssh_path: ~/bin/ssh
+ssh_options: -oSomeDummyUserOption=yes
+""")
+
+            cfgfile.flush()
+            parser = OptionParser("dummy")
+            parser.install_display_options(verbose_options=True)
+            parser.install_connector_options()
+            options, _ = parser.parse_args([])
+            config = ClushConfig(options) # filename=None to use defaults!
+            self.assertEqual(config.color, WHENCOLOR_CHOICES[0])
+            self.assertEqual(config.verbosity, VERB_VERB) # takes biggest
+            self.assertEqual(config.fanout, 42)
+            self.assertEqual(config.connect_timeout, 14)
+            self.assertEqual(config.command_timeout, 0)
+            self.assertEqual(config.ssh_user, 'trump')
+            self.assertEqual(config.ssh_path, '~/bin/ssh')
+            self.assertEqual(config.ssh_options, '-oSomeDummyUserOption=yes')
+            cfgfile.close()
+
+        finally:
+            if xdg_config_home_save:
+                os.environ['XDG_CONFIG_HOME'] = xdg_config_home_save
+            else:
+                del os.environ['XDG_CONFIG_HOME']
+            shutil.rmtree(dname, ignore_errors=True)