Add SCB support for ssh operations (#108)

Shell Control Box (SCB) is an activity monitoring appliance from Balabit (now One Identity) that controls privileged access to remote servers.
Added support for using ops with SCB for the following operations: ssh, tunnel, proxy, ansible play, run and sync

README.md

@@ -407,6 +407,13 @@ optional arguments:
   --nossh               Port tunnel a machine that does not have SSH. Implies
                         --ipaddress, and --tunnel; requires --local and
                         --remote
+  --keygen              Create a ssh keys pair to use with this infrastructure
+  --noscb               Disable use of Shell Control Box (SCB) even it is
+                        enabled in the cluster config
+  --auto_scb_port       When using Shell Control Box (SCB) and creating a
+                        proxy,a random port is generated, which will be used
+                        in the ssh config for all playbook, run and sync
+                        operations
 
     Examples:
         # SSH using current username as remote username
@@ -433,6 +440,19 @@ optional arguments:
 
         # Create a proxy to a remote server that listens on a local port
         ops clusters/qe1.yaml ssh --proxy --local 8080 bastion
+
+        # In case Shell Control Box (SCB) is configured and enabled on the cluster a proxy which
+        # will be used by all ops play, run and sync operations, can be created either using
+        # either the port configured the cluster config file or an auto generated port.
+        # In this case --local param must not be used
+        # Example for using the port configured in the cluster config
+        ops clusters/qe1.yaml ssh bastion --proxy
+        # Example for using the auto generated port
+        ops clusters/qe1.yaml ssh bastion --proxy --auto_scb_port
+
+
+        # Disable use of Shell Control Box (SCB) even it is enabled in the cluster config
+        ops clusters/qe1.yaml ssh bastion --noscb
 ```
 
 #### SSHPass
@@ -484,14 +504,46 @@ In case you want to use the OSX Keychain to store your password and reuse across
 
 1. Run `ops` tool
 
+#### SCB
+Shell Control Box (SCB) is an activity monitoring appliance from Balabit (now One Identity) that controls privileged access to remote servers.
+`ops` has support for using SCB as ssh proxy for the following operations: `ssh, tunnel, proxy, ansible play, run and sync`
+
+In order to use SCB an extra section needs to be added to the cluster config file:
+```
+scb:
+  enabled: true
+  host: "scb.example.com"
+  proxy_port: 2222 # optional
+```
+Having this config all ssh operations will be done via the scb host, unless the `--noscb` flag is used.
+
+When using `SCB`, `SSHPass` will not be used.
+
+For ansible `play`, `run` and `sync` operations to work via SCB a proxy needs to be created first and then run `ops` in a different terminal window or tab:
+```
+# 1. Create a proxy in a terminal window
+# Example for using the port configured in the cluster config
+ops clusters/qe1.yaml ssh bastion --proxy
+# Example for using the auto generated port
+ops clusters/qe1.yaml ssh bastion --proxy --auto_scb_port
+
+# 2. Run the play/run/sync command normally in a different terminal window or tab
+# A message will indicate the scb proxy is used
+ops clusters/qe1.yaml play ansible/plays/cluster/configure.yaml
+...
+Connecting via scb proxy at 127.0.0.1:2222.
+This proxy should have already been started and running in a different terminal window.
+If there are connection issues double check that the proxy is running.
+...
+```
 
 ### Play
 
 Run an ansible playbook.
 
 ```
 usage: ops cluster_config_path play [-h] [-e EXTRA_VARS] [--ask-sudo-pass]
-                                    [--limit LIMIT]
+                                    [--limit LIMIT] [--noscb]
                                     playbook_path
                                     [ansible_args [ansible_args ...]]
 
@@ -506,6 +558,8 @@ optional arguments:
   --ask-sudo-pass       Ask sudo pass for commands that need sudo
   --limit LIMIT         Limit run to a specific server subgroup. Eg: --limit
                         newton-dcs
+  --noscb               Disable use of Shell Control Box (SCB) even if it is
+                        enabled in the cluster config
 
     Examples:
         # Run an ansible playbook
@@ -530,6 +584,7 @@ Run a bash command on the selected nodes.
 
 ```
 usage: ops cluster_config_path run [-h] [--ask-sudo-pass] [--limit LIMIT]
+                                   [--noscb]
                                    host_pattern shell_command
                                    [extra_args [extra_args ...]]
 
@@ -543,6 +598,8 @@ optional arguments:
   --ask-sudo-pass  Ask sudo pass for commands that need sudo
   --limit LIMIT    Limit run to a specific server subgroup. Eg: --limit
                    newton-dcs
+  --noscb          Disable use of Shell Control Box (SCB) even if it is
+                   enabled in the cluster config
 
     Examples:
         # Last 5 installed packages on each host
@@ -562,7 +619,8 @@ optional arguments:
 Performs `rsync` to/from a given set of nodes.
 
 ```
-usage: ops cluster_config_path sync [-h] [-l USER] src dest [opts [opts ...]]
+usage: ops cluster_config_path sync [-h] [-l USER] [--noscb]
+                                    src dest [opts [opts ...]]
 
 positional arguments:
   src                   Source dir
@@ -572,6 +630,8 @@ positional arguments:
 optional arguments:
   -h, --help            show this help message and exit
   -l USER, --user USER  Value for remote user that will be used for ssh
+  --noscb               Disable use of Shell Control Box (SCB) even if it is
+                        enabled in the cluster config
 
         rsync wrapper for ops inventory conventions
 

src/ops/cli/parser.py

@@ -109,5 +109,8 @@ def configure_common_ansible_args(parser):
                         help='Ask sudo pass for commands that need sudo')
     parser.add_argument('--limit', type=str,
                         help='Limit run to a specific server subgroup. Eg: --limit newton-dcs')
+    parser.add_argument('--noscb', action='store_false', dest='use_scb',
+                        help='Disable use of Shell Control Box (SCB) even if '
+                             'it is enabled in the cluster config')
 
     return parser

src/ops/cli/playbook.py

@@ -10,11 +10,13 @@
 
 from .parser import SubParserConfig
 from .parser import configure_common_ansible_args, configure_common_arguments
+from ops.inventory.sshconfig import SshConfigGenerator
 import getpass
 import logging
 
 logger = logging.getLogger(__name__)
 
+
 class PlaybookParserConfig(SubParserConfig):
     def get_name(self):
         return 'play'
@@ -70,9 +72,12 @@ def __init__(self, ops_config, root_dir, inventory_generator,
 
     def run(self, args, extra_args):
         logger.info("Found extra_args %s", extra_args)
-        inventory_path, ssh_config_path = self.inventory_generator.generate()
+        inventory_path, ssh_config_paths = self.inventory_generator.generate()
+        ssh_config_path = SshConfigGenerator.get_ssh_config_path(self.cluster_config,
+                                                                 ssh_config_paths,
+                                                                 args.use_scb)
+        ssh_config = f"ANSIBLE_SSH_ARGS='-F {ssh_config_path}'"
 
-        ssh_config = "ANSIBLE_SSH_ARGS='-F %s'" % ssh_config_path
         ansible_config = "ANSIBLE_CONFIG=%s" % self.ops_config.ansible_config_path
 
         # default user: read from cluster then from ops config then local user

src/ops/cli/run.py

@@ -10,6 +10,7 @@
 
 import logging
 from .parser import configure_common_ansible_args, SubParserConfig
+from ops.inventory.sshconfig import SshConfigGenerator
 
 logger = logging.getLogger(__name__)
 
@@ -65,9 +66,11 @@ def __init__(self, ops_config, root_dir, inventory_generator,
 
     def run(self, args, extra_args):
         logger.info("Found extra_args %s", extra_args)
-        inventory_path, ssh_config_path = self.inventory_generator.generate()
+        inventory_path, ssh_config_paths = self.inventory_generator.generate()
         limit = args.host_pattern
-
+        ssh_config_path = SshConfigGenerator.get_ssh_config_path(self.cluster_config,
+                                                                 ssh_config_paths,
+                                                                 args.use_scb)
         extra_args = ' '.join(args.extra_args)
         command = """cd {root_dir}
 ANSIBLE_SSH_ARGS='-F {ssh_config}' ANSIBLE_CONFIG={ansible_config_path} ansible -i {inventory_path} '{limit}' \\

src/ops/cli/ssh.py

@@ -14,6 +14,7 @@
 from .parser import SubParserConfig
 from .parser import configure_common_arguments
 from ansible.inventory.host import Host
+from ops.inventory.sshconfig import SshConfigGenerator
 
 from . import err
 import sys
@@ -67,17 +68,30 @@ def configure(self, parser):
         parser.add_argument(
             '--proxy',
             action="store_true",
-            help="Use SSH proxy, must pass --local")
+            help="Use SSH proxy, must pass --local or when using scb, either --auto_scb_port or "
+                 "no extra option which will use scb.proxy_port from cluster_config")
         parser.add_argument(
             '--nossh',
             action="store_true",
-            help="Port tunnel a machine that does not have SSH. " 
+            help="Port tunnel a machine that does not have SSH. "
                  "Implies --ipaddress, and --tunnel; requires --local and --remote"
                 )
         parser.add_argument(
             '--keygen',
             action='store_true',
             help='Create a ssh keys pair to use with this infrastructure')
+        parser.add_argument(
+            '--noscb',
+            action='store_false',
+            dest='use_scb',
+            help='Disable use of Shell Control Box (SCB) even if it is '
+                 'enabled in the cluster config')
+        parser.add_argument(
+            '--auto_scb_port',
+            action='store_true',
+            help='When using Shell Control Box (SCB) and creating a proxy,'
+                 'a random port is generated, which will be used in the ssh config '
+                 'for all playbook, run and sync operations')
 
     def get_help(self):
         return 'SSH or create an SSH tunnel to a server in the cluster'
@@ -111,6 +125,18 @@ def get_epilog(self):
         # Create a proxy to a remote server that listens on a local port
         ops clusters/qe1.yaml ssh --proxy --local 8080 bastion
         ops clusters/qe1.yaml ssh --proxy --local 0.0.0.0:8080 bastion
+
+        # In case Shell Control Box (SCB) is configured and enabled on the cluster a proxy which
+        # will be used by all ops play, run and sync operations, can be created either using
+        # either the port configured the cluster config file or an auto generated port.
+        # In this case --local param must not be used
+        # Example for using the port configured in the cluster config
+        ops clusters/qe1.yaml ssh bastion --proxy
+        # Example for using the auto generated port
+        ops clusters/qe1.yaml ssh bastion --proxy --auto_scb_port
+
+        # Disable use of Shell Control Box (SCB) even it is enabled in the cluster config
+        ops clusters/qe1.yaml ssh bastion --noscb
         '''
 
 
@@ -162,9 +188,19 @@ def run(self, args, extra_args):
                 err('When using --tunnel or --nossh both the --local and --remote parameters are required')
                 sys.exit(2)
 
+        scb_settings = self.cluster_config.get('scb', {})
+        scb_enabled = scb_settings.get('enabled') and args.use_scb
+        scb_host = scb_settings.get('host') or self.ops_config.get('scb.host')
+        scb_proxy_port = scb_settings.get('proxy_port')
+
+        if scb_enabled and not scb_host:
+            err('When scb is enabled scb_host is required!')
+            sys.exit(2)
+
         if args.proxy:
-            if args.local is None:
-                err('When using --proxy the --local parameter is required')
+            if args.local is None and (args.auto_scb_port is False and not scb_proxy_port):
+                err('When using --proxy the --local parameter is required if not using '
+                    '--auto_scb_port and scb.proxy_port is not configured in the cluster config')
                 sys.exit(2)
 
         group = "%s,&%s" % (self.cluster_name, args.role)
@@ -203,12 +239,12 @@ def run(self, args, extra_args):
             bastion = self.ansible_inventory.get_hosts(
                 'bastion')[0].vars.get('ansible_ssh_host')
             host = Host(name=args.role)
-            ssh_host = '{}--{}'.format(bastion, host.name)
+            ssh_host = f'{bastion}--{host.name}'
         ssh_user = self.cluster_config.get('ssh_user') or self.ops_config.get(
             'ssh.user') or getpass.getuser()
         if args.user:
             ssh_user = args.user
-        if ssh_user and not '-l' in args.ssh_opts:
+        if ssh_user and '-l' not in args.ssh_opts:
             args.ssh_opts.extend(['-l', ssh_user])
 
         if args.nossh:
@@ -224,22 +260,44 @@ def run(self, args, extra_args):
         ssh_config = args.ssh_config or self.ops_config.get(
             'ssh.config') or self.ansible_inventory.get_ssh_config()
 
+        scb_ssh_host = None
+        if scb_enabled:
+            # scb->bastion->host vs scb->bastion
+            scb_delimiter = "--" if "--" in ssh_host else "@"
+            scb_ssh_host = f"{ssh_host}{scb_delimiter}{scb_host}"
+
         if args.tunnel:
             if args.ipaddress:
                 host_ip = host.vars.get('private_ip_address')
             else:
                 host_ip = 'localhost'
-            command = "ssh -F %s %s -4 -N -L %s:%s:%d" % (
-                ssh_config, ssh_host, args.local, host_ip, args.remote)
+            if scb_enabled:
+                command = f"ssh -F {ssh_config} {ssh_user}@{scb_ssh_host} " \
+                          f"-4 -T -L {args.local}:{host_ip}:{args.remote:d}"
+            else:
+                command = f"ssh -F {ssh_config} {ssh_host} " \
+                          f"-4 -N -L {args.local}:{host_ip}:{args.remote:d}"
         else:
-            command = "ssh -F %s %s" % (ssh_config, ssh_host)
+            if scb_enabled:
+                command = f"ssh -F {ssh_config} {ssh_user}@{scb_ssh_host}"
+            else:
+                command = f"ssh -F {ssh_config} {ssh_host}"
 
         if args.proxy:
-            command = "ssh -F %s %s -4 -N -D %s -f -o 'ExitOnForwardFailure yes'" % (
-                ssh_config, ssh_host, args.local)
+            if scb_enabled:
+                proxy_port = args.local or SshConfigGenerator.generate_ssh_scb_proxy_port(
+                    self.ansible_inventory.generated_path.rstrip("/inventory"),
+                    args.auto_scb_port,
+                    scb_proxy_port
+                )
+                command = f"ssh -F {ssh_config} {ssh_user}@{scb_ssh_host} " \
+                          f"-4 -T -D {proxy_port} -o 'ExitOnForwardFailure yes'"
+            else:
+                command = f"ssh -F {ssh_config} {ssh_host} " \
+                          f"-4 -N -D {args.local} -f -o 'ExitOnForwardFailure yes'"
 
         if args.ssh_opts:
-            command += " " + " ".join(args.ssh_opts)
+            command = f"{command} {' '.join(args.ssh_opts)}"
 
         # Check if optional sshpass is available and print info message
         sshpass_path = os.path.expanduser("~/bin/sshpass")

src/ops/cli/sync.py

@@ -14,9 +14,11 @@
 
 from .parser import SubParserConfig
 from . import *
+from ops.inventory.sshconfig import SshConfigGenerator
 
 logger = logging.getLogger(__name__)
 
+
 class SyncParserConfig(SubParserConfig):
     def configure(self, parser):
         parser.add_argument(
@@ -26,6 +28,9 @@ def configure(self, parser):
             help='Value for remote user that will be used for ssh')
         parser.add_argument('src', type=str, help='Source dir')
         parser.add_argument('dest', type=str, help='Dest dir')
+        parser.add_argument('--noscb', action='store_false', dest='use_scb',
+                            help='Disable use of Shell Control Box (SCB) '
+                                 'even if it is enabled in the cluster config')
         parser.add_argument(
             'opts',
             default=['-va --progress'],
@@ -68,10 +73,13 @@ def __init__(self, cluster_config, root_dir,
 
     def run(self, args, extra_args):
         logger.info("Found extra_args %s", extra_args)
-        inventory_path, ssh_config_path = self.inventory_generator.generate()
+        inventory_path, ssh_config_paths = self.inventory_generator.generate()
         src = PathExpr(args.src)
         dest = PathExpr(args.dest)
 
+        ssh_config_path = SshConfigGenerator.get_ssh_config_path(self.cluster_config,
+                                                                 ssh_config_paths,
+                                                                 args.use_scb)
         if src.is_remote and dest.is_remote:
             display(
                 'Too remote expressions are not allowed',

src/ops/data/ssh/ssh.config

@@ -9,6 +9,16 @@ Host *
     StrictHostKeyChecking  no
     TCPKeepAlive           yes
 
+Host *--*--*
+    ForwardAgent           yes
+    LogLevel               QUIET
+    ProxyCommand           ssh -o StrictHostKeyChecking=no -o ForwardAgent=yes -A %r@$(echo %h | sed -e 's/--.*//g')@$(echo %h | sed -e 's/.*--//g') nc $(echo %h | sed -e 's/.*--\(.*\)--.*/\1/') %p
+    SendEnv                LANG LC_*
+    ServerAliveCountMax    2
+    ServerAliveInterval    30
+    StrictHostKeyChecking  no
+    TCPKeepAlive           yes
+
 Host *--*
     ForwardAgent           yes
     LogLevel               QUIET