Skip to content
This repository

command-line tool multiput #171

Open
wants to merge 2 commits into from

1 participant

Stephen A. Balaban
Stephen A. Balaban

1) Added put functionality with multiple nodes using the --multi or -m option.
I hope to soon implement a put command with an --all option that will put
the file to all active nodes in a cluster.
2) I updated the Sphinx documentation to reflect these changes.

Stephen A. Balaban stephenbalaban command-line tool:
1) Added put functionality with multiple nodes using the --multi or -m option.
   I hope to soon implement a put command with an --all option that will put
   the file to all active nodes in a cluster.
2) I updated the Sphinx documentation to reflect these changes.
64155ad
Stephen A. Balaban stephenbalaban Forgot to run ./check.py (code now passes) [sorry about that] 57fde90
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 1 author.

Nov 16, 2012
Stephen A. Balaban stephenbalaban command-line tool:
1) Added put functionality with multiple nodes using the --multi or -m option.
   I hope to soon implement a put command with an --all option that will put
   the file to all active nodes in a cluster.
2) I updated the Sphinx documentation to reflect these changes.
64155ad
Nov 17, 2012
Stephen A. Balaban stephenbalaban Forgot to run ./check.py (code now passes) [sorry about that] 57fde90
This page is out of date. Refresh to see the latest.
5 docs/sphinx/manual/putget.rst
Source Rendered
@@ -23,6 +23,11 @@ To copy files to a different cluster node, use the ``--node`` (``-n``) option::
23 23
24 24 $ starcluster put mycluster --node node001 /local/path /remote/path
25 25
  26 +To copy files to multiple cluster nodes, provide comma separated nodes to the
  27 +``--multi`` (``-m``) option::
  28 +
  29 + $ starcluster put mycluster --multi master,node001 /local/path /remote/path
  30 +
26 31 *********************************
27 32 Copying Data from a Cluster (get)
28 33 *********************************
41 starcluster/commands/put.py
@@ -4,6 +4,18 @@
4 4 from completers import ClusterCompleter
5 5
6 6
  7 +def list_csv(csv_string):
  8 + """
  9 + For parsing comma separated values in --multi e.g. --multi master,node001
  10 + CommaSeparatedValues -> Nodename Generator
  11 +
  12 + Removes any empty strings from trailing commas
  13 + >>> list_csv("master,node001,")
  14 + ["master", "node001"]
  15 + """
  16 + return filter(bool, [name.strip() for name in csv_string.split(',')])
  17 +
  18 +
7 19 class CmdPut(ClusterCompleter):
8 20 """
9 21 put [options] <cluster_tag> [<local_file_or_dir> ...] <remote_destination>
@@ -24,6 +36,10 @@ class CmdPut(ClusterCompleter):
24 36 # Copy a file or dir to a node (node001 in this example)
25 37 $ starcluster put mycluster --node node001 /local/path /remote/path
26 38
  39 + # Copy a file or dir to multiple nodes (master and node001 in this
  40 + # example)
  41 + $ starcluster put mycluster -m master,node001 /local/path /remote/path
  42 +
27 43
28 44 This will copy a file or directory to the remote server
29 45 """
@@ -34,6 +50,17 @@ def addopts(self, parser):
34 50 help="Transfer files as USER ")
35 51 parser.add_option("-n", "--node", dest="node", default="master",
36 52 help="Transfer files to NODE (defaults to master)")
  53 + parser.add_option("-m", "--multi", dest="multi", default=None,
  54 + help="Transfer files to multiple NODEs (defaults to"
  55 + + "master)")
  56 +
  57 + def put(self, node, rpath, lpaths):
  58 + if self.opts.user:
  59 + node.ssh.switch_user(self.opts.user)
  60 + if len(lpaths) > 1 and not node.ssh.isdir(rpath):
  61 + msg = "Remote path in %s does not exist: %s" % (node, rpath)
  62 + raise exception.BaseException(msg)
  63 + node.ssh.put(lpaths, rpath)
37 64
38 65 def execute(self, args):
39 66 if len(args) < 3:
@@ -47,10 +74,10 @@ def execute(self, args):
47 74 raise exception.BaseException(
48 75 "Local file or directory does not exist: %s" % lpath)
49 76 cl = self.cm.get_cluster(ctag)
50   - node = cl.get_node_by_alias(self.opts.node)
51   - if self.opts.user:
52   - node.ssh.switch_user(self.opts.user)
53   - if len(lpaths) > 1 and not node.ssh.isdir(rpath):
54   - raise exception.BaseException("Remote path does not exist: %s" %
55   - rpath)
56   - node.ssh.put(lpaths, rpath)
  77 + if self.opts.multi:
  78 + nodes = [cl.get_node_by_alias(nodename) for nodename in
  79 + list_csv(self.opts.multi)]
  80 + else:
  81 + nodes = [cl.get_node_by_alias(self.opts.node)]
  82 + for node in nodes:
  83 + self.put(node, rpath, lpaths)

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.