Add 'tahoe invite' and 'tahoe create-node --join' commands

This opens a wormhole and sends appropriate JSON down
it to a tahoe-gui using a wormhole server running on
tahoe-lafs.org

The other end uses the 'tahoe create-node' command (with
new --join option) to read the configuration JSON from
a 'tahoe invite' command

docs/index.rst

@@ -13,6 +13,7 @@ Contents:
    about
    INSTALL
    running
+   magic-wormhole-invites
    configuration
    architecture
 

docs/magic-folder-howto.rst

@@ -1,3 +1,5 @@
+.. _magic-folder-howto:
+
 =========================
 Magic Folder Set-up Howto
 =========================

docs/magic-wormhole-invites.rst

@@ -0,0 +1,73 @@
+**********************
+Magic Wormhole Invites
+**********************
+
+Magic Wormhole
+==============
+
+`magic wormhole`_ is a server and a client which together use Password
+Authenticated Key Exchange (PAKE) to use a short code to establish a
+secure channel between two computers. These codes are one-time use and
+an attacker gets at most one "guess", thus allowing low-entropy codes
+to be used.
+
+.. _magic wormhole: https://github.com/warner/magic-wormhole#design
+
+
+Invites and Joins
+=================
+
+Inside Tahoe-LAFS we are using a channel created using `magic
+wormhole`_ to exchange configuration and the secret fURL of the
+Introducer with new clients. In the future, we would like to make the
+Magic Folder (:ref:`Magic Folder HOWTO <magic-folder-howto>`) invites and joins work this way
+as well.
+
+This is a two-part process. Alice runs a grid and wishes to have her
+friend Bob use it as a client. She runs ``tahoe invite bob`` which
+will print out a short "wormhole code" like ``2-unicorn-quiver``. You
+may also include some options for total, happy and needed shares if
+you like.
+
+Alice then transmits this one-time secret code to Bob. Alice must keep
+her command running until Bob has done his step as it is waiting until
+a secure channel is established before sending the data.
+
+Bob then runs ``tahoe create-client --join <secret code>`` with any
+other options he likes. This will "use up" the code establishing a
+secure session with Alice's computer. If an attacker tries to guess
+the code, they get only once chance to do so (and then Bob's side will
+fail). Once Bob's computer has connected to Alice's computer, the two
+computers performs the protocol described below, resulting in some
+JSON with the Introducer fURL, nickname and any other options being
+sent to Bob's computer. The ``tahoe create-client`` command then uses
+these options to set up Bob's client.
+
+
+
+Tahoe-LAFS Secret Exchange
+==========================
+
+The protocol that the Alice (the one doing the invite) and Bob (the
+one being invited) sides perform once a magic wormhole secure channel
+has been established goes as follows:
+
+Alice and Bob both immediately send an "abilities" message as
+JSON. For Alice this is ``{"abilities": {"server-v1": {}}}``. For Bob,
+this is ``{"abilities": {"client-v1": {}}}``.
+
+After receiving the message from the other side and confirming the
+expected protocol, Alice transmits the configuration JSON::
+
+    {
+        "needed": 3,
+        "total": 10,
+        "happy": 7,
+        "nickname": "bob",
+        "introducer": "pb://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx@example.com:41505/yyyyyyyyyyyyyyyyyyyyyyy"
+    }
+
+Both sides then disconnect.
+
+As you can see, there is room for future revisions of the protocol but
+as of yet none have been sketched out.

docs/running.rst

@@ -26,6 +26,39 @@ grid`_ as you only need to create a client node. When you want to create your
 own grid you'll need to create the introducer and several initial storage
 nodes (see the note about small grids below).
 
+
+Being Introduced to a Grid
+--------------------------
+
+A collection of Tahoe servers is called a Grid and usually has 1
+Introducer (but sometimes more, and it's possible to run with zero). The
+Introducer announces which storage servers constitute the Grid and how to
+contact them. There is a secret "fURL" you need to know to talk to the
+Introducer.
+
+One way to get this secret is using traditional tools such as encrypted
+email, encrypted instant-messaging, etcetera. It is important to transmit
+this fURL secretly as knowing it gives you access to the Grid.
+
+An additional way to share the fURL securely is via `magic
+wormhole`_. This uses a weak one-time password and a server on the
+internet (at `wormhole.tahoe-lafs.org`) to open a secure channel between
+two computers. In Tahoe-LAFS this functions via the commands `tahoe
+invite` and `tahoe create-client --join`. A person who already has access
+to a Grid can use `tahoe invite` to create one end of the `magic
+wormhole`_ and then transmits some JSON (including the Introducer's
+secret fURL) to the other end. `tahoe invite` will print a one-time
+secret code; you must then communicate this code to the person who will
+join the Grid.
+
+The other end of the `magic wormhole`_ in this case is `tahoe
+create-client --join <one-time code>`, where the person being invited
+types in the code they were given. Ideally, this code would be
+transmitted securely. It is, however, only useful exactly once. Also, it
+is much easier to transcribe by a human. Codes look like
+`7-surrender-tunnel` (a short number and two words).
+
+
 Running a Client
 ----------------
 
@@ -38,6 +71,11 @@ To construct a client node, run “``tahoe create-client``”, which will create
 it will do is connect to the introducer and get itself connected to all other
 nodes on the grid.
 
+Some Grids use "magic wormhole" one-time codes to configure the basic
+options. In such a case you use ``tahoe create-client --join
+<one-time-code>`` and do not have to do any of the ``tahoe.cfg`` editing
+mentioned above.
+
 By default, “``tahoe create-client``” creates a client-only node, that
 does not offer its disk space to other nodes. To configure other behavior,
 use “``tahoe create-node``” or see :doc:`configuration`.
@@ -47,6 +85,7 @@ On Unix, you can run it in the background instead by using the
 “``tahoe start``” command. To stop a node started in this way, use
 “``tahoe stop``”. ``tahoe --help`` gives a summary of all commands.
 
+
 Running a Server or Introducer
 ------------------------------
 
@@ -67,6 +106,13 @@ URL the other nodes must use in order to connect to this introducer.
 (Note that “``tahoe run .``” doesn't work for introducers, this is a
 known issue: `#937`_.)
 
+You can distribute your Introducer fURL securely to new clients by using
+the ``tahoe invite`` command. This will prepare some JSON to send to the
+other side, request a `magic wormhole`_ code from
+``wormhole.tahoe-lafs.org`` and print it out to the terminal. This
+one-time code should be transmitted to the user of the client, who can
+then run ``tahoe create-client --join <one-time-code>``.
+
 Storage servers are created the same way: ``tahoe create-node
 --hostname=HOSTNAME .`` from a new directory. You'll need to provide the
 introducer FURL (either as a ``--introducer=`` argument, or by editing
@@ -79,6 +125,7 @@ Tahoe-LAFS.
 .. _public test grid: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TestGrid
 .. _TestGrid page: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TestGrid
 .. _#937:  https://tahoe-lafs.org/trac/tahoe-lafs/ticket/937
+.. _magic wormhole: https://magic-wormhole.io/
 
 
 A note about small grids

integration/test_servers_of_happiness.py

@@ -24,8 +24,8 @@ def test_upload_immutable(reactor, temp_dir, introducer_furl, flog_gatherer, sto
 
     node_dir = join(temp_dir, 'edna')
 
-    print("waiting 5 seconds unil we're maybe ready")
-    yield task.deferLater(reactor, 5, lambda: None)
+    print("waiting 10 seconds unil we're maybe ready")
+    yield task.deferLater(reactor, 10, lambda: None)
 
     # upload a file, which should fail because we have don't have 7
     # storage servers (but happiness is set to 7)

src/allmydata/_auto_deps.py

@@ -91,6 +91,9 @@
     "PyYAML >= 3.11",
 
     "six >= 1.10.0",
+
+    # for 'tahoe invite' and 'tahoe join'
+    "magic-wormhole >= 0.10.2",
 ]
 
 # Includes some indirect dependencies, but does not include allmydata.

src/allmydata/scripts/common.py

@@ -1,6 +1,8 @@
 
 import os, sys, urllib, textwrap
 import codecs
+from ConfigParser import NoSectionError
+from os.path import join
 from twisted.python import usage
 from allmydata.util.assertutil import precondition
 from allmydata.util.encodingutil import unicode_to_url, quote_output, \
@@ -103,6 +105,26 @@ def getSynopsis(self):
 DEFAULT_ALIAS = u"tahoe"
 
 
+def get_introducer_furl(nodedir, config):
+    """
+    :return: the introducer FURL for the given node (no matter if it's
+        a client-type node or an introducer itself)
+    """
+    try:
+        introducer_furl = config.get('client', 'introducer.furl')
+    except NoSectionError:
+        # we're not a client; maybe this is running *on* the introducer?
+        try:
+            with open(join(nodedir, "private", "introducer.furl"), "r") as f:
+                introducer_furl = f.read().strip()
+        except IOError:
+            raise Exception(
+                "Can't find introducer FURL in tahoe.cfg nor "
+                "{}/private/introducer.furl".format(nodedir)
+            )
+    return introducer_furl
+
+
 def get_aliases(nodedir):
     aliases = {}
     aliasfile = os.path.join(nodedir, "private", "aliases")

src/allmydata/scripts/create_node.py

@@ -1,11 +1,14 @@
 import os
+import json
+
 from twisted.internet import reactor, defer
 from twisted.python.usage import UsageError
 from allmydata.scripts.common import BasedirOptions, NoDefaultBasedirOptions
 from allmydata.scripts.default_nodedir import _default_nodedir
 from allmydata.util.assertutil import precondition
-from allmydata.util.encodingutil import listdir_unicode, argv_to_unicode, quote_local_unicode_path
+from allmydata.util.encodingutil import listdir_unicode, argv_to_unicode, quote_local_unicode_path, get_io_encoding
 from allmydata.util import fileutil, i2p_provider, iputil, tor_provider
+from wormhole import wormhole
 
 
 dummy_tac = """
@@ -80,7 +83,7 @@ def validate_where_options(o):
     else:
         # no --location and --port? expect --listen= (maybe the default), and
         # --listen=tcp requires --hostname. But --listen=none is special.
-        if o['listen'] != "none":
+        if o['listen'] != "none" and o.get('join', None) is None:
             listeners = o['listen'].split(",")
             for l in listeners:
                 if l not in ["tcp", "tor", "i2p"]:
@@ -155,6 +158,7 @@ class CreateClientOptions(_CreateBaseOptions):
         ("shares-needed", None, 3, "Needed shares required for uploaded files."),
         ("shares-happy", None, 7, "How many servers new files must be placed on."),
         ("shares-total", None, 10, "Total shares required for uploaded files."),
+        ("join", None, None, "Join a grid with the given Invite Code."),
         ]
 
     # This is overridden in order to ensure we get a "Wrong number of
@@ -325,6 +329,45 @@ def write_client_config(c, config):
     c.write("enabled = false\n")
     c.write("\n")
 
+
+@defer.inlineCallbacks
+def _get_config_via_wormhole(config):
+    out = config.stdout
+    print >>out, "Opening wormhole with code '{}'".format(config['join'])
+    relay_url = config.parent['wormhole-server']
+    print >>out, "Connecting to '{}'".format(relay_url)
+
+    wh = wormhole.create(
+        appid=config.parent['wormhole-invite-appid'],
+        relay_url=relay_url,
+        reactor=reactor,
+    )
+    code = unicode(config['join'])
+    wh.set_code(code)
+    yield wh.get_welcome()
+    print >>out, "Connected to wormhole server"
+
+    intro = {
+        u"abilities": {
+            "client-v1": {},
+        }
+    }
+    wh.send_message(json.dumps(intro))
+
+    server_intro = yield wh.get_message()
+    server_intro = json.loads(server_intro)
+
+    print >>out, "  received server introduction"
+    if u'abilities' not in server_intro:
+        raise RuntimeError("  Expected 'abilities' in server introduction")
+    if u'server-v1' not in server_intro['abilities']:
+        raise RuntimeError("  Expected 'server-v1' in server abilities")
+
+    remote_data = yield wh.get_message()
+    print >>out, "  received configuration"
+    defer.returnValue(json.loads(remote_data))
+
+
 @defer.inlineCallbacks
 def create_node(config):
     out = config.stdout
@@ -344,6 +387,39 @@ def create_node(config):
         os.mkdir(basedir)
     write_tac(basedir, "client")
 
+    # if we're doing magic-wormhole stuff, do it now
+    if config['join'] is not None:
+        try:
+            remote_config = yield _get_config_via_wormhole(config)
+        except RuntimeError as e:
+            print >>err, str(e)
+            defer.returnValue(1)
+
+        # configuration we'll allow the inviter to set
+        whitelist = [
+            'shares-happy', 'shares-needed', 'shares-total',
+            'introducer', 'nickname',
+        ]
+        sensitive_keys = ['introducer']
+
+        print >>out, "Encoding: {shares-needed} of {shares-total} shares, on at least {shares-happy} servers".format(**remote_config)
+        print >>out, "Overriding the following config:"
+
+        for k in whitelist:
+            v = remote_config.get(k, None)
+            if v is not None:
+                # we're faking usually argv-supplied options :/
+                if isinstance(v, unicode):
+                    v = v.encode(get_io_encoding())
+                config[k] = v
+                if k not in sensitive_keys:
+                    if k not in ['shares-happy', 'shares-total', 'shares-needed']:
+                        print >>out, "  {}: {}".format(k, v)
+                else:
+                    print >>out, "  {}: [sensitive data; see tahoe.cfg]".format(k)
+            else:
+                print >>out, "option '{}' not whitelisted; removing".format(k)
+
     fileutil.make_dirs(os.path.join(basedir, "private"), 0700)
     with open(os.path.join(basedir, "tahoe.cfg"), "w") as c:
         yield write_node_config(c, config)

src/allmydata/scripts/runner.py

@@ -7,7 +7,7 @@
 
 from allmydata.scripts.common import get_default_nodedir
 from allmydata.scripts import debug, create_node, startstop_node, cli, \
-    stats_gatherer, admin, magic_folder_cli
+    stats_gatherer, admin, magic_folder_cli, tahoe_invite
 from allmydata.util.encodingutil import quote_output, quote_local_unicode_path, get_io_encoding
 
 def GROUP(s):
@@ -47,6 +47,8 @@ class Options(usage.Options):
                     + GROUP("Using the file store")
                     +   cli.subCommands
                     +   magic_folder_cli.subCommands
+                    + GROUP("Grid Management")
+                    +   tahoe_invite.subCommands
                     )
 
     optFlags = [
@@ -56,6 +58,8 @@ class Options(usage.Options):
     ]
     optParameters = [
         ["node-directory", "d", None, NODEDIR_HELP],
+        ["wormhole-server", None, u"ws://wormhole.tahoe-lafs.org:4000/v1", "The magic wormhole server to use.", unicode],
+        ["wormhole-invite-appid", None, u"tahoe-lafs.org/invite", "The appid to use on the wormhole server.", unicode],
     ]
 
     def opt_version(self):
@@ -139,6 +143,8 @@ def dispatch(config,
         # same
         f0 = magic_folder_cli.dispatch[command]
         f = lambda so: threads.deferToThread(f0, so)
+    elif command in tahoe_invite.dispatch:
+        f = tahoe_invite.dispatch[command]
     else:
         raise usage.UsageError()