Improve install output #38

fortuna · 2018-03-23T21:41:21Z

With this change we display the step before it's done, making it clearer what's going on.
We are now also able to detect if the firewall is blocking the connection.

This should address many of the issues users are having.

@sandrigo FYI

Sample output

> Verifying that Docker is installed .......... OK
> Creating persistent state dir ............... OK
> Generating secret key ....................... OK
> Generating TLS certificate .................. OK
> Generating SHA-256 certificate fingerprint .. OK
> Starting Shadowbox .......................... OK
> Starting Watchtower ......................... OK
> Waiting for Shadowsocks to be healthy ....... OK
> Creating first user ......................... OK
> Adding API URL to config .................... OK
> Checking firewall ........................... BLOCKED

CONGRATULATIONS! Your Outline Server is up and running.

You won’t be able to access it externally, despite your server being correctly
set up, because this host machine has a firewall that is preventing incoming
connections to ports 12380 and 43732.

If you plan to have a single Access Key to access your server, opening those 
ports on TCP and UDP should suffice. If you plan on adding additional Access
Keys, you’ll have to open ports 1024 through 65535 on your firewall since the
Outline Server may allocate any of those ports to new Access Keys.


To manage your Outline Server, please copy the following text (including curly brackets) into Step 2 of the Outline Manager interface:

{
  "apiUrl": "https://100.000.000.000:12380/XXXXXXXX",
  "certSha256": "008F471BECDB3755936A476015DA8E756D1E1E71F764F83C4E7277EF482E0B60"
}

trevj · 2018-03-26T13:37:48Z

Note to self on how to test this:

curl -SL https://raw.githubusercontent.com/Jigsaw-Code/outline-server/fortuna-docker/src/server_manager/install_scripts/install_server.sh | bash

trevj · 2018-03-26T13:40:22Z

src/server_manager/install_scripts/install_server.sh

@@ -235,11 +268,17 @@ install_shadowbox() {
  # require new dependencies.
  cat <<END_OF_SERVER_OUTPUT

-Please copy the following configuration to your Outline Manager:
+CONGRATULATIONS! Your Outline Server is up and running.


Outline Server -> Outline server

(and throughout)

trevj · 2018-03-26T13:43:09Z

src/server_manager/install_scripts/install_server.sh

+set up, because this host machine has a firewall that is preventing incoming
+connections to ports ${SB_API_PORT} and ${ACCESS_KEY_PORT}.
+
+If you plan to have a single Access Key to access your server, opening those 


Access Key -> access key

trevj · 2018-03-26T13:45:45Z

src/server_manager/install_scripts/install_server.sh

+  log_for_sentry "Setting SB_PUBLIC_IP"
+  # TODO(fortuna): Make sure this is IPv4
+  readonly SB_PUBLIC_IP=${SB_PUBLIC_IP:-$(curl -4s https://ipinfo.io/ip)}
+


extra blank line

trevj · 2018-03-26T13:46:49Z

src/server_manager/install_scripts/install_server.sh


-  log_step "Creating first user"
+function check_firewall() {
+  if ! curl --max-time 5 --cacert "${SB_CERTIFICATE_FILE}" -s "${PUBLIC_API_URL}/access-keys" >/dev/null; then


It's pretty funny that we import the cert only for the connectivity check, not for creating the first user.

Maybe we shouldn't. The point of using localhost (which requires that we do --insecure) is that we avoid hitting firewall issues and get to install the server completely. The check for firewall can be done last, such that if the admin fixes it post-installation, the Server will work seamlessly.

I thought accessing via localhost was orthogonal to using the cert, or does the cert include the public IP?

The cert has the public ip only, so we need to use insecure mode.

We could potentially add localhost to the cert, but we need to figure it out.
It's unclear whether we should worry about a localhost connection being intercepted. You could have a malicious app take over all ports, but I think that may be outside our threat model.
In any case, adding too many options to the cert can make it fingerprintable.

I think it's important to test with the certs, to make sure it's set up properly.

Seems totally reasonable, and well worth a comment in the source :-)

trevj · 2018-03-26T13:49:06Z

src/server_manager/install_scripts/install_server.sh

+
+
+  if [[ ! "$SB_PUBLIC_IP" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+    echo "Invalid IP lookup result: $SB_PUBLIC_IP"


log_for_error, no?

trevj · 2018-03-26T13:51:28Z

src/server_manager/install_scripts/install_server.sh


-  log_step "Creating first user"
+function check_firewall() {


I would expect most firewalls to allow connections from localhost addresses, even if they're blocked from the outside. This will have a lot of false positives - but I guess as a placeholder it's okay.

I agree this may have some false positives. I have no idea how much, but this is definitely an improvement to what we have now.

I've added extra text about the router of cloud provider blocking in case that happens.

@dborkan had the excellent point that he's seen the converse: a firewall that allows connections from localhost but not externally. In that case, this advice is worse than what we have. I think you're right to just always tell the user which ports to open.

trevj · 2018-03-26T13:52:02Z

src/server_manager/install_scripts/install_server.sh

+
+  readonly PUBLIC_API_URL="https://${SB_PUBLIC_IP}:${SB_API_PORT}/${SB_API_PREFIX}"
+  readonly LOCAL_API_URL="https://localhost:${SB_API_PORT}/${SB_API_PREFIX}"
+  run_step "Waiting for Shadowsocks to be healthy" wait_shadowbox


Shadowbox to be healthy, right?

Changed to Outline Server

trevj · 2018-03-26T13:53:25Z

src/server_manager/install_scripts/install_server.sh

+
+${FIREWALL_STATUS}
+To manage your Outline Server, please copy the following text (including curly
+brackets) into Step 2 of the Outline Manager interface:


Presumably they've just come from the Outline Manager; I would re-word to something like: "into the Outline Manager".

The same screen on the Manager shows both Step 1 and Step 2. I think mentioning Step 2 may feel excessive, but makes it more explicit. The text was suggested by Santi.

I'm not going to hold this up on worsmithing but FYI "the Outline Manager interface" is redundant - it's just "the Outline Manager".

trevj · 2018-03-26T13:55:30Z

src/server_manager/install_scripts/install_server.sh

+     FIREWALL_STATUS="\
+You won’t be able to access it externally, despite your server being correctly
+set up, because this host machine has a firewall that is preventing incoming
+connections to ports ${SB_API_PORT} and ${ACCESS_KEY_PORT}.


I think right now you're only checking the Shadowbox port, no?

Yes. I'm assuming that both are blocked together.

trevj · 2018-03-26T18:39:48Z

@fortuna Please consider a squash commit, with a list summarising all of the changes (for example, you removed the call to ufw which is obviously a significant change but one I still missed first time around).

trevj · 2018-03-26T18:40:28Z

A few open issues which may benefit from this change:

Improve install output

6676f74

fortuna self-assigned this Mar 23, 2018

fortuna requested a review from trevj March 23, 2018 21:41

fortuna added the server/advanced install Issues with manual/advanced server installation label Mar 23, 2018

fortuna added 2 commits March 23, 2018 17:41

Merge branch 'master' into fortuna-docker

696479d

Show ports to open

10da623

fortuna mentioned this pull request Mar 23, 2018

Installation struck on Starting Watchtower OK #25

Closed

fortuna added 3 commits March 23, 2018 18:23

Wrap columns

feab2fd

Minor fix to spacing

f57603e

Merge branch 'master' into fortuna-docker

1d42b83

trevj suggested changes Mar 26, 2018

View reviewed changes

fortuna added 2 commits March 26, 2018 13:36

Address review comments

3c1c5b9

Add better docker test

8a04761

trevj approved these changes Mar 26, 2018

View reviewed changes

Merge branch 'master' into fortuna-docker

20969d7

fortuna merged commit d8d196e into master Mar 26, 2018

fortuna deleted the fortuna-docker branch March 26, 2018 21:54

This was referenced Mar 26, 2018

Tweak message #52

Merged

Stuck on "Starting Watchtower ......................... OK" #27

Closed

Server setup stuck at "starting watchtower" #22

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve install output #38

Improve install output #38

fortuna commented Mar 23, 2018 •

edited

trevj commented Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

ghost Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018 •

edited

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

trevj Mar 26, 2018

fortuna Mar 26, 2018

trevj Mar 26, 2018

trevj commented Mar 26, 2018

trevj commented Mar 26, 2018



		if [[ ! "$SB_PUBLIC_IP" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
		echo "Invalid IP lookup result: $SB_PUBLIC_IP"

Improve install output #38

Improve install output #38

Conversation

fortuna commented Mar 23, 2018 • edited

trevj commented Mar 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevj Mar 26, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevj commented Mar 26, 2018

trevj commented Mar 26, 2018

fortuna commented Mar 23, 2018 •

edited

trevj Mar 26, 2018 •

edited