Implement 'ngrok' (or similar) to allow web access to a local container.

[owenbush]

Feature Request

I'd love to see something like https://ngrok.com/ available to use with ddev to allow web access to a local container. I used this previously to allow Recurly to use webhooks to post data back to my local machine during testing and it was a lifesaver. Another good example of a use case is to be able to use something like BrowserStack to check a local site in multiple browsers prior to deploying to a web-accessible staging/production environment.

What you expected to happen:

ddev ngrok --auth="username:password" 80
Would expose the webcontainer port 80 over the internet behind simple HTTP authentication to prevent prying eyes.

Related source links or issues:

https://ngrok.com/

[rickmanelius]

Thanks, @owenbush! I also followed up with Owen in Slack regarding some additional details that are in his company's internal documentation. He sent the link over and I'll pull some of the pertinent information into this thread for additional context.

[rickmanelius]

PREREQUISITES

First of all, there should be the following file in the root of the repository ngrok-setup.sh

If that file does not exist then you can create it with 755 permissions and with the following content

if [ ! -f ./ngrok ]; then
  echo "Downloading ngrok..."
  curl -0 'https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip' > ngrok.zip
  echo "Unzipping ngrok..."
  unzip ngrok.zip
  echo "Removing zip..."
  rm ngrok.zip
  echo "Opening ngrok.com..."
  open https://dashboard.ngrok.com/user/login
fi

STEP 1

Navigate to the root of your repository and run the following command $ ./ngrok-setup.sh

STEP 2

When you ran the command in Step 1 your browser should have automatically opened up the ngrok website in your default browser. If this did not happen then navigate to:
https://dashboard.ngrok.com/user/login

STEP 3

If you do not have an account with ngrok then register for one, they are free. If you do have an account log in to your account and navigate to
https://dashboard.ngrok.com/get-started

STEP 4

Once on your dashboard you should see a command you need to execute to authenticate your ngrok binary that we downloaded earlier with the ngrok services. To execute this command you will need to be SSH'd on to your container.

$ ssh [sitename]
/var/www/html/docroot# cd ..
/var/www/html# ./ngrok authtoken [auth-token-from-ngrok.com]

STEP 5

Once you have successfully authenticated your ngrok binary with the ngrok website you can now make use of the functionality to expose your site over the internet. 
Note: You should always specify basic authentication to secure your exposed tunnel.

/var/www/html# ./ngrok http -auth="username:password" 80
ngrok by @inconshreveable                                                            (Ctrl+C to quit)

Session Status                online
Account                       owen@company.com (Plan: Free)
Version                       2.2.4
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://c08af649.ngrok.io -> localhost:80
Forwarding                    https://c08af649.ngrok.io -> localhost:80

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

In the above output, your web-accessible tunnel to your site is  http://c08af649.ngrok.io

STEP 6

Your tunnel will remain available and active so long as you keep the window with the running ngrok binary open. If you wish to disable your tunnel you can do so at any time by pressing CTRL+C to quit.

[rfay]

@owenbush It seems to me there wouldn't be anything required of ddev to accomplish this.

If I do a docker ps with my site running I see
31abe125d844 drud/nginx-php-fpm7-local:v0.7.3 "/start.sh" 4 hours ago Up About an hour (healthy) 443/tcp, 0.0.0.0:32770->80/tcp, 0.0.0.0:32769->8025/tcp ddev-randyfay-web

And if I hit localhost:32770 with the web browser everything is there just peachy. Doesn't that mean that ngrok would work fine as is?

I will open an issue for ddev describe to report the (unrouted) web port (#376)

[owenbush]

@rfay, the issue is that localhost:32270 is not web-accessible for a third party service/server. For example a webhook callback from Recurly 'localhost' wouldn't map to my local machine. ngrok the service creates a DNS resolvable public URL for access like: http://c08af649.ngrok.io

@rickmanelius the one thing I will state about those instructions I created before is that they were for a one-time setup in a local repo and it worked fine, but what would be ideal is that ngrok is baked into ddev and we therefore don't need to run that custom setup script I created.

[rfay]

Right @owenbush - so for example, you can use https://562fa535.ngrok.io/ right now to hit my local ddev instance for randyfay.ddev.local. And it will answer to any hostname with no problems.

I could even set up a CNAME to the ngrok URL and things would work just fine.

Let me know what I'm missing here.

[owenbush]

@rfay Yea setting up ngrok manually is completely doable. I have done is myself before. I just felt it was a nice out-of-the-box feature for ddev to facilitate. There is nothing particularly difficult about setting up ngrok manually if you have CLI experience and I could certainly put together a set of instructions for use with ddev if that was desired.

[rfay]

For the record, all I did was:

1. install ngrok with homebrew (brew cask install ngrok)
2. Create an ngrok account at ngrok.io (turned out I could have just clicked the github link)
3. issue the offered ngrok authtoken command
4. ngrok http 32770 (where 32770 was provided by the docker ps | grep nginx reported mapping to port 80.
5. Remove the $base_url from settings.php (for css to work)

[owenbush]

Yea, its pretty simple for those who are comfortable doing that. My thought was more for those who were not, but wanted a seamless way of doing so with ddev for testing, webhook or sharing reasons. By having this baked into ddev and having a ddev command to produce a web-accessible URL they can use however they choose would save them from having to install ngrok themselves.

ddev ngrok run

Which would display something like
http://c08af649.ngrok.io

As I said, I know ddev/docker is compatible and that is not where I was heading with this feature request. The feature request was to somehow have ngrok part of the core ddev offering so people do not need to worry, or even know, about what is going on under the hood.

I understand if this is a bit too bespoke a thing to bake in, it would be a nice to have for me. But I can manually set it up if not.

[rfay]

I also very much appreciate you motivating me to finally give ngrok a try. It was awesomely more easier than the old ssh tunnel tricks I used to use for this sort of thing. Of course... I have to trust ngrok.io, which is a lot to ask. But so easy! Thanks for getting me over the hump!

[rickmanelius]

Related to #416

[rickmanelius]

Just noting that this feature has come up a few times in recent conversation. I think this would be of value in parallel with the hosting provider conversations because it provides yet another way to get work viewable upstream. Tagging w/v1.4.0 for now.

[beathorst]

There is another service calls http://xip.io e.g is used by https://www.mamp.info/en/mamp-pro/

[rfay]

Some general related notes:

There is a good summary of techniques on https://medium.com/botfuel/how-to-expose-a-local-development-server-to-the-internet-c31532d741cc and I see a number of others - It suggests:

• ngrok
• localtunnel
• Just proxying with an ssh tunnel.
  So there are at least two variants of what you're talking about:

1. Just proxying to an exposed port on your host machine for access by another machine on your local network.
2. Proxying your local development environment to a host on the internet for access from anywhere.
   For either approach, we need to figure out what port to proxy.
   If you can just use http, I'd proxy the localhost port (which goes directly to the web container and doesn't care what the hostname in the URL is). So if ddev describe shows http://d7git.ddev.local:8080, https://d7git.ddev.local:8443, http://127.0.0.1:32827, use the 127.0.0.1 port (32827 in this case). If you can do this, you won't have to fiddle with faking the hostname on the host from which you're going to access this.
   So for option 1 (just exposing on another port on your machine), use any of these technqiues. I'll use the socat approach on macOS (brew install socat).
   socat tcp-listen:8889,reuseaddr,fork tcp:localhost:32827
   where 32827 is the port listed by ddev describe as localhost access, and 8889 is the port you want to expose to others. Then find out your local network IP address (Use ifconfig or other techniques) and others can access your ddev project with that. For example, my setup today would be http://10.150.150.87:32827/
3. Proxying your project for somebody else to use on the internet, via ssh tunneling:
   ssh -R :9101:localhost:32827 user@host.example.com
   That will tunnel your local port 32827 (check your own ddev describe for this) to port 9101 on the remote host.example.com. Note that you'll likely have to

• Change the firewall config to enable access to that port on the remote host
• Enable GatewayPorts yes in the host's sshd config.

[rfay]

Another option for this is https://github.com/alexellis/inlets

[bennlich]

Hmm. What if I don't really care about security, and I just want to be able to send people the IP address of the droplet I'm developing on. Seems like there should be an easy way to modify the ddev nginx config to respond to incoming requests? It's already listening on port 80, right?

[rfay]

Hi @bennlich - We've had router_bind_all_interfaces: true as an option for the global_config.yaml for some time, that allows anybody on your local network to access things. But it still requires name resolution because the router needs to know the name of the project you're hitting. So you have to add PROJECTNAME.ddev.site to the client machine, or run your own DNS, or ...

If you'd rather access a particular project directly by port, not using the name of the project, you can do that with the strategy in #2076 (comment)