make shell (docker-compose) fails on a host behind a proxy

[ffledgling]

Following the instructions on the getting started page on a Fedora 24 machine behind a proxy.

The host machine has both http_proxy and https_proxy set and exported.
Running make shell on such a host fails like so:

$ make shell
docker-compose -f tooling/docker/docker-compose.yml build es
Building es
Step 1 : FROM elasticsearch:1.4
 ---> 76f6ea8043c7
Step 2 : RUN plugin install elasticsearch/elasticsearch-lang-javascript/2.4.1
 ---> Running in cfe94f08f0c8
-> Installing elasticsearch/elasticsearch-lang-javascript/2.4.1...
Trying http://download.elasticsearch.org/elasticsearch/elasticsearch-lang-javascript/elasticsearch-lang-javascript-2.4.
1.zip...
Trying http://search.maven.org/remotecontent?filepath=elasticsearch/elasticsearch-lang-javascript/2.4.1/elasticsearch-l
ang-javascript-2.4.1.zip...
Trying https://oss.sonatype.org/service/local/repositories/releases/content/elasticsearch/elasticsearch-lang-javascript
/2.4.1/elasticsearch-lang-javascript-2.4.1.zip...
Trying https://github.com/elasticsearch/elasticsearch-lang-javascript/archive/2.4.1.zip...
Trying https://github.com/elasticsearch/elasticsearch-lang-javascript/archive/master.zip...
Failed to install elasticsearch/elasticsearch-lang-javascript/2.4.1, reason: failed to download out of all possible loc
ations..., use --verbose to get detailed information
ERROR: Service 'es' failed to build: The command '/bin/sh -c plugin install elasticsearch/elasticsearch-lang-javascript
/2.4.1' returned a non-zero code: 74
makefile:74: recipe for target 'docker_es' failed
make: *** [docker_es] Error 1

Exporting http_proxy and https_proxy in docker-compose like so:

environment:
  - http_proxy http://my.proxy:port
  - https_proxy https://my.proxy:port

This does not work as expected

The following line in es/Dockerfile still fails.

This appears to be a bug in elastic search, with an open issue. The work around mentioned there indicates one needs to explicitly pass the proxy variables to the plugin command in the line mentioned above.

After applying the fixes, The diff looks like this:

diff --git a/tooling/docker/es/Dockerfile b/tooling/docker/es/Dockerfile
index 7dffcec..af6bd36 100644
--- a/tooling/docker/es/Dockerfile
+++ b/tooling/docker/es/Dockerfile
@@ -1,4 +1,6 @@
 FROM elasticsearch:1.4

-RUN plugin install elasticsearch/elasticsearch-lang-javascript/2.4.1
+ENV proxy_port 3128
+ENV proxy_host my.proxy.com
+RUN plugin install elasticsearch/elasticsearch-lang-javascript/2.4.1 -DproxyPort=$proxy_port -DproxyHost=$proxy_host

A similar problem exists with the dev container when it runs set_up_ubuntu.sh which does apt-get operations.

Fix for that is:

diff --git a/tooling/docker/dev/Dockerfile b/tooling/docker/dev/Dockerfile
index e46dd50..c5946e0 100644
--- a/tooling/docker/dev/Dockerfile
+++ b/tooling/docker/dev/Dockerfile
@@ -13,6 +13,9 @@ FROM ubuntu@sha256:0ca448cb174259ddb2ae6e213ebebe7590862d522fe38971e1175faedf0b6

 MAINTAINER Erik Rose <erik@mozilla.com>

+ENV http_proxy http://my.proxy:port
+ENV https_proxy https://my.proxy:port
+
 COPY set_up_ubuntu.sh /tmp/
 RUN /tmp/set_up_ubuntu.sh

Is there a sane/better way to support hosts that are behind a proxy without having to cause trouble for hosts that are not?

[erikrose]

Hi, Anhad.

The solution here is probably to use docker build args, as at http://stackoverflow.com/a/34545644. docker-compose recently grew support for them; the Stack Overflow comment has old information. So, a patch that adds build args for http_proxy and https_proxy would be most welcome! If you figure out how to do that, a fix for https://bugzilla.mozilla.org/show_bug.cgi?id=1282932 would also be easy.

Thanks for the excellent bug report!

[AlexPham3011]

Hi Erik.

Sorry for asking a dumb question, but does docker is mandatory for DXR installation ?

I am running on Ubuntu 14.04 Virtual Box

Thank you.

Alex

[erikrose]

No, but there are so many dependencies that it makes it a lot easier.