Docker-compose build behind a HTTP proxy

[2xyo]

Problem to Solve

Building OpenCTI behind a HTTP proxy is currently failing.

Actual status

$ docker-compose up
...
integration_1    | Collecting requests (from -r requirements.txt (line 1))
integration_1    |   Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f966c0e6438>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/requests/
worker_1         | Collecting requests (from -r requirements.txt (line 1))
worker_1         |   Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f5f984cf748>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/requests/
...

Current Workaround

1. Move all the pip install from the entrypoiny.sh to the associated Dockerfile
2. Add build: ./integration and build: ./worker to docker-compose.yml

3. $ docker-compose build \
   --build-arg "HTTP_PROXY=$HTTP_PROXY" \
   --build-arg "HTTPS_PROXY=$HTTPS_PROXY" \
   --build-arg "http_proxy=$HTTP_PROXY" \
   --build-arg "https_proxy=$HTTPS_PROXY" 
   

4. Enjoy

Proposed Solution

See workaround 1.

[soreno123]

I've solved using this suggestion:
https://docs.docker.com/config/daemon/systemd/

Create a systemd drop-in directory for the docker service:

$ sudo mkdir -p /etc/systemd/system/docker.service.d
Create a file called /etc/systemd/system/docker.service.d/http-proxy.conf that adds the HTTP_PROXY environment variable:

[Service]
Environment="HTTP_PROXY=http://proxy.example.com:80/"

Or, if you are behind an HTTPS proxy server, create a file called /etc/systemd/system/docker.service.d/https-proxy.conf that adds the HTTPS_PROXY environment variable:

[Service]
Environment="HTTPS_PROXY=https://proxy.example.com:443/"

Regards,

[2xyo]

My bad, I forgot to mention that I already have this setup:

$ cat /etc/systemd/system/docker.service.d/http-proxy.conf 
[Service]
Environment="HTTP_PROXY=http://proxy.corp:3128/"

This environment variable is used by the docker daemon to download images for example but not inside the container.

[SamuelHassine]

@2xyo: Thanks for your report. We will install the Python dependencies directly in the image in the next release.

[SandVonSchaewen]

Configuration for Windows with a Local Proxy like CNTLM

1. Close Docker daemon
2. Open the %User% .docker\config.json File
3. Replace the Entries for proxy which might point to localhost to this:

{ "proxies":{ "default":{ "httpProxy":"http://host.docker.internal:YourProxyPort(3128)", "httpsProxy":"http://host.docker.internal:YourProxyPort(3128)", "noProxy":"localhost" } }

4. Restart Docker daemon

If you now set up new containers the Environment variables should now set corretly and work out of the box.
You can check this with the printenv comand if you setup linux in a container.