Documentation on using exposed Remote Docker Host without docker-machine

[Mario-Hofstaetter]

To start off with, thank you for your work.

I'd like to clearify a few things, and maybe help other users who also struggle on the same topic.

Running FluentDocker 2.7.1

I've spent several hours trying to find out

how to easily use remote docker hosts WITHOUT using docker-machine, and WITHOUT SSH keys.

In our local dev & test environment with have several docker host server daemons exposed (yes, unsecure) like so:

ax-docker01:2375
ax-docker02:2375
ax-docker03:2375

etc.

After looking at the source code I succeeded in using:

var myUri = new DockerUri("ax-docker03:2375");
var param = new ContainerCreateParams { Name = "JustRunIt" };
var imageId = "885b58d646ca3cfd7714963d47a64a027";
var resp = myUri.Run(imageId, param);

and also

var _dockerHost = new DockerHostService("docker03", isNative: true, stopWhenDisposed:true, dockerUri: "ax-docker03:2375");
var p = new ContainerCreateParams { Name = "container1" };
var c1 = _dockerHost.Create(imageId, forcePull: false, prms: p);
c1.Start();

These gave me access to two (different) sets of commands and possibilities.

But I was unable to use your FluentAPI with them, I just found no way to tell it to use my HostService or URI.
The README.md just states that "FluentDocker supports connection to remote docker daemons" and goes on with docker-machine instructions.

Then, after seeing some github issue here, I discovered this working approach:

// !!!!
Environment.SetEnvironmentVariable("DOCKER_HOST", $"{myDockerHostServer}:2375");

using (var container1 = new Builder().UseContainer().ReuseIfExists()
                        .WithName("fluent1").UseImage("885b58d646ca3cfd7714963d47a64a027")
                        .ExposePort(8080)
                        .Build().Start())
{
    // DO STUFF
}

This does exactly what I need, with no hassle. Nowhere in README.md does it say you can use the DOCKER_HOST ENV variable like that 😮

@mariotoffia Is this the legitimate, supported way of doing this?
Or am I missing something else?

If I find time for it I would create a pull request to expand the README on this ?

Secondly:
Using the above way, If one tries this:

.ExposePort(20000, 8080).WaitForPort("8080/tcp", 5000 /* millisec */)

It fails with:

Ductus.FluentDocker.Common.FluentDockerException: 'Timeout waiting for service at = 127.0.0.1 port = 20000'

Is this expected? Shouldn't it also use the DOCKER_HOST ?

I then discovered the address parameter, and tried this:

.ExposePort(20000, 8080).WaitForPort("8080/tcp", 5000 /* millisec */ , address: myDockerHostServer)

(myDockerHostServer being ax-docker03)

which fails with:

System.FormatException: 'An invalid IP address was specified.'

Finally, using the IP address of the host server:

.ExposePort(20000, 8080).WaitForPort("8080/tcp", 5000 /* millisec */, address: myDockerHostIp)

This is successful. Again, is this the correct way to do this?

Thank you.

[mariotoffia]

@Mario-Hofstaetter Thanks for your comments. Yes, the DOCKER_HOST environment variable that is used for overriding the mechanism just for the use-case you described above. However, I've set this a a feature since it would make sense to have the ability to set a custom Docker Daemon Uri per builder chain without involving remoting.

If you find time to do a writeup on this in the Readme.md file please do! :) - However, it needs to to have a clause of discurage since bypassing any sort of security measure will sooner or later make your system compromised!

WaitForPort do a docker inspect on the container and gets the endpoint for that particular container. It seems to have failed (I have trouble to determine the issue with this little information). However, as you found the way to specify a IP address (not dns address for now).

If you can, please give more diagnostics around

1. docker inspect
2. do you have docker as DNS entry (try ping docker)

So I may try to reproduce the IPAddress.Loopback problem.

But in summary: Both of your usage is correct and will be maintained in this library for a forseeable future.

Cheers,
Mario

[Mario-Hofstaetter]

Thank you for answering.

@README: will try to get a pull request going the next days.

@WaitForPort:
The local client Machine is Windows 10 Prof (10.17.11.50) , the docker hosts are CentOS VMs, version information via cmd:

C:\>set DOCKER_HOST=ax-docker03:2375
C:\>docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        c89750f8
 Built:             06/09/2019 17:10:42
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Pinging docker on the Windows 10 Machine results in an error, since Docker Desktop for Windows 10 is currently not installed (fails to install...).
However, the client docker.exe is installed via the docker-cli chocolatey package.

There is also this leftover in the windows hosts file:

# Added by Docker for Windows
10.17.11.50 host.docker.internal
10.17.11.50 gateway.docker.internal
# End of section

Using that host and

using (IContainerService container1 = new Builder().UseContainer().ReuseIfExists()
                        .WithName("fluent1").UseImage("885b58d646ca3cfd7714963d47a64a027")
                        .ExposePort(20000, 8080).WaitForPort("8080/tcp", 120000)
                        .Build().Start())

Running in cmd with DOCKER_HOST=ax-docker03:2375

C:\>docker inspect fluent1>inspect-fluent1.json
C:\>docker port fluent1>port-fluent1.txt

The docker-daemon is putting out 0.0.0.0 as IP.
FluentDocker would need to replace it with the content of DOCKER_HOST in that case?

inspect-fluent1.json:

[
    {
        "Id": "a78035decdbca398f931f9a57dafe05a5e894f10e926cae3c54466506b9224f8",
        "Created": "2019-09-02T12:07:27.029757133Z",
        "Path": "./docker-entrypoint.sh",
        "Args": [
            "bin/myapp"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 29954,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-09-02T12:07:28.156751298Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:885b58d646ca3cfd7714963d47a64a02784e9d41c99881db80e183a1c3a5f3c5",
        "ResolvConfPath": "/var/lib/docker/containers/a78035decdbca398f931f9a57dafe05a5e894f10e926cae3c54466506b9224f8/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/a78035decdbca398f931f9a57dafe05a5e894f10e926cae3c54466506b9224f8/hostname",
        "HostsPath": "/var/lib/docker/containers/a78035decdbca398f931f9a57dafe05a5e894f10e926cae3c54466506b9224f8/hosts",
        "LogPath": "/var/lib/docker/containers/a78035decdbca398f931f9a57dafe05a5e894f10e926cae3c54466506b9224f8/a78035decdbca398f931f9a57dafe05a5e894f10e926cae3c54466506b9224f8-json.log",
        "Name": "/fluent1",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {
                "8080/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "20000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "shareable",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/78d0b073b31a42507989db174cd4b113d1e57a0948831957cbc3d513732e9266-init/diff:/var/lib/docker/overlay2/718de9095c58c912b37c1647ba7648b0f56893470b70a24480393268125d86d3/diff:/var/lib/docker/overlay2/307494b331b2258af29d889416bf86a96e40699ee813a059a8654a25d1f246c1/diff:/var/lib/docker/overlay2/be1a41c9e8ba6104d2d1e8e66e00fe791f4e45a602829ad00630048e09971868/diff:/var/lib/docker/overlay2/374634d3152221eec9f7298954cf352b9ce0843e44f351fc09e390f260fcceff/diff:/var/lib/docker/overlay2/5ed61f7ac1173c265e53fb3daba0baffd75533518462626ce2880fcd09ee500b/diff",
                "MergedDir": "/var/lib/docker/overlay2/78d0b073b31a42507989db174cd4b113d1e57a0948831957cbc3d513732e9266/merged",
                "UpperDir": "/var/lib/docker/overlay2/78d0b073b31a42507989db174cd4b113d1e57a0948831957cbc3d513732e9266/diff",
                "WorkDir": "/var/lib/docker/overlay2/78d0b073b31a42507989db174cd4b113d1e57a0948831957cbc3d513732e9266/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "a78035decdbc",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "2011/tcp": {},
                "2012/tcp": {},
                "4199/tcp": {},
                "8080/tcp": {},
                "8081/tcp": {},
                "8091/tcp": {},
                "917/udp": {},
                "9200/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "myapp_SHM_SIZE=64MB"
            ],
            "Cmd": [
                "bin/myapp"
            ],
            "Image": "885b58d646ca3cfd7714963d47a64a027",
            "Volumes": null,
            "WorkingDir": "/myapp",
            "Entrypoint": [
                "./docker-entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "com.company.build_timestamp": "2019-07-24 13:48:50",
                "com.company.product": "myapp",
                "com.company.version_base_dockerfile": "1.0.0",
                "com.company.version_dockerfile": "1.1.0",
                "maintainer": "me@company.com",
                "org.label-schema.build-date": "20181204",
                "org.label-schema.license": "GPLv2",
                "org.label-schema.name": "CentOS Base Image",
                "org.label-schema.schema-version": "1.0",
                "org.label-schema.vendor": "CentOS"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "85db5b9a1612736ffce315261ec00de48e4701ff07c6e99c79b25666b91da1cf",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "2011/tcp": null,
                "2012/tcp": null,
                "4199/tcp": null,
                "8080/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "20000"
                    }
                ],
                "8081/tcp": null,
                "8091/tcp": null,
                "917/udp": null,
                "9200/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/85db5b9a1612",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "0a82b1edb193e427bf259822eecc41a9b1f66285fb496cef5eaf8a700de00140",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "cea175cb9cb25cf8bd803c559133fcedf99da6d3d2756517bb3c929d14492b20",
                    "EndpointID": "0a82b1edb193e427bf259822eecc41a9b1f66285fb496cef5eaf8a700de00140",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

port-fluent1.txt:

8080/tcp -> 0.0.0.0:20000

What other information would be of assistence for you?

---

[EDIT] I looked at the source and understand why this is the case. Maybe I'll PR it as well :-) You may ignore this following section for now.

By the way, there also seems to be an issue with the millisTimeout parameter of WaitForPort.
It waits significantly longer than the specified timeout.
Should I create another issue for this?

I surrounded the using with try catch and used a Stopwatch to measure the time.

WaitForPort call	Elapsed Time till exception in seconds
Without WaitForPort	~ 2 seconds'
.WaitForPort("8080/tcp", 1000)	7 seconds
.WaitForPort("8080/tcp", 2000)	10 seconds
.WaitForPort("8080/tcp", 4000)	17 seconds
.WaitForPort("8080/tcp", 10000)	35 seconds
.WaitForPort("8080/tcp", 30000)	95 seconds

Times are repeatable.

Cheers,
(Also :-) ) Mario

[mariotoffia]

Hi Mario ;) - I think that the resolution may need a bit more flexibility so you as a user may have a say in how it will resolve and fall back. I have filed a Issue #107 to accomodate just that.

...and I really need to check the wait routine :)

Thanks for all info,
Mario

[Mario-Hofstaetter]

Great thanks,

the wait routine uses a while loop with

Thread.Sleep(1000);
totalWait += 1000;

in case of connect failure, which does not take into account the time already used by the TCP Socket timeout (I think).

I planned to change that to use instead a Stopwatch to measure the total time that hat really passed, or better use a System.Threading.Tasks.Task for the Connect loop and

task.Wait(millisTimeout);

to terminate it. For the Pull Request I want to implement an integration test for before-after testing.

I won't have time for that until weekend.

Cheers

[mariotoffia]

Thanks!

I've comitted a fix so it should handle this within waitTime (timeout + 1s) (except when socket hangs over duetime) for now. I'm trying to avoid as much threading or otherwise complexity in this library (as long as I can).

Cheers,
Mario