How to client.Close()?

[didip]

I must have missed the obvious thing, how do I close docker client connection?

[fsouza]

Hi @didip, since go-dockerclient will just send HTTP requests, it forwards handling of connections to the default HTTP client in the Go standard library.

You can also customize the HTTP client to be used and customize when to close connections.

[didip]

Somewhat related to my original question. I've noticed that memory consumptions grow when I loop and fetch containers/images data. The pseudo-code looks like this:

var client = docker.NewClient()
for {
    client.GetContainers()
    time.sleep(5 * time.Minute)
}

the go-dockerclient code itself looks sane, you guys call res.Body.Close() whenever needed.

Should I create a new separate issue for this?

[fsouza]

Hey @didip, I think we can keep this issue, for further investigation. I will try to reproduce and fix the issue locally.

Thanks for reporting!

[fsouza]

@didip I've created this small reproducer:

package main

import (
    "flag"
    "fmt"
    "sync"
    "time"

    "github.com/fsouza/go-dockerclient"
)

var (
    floods   int
    interval time.Duration
)

func init() {
    flag.IntVar(&floods, "f", 4, "number of concurrent floods")
    flag.DurationVar(&interval, "i", time.Second, "interval between requests")
    flag.Parse()
}

func flood(wg *sync.WaitGroup, client *docker.Client, containerID string) {
    var (
        container *docker.Container
        err       error
    )
    for {
        container, err = client.InspectContainer(containerID)
        time.Sleep(interval)
        fmt.Println(container.Name, err)
    }
}

func main() {
    endpoint := flag.Arg(0)
    client, err := docker.NewClient(endpoint)
    if err != nil {
        panic(err)
    }
    var wg sync.WaitGroup
    wg.Add(floods)
    for i := 0; i < floods; i++ {
        go flood(&wg, client, flag.Arg(1))
    }
    wg.Wait()
}

Running with 8 goroutines and 1ms interval, it rapidly grows to ~6.6MB of memory usage, and then stop growing. I think it should not grow that much, but it's indeed stopping growing. I've tried with 64 goroutines and reached the top around 8.0MB. I will do some profiling, but could you share more information on your environment, like the OS, Docker and Go version?

Here is the Docer server environment that I tested:

% uname -a
Linux vagrant 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
% docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef

Here is the environment where I ran the client code:

% uname -a
Darwin xikinbook.local 14.1.0 Darwin Kernel Version 14.1.0: Mon Dec 22 23:10:38 PST 2014; root:xnu-2782.10.72~2/RELEASE_X86_64 x86_64
% go version
go version devel +1fda57b Wed Feb 25 00:52:03 2015 +0000 darwin/amd64

Thank you again!

[didip]

Thanks for investigating this so quickly!

uname -a
Linux chi-ops-docker-2.nr-ops.net 3.10.39-1.el6.elrepo.x86_64 #1 SMP Tue May 6 13:04:24 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux

docker version
Client version: 1.2.0
Client API version: 1.14
Go version (client): go1.3.3
Git commit (client): fa7b24f/1.2.0
OS/Arch (client): linux/amd64
Server version: 1.2.0
Server API version: 1.14
Go version (server): go1.3.3
Git commit (server): fa7b24f/1.2.0

Go version is: 1.4.1

[didip]

I can also show the code: https://github.com/resourced/resourced/blob/master/libdocker/libdocker.go

• Line 40 & Line 56: We pool docker connection in global map.
• Then here: We get the containers data. Each reader.Run() is executed inside its own goroutine.

Globally pooling docker connections helps the problem a bit, but memory is still growing.

[didip]

We found the memory leak and its in our code. Sorry for the false alarm.

[fsouza]

Hi @didip, sorry for the delay, and thanks for the feedback, please feel free to open new issues in the future!