Timeout waiting for connection from pool

[rohitkadam19]

I am trying to pull an image whose size is around 700 MB and it takes around 3-4 minutes if I pull it using command. I am pulling same using docker-java library. Getting following error.

DockerClient dockerClient = DockerClientBuilder.getInstance("http://localhost:2375").build();
dockerClient.pullImageCmd("example-image").withTag("latest").exec();

javax.ws.rs.ProcessingException: org.apache.http.conn.ConnectionPoolTimeoutException: Timeout     waiting for connection from pool
at com.github.dockerjava.jaxrs.connector.ApacheConnector.apply(ApacheConnector.java:490)
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:246)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:667)
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:664)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:297)

And after exception it downloads the image. I am not able to see any method which will wait till complete image download.

Please help!

[marcuslinke]

The ConnectionPoolTimeoutException occurs if there are no more connections available in the connection pool . The default setting is the have only 2 connections in the pool, so either ensure that all former connections are closed (by closing the InputStream response object after processing) or adjust the pool size via DockerClientConfig#setMaxPerRouteConnections.

[marcuslinke]

FYI: The pull image API will be changed with the next release to handle the stream processing more easily.

[rohitkadam19]

How would I know all layers of images has been downloaded successfully? After that I can close InputStream response object and can pull another image. Also I will try with changing pool size DockerClientConfig#setMaxPerRouteConnections

[marcuslinke]

OK, thats the problem because of I've decided to refactor the API. Currently (with latest 1.4.0 release) it needs some dirty helper code to process the stream. Take a look at

https://github.com/docker-java/docker-java/blob/docker-java-1.4.0/src/test/java/com/github/dockerjava/core/command/PullImageCmdImplTest.java#L92

As you can see, it needs a function asString(InputStream response) that consumes the stream until EOF and collects it into a string to check if the operation succeeds:

public static String asString(InputStream response) {

        StringWriter logwriter = new StringWriter();

        try {
            LineIterator itr = IOUtils.lineIterator(response, "UTF-8");

            while (itr.hasNext()) {
                String line = itr.next();
                logwriter.write(line + (itr.hasNext() ? "\n" : ""));
                LOG.info("line: " + line);
            }
            response.close();

            return logwriter.toString();
        } catch (IOException e) {
            throw new RuntimeException(e);
        } finally {
            IOUtils.closeQuietly(response);
        }
    }

With the next release the API will be callback-driven to simplify the stream consumption.

[rohitkadam19]

Ok, Thanks a lot for this. I will wait for next release, for now will have some workaround.

Is it available in SNAPSHOT? I can try with it.

[marcuslinke]

Ok. So closing for now.