PoolingClientConnectionManager class def not found

[dparis]

I'm cleaning up a lein project's deps, and after shuffling around some namespaces and updating the amazonica project.clj dependency to use 0.1.22, I'm getting a new error when calling s3/get-object:

java.lang.NoClassDefFoundError: org/apache/http/impl/conn/PoolingClientConnectionManager
 at com.amazonaws.http.ConnectionManagerFactory.createPoolingClientConnManager (ConnectionManagerFactory.java:26)
    com.amazonaws.http.HttpClientFactory.createHttpClient (HttpClientFactory.java:87)
    com.amazonaws.http.AmazonHttpClient.<init> (AmazonHttpClient.java:121)
    com.amazonaws.AmazonWebServiceClient.<init> (AmazonWebServiceClient.java:66)
    com.amazonaws.services.s3.AmazonS3Client.<init> (AmazonS3Client.java:304)
    com.amazonaws.services.s3.AmazonS3Client.<init> (AmazonS3Client.java:286)
    sun.reflect.NativeConstructorAccessorImpl.newInstance0 (NativeConstructorAccessorImpl.java:-2)
    sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:57)
    sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)
    java.lang.reflect.Constructor.newInstance (Constructor.java:526)
    clojure.lang.Reflector.invokeConstructor (Reflector.java:180)
    amazonica.core$create_client.invoke (core.clj:139)
    amazonica.core$amazon_client_STAR_.invoke (core.clj:187)
    clojure.lang.AFn.applyToHelper (AFn.java:167)
    clojure.lang.AFn.applyTo (AFn.java:151)
    clojure.core$apply.invoke (core.clj:617)
    clojure.core$memoize$fn__5049.doInvoke (core.clj:5735)
    clojure.lang.RestFn.invoke (RestFn.java:436)
    amazonica.core$candidate_client.invoke (core.clj:661)
    amazonica.core$fn_call$fn__1670.invoke (core.clj:671)
    clojure.lang.Delay.deref (Delay.java:33)
    clojure.core$deref.invoke (core.clj:2128)
    amazonica.core$fn_call$fn__1672.invoke (core.clj:674)
    amazonica.core$intern_function$fn__1687.doInvoke (core.clj:718)
.
.
.

I tried resetting my amazonica dependency back to the previous version 0.1.15, but I'm still getting the error. It looks like the aws-sdk jar updated the ConnectionManagerFactory code when it bumped to version 1.5.0, which amazonica was including a bit before version 0.1.15 if I'm not mistaken.

Any idea why this is happening now? As part of my project refactoring, I removed some dependencies from the project. Is there any reason why amazonica would be dependent on any other libraries being present in the project?

[dparis]

New data point: I created a new empty lein app project and added [amazonica "0.1.22"] as the only project dependency. I then nuked my .m2 directory and ran lein deps on my real project. After all of the dependencies had been resolved, I then ran lein deps on the new empty project and sure enough it grabbed a different version of org.apache.httpclient and org.apache.httpcore.

Here are the org.apache libs pulled in by my app:

Retrieving org/apache/commons/commons-parent/5/commons-parent-5.pom from central
Retrieving org/apache/apache/4/apache-4.pom from central
Retrieving org/apache/httpcomponents/httpclient/4.1/httpclient-4.1.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-client/4.1/httpcomponents-client-4.1.pom from central
Retrieving org/apache/httpcomponents/project/4.1.1/project-4.1.1.pom from central
Retrieving org/apache/httpcomponents/httpcore/4.1/httpcore-4.1.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-core/4.1/httpcomponents-core-4.1.pom from central
Retrieving org/apache/commons/commons-parent/11/commons-parent-11.pom from central
Retrieving org/apache/httpcomponents/httpcore/4.1/httpcore-4.1.jar from central
Retrieving org/apache/httpcomponents/httpclient/4.1/httpclient-4.1.jar from central
Retrieving org/apache/httpcomponents/httpclient/4.2/httpclient-4.2.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-client/4.2/httpcomponents-client-4.2.pom from central
Retrieving org/apache/httpcomponents/project/5/project-5.pom from central
Retrieving org/apache/httpcomponents/httpcore/4.2/httpcore-4.2.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-core/4.2/httpcomponents-core-4.2.pom from central
Retrieving org/apache/httpcomponents/project/6/project-6.pom from central
Retrieving org/apache/commons/commons-parent/22/commons-parent-22.pom from central
Retrieving org/apache/apache/9/apache-9.pom from central
Retrieving org/apache/lucene/lucene-core/3.1.0/lucene-core-3.1.0.pom from central
Retrieving org/apache/lucene/lucene-parent/3.1.0/lucene-parent-3.1.0.pom from central
Retrieving org/apache/lucene/lucene-solr-grandparent/3.1.0/lucene-solr-grandparent-3.1.0.pom from central
Retrieving org/apache/apache/8/apache-8.pom from central
Retrieving org/apache/lucene/lucene-analyzers/3.1.0/lucene-analyzers-3.1.0.pom from central
Retrieving org/apache/commons/commons-parent/20/commons-parent-20.pom from central
Retrieving org/apache/httpcomponents/httpclient/4.2.1/httpclient-4.2.1.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-client/4.2.1/httpcomponents-client-4.2.1.pom from central
Retrieving org/apache/httpcomponents/httpcore/4.2.1/httpcore-4.2.1.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-core/4.2.1/httpcomponents-core-4.2.1.pom from central
Retrieving org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom from central
Retrieving org/apache/commons/commons-parent/17/commons-parent-17.pom from central
Retrieving org/apache/apache/7/apache-7.pom from central
Retrieving org/apache/apache/3/apache-3.pom from central
Retrieving org/apache/commons/commons-parent/9/commons-parent-9.pom from central
Retrieving org/apache/commons/commons-lang3/3.1/commons-lang3-3.1.pom from central
Retrieving org/apache/httpcomponents/httpclient/4.2.2/httpclient-4.2.2.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-client/4.2.2/httpcomponents-client-4.2.2.pom from central
Retrieving org/apache/httpcomponents/httpcore/4.2.2/httpcore-4.2.2.pom from central
Retrieving org/apache/httpcomponents/httpcomponents-core/4.2.2/httpcomponents-core-4.2.2.pom from central
Retrieving org/apache/httpcomponents/httpmime/4.2.2/httpmime-4.2.2.pom from central
Retrieving org/apache/commons/commons-parent/26/commons-parent-26.pom from central
Retrieving org/apache/commons/commons-parent/25/commons-parent-25.pom from central
Retrieving org/apache/commons/commons-parent/24/commons-parent-24.pom from central
Retrieving org/apache/xmlgraphics/batik-dom/1.7/batik-dom-1.7.pom from central
Retrieving org/apache/xmlgraphics/batik-css/1.7/batik-css-1.7.pom from central
Retrieving org/apache/xmlgraphics/batik-ext/1.7/batik-ext-1.7.pom from central
Retrieving org/apache/xmlgraphics/batik-util/1.7/batik-util-1.7.pom from central
Retrieving org/apache/xmlgraphics/batik-xml/1.7/batik-xml-1.7.pom from central
Retrieving org/apache/xmlgraphics/batik-svggen/1.7/batik-svggen-1.7.pom from central
Retrieving org/apache/xmlgraphics/batik-awt-util/1.7/batik-awt-util-1.7.pom from central
Retrieving org/apache/poi/poi-ooxml/3.9/poi-ooxml-3.9.pom from central
Retrieving org/apache/poi/poi/3.9/poi-3.9.pom from central
Retrieving org/apache/poi/poi-ooxml-schemas/3.9/poi-ooxml-schemas-3.9.pom from central
Retrieving org/apache/xmlbeans/xmlbeans/2.3.0/xmlbeans-2.3.0.pom from central
Retrieving org/apache/abdera/abdera-i18n/1.1.1/abdera-i18n-1.1.1.pom from central
Retrieving org/apache/abdera/abdera/1.1.1/abdera-1.1.1.pom from central
Retrieving org/apache/geronimo/specs/geronimo-activation_1.0.2_spec/1.1/geronimo-activation_1.0.2_spec-1.1.pom from central
Retrieving org/apache/geronimo/specs/specs/1.1/specs-1.1.pom from central
Retrieving org/apache/apache/1/apache-1.pom from central
Retrieving org/apache/lucene/lucene-core/3.1.0/lucene-core-3.1.0.jar from central
Retrieving org/apache/lucene/lucene-analyzers/3.1.0/lucene-analyzers-3.1.0.jar from central
Retrieving org/apache/httpcomponents/httpmime/4.2.2/httpmime-4.2.2.jar from central
Retrieving org/apache/xmlgraphics/batik-dom/1.7/batik-dom-1.7.jar from central
Retrieving org/apache/commons/commons-lang3/3.1/commons-lang3-3.1.jar from central
Retrieving org/apache/xmlgraphics/batik-css/1.7/batik-css-1.7.jar from central
Retrieving org/apache/xmlgraphics/batik-ext/1.7/batik-ext-1.7.jar from central
Retrieving org/apache/xmlgraphics/batik-svggen/1.7/batik-svggen-1.7.jar from central
Retrieving org/apache/xmlgraphics/batik-awt-util/1.7/batik-awt-util-1.7.jar from central
Retrieving org/apache/xmlgraphics/batik-util/1.7/batik-util-1.7.jar from central
Retrieving org/apache/xmlgraphics/batik-xml/1.7/batik-xml-1.7.jar from central
Retrieving org/apache/poi/poi-ooxml/3.9/poi-ooxml-3.9.jar from central
Retrieving org/apache/poi/poi/3.9/poi-3.9.jar from central
Retrieving org/apache/poi/poi-ooxml-schemas/3.9/poi-ooxml-schemas-3.9.jar from central
Retrieving org/apache/xmlbeans/xmlbeans/2.3.0/xmlbeans-2.3.0.jar from central
Retrieving org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar from central
Retrieving org/apache/abdera/abdera-i18n/1.1.1/abdera-i18n-1.1.1.jar from central
Retrieving org/apache/geronimo/specs/geronimo-activation_1.0.2_spec/1.1/geronimo-activation_1.0.2_spec-1.1.jar from central

and here are the further libs pulled in by the empty project:

Retrieving org/apache/httpcomponents/httpclient/4.2/httpclient-4.2.jar from central
Retrieving org/apache/httpcomponents/httpcore/4.2/httpcore-4.2.jar from central

Still not sure I understand why it's happening, but I can at least show that it's likely related to a version conflict somewhere in the load path. It looks like httpcore added support for the connection pooling in 4.2, so if my app is pulling in 4.1 and amazonica is seeing that instead of the 4.2 version, that might explain the NoClassDefFoundError.

[dparis]

I just discovered the joys of lein deps :tree and solved my issue (along with, apparently, a number of other potential issues with different libs shadowing indirect dependencies). I've explicitly added [org.apache.httpcomponents/httpclient "4.2.2"] to my direct project dependencies and amazonica seems to be functioning again. Hopefully no incompatible changes between 4.1 and 4.2.2 that cause regressions elsewhere in the app, but at least the core functionality is working again.

Live and learn, I suppose. Sorry for the spam.