Trying to write metadata with JacksonHandle causes IOException: "Attempted write to closed stream"

[nikos]

When using Java Client API version 2.0-3 with JDK 1.7.0-67 and attempting to write a document with the JSON Document Manager including a metadata handle an java.io.IOException is thrown.

Sample java code (from ProductRepositoryJSON.java) to demonstrate the problem:

        DocumentMetadataHandle metadata = new DocumentMetadataHandle();
        metadata.getCollections().add(COLLECTION_REF);

        JacksonHandle writeHandle = new JacksonHandle();
        JsonNode writeDocument = writeHandle.getMapper().convertValue(product, JsonNode.class);
        writeHandle.set(writeDocument);

        jsonDocumentManager.write(getDocId(product.getSku()), metadata, writeHandle);

The full stacktrace:

java.io.IOException: Attempted write to closed stream.
    at org.apache.http.impl.io.ChunkedOutputStream.write(ChunkedOutputStream.java:178)
    at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
    at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
    at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
    at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
    at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
    at java.io.BufferedWriter.flush(BufferedWriter.java:254)
    at com.sun.jersey.multipart.impl.MultiPartWriter.writeTo(MultiPartWriter.java:232)
    at com.sun.jersey.multipart.impl.MultiPartWriter.writeTo(MultiPartWriter.java:71)
    at com.sun.jersey.api.client.RequestWriter$RequestEntityWriterImpl.writeRequestEntity(RequestWriter.java:231)
    at com.sun.jersey.client.apache4.ApacheHttpClient4Handler$2.writeTo(ApacheHttpClient4Handler.java:262)
    at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:89)
    at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:108)
    at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:117)
    at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:265)
    at org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:231)
    at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:236)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121)
    at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:115)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
    at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
    at com.marklogic.client.impl.DigestChallengeFilter.handle(DigestChallengeFilter.java:34)
    at com.sun.jersey.api.client.filter.HTTPDigestAuthFilter.handle(HTTPDigestAuthFilter.java:493)
    at com.sun.jersey.api.client.Client.handle(Client.java:648)
    at com.sun.jersey.api.client.WebResource.handle(WebResource.java:680)
    at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
    at com.sun.jersey.api.client.WebResource$Builder.put(WebResource.java:537)
    at com.marklogic.client.impl.JerseyServices.putPostDocumentImpl(JerseyServices.java:1238)
    at com.marklogic.client.impl.JerseyServices.putDocument(JerseyServices.java:954)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:453)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:388)
    at com.marklogic.client.impl.DocumentManagerImpl.write(DocumentManagerImpl.java:358)
    at de.nava.mlsample.service.ProductRepositoryJSON.add(ProductRepositoryJSON.java:53)
    at de.nava.mlsample.web.SampleRESTwithJSONController.createProduct(SampleRESTwithJSONController.java:33)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:863)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:280)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.boot.actuate.autoconfigure.MetricFilterAutoConfiguration$MetricsFilter.doFilterInternal(MetricFilterAutoConfiguration.java:89)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
    at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1040)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1720)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1679)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:745)

[sammefford]

Niko, thanks for submitting this issue. I've pushed a fix. Our process is to have QA test & validate the fix before we ship it. Since our patch release, 7.0.4, is almost out the door, it may be too late to include this. I'll see what can be done. But worst case scenario, if we have the code on the b2_0 branch and you can build with the fix, does that get you what you need for the short term?

[nikos]

Thanks very much, Sam, for your fast response and fixing this problem. Pulling from the b2_0 branch and build the Java client on my own isn't a big deal. Thanks again.

[nikos]

Just to get a better understanding: is there a 1:1 relation between the Java Client API version (2.0-3) and ML server version (7.0.3)? I just wonder if a new maven version of Java Client API is possible (say 2.0-3.1) to be released to maven...

[sammefford]

Yes, for now there is that 1:1 relation between the Java Client API version (2.0-3) and ML server version (7.0.3). Whether or not we release a new 2.0-3.1 or 2.0-4 to align with 7.0.4, we'd like to have full QA on our releases. We're discussing internally what makes the most sense in this case.

If any other customers run across this issue but aren't as able as Niko to build the b2_0 branch, here is a work-around. After instantiating a new JacksonHandle, configure the ObjectMapper with AUTO_CLOSE set to false for both the JsonGenerator and the JsonParser:
JacksonHandle writeHandle = new JacksonHandle();
writeHandle.getMapper().configure(JsonGenerator.Feature.AUTO_CLOSE_TARGET, false);
writeHandle.getMapper().configure(JsonParser.Feature.AUTO_CLOSE_SOURCE, false);

This is exactly what the patched version of the JacksonHandle does for you.

[sravanr]

added a functional test in Master