ApiListener: Produces=Detect does not set appropriate content-type : application/json header (7.8.1)

[Laurens-makel]

❗ please do not add sensitive information in issues, you can provide extra information via email using issue number as reference ❗

Describe the issue
We have an adapter which produces either JSON or a PDF.

When using skipValidation="true" on the Exit related to the happy flow and PDF output and no produces attribute on the ApiListener, this will result in the following:

• PDF: application/pdf
• JSON: */*

When using produces="JSON" in this situation causes the following:

• PDF: application/json;charset=utf-8;
• JSON: application/json;charset=utf-8;

When using produces="DETECT":

• PDF: application/pdf
• JSON: application/octet-stream

Expected behaviour:

• PDF: application/pdf
• JSON: application/json;charset=utf-8;

I believe that this could all be prevented if #4420 was implemented and produces="PDF, JSON" could be used...

Besides that, DETECT should properly detect when the output is JSON.

Reporter
Laurens

[nielsm5]

When using the json flow, do you see something in the logging that it cannot compute the content-type?
It's very strange that it wouldn't be able to compute text content unless something else has explicitly set the content type and thus it won't compute.

[Laurens-makel]

When using the json flow, do you see something in the logging that it cannot compute the content-type? It's very strange that it wouldn't be able to compute text content unless something else has explicitly set the content type and thus it won't compute.

Hmm deze fout in ieder geval bij het ontvangen van de GET.. (ja timestamps zijn anders, zat wat te kutten met github..)

2023-08-11 15:37:52,798 INFO [http-nio-80-exec-6] 41C rest.ApiListenerServlet - ApiListenerServlet dispatching uri [/user] and method [GET]
2023-08-11 15:37:52,799 DEBUG [http-nio-80-exec-6] 41C rest.ApiListenerServlet - Evaluating preconditions for listener[listener of [GetUsers]] etagKey[Ibis4Navigator_LOC_/user]
2023-08-11 15:37:52,799 DEBUG [http-nio-80-exec-6] 41C rest.ApiListenerServlet - found etag value[a0fd2921e7eec67198be44b127bfe5c9] for key[Ibis4Navigator_LOC_/user]
2023-08-11 15:37:52,799 WARN [http-nio-80-exec-6] 41C stream.MessageContext - unable to parse mimetype from string [null]
2023-08-11 15:30:39,956 WARN [http-nio-80-exec-2] 37C stream.MessageContext - unable to parse mimetype from string [null]
org.springframework.util.InvalidMimeTypeException: Invalid mime type "null": 'mimeType' must not be empty
      at org.springframework.util.MimeTypeUtils.parseMimeType(MimeTypeUtils.java:203) ~[spring-core-5.3.29.jar:5.3.29]
      at org.springframework.util.MimeType.valueOf(MimeType.java:615) ~[spring-core-5.3.29.jar:5.3.29]
      at nl.nn.adapterframework.stream.MessageContext.withMimeType(MessageContext.java:76) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.util.MessageUtils.getContext(MessageUtils.java:59) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.util.MessageUtils.parseContentAsMessage(MessageUtils.java:101) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.http.rest.ApiListenerServlet.handleRequest(ApiListenerServlet.java:538) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.http.rest.ApiListenerServlet.service(ApiListenerServlet.java:189) [ibis-adapterframework-core-7.8.1.jar:7.8.1]2023-08-11 15:30:39,956 WARN [http-nio-80-exec-2] 37C stream.MessageContext - unable to parse mimetype from string [null]
org.springframework.util.InvalidMimeTypeException: Invalid mime type "null": 'mimeType' must not be empty
      at org.springframework.util.MimeTypeUtils.parseMimeType(MimeTypeUtils.java:203) ~[spring-core-5.3.29.jar:5.3.29]
      at org.springframework.util.MimeType.valueOf(MimeType.java:615) ~[spring-core-5.3.29.jar:5.3.29]
      at nl.nn.adapterframework.stream.MessageContext.withMimeType(MessageContext.java:76) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.util.MessageUtils.getContext(MessageUtils.java:59) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.util.MessageUtils.parseContentAsMessage(MessageUtils.java:101) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.http.rest.ApiListenerServlet.handleRequest(ApiListenerServlet.java:538) [ibis-adapterframework-core-7.8.1.jar:7.8.1]
      at nl.nn.adapterframework.http.rest.ApiListenerServlet.service(ApiListenerServlet.java:189) [ibis-adapterframework-core-7.8.1.jar:7.8.1

Echter geen fouten bij het geven van het response.. zie ook nergens debug logging voorbij komen over welke values hij nu kiest...
De JSON is geproduceerd door de Json2XmlValidator als laatste pipe..

[Laurens-makel]

De debugger laat ook zien dat application/octet-stream het gedecteerde type is

[nielsm5]

Het probleem van de bovenstaande stacktrace heb ik reeds opgelost in #5259.
Die komt uit het binnenkomende bericht, en daar hebben we helaas verder niets aan 😞 .

Ik heb veel logging toegevoegd (ook o.a. in #5259), ik hoop dat met deze extra logging we dit beter kunnen pin-pointen.

[nielsm5]

Implementing #4420 wont solve this issue because you would still need a way to differentiate between the two mimetypes.
DETECT doesn't work because it requires a filename which isn't present. I've yet to think of a good way to fix this.