Incidental output (to *out*) during JSON streaming can produce invalid JSON

[ordnungswidrig]

Description

Output written to stdout, e.g. debug logging can accidently land in the json response create with io.pedestal.http/json-response.

Expected Behavior

If a lazy sequence generates output to stdout during realization and is used as a value in a json response, the output to stdout is actually written to the http response outputstream because *out* is rebound during the execution of json-response.

Actual Behavior

*out* should never be rebound without explicit notice to the user.

Steps to reproduce

(defn test-lazy-seq-stdout [req]
  (->
   (for [i (range 100)] (do (print "I is" i) i))
   (http/json-response)
   (response/status 200)))

Response body is like this:

I is 0I is 1I is 2I is 3I is 4I is 5I is 6I is 7I is 8I is 9I is 10I is 11I is 12I is 13I is 14I is 15I is 16I is 17I is 18I is 19I is 20I is 21I is 22I is 23I is 24I is 25I is 26I is 27I is 28I is 29I is 30I is 31I is 32I is 33I is 34I is 35I is 36I is 37I is 38I is 39I is 40I is 41I is 42I is 43I is 44I is 45I is 46I is 47I is 48I is 49I is 50I is 51I is 52I is 53I is 54I is 55I is 56I is 57I is 58I is 59I is 60I is 61I is 62I is 63I is 64I is 65I is 66I is 67I is 68I is 69I is 70I is 71I is 72I is 73I is 74I is 75I is 76I is 77I is 78I is 79I is 80I is 81I is 82I is 83I is 84I is 85I is 86I is 87I is 88I is 89I is 90I is 91I is 92I is 93I is 94I is 95I is 96I is 97I is 98I is 99[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99]

Environment

Arachne on pedestal via
[org.arachne-framework/pedestal-assets "0.1.0-master-0006-46abdf0"]

Operating System (including version).

Darwin monoid.h.c6e.de 16.3.0 Darwin Kernel Version 16.3.0: Sun Oct 23 14:50:01 PDT 2016; root:xnu-3789.30.76~4/RELEASE_X86_64 x86_64

Your current Leiningen version (lein --version)

Pedestal version

0.5.2

[ohpauleez]

Hi @ordnungswidrig -- thanks for reporting the issue!

json-response is typically meant for other interceptors that shape the final response and not intended to be used as a general utility in user code. Its specific use (like the other data-writers/printers) is to stream output directly into the Response body. Your inadvertent use is the actual targeted use of that function (specifically around sequences that need to print/write piece-by-piece to the Response stream)

Is there a reason why you're explicit calling json-response instead of using the json-body interceptor.

[ordnungswidrig]

Thanks for pointing out, better to use the interceptor. Nevertheless the behaviour of json-response as a public function seemed weird to me and a warning in the doc string might be appropriate.

[ohpauleez]

I agree we should document this split (utility functions vs interceptors) better and explain the use case within the doc strings. Leaving this issue open to address that. Thanks @ordnungswidrig

[hlship]

Documentation, explicitly marking things as deprecated, and actual removal would be an improvement.

[hlship]

I've fixed the buggy part (random output mixed into the JSON stream) here: 39cc924