-
Notifications
You must be signed in to change notification settings - Fork 408
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Always parse json in strict mode #507
Conversation
resp-array (client/get (localhost "/json-array") {:as :json}) | ||
resp-array-strict (client/get (localhost "/json-array") {:as :json-strict}) | ||
resp-large-array (client/get (localhost "/json-large-array") {:as :json}) | ||
resp-large-array-strict (client/get (localhost "/json-large-array") {:as :json-strict}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added these additional test cases to replicate the bug.
@rymndhng it looks like
|
a626a5e
to
3cdbd80
Compare
Thanks for the poke! I've rebased against master and I've put back the logic for detecting if the stream has content. |
3cdbd80
to
8561efa
Compare
I have rebased the PR against the latest |
8561efa
to
0333e9e
Compare
Thanks for following up :) |
Fixes #489.
This solution contains the following changes:
Reduces the number of options for JSON coercions
An assumption I'm going to make is that most folks who work with
clj-http
don't really intend to get a lazy sequence back. They want the JSOn value back (with no additional side-effects). By making all the:json
options strict, we reduce the number of options from 4 to 2.:json-strict
becomes an undocumented alias for:json
for backwards compatibility, and similarly:json-strict-string-keys
is aliased to:json-string-keys
.This problem started surfacing when introducing #475 and users were caught off guard by confusing error messages due to lazy IO (see #489).
Document how to incrementally parse JSON to avoid unintended lazy IO
Mixing lazy sequences and IO is hrad to grok. For clj-http to incrementally parse JSON and close the underlying reader reliably, it would need to add additional options (similar to clojure.java.jdbc) such as
row-fn
andresultset-fn
in order to process each row. Adding more options increases complexity for users to figure out when/why these options should matter. It'd also add complexity for documentation.I personally think Incremental JSON parsing is an advanced feature which most users do not need to be exposed to. As an alternative, we should provide some documentation for how to incrementally parse JSON correct. This PR provides this documentation for advanced users who want this functionality.