Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add ability to parse parts of json #134

Closed
wants to merge 7 commits into from

Conversation

DeLaGuardo
Copy link

@DeLaGuardo DeLaGuardo commented Oct 12, 2018

New core function to parse parts of json from the stream (potentially large one)

(parse-stream rdr nil nil [#{"some"} #{"nested"} #{"attributes"}])
(parse-stream rdr nil nil [#{:using} #{:key-fn}] true)

Few more tests added to track the performance.

Right now parts can be retrieved by the path passed as a set of predicates as a second argument.

Lazy seq will return when the path points to the key which contains the array and argument detach-children? is true.

;; {"foo": [{"bar": 1}, {"bar": 2}]}
(take 1 (parse-stream rdr nil nil [#{"foo"}] true))
;; => ({"bar" 1})

And you can continue the path after array pointer to return partial objects from array as lazy seq.

(def jstr "{\"foo\": [{\"bar\": 1}, {\"bar\": 2, \"baz\": 3}]}")
(def x (parse-string jstr nil nil [#{"foo"} #{"bar"}] true))
(type x) ;; => clojure.lang.LazySeq
(second x) ;; => {"bar" 2}

As you can see it works with parse-string as well (and with all other public functions)

Should fix #62

TODO:

  • consider to use jsonPath to build a list of predicates from the string

@DeLaGuardo DeLaGuardo changed the title Add parsed-partial function [WIP] Add parsed-partial function Oct 12, 2018
Star should be an indicator of "every element from array lazily"
Update all public functions to have two extra arguments:

  * pred-xs for a collection of functions (f Any x -> Boolean y)
  * detach-children? - boolean

Collection of predicates will be used to check if key/value pair from object
need to be kept in resulted map or not. Predicates are position sencitive -
representing the level of nesting (left to right).

detach-children? option should be used to return lazy collection without parent
elements. E.g. for that json: {"foo": [1, 2, 3]} and for that pred-xs:
[#{"foo"}] with `true` as a detach-children? lazy-seq with '(1 2 3) will return.
@DeLaGuardo DeLaGuardo changed the title [WIP] Add parsed-partial function [WIP] Add ability to parse parts of json Oct 15, 2018
@DeLaGuardo DeLaGuardo closed this Oct 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Can't lazily parse large JSON objects
1 participant