Small but useful wrapper above the erlang's JSON parsing toolkit jsx to stream json elements from an Elixir binary stream.
stream_path = ["data", 1, "actions"]
{actions_stream, doc_fun} =
"example.json"
|> File.stream!([],2048)
|> JSONStream.stream(stream_path)
Enum.to_list(actions_stream )
# Will output:
# [
# %{"link" => "http://www.facebook.com/X998/posts/Y998", "name" => "Like"},
# %{"link" => "http://www.facebook.com/X998/posts/Y998", "name" => "Comment"}
# ]
doc_fun.()
# Will output:
# %{
# "data" => [
# %{
# "actions" => [
# %{
# "link" => "http://www.facebook.com/X999/posts/Y999",
# "name" => "Comment"
# },
# %{"link" => "http://www.facebook.com/X999/posts/Y999", "name" => "Like"}
# ],
# "created_time" => "2010-08-02T21:27:44+0000",
# "from" => %{"id" => "X12", "name" => "Tom Brady"},
# "id" => "X999_Y999",
# "message" => "Looking forward to 2010!",
# "type" => "status",
# "updated_time" => "2010-08-02T21:27:44+0000"
# },
# %{
# "actions" => [],
# "created_time" => "2010-08-02T21:27:44+0000",
# "from" => %{"id" => "X18", "name" => "Peyton Manning"},
# "id" => "X998_Y998",
# "message" => "Where's my contract?",
# "type" => "status",
# "updated_time" => "2010-08-02T21:27:44+0000"
# }
# ]
# }stream_path describe the JSON path (string for map key and integer
for array index), to the Array/Object you want to stream Element/{key,value}.
actions_stream is the stream of element in case stream_path
targets an array, or a stream of {k,v} in case stream_path
targets a map.
doc_fun is a function ()->doc where doc will be the whole
parsed json document, but without the elements from the stream. It
will be accessible only after the whole stream will be runned.
Else, doc_fun.()-> :stream_not_finished.
The package can be installed as:
- Add json_stream to your list of dependencies in
mix.exs:
def deps do
[{:json_stream, "~> 0.0.2"}]
end- Ensure json_stream is started before your application:
def application do
[applications: [:json_stream]]
end