Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading concatenated json's from an istream #210

Closed
habemus-papadum opened this issue Feb 19, 2016 · 8 comments
Closed

Reading concatenated json's from an istream #210

habemus-papadum opened this issue Feb 19, 2016 · 8 comments

Comments

@habemus-papadum
Copy link

Hi,
Your library is great (and because I am lazy about being lazy, I've typed nlohmann::json so many times that I feel like I practically know you)

I have a file of multiple concatenated, pretty printed json objects that I want to parse. Is there a way with the current library to split a string in individual json which than can be dealt with as usual (of course, any other reasonable way of skinning the same cat would be appreciated.) Otherwise, I've seen json PCRE regexes that might also do the trick. Do think that is the way to go?

Thanks!

@gregmarr
Copy link
Contributor

Is there anything between the json objects besides whitespace, or maybe newlines only between the objects but nowhere else? Since you say pretty printed, I'm guessing the newlines is no. Do you have a small sample you can share?

@habemus-papadum
Copy link
Author

{
  "pi": 3.141,
  "happy": true,
  "name": "Niels",
  "nothing": null,
  "answer": {
    "everything": 42
  },
}
{
  "pi": 3.141,
  "happy": true,
  "name": "Niels",
  "nothing": null,
  "answer": {
    "everything": 42
  },
}

... would be an example.

basically I am trying to slurp in data from a process which is periodically appending
objects to a file (and I don't want to force the json to be single line to make manual testing/debugging easier) -- my process wakes up, slurps the data, truncates the file, and the process repeats (processes use file locking to avoid races, etc)

thanks!

@whackashoe
Copy link
Contributor

One thing you could try is iterate over the characters and count how many { you have and every time you encounter a } subtract from that counter. Once the counter is 0 you know the end of the first json object.

@habemus-papadum
Copy link
Author

arg, of course. that sounds good. Thanks for the help!

@whackashoe
Copy link
Contributor

No prob, have a nice weekend!

@nlohmann
Copy link
Owner

Hi @lilinjn, does the idea of @whackashoe help you?

The format you are using reminds me of JSON Lines, but your input seems to be pretty-printed. I have currently no plans supporting such an extension.

@habemus-papadum
Copy link
Author

Yes -- I think it's simpler to roll my own reader based on @whackashoe's
point. Thanks everyone.

On Sat, Feb 20, 2016 at 5:37 AM, Niels notifications@github.com wrote:

Hi @lilinjn https://github.com/lilinjn, does the idea of @whackashoe
https://github.com/whackashoe help you?

The format you are using reminds me of JSON Lines http://jsonlines.org,
but your input seems to be pretty-printed. I have currently no plans
supporting such an extension.


Reply to this email directly or view it on GitHub
#210 (comment).

@nlohmann
Copy link
Owner

Ok, then thanks @whackashoe for helping!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants