-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Report JSON parsing errors. #42
Comments
I think this may be a variation on #39. @sourabhnk is this problem repeatable. For example if you rehydrate the same tweet ids and try to generate the CSV does it throw the same error? |
Also, would you be willing to send me your ids and jsonl file that you generated when the error was thrown? You can attach them here or send them to ehs@pobox.com Thanks! |
Hi Ed, |
Yes, if the JSON file is empty that could definitely be a problem. But Hydrator should handle that situation. It hasn't come up before. |
I have shared the csv files by mail. #39 didn't help much in my case. |
@sourabhnk it appears that your tweet ids have been corrupted by opening them with Excel and saving them. Excel is unable to handle the large numbers and overflows the correct value so that it ends with zeros instead of the correct numbers. It is apparent when looking at your files because all the ids four zeros. This means that none of the tweets are able to hydrate, and when you go to convert the JSON to CSV there is nothing to convert. The Hydrator should report when the JSON file is empty. So I'm going to leave this issue open until it does that. |
Yes Ed, I had extracted tsv file and used Excel to save it in csv. Probably that could be the cause. |
Awesome, thank you @sourabhnk. The reporting issue isn't something you did wrong, it is an improvement we need to make in the application! |
Hi Ed,
Having said this, I have almost resolved the issue. If you would like, we can close this thread. |
Thanks for the update @sourabhnk. I am starting to think we should encourage people to share tweet ids in a quoted form in .csv files to prevent this sort of thing from happening. You arent the first person to run into this problem and won't be the last! I'd like to keep this issue open until the Hydrator better reports problems with empty or malformed JSON files. Thank you for your help in diagnosing the problem! |
Getting this error when i run a small sample of ~300 tweet IDs. None of the IDs are getting hydrated.
![hydratorerror](https://user-images.githubusercontent.com/61457807/82531087-4713d500-9b5c-11ea-8b5a-6b42df3e16c2.JPG)
This is what the tool shows:
The text was updated successfully, but these errors were encountered: