Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Reading stream with json.NewDecoder fails with "invalid character '\r' in string literal" if large base64 byte present #39874
Dear Go team,
first off, thank you for this amazingly productive language :) I'm writing you because I believe that I have found an issue with the streaming JSON decoder in the standard library.
This crashes with "invalid character '\r' in string literal":
if presented with this input data:
In contrast, this main function works, because the JSON is now accumulated in the scanner before parsing:
Example data that works in both cases would be
In my opinion, the issue is the space character invented here:
in combination with reading the input in 512 byte chunks here:
So while parsing with the streaming JSON decoder, it will only read up until
(or exactly 512 bytes)
As expected, changing
please fill out the issue template when filing reports
this is more likely a limitation of your terminal emulator than an issue with json, related: https://unix.stackexchange.com/questions/204815/terminal-does-not-accept-pasted-or-typed-lines-of-more-than-1024-characters
Thank you for pointing that out. I agree with you that this is most likely an issue with my terminal when data is being read from it with a small buffer, like the default 512 that json.NewDecoder uses.
In other words, the exact same terminal emulator works if I force Go to read from it with a larger 2048 byte buffer.
I'll close this issue, then. Let's hope that the next person observing the problem will find this thread so that they can pinpoint the cause more quickly than me.