Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Fix TCP buffer parsing to find JSON statements to correctly #15591
Fix TCP buffer parsing to find JSON statements by correctly ignoring escaped quotes, and brackets
This is a simple fix to issue #15562 - where sending some JSON requests via TCP raw sockets (that is to port 9090) either fail or hang before even being passed to rapidjson for parsing. The same JSON requests made via webserver do not have this issue.
This looks like standard TCP protocol parsing to me, corrcting the approach we already take, and I see no reason not to fix the issue in this way. We could do the whole thing differently of course, but that kind of refactor is best left until someone has the time and interest to do so.
@MilhouseVH care to test it out?
referenced this pull request
Feb 23, 2019
@DaveTBlake Thanks for having a go at this. I've given it a very quick test and yes, it works for the example request, when using double-quotes
However the parse errors are still there if you swap
as this is equally valid JSON, but fails to parse as before, so I think the
For example, this request will hang with no response (the strings all use apostrophes, but the
I suppose it's also possible that a malformed request with unbalanced single/double quotes etc. might easily hang the TCP server, but I suppose that is possible already with unbalanced braces.
I'm not sure how complicated/safe/reliable this kind of single/double quote parsing might be as it's unlikely to be trivial given all the possible permutations but I'll let you be the judge of that!
Thanks for testing @MilhouseVH (you had me for a moment with ' - all my testing to pot! But no, phew)
Thanks for catch @rmrector - away from computer today, but will implement once I'm back at it.
@pkerling unit tests sounds a find idea, especially if you would like to help out with that. The test units and I have not met, let alone sending JSON via TCP using them
I can give you some hints, sure. I think it would be fine to test just the behavior you changed here, i.e. how multiple requests are split. That is much more easily doable than going through the whole TCP stack. For that to work, I guess it would be practical to separate the request splitting part from the TCP server, so it can be tested on its own.
Thanks, I'd always thought single and double quotes were interchangeable with JSON but apparently not, so that does simplify the fix!
I'll include this change in nightlies and report any further feedback.