Fix TCP buffer parsing to find JSON statements to correctly

[DaveTBlake]

Fix TCP buffer parsing to find JSON statements by correctly ignoring escaped quotes, and brackets {, }, [, ], when inside quote pairs.

This is a simple fix to issue #15562 - where sending some JSON requests via TCP raw sockets (that is to port 9090) either fail or hang before even being passed to rapidjson for parsing. The same JSON requests made via webserver do not have this issue.
An example request (with } in a parameter value) is

{"jsonrpc": "2.0", "params": {"directory": "/etc/abc}", "media": "files", 
"properties": ["file", "lastmodified"]}, "method": "Files.GetDirectory", "id": "libDirectory"}

This looks like standard TCP protocol parsing to me, corrcting the approach we already take, and I see no reason not to fix the issue in this way. We could do the whole thing differently of course, but that kind of refactor is best left until someone has the time and interest to do so.

@MilhouseVH care to test it out?

[MilhouseVH]

@DaveTBlake Thanks for having a go at this. I've given it a very quick test and yes, it works for the example request, when using double-quotes

However the parse errors are still there if you swap " for ' in the example request, ie.:

{'jsonrpc': '2.0', 'params': {'directory': '/storage/abc}', 'media': 'files', 'properties': ['file', 'lastmodified']}, 'method': 'Files.GetDirectory', 'id': 'libDirectory'}

as this is equally valid JSON, but fails to parse as before, so I think the inString determination will also need to consider strings enclosed by apostrophes and/or double-quotes, and requests that might use a mixture of apostrophe/double-quote strings, and of course handle apostrophes within double-quote strings (and vice versa)...

For example, this request will hang with no response (the strings all use apostrophes, but the directory property includes an embedded double-quote):

{'jsonrpc': '2.0', 'params': {'directory': '/storage/abc"', 'media': 'files', 'properties': ['file', 'lastmodified']}, 'method': 'Files.GetDirectory', 'id': 'libDirectory'}

I suppose it's also possible that a malformed request with unbalanced single/double quotes etc. might easily hang the TCP server, but I suppose that is possible already with unbalanced braces.

I'm not sure how complicated/safe/reliable this kind of single/double quote parsing might be as it's unlikely to be trivial given all the possible permutations but I'll let you be the judge of that!

[rmrector]

I'm cool with this quick fix in general, but I see one small gotcha.

[rmrector]

However the parse errors are still there if you swap " for ' in the example request

@MilhouseVH Nah, that's not valid JSON, JSON keeps it simple by requiring double-quoted strings.

[yol]

@DaveTBlake What do you think about adding unit tests? This is the kind of code that makes it quite easy to include some and could really benefit.

[DaveTBlake]

Thanks for testing @MilhouseVH (you had me for a moment with ' - all my testing to pot! But no, phew)

Thanks for catch @rmrector - away from computer today, but will implement once I'm back at it.

@pkerling unit tests sounds a find idea, especially if you would like to help out with that. The test units and I have not met, let alone sending JSON via TCP using them

[yol]

@pkerling unit tests sounds a find idea, especially if you would like to help out with that. The test units and I have not met, let alone sending JSON via TCP using them

I can give you some hints, sure. I think it would be fine to test just the behavior you changed here, i.e. how multiple requests are split. That is much more easily doable than going through the whole TCP stack. For that to work, I guess it would be practical to separate the request splitting part from the TCP server, so it can be tested on its own.
Let me know if you need further help.

[MilhouseVH]

@MilhouseVH Nah, that's not valid JSON, JSON keeps it simple by requiring double-quoted strings.

Thanks, I'd always thought single and double quotes were interchangeable with JSON but apparently not, so that does simplify the fix!

I'll include this change in nightlies and report any further feedback.

[DaveTBlake]

I'll include this change in nightlies and report any further feedback.

Thanks @MilhouseVH, PR updated with Ryan's correction

[MartijnKaijser]

Any objections?

[MilhouseVH]

There have been no complaints so far from any test users, so no objection from me.

[DaveTBlake]

Support from Millhouse and no objections, time for merge :)