< and > symbols replaced by their unicode codepoint values

[rdilipk]

I have a sql_insert processor that inserts a JSON document into a table (postgres) with a column of type json. I find that all the < and > characters replaced by \u003c and \u003e respectively. I don't see this problem if such documents are inserted into postgres by any other means and I don't think JSON itself requires any such transformation. Is this a known issue?

[rdilipk]

This seems like the way json.Marshal works in Go. Unmarshalling gets rid of it. So never mind.

[rdilipk]

Thanks @mihaitodor for getting back to me. My args_mapping does currently have a .string() call. I will remove it on Monday and verify if the escaping goes away. Thanks for tracking this down! On the issue of why json.Marshal escapes like this by default, there's a thread about it here. Seems to have been closed as WAD.

While I have your attention, can I request you to take a quick peek at this one too?

Thanks!

[mihaitodor]

Seems to have been closed as WAD.

Thanks for digging up that thread. I'm not too surprised, but glad they at least provided some means to turn this off.

While I have your attention, can I request you to take a quick peek at #1247 one too?

Oops, was meaning to get to that one too. I'll reply there. Thanks for the reminder!

[rdilipk]

@mihaitodor I think this isn't as straightforward as I thought it would be. Especially if I am trying to carve out a json snippet from a larger json document. If I modify the example you used as:

{"baz":{"foo": "<bar>"}}

Then, the only way I can think of to set up args mapping is:

args_mapping = root = [this.baz.string()]
or
args_mapping = root = [json().baz.string()]

IOW, I cannot get rid of the string() call. Without that, the postgres json type and this.baz (or json().baz) end up being incompatible types. I can't see how content() works here as it seems to refer to the entire json document.

[mihaitodor]

Try using https://www.benthos.dev/docs/guides/bloblang/methods#bytes instead of string() and I hope that does the trick for now.

[mihaitodor]

Actually, scratch that. I think the only way to work around it right now is to replace those escape codes as I described in this new issue: #1253, but maybe I'm missing something.