Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse/Encode logfmt is not idempotent when values have double quotes #777

Open
srstrickland opened this issue Mar 30, 2024 · 3 comments
Open
Labels
type: bug A code related bug vrl: stdlib Changes to the standard library

Comments

@srstrickland
Copy link

In general, any encode/decode function pairs should be idempotent; that is, this should always be true:

x == decode(encode(x))

It appears to not be the case with encode_logfmt and parse_logfmt, as demonstrated here, where result.key has additional escape characters.

Taken to the extreme:

.obj.key = s'this has a " quote'
parse_logfmt!(encode_logfmt(parse_logfmt!(encode_logfmt(parse_logfmt!(encode_logfmt(parse_logfmt!(encode_logfmt(parse_logfmt!(encode_logfmt(.obj))))))))))

yields:

{ "key": "this has a \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\" quote" }

This isn't really causing me any headaches, and the fact that it hasn't been raised yet probably indicates the same for other folks, just thought it was worth bringing up, as I stumbled on it today.

@jszwedko jszwedko added type: bug A code related bug vrl: stdlib Changes to the standard library labels Apr 1, 2024
@drmason13
Copy link
Contributor

Thanks for raising this, I think you're absolutely right about the principle. I would like to take a stab at fixing this.

@drmason13
Copy link
Contributor

drmason13 commented May 7, 2024

I got a test reproducing the error this evening. The problem is the same in both directions:
encode(decode())
decode(encode())

decoding does not remove escape characters.
encoding escapes " and \ characters. EDIT: also \n characters.
Either decoding should parse and remove the escape characters, or encoding should not add escape characters.

I've been testing the encode/decode value functions, so I don't think it's just the logfmt functions.

@drmason13
Copy link
Contributor

Adding some parsing escape removal to parsing sounds like the right fix to me.

It would be a breaking change for people who are relying on escape characters being perserved when parsing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug A code related bug vrl: stdlib Changes to the standard library
Projects
None yet
Development

No branches or pull requests

3 participants