HTML Encoded entities in Logs to DataDog

[JanThiel]

Describe the bug
Hey @Pierre-Lannoy, its me again ;-) Hope you are doing well!

Since the last update to decalog 3.4.1 we encounter HTML encoded entities in our DataDog logs:

Alert [1] Fatal error (E_ERROR): “Maximum execution time of 45 seconds exceeded“ at `./wp-content/plugins/decalog/includes/system/class-sharedmemory.php:149`.

The “ and ” are new. It also happens for other HTML encodeable entities whenever they are part of the submitted log message.

I am quite certain this was not the case before we updated to 3.4.1. Yet I cannot completely rule out a change on Datadog side.

To Reproduce
Steps to reproduce the behavior:

1. Connect against Datadog
2. Break something and trigger an error

Expected behavior
The string should appear in DataDog in its natural beauty without these crude transformations.

[Pierre-Lannoy]

Hi @JanThiel !
I'm going well (since Eunice has gone to see what's going on elsewhere), thanks… Hope you too :)

Regarding your issue, that's very strange. The only remembrance I have concerning this type of subject is this commit. But it's about New Relic, not DataDog.

Let me investigate it…

[JanThiel]

If it helps, it is not just the message, it is also within the trace information:

{"call":"WP_REST_Posts_Controller->prepare_item_for_response(WP_Post, WP_REST_Request)","file":"./wp-includes/rest-api/endpoints/class-wp-rest-posts-controller.php:367"},
--
{"call":"WP_REST_Posts_Controller->get_items(WP_REST_Request)","file":"./wp-includes/rest-api/class-wp-rest-server.php:1140"},

Seems to be something more general.

[Pierre-Lannoy]

Yes. It seems Datadog have changed something on its side.
The message is simple to fix. It's many more complex with stack trace.

I will try to globally fix it for the next version - even if I don't know how at this time 🤣

[JanThiel]

Let me raise a ticket with Datadog and see what they reply :-)

[Pierre-Lannoy]

Ah thanks. That's a good idea.

[freshuk]

hi guys, i can confirm same issue with datadog

[Pierre-Lannoy]

Hi @JanThiel !
Did you have some news from Datadog?

[JanThiel]

@Pierre-Lannoy Nope ... awaiting feedback from their technical department.

[JanThiel]

@Pierre-Lannoy Here is the feedback from DD. Can you suggest a way how to deliver what they request?

Hello Jan,

Thank you for providing me with your codebase. I reached out to our Logs team and found out that these (&ldquo) are HTML encoded characters [example].

To resolve this,
would you be able to check your raw logs?

• If your application is not storing raw logs in any file, is it possible to capture or store them in a place so that you can check how your raw logs are formatted?
• If you are seeing these HTML encoded characters, can you configure you logger or application so that these characters are not generated in your logs?

Please let me know if you have any questions on this.

[Pierre-Lannoy]

Hello @JanThiel and thanks for letting me know about DataDog's answer.
What is really strange (and why I doesn't understand their point) is DecaLog doesn't encode or decode HTML entities by itself. The record format has not changed since DecaLog 3.2.0 (2021-09-07). The only possible thing is they changed something from their side regarding json decoding/parsing.

Well, anyway, they're not going to do a rollback. So it's up to DecaLog to adapt…

I will make a POC to decode HTML entities for all fields sent to DataDog and will let you know.

[JanThiel]

I understood their request just as a request to prove that Decalog does not encode.
Is there an easy way (without code hacks) to dump a request before it is sent to Datadog?

[Pierre-Lannoy]

Wowowowo. Found it! It's coming from DecaLog and it was introduced by PHP 8.1 compatibility. So now, I have choice to 8.1 compatibility or correct strings 🤣
Let me investigate it and see how I can work around…

[Pierre-Lannoy]

OK. That's fixed. Will be in the next release (3.5.0).
Sorry for the confused resolution of this issue. And thanks again to have reported it…

[JanThiel]

Glad to hear you found and fixed it 👍🎉
Thanks again 😊

[Pierre-Lannoy]

3.5.0 is out! Can you tell me if it's ok for you now?

[JanThiel]

Thank you very much 😊 I will get back with feedback within the next couple of days

[JanThiel]

I can confirm this is fixed with 3.5.0 🥳

[Pierre-Lannoy]

Great! Thanks for the feedback 👍🏼