[Fix] Fix structured logs in cronjobs

[ViniciustCosta]

Background

In #4781, it was necessary to add a logs filter to propagate the structured fields into GCP Logging for cronjobs in GAE/GKE (this is an unfortunate workaround due to different log configurations set depending on the environment).

However, the way this filter was implemented broke some of the logs, when non-serializable arguments were passed to the python logging extras fields. Thus, we disabled it in #4794.

Description

This fix updates the logs filter (responsible for adding extras into the json_fields for GCP logging) by checking whether the values in logs extras are JSON serializable and calling the _handle_unserializable() function for the negative cases, which simply converts them into strings.

By doing this, we avoid the type errors that occurred in GCP (TypeError: Object is not JSON serializable) while also maintaining the structure of the json for indexed queries.

Tests

In order to test in the GKE environment, I leveraged our development environment with candidate release. I deployed a debug cronjob (debug-logs) calling the same log entry that broke the cleanup cronjob before.

The test results, before and after landing the fix, were:

• Current fail state (link to logs):
  
  

• After the fix (link to logs):
  
  

• Also, in order to verify if the structured json fields were correctly maintained, I added a more complex log entry:

  logs.warning('Test structured logs.', test_dict_field={'key1':'v1', 'key2':'v2'}, test_set_field={'1', '2'}

  

b/417782114