PostgresToGCSOperator does not allow nested JSON

[xfoursea]

Apache Airflow Provider(s)

google

Versions of Apache Airflow Providers

apache-airflow-providers-google==6.3.0

Apache Airflow version

2.1.4

Operating System

macOS Big Sur version 11.6.1

Deployment

Composer

Deployment details

No response

What happened

Postgres JSON column output contains extra \:

{"info": "{\"phones\": [{\"type\": \"mobile\", \"phone\": \"001001\"}, {\"type\": \"fix\", \"phone\": \"002002\"}]}", "name": null}

While in the previous version the output looks like
{"info": {"phones": [{"phone": "001001", "type": "mobile"}, {"phone": "002002", "type": "fix"}]}, "name": null}

The introduced extra \ will cause JSON parsing error in following GCSToBigQueryOperator

What you think should happen instead

The output should NOT contain extra \:
{"info": {"phones": [{"phone": "001001", "type": "mobile"}, {"phone": "002002", "type": "fix"}]}, "name": null}

It is caused by this new code change in https://github.com/apache/airflow/blob/main/airflow/providers/google/cloud/transfers/postgres_to_gcs.py
should comment out this block

if isinstance(value, dict):
return json.dumps(value)

How to reproduce

Try to output a Postgres table with JSON column --- you may use the the info above as example.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[eladkal]

@pierrejeambrun would you like to take a look at this?

[pierrejeambrun]

Hello guys,

Yes I can take a look @eladkal, thank you :)