Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-6505] Let emoji encoded properly for json.dumps() #7399

Merged
merged 1 commit into from Feb 13, 2020

Conversation

damon09273
Copy link
Contributor

@damon09273 damon09273 commented Feb 11, 2020

Make sure you have checked all steps below.

Jira

Description

Emoji can't be encoded properly when json.dumps() and 'UTF-8', problem fixed by adding parameter ensure_ascii=False.
In line BaseSQLToGoogleCloudStorageOperato.

For example

the emoji 🍻 encoded differentially when in use or not use ensure_ascii=False.

In Use (correct UTF-8 encode): "\xf0\x9f\x8d\xbb"

Not Use:"\\ud83c\\udf7b"

Ref: https://stackoverflow.com/questions/51183947/python-json-dumps-doesnt-encode-emojis-properly

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

Make sure you have checked all steps below.

Jira

  • My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
    • https://issues.apache.org/jira/browse/AIRFLOW-XXX
    • In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.
    • In case you are proposing a fundamental code change, you need to create an Airflow Improvement Proposal (AIP).
    • In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.

Description

  • Here are some details about my PR, including screenshots of any UI changes:

Tests

  • My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

@boring-cyborg boring-cyborg bot added the provider:google Google (including GCP) related issues label Feb 11, 2020
@damon09273 damon09273 force-pushed the fix_emoji_encoding_for_sql_to_gcs branch from f48c0e4 to 2f78a4a Compare February 11, 2020 08:06
@damon09273 damon09273 changed the title Let emoji encoded properly for json.dumps() [AIRFLOW-6505] Let emoji encoded properly for json.dumps() Feb 11, 2020
@mik-laj
Copy link
Member

mik-laj commented Feb 12, 2020

Travis is sad. Can you fix it?

@damon09273 damon09273 force-pushed the fix_emoji_encoding_for_sql_to_gcs branch 4 times, most recently from 09e5181 to 1770366 Compare February 13, 2020 08:09
@damon09273 damon09273 force-pushed the fix_emoji_encoding_for_sql_to_gcs branch from 1770366 to 1e38000 Compare February 13, 2020 08:26
@codecov-io
Copy link

codecov-io commented Feb 13, 2020

Codecov Report

Merging #7399 into master will decrease coverage by 54.43%.
The diff coverage is 0%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #7399       +/-   ##
===========================================
- Coverage    86.6%   32.17%   -54.44%     
===========================================
  Files         873      873               
  Lines       40725    40855      +130     
===========================================
- Hits        35271    13145    -22126     
- Misses       5454    27710    +22256
Impacted Files Coverage Δ
...low/providers/google/cloud/operators/sql_to_gcs.py 24.75% <0%> (-67.33%) ⬇️
...low/contrib/operators/wasb_delete_blob_operator.py 0% <0%> (-100%) ⬇️
...ing_platform/example_dags/example_display_video.py 0% <0%> (-100%) ⬇️
airflow/contrib/hooks/vertica_hook.py 0% <0%> (-100%) ⬇️
airflow/contrib/sensors/__init__.py 0% <0%> (-100%) ⬇️
airflow/hooks/mssql_hook.py 0% <0%> (-100%) ⬇️
airflow/hooks/webhdfs_hook.py 0% <0%> (-100%) ⬇️
airflow/contrib/sensors/emr_base_sensor.py 0% <0%> (-100%) ⬇️
...irflow/contrib/operators/slack_webhook_operator.py 0% <0%> (-100%) ⬇️
airflow/hooks/jdbc_hook.py 0% <0%> (-100%) ⬇️
... and 747 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e00243...1e38000. Read the comment docs.

@damon09273
Copy link
Contributor Author

@mik-laj Fixed, thank you.

@mik-laj mik-laj merged commit 2381c82 into apache:master Feb 13, 2020
galuszkak pushed a commit to FlyrInc/apache-airflow that referenced this pull request Mar 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
provider:google Google (including GCP) related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants