Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 fix: Normalization date-time should handle empty strings "" #6379

Merged
merged 26 commits into from Oct 8, 2021
Merged

🐛 fix: Normalization date-time should handle empty strings "" #6379

merged 26 commits into from Oct 8, 2021

Conversation

annalvova05
Copy link
Contributor

@annalvova05 annalvova05 commented Sep 22, 2021 •
edited by antixar

What

Fix normalization date-time should handle empty strings "" #5480

How

Describe the solution

Recommended reading order

base-normalization/normalization/transform_catalog/stream_processor.py
base-normalization/dbt-project-template/macros/cross_db_utils/type_conversions.sql

Pre-merge Checklist

Updating a connector

Community member or Airbyter

  • Grant edit access to maintainers (instructions)
  • Secrets in the connector's spec are annotated with airbyte_secret
  • Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
  • Code reviews completed
  • Documentation updated
    • Connector's README.md
    • Connector's bootstrap.md. See description and examples
    • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
  • PR name follows PR naming conventions
  • Connector version bumped like described here

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

  • Create a non-forked branch based on this PR and test the below items on it
  • Build is successful
  • Credentials added to Github CI. Instructions.
  • /test connector=connectors/<name> command is passing.
  • New Connector version released on Dockerhub by running the /publish command described here

@annalvova05 annalvova05 self-assigned this Sep 22, 2021
@github-actions github-actions bot added area/worker Related to worker normalization labels Sep 22, 2021
@annalvova05
Copy link
Contributor Author

annalvova05 commented Sep 22, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1262285863
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1262285863
🐛 https://gradle.com/s/sijhl3t4esopu

@jrhizor jrhizor temporarily deployed to more-secrets September 22, 2021 14:52 Inactive
@annalvova05 annalvova05 temporarily deployed to more-secrets September 22, 2021 15:56 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Sep 22, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1262532594
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1262532594
🐛 https://gradle.com/s/aoyiszeszro6k

@jrhizor jrhizor temporarily deployed to more-secrets September 22, 2021 15:59 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Sep 23, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1264734776
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1264734776
🐛 https://gradle.com/s/asbqws7hhqdzm

@annalvova05 annalvova05 temporarily deployed to more-secrets September 23, 2021 06:33 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets September 23, 2021 06:35 Inactive
@annalvova05 annalvova05 temporarily deployed to more-secrets September 23, 2021 10:36 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Sep 23, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1265454087
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1265454087
🐛 https://gradle.com/s/jwur5xr3he5gm

@jrhizor jrhizor temporarily deployed to more-secrets September 23, 2021 10:39 Inactive
@annalvova05
Merge remote-tracking branch 'origin/master' into alvova/5480-fix-emp…
2b86610 
…ty-strings-normalization

# Conflicts:
#	airbyte-integrations/bases/base-normalization/integration_tests/normalization_test_output/snowflake/test_primary_key_streams/final/airbyte_tables/TEST_NORMALIZATION/DEDUP_EXCHANGE_RATE_SCD.sql
#	airbyte-integrations/bases/base-normalization/integration_tests/normalization_test_output/snowflake/test_primary_key_streams/final/airbyte_tables/TEST_NORMALIZATION/EXCHANGE_RATE.sql
#	airbyte-integrations/bases/base-normalization/integration_tests/normalization_test_output/snowflake/test_primary_key_streams/models/generated/airbyte_ctes/TEST_NORMALIZATION/DEDUP_EXCHANGE_RATE_AB2.sql
#	airbyte-integrations/bases/base-normalization/integration_tests/normalization_test_output/snowflake/test_primary_key_streams/models/generated/airbyte_ctes/TEST_NORMALIZATION/EXCHANGE_RATE_AB2.sql
#	airbyte-integrations/bases/base-normalization/normalization/transform_catalog/stream_processor.py
@annalvova05
fix for snowflake
a699acf
@annalvova05 annalvova05 temporarily deployed to more-secrets September 23, 2021 11:56 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Sep 23, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1265687924
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1265687924
🐛 https://gradle.com/s/d7pqpv63lddvs

@jrhizor jrhizor temporarily deployed to more-secrets September 23, 2021 11:58 Inactive
@annalvova05
Merge remote-tracking branch 'origin/master' into alvova/5480-fix-emp…
d46028b 
…ty-strings-normalization

# Conflicts:
#	airbyte-integrations/bases/base-normalization/Dockerfile
#	airbyte-workers/src/main/java/io/airbyte/workers/normalization/DefaultNormalizationRunner.java
@annalvova05
fix for mysql
34dd6db
@annalvova05
Copy link
Contributor Author

annalvova05 commented Sep 23, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1266502230
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1266502230
🐛 https://gradle.com/s/iw5gcpgcuvgps

@annalvova05 annalvova05 temporarily deployed to more-secrets September 23, 2021 15:42 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 7, 2021 10:43 Inactive
@annalvova05
Merge remote-tracking branch 'origin/master' into alvova/5480-fix-emp…
5d692a8 
…ty-strings-normalization

# Conflicts:
#	airbyte-integrations/bases/base-normalization/Dockerfile
#	airbyte-integrations/bases/base-normalization/dbt-project-template/macros/cross_db_utils/type_conversions.sql
#	airbyte-integrations/bases/base-normalization/integration_tests/dbt_integration_test.py
#	airbyte-integrations/bases/base-normalization/normalization/transform_catalog/stream_processor.py
#	airbyte-workers/src/main/java/io/airbyte/workers/normalization/DefaultNormalizationRunner.java
@annalvova05
add datetime normalization for mssql
81bcf4f
@annalvova05
Copy link
Contributor Author

annalvova05 commented Oct 7, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1317193394
❌ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1317193394
🐛 https://gradle.com/s/rl4cakwonttxk

@annalvova05 annalvova05 temporarily deployed to more-secrets October 7, 2021 17:12 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 7, 2021 17:13 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Oct 8, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319421614
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319421614
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 367    215    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     39    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1070    410    62%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 367    215    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     39    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1070    410    62%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     12    92%
	 normalization/transform_catalog/destination_name_transformer.py     120      4    97%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 367     32    91%
	 normalization/transform_catalog/table_name_registry.py              174     51    71%
	 normalization/transform_catalog/transform.py                         45     30    33%
	 normalization/transform_catalog/utils.py                             33      0   100%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     48    68%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1070    183    83%

@annalvova05 annalvova05 temporarily deployed to more-secrets October 8, 2021 07:45 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 8, 2021 07:46 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Oct 8, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319615777
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319615777
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370    218    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     39    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1073    413    62%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370    218    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     39    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1073    413    62%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     12    92%
	 normalization/transform_catalog/destination_name_transformer.py     120      4    97%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370     34    91%
	 normalization/transform_catalog/table_name_registry.py              174     51    71%
	 normalization/transform_catalog/transform.py                         45     30    33%
	 normalization/transform_catalog/utils.py                             33      0   100%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     48    68%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1073    185    83%

@annalvova05 annalvova05 temporarily deployed to more-secrets October 8, 2021 08:47 Inactive
@jrhizor jrhizor temporarily deployed to more-secrets October 8, 2021 08:48 Inactive
@github-actions github-actions bot added the area/connectors Connector related issues label Oct 8, 2021
@annalvova05 annalvova05 temporarily deployed to more-secrets October 8, 2021 09:07 Inactive
@annalvova05
Merge remote-tracking branch 'origin/master' into alvova/5480-fix-emp…
86b48e8 
…ty-strings-normalization

# Conflicts:
#	docs/understanding-airbyte/basic-normalization.md
@annalvova05
bump version
c8e2529
@annalvova05 annalvova05 temporarily deployed to more-secrets October 8, 2021 09:21 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Oct 8, 2021 •
edited by github-actions bot

/test connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319744789
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319744789
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        74      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       18     11    39%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 47     20    57%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  860    419    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370    218    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     39    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1073    413    62%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     77    46%
	 normalization/transform_catalog/destination_name_transformer.py     120      6    95%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370    218    41%
	 normalization/transform_catalog/table_name_registry.py              174     34    80%
	 normalization/transform_catalog/transform.py                         45     26    42%
	 normalization/transform_catalog/utils.py                             33      7    79%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     39    74%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1073    413    62%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                              Stmts   Miss  Cover
	 -------------------------------------------------------------------------------------
	 main_dev_transform_catalog.py                                         3      3     0%
	 main_dev_transform_config.py                                          3      3     0%
	 normalization/__init__.py                                             4      0   100%
	 normalization/destination_type.py                                    12      0   100%
	 normalization/transform_catalog/__init__.py                           2      0   100%
	 normalization/transform_catalog/catalog_processor.py                143     12    92%
	 normalization/transform_catalog/destination_name_transformer.py     120      4    97%
	 normalization/transform_catalog/reserved_keywords.py                 11      0   100%
	 normalization/transform_catalog/stream_processor.py                 370     32    91%
	 normalization/transform_catalog/table_name_registry.py              174     51    71%
	 normalization/transform_catalog/transform.py                         45     30    33%
	 normalization/transform_catalog/utils.py                             33      0   100%
	 normalization/transform_config/__init__.py                            2      0   100%
	 normalization/transform_config/transform.py                         151     48    68%
	 -------------------------------------------------------------------------------------
	 TOTAL                                                              1073    183    83%

@jrhizor jrhizor temporarily deployed to more-secrets October 8, 2021 09:28 Inactive
@annalvova05
Copy link
Contributor Author

annalvova05 commented Oct 8, 2021 •
edited by github-actions bot

/publish connector=bases/base-normalization

🕑 bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319868581
✅ bases/base-normalization https://github.com/airbytehq/airbyte/actions/runs/1319868581

@jrhizor jrhizor temporarily deployed to more-secrets October 8, 2021 10:07 Inactive
@annalvova05 annalvova05 temporarily deployed to more-secrets October 8, 2021 10:46 Inactive
@annalvova05 annalvova05 merged commit ec68f47 into master Oct 8, 2021
@annalvova05 annalvova05 deleted the alvova/5480-fix-empty-strings-normalization branch October 8, 2021 10:57
@jrhizor jrhizor mentioned this pull request Oct 8, 2021
Merged
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
@annalvova05 @ChristopheDuong @schlattk
🐛 fix: Normalization date-time should handle empty strings "" (airbyt…
c038311 
…ehq#6379)

* add empty string normalization for postgres

* add empty string normalization for destinations

* fix

* fix

* fix

* fix for snowflake

* fix for mysql

* fix normalization for mysql

* upd doc

* upd doc

* Update airbyte-integrations/bases/base-normalization/integration_tests/dbt_integration_test.py

Co-authored-by: Christophe Duong <christophe.duong@gmail.com>

* Update airbyte-integrations/bases/base-normalization/integration_tests/dbt_integration_test.py

Co-authored-by: Christophe Duong <christophe.duong@gmail.com>

* bump version

* bump version

* add datetime normalization for mssql

* upd row count for mssql

* upd

* bump version

* upd docs for 0.1.50 normalization version

Co-authored-by: Christophe Duong <christophe.duong@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Phlair Phlair Phlair left review comments

@ChristopheDuong ChristopheDuong ChristopheDuong approved these changes

@antixar antixar antixar approved these changes

Assignees

@annalvova05 annalvova05

Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation area/worker Related to worker normalization
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Normalization date-time should handle empty strings ""
6 participants
@annalvova05 @CLAassistant @ChristopheDuong @Phlair @antixar @jrhizor