Skip to content

use replace function only when null is present - #21523

Merged
Subodh Chaturvedi (subodh1810) merged 4 commits into
masterfrom
fix-replace-null-call
Jan 27, 2023
Merged

Subodh Chaturvedi (subodh1810) merged 4 commits into
masterfrom
fix-replace-null-call

Conversation

@subodh1810

Copy link
Copy Markdown
Contributor

The method cursorCandidate.replaceAll("\u0000", ""); is costly and eats upto 2.6% of total CPU time in syncs. We should only call it if there is null present in the cursor value.

See : #20424 (comment)

@github-actions

github-actions Bot commented Jan 18, 2023 •

Copy link
Copy Markdown
Contributor

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

✅ Sources (25)

Connector Version Changelog Publish
source-alloydb 1.0.36 ✅ ✅
source-alloydb-strict-encrypt 1.0.36 🔵
(ignored)
🔵
(ignored)
source-bigquery 0.2.3 ✅ ✅
source-clickhouse 0.1.15 ✅ ✅
source-clickhouse-strict-encrypt 0.1.15 🔵
(ignored)
🔵
(ignored)
source-cockroachdb 0.1.19 ✅ ✅
source-cockroachdb-strict-encrypt 0.1.19 🔵
(ignored)
🔵
(ignored)
source-db2 0.1.17 ✅ ✅
source-db2-strict-encrypt 0.1.17 🔵
(ignored)
🔵
(ignored)
source-dynamodb 0.1.0 ✅ ✅
source-jdbc 0.3.5 🔵
(ignored)
🔵
(ignored)
source-mongodb-strict-encrypt 0.1.19 🔵
(ignored)
🔵
(ignored)
source-mongodb-v2 0.1.19 ✅ ✅
source-mssql 0.4.28 ✅ ✅
source-mssql-strict-encrypt 0.4.28 🔵
(ignored)
🔵
(ignored)
source-mysql 1.0.21 ✅ ✅
source-mysql-strict-encrypt 1.0.21 🔵
(ignored)
🔵
(ignored)
source-oracle 0.3.22 ✅ ✅
source-oracle-strict-encrypt 0.3.22 🔵
(ignored)
🔵
(ignored)
source-postgres 1.0.42 ✅ ✅
source-postgres-strict-encrypt 1.0.42 🔵
(ignored)
🔵
(ignored)
source-redshift 0.3.16 ✅ ✅
source-scaffold-java-jdbc 0.1.0 🔵
(ignored)
🔵
(ignored)
source-snowflake 0.1.29 ✅ ✅
source-tidb 0.2.2 ✅ ✅
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector Version Changelog Publish
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)

Category Status Actionable Item
Version ❌
mismatch
The version of the connector is different from its normal variant. Please bump the version of the connector.
⚠
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog ⚠
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
❌
changelog missing
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish ⚠
not in seed
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
❌
diff seed version
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@subodh1810

Subodh Chaturvedi (subodh1810) commented Jan 18, 2023 •

Copy link
Copy Markdown
Contributor Author

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3949753893
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3949753893
Python tests coverage:

	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          12      4    67%   16-19
	 source_acceptance_test/config.py                       141      5    96%   87, 93, 239, 243-244
	 source_acceptance_test/conftest.py                     211     95    55%   36, 42-44, 49, 54, 77, 83, 89-91, 110, 115-117, 123-125, 131-132, 137-138, 143, 149, 158-167, 173-178, 193, 217, 248, 254, 262-267, 275-285, 293-306, 311-317, 324-335, 342-358
	 source_acceptance_test/plugin.py                        69     25    64%   22-23, 31, 36, 120-140, 144-148
	 source_acceptance_test/tests/test_core.py              402    115    71%   53, 58, 93-104, 109-116, 120-121, 125-126, 308, 346-363, 376-387, 391-396, 402, 435-440, 478-485, 528-530, 533, 598-606, 618-621, 626, 682-683, 689, 692, 728-738, 751-776
	 source_acceptance_test/tests/test_incremental.py       160     14    91%   58-65, 70-83, 246
	 source_acceptance_test/utils/asserts.py                 39      2    95%   62-63
	 source_acceptance_test/utils/common.py                  94     10    89%   16-17, 32-38, 72, 75
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       133     33    75%   24-27, 46-47, 50-54, 57-58, 73-75, 78-80, 83-85, 88-90, 93-95, 124-125, 159-161, 208
	 source_acceptance_test/utils/json_schema_helper.py     107     13    88%   30-31, 38, 41, 65-68, 96, 120, 192-194
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1609    339    79%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:94: The previous and actual specifications are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_core.py:377: The previous and actual discovered catalogs are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/tests/test_incremental.py:22: `future_state` has a bypass reason, skipping.
================== 54 passed, 6 skipped in 718.52s (0:11:58) ===================

@subodh1810

Subodh Chaturvedi (subodh1810) commented Jan 27, 2023 •

Copy link
Copy Markdown
Contributor Author

/publish connector=connectors/source-postgres-strict-encrypt

🕑 Publishing the following connectors:
connectors/source-postgres-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/4022574089


Connector Did it publish? Were definitions generated?
connectors/source-postgres-strict-encrypt ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@subodh1810

Subodh Chaturvedi (subodh1810) commented Jan 27, 2023 •

Copy link
Copy Markdown
Contributor Author

/publish connector=connectors/source-postgres

🕑 Publishing the following connectors:
connectors/source-postgres
https://github.com/airbytehq/airbyte/actions/runs/4022574459


Connector Did it publish? Were definitions generated?
connectors/source-postgres ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@subodh1810
Subodh Chaturvedi (subodh1810) temporarily deployed to more-secrets January 27, 2023 07:55 — with GitHub Actions Inactive
@subodh1810
Subodh Chaturvedi (subodh1810) temporarily deployed to more-secrets January 27, 2023 07:55 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage 24% ❌

This branch was previously deployed

1 inactive deployment
more-secrets — 4a022f11 Deployed Jan 27, 2023 by octavia-squidington-iii
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants