Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🐛 Source MySQL/MsSQL Disable index logging for MySQL #25740

Merged
merged 11 commits into from
May 3, 2023
Merged

🐛 Source MySQL/MsSQL Disable index logging for MySQL #25740

merged 11 commits into from
May 3, 2023

Conversation

akashkulk
Copy link
Contributor

@akashkulk akashkulk commented May 1, 2023 •
edited
Loading

Context : #25729 (comment)

Disable this until we have RCA. Re-opening #25740 (comment), Closes #25660

@github-actions
Copy link
Contributor

github-actions bot commented May 1, 2023 •
edited by akashkulk
Loading

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan and you've followed all steps in the Breaking Changes Checklist
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • You, or an Airbyter, have run /test successfully on this PR - or on a non-forked branch
  • You, or an Airbyter, have run /publish successfully on this PR - or on a non-forked branch
  • You've updated the connector's metadata.yaml file (new!)
  • The Octavia bot updated the source_definitions.yaml or destination_definitions.yaml, or you ran processResources manually (deprecated)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@akashkulk akashkulk marked this pull request as ready for review May 1, 2023 23:32
@akashkulk akashkulk requested a review from a team as a code owner May 1, 2023 23:32
@akashkulk
Copy link
Contributor Author

akashkulk commented May 1, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4855954248
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4855954248
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
=================== 68 passed, 5 skipped in 84.45s (0:01:24) ===================

@akashkulk
Copy link
Contributor Author

akashkulk commented May 1, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4855954832
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4855954832
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 22 passed, 6 skipped, 30 warnings in 17.60s ==================

@akashkulk akashkulk mentioned this pull request May 1, 2023
Closed
@github-actions
Copy link
Contributor

github-actions bot commented May 1, 2023 •
edited
Loading

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

✅ Sources (22)
Connector Version Changelog Publish
source-alloydb 2.0.28 ✅ ✅
source-alloydb-strict-encrypt 2.0.28 🔵
(ignored)		 🔵
(ignored)
source-bigquery 0.2.3 ✅ ✅
source-clickhouse 0.1.17 ✅ ✅
source-clickhouse-strict-encrypt 0.1.17 🔵
(ignored)		 🔵
(ignored)
source-cockroachdb 0.1.22 ✅ ✅
source-cockroachdb-strict-encrypt 0.1.22 🔵
(ignored)		 🔵
(ignored)
source-db2 0.1.19 ✅ ✅
source-db2-strict-encrypt 0.1.19 🔵
(ignored)		 🔵
(ignored)
source-mssql 1.0.16 ✅ ✅
source-mssql-strict-encrypt 1.0.16 🔵
(ignored)		 🔵
(ignored)
source-mysql 2.0.20 ✅ ✅
source-mysql-strict-encrypt 2.0.20 🔵
(ignored)		 🔵
(ignored)
source-oracle 0.3.24 ✅ ✅
source-oracle-strict-encrypt 0.3.24 🔵
(ignored)		 🔵
(ignored)
source-postgres 2.0.28 ✅ ✅
source-postgres-strict-encrypt 2.0.28 🔵
(ignored)		 🔵
(ignored)
source-redshift 0.3.16 ✅ ✅
source-scaffold-java-jdbc 0.1.0 🔵
(ignored)		 🔵
(ignored)
source-snowflake 0.1.34 ✅ ✅
source-teradata 0.1.0 ✅ ✅
source-tidb 0.2.4 ✅ ✅
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)
Connector Version Changelog Publish
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)
Category Status Actionable Item
Version ❌
mismatch		 The version of the connector is different from its normal variant. Please bump the version of the connector.
⚠
doc not found		 The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog ⚠
doc not found		 The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
❌
changelog missing		 There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish ⚠
not in seed		 The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
❌
diff seed version		 The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@akashkulk akashkulk mentioned this pull request May 1, 2023
Closed
@akashkulk
Copy link
Contributor Author

akashkulk commented May 1, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/4856080126
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/4856080126
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 22 passed, 6 skipped, 30 warnings in 17.93s ==================

prateekmukhedkar
prateekmukhedkar approved these changes May 2, 2023
View reviewed changes
Copy link
Contributor

@prateekmukhedkar prateekmukhedkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Don't we want to publish source-postgres also?

@akashkulk
Copy link
Contributor Author

Approving. Don't we want to publish source-postgres also?

There isn't any logical change in source-postgres here. I'm actually just removing the common logging and moving it into Postgres override. AFAIK in the dashboards, we are only seeing errors in source-mysql.

My preference is to keep the source-postgres logging as it's been useful in helping customers identify issues with their indexes, debugging slow queries, etc

@akashkulk akashkulk changed the title Disable index logging for MySQL 🐛 Source MySQL/MsSQL Disable index logging for MySQL May 2, 2023
@akashkulk
Copy link
Contributor Author

/publish connector=connectors/source-mssql run-tests=false

@akashkulk
Copy link
Contributor Author

/publish connector=connectors/source-mssql-strict-encrypt run-tests=false

@akashkulk
Copy link
Contributor Author

/publish connector=connectors/source-mysql run-tests=false

@akashkulk
Copy link
Contributor Author

akashkulk commented May 2, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-mysql run-tests=false

🕑 Publishing the following connectors:
connectors/source-mysql
https://github.com/airbytehq/airbyte/actions/runs/4865275826


Connector Version Did it publish? Were definitions generated?
connectors/source-mysql 2.0.20 ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@akashkulk
Copy link
Contributor Author

akashkulk commented May 2, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-mysql-strict-encrypt run-tests=false

🕑 Publishing the following connectors:
connectors/source-mysql-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/4865548699


Connector Version Did it publish? Were definitions generated?
connectors/source-mysql-strict-encrypt 2.0.20 ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@akashkulk
Copy link
Contributor Author

akashkulk commented May 2, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-mssql run-tests=false

🕑 Publishing the following connectors:
connectors/source-mssql
https://github.com/airbytehq/airbyte/actions/runs/4865665764


Connector Version Did it publish? Were definitions generated?
connectors/source-mssql 1.0.15 ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@akashkulk
Copy link
Contributor Author

akashkulk commented May 2, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-mssql run-tests=false

🕑 Publishing the following connectors:
connectors/source-mssql
https://github.com/airbytehq/airbyte/actions/runs/4865694755


Connector Version Did it publish? Were definitions generated?
connectors/source-mssql 1.0.15 ❌ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@akashkulk
Copy link
Contributor Author

akashkulk commented May 3, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-mssql run-tests=false

🕑 Publishing the following connectors:
connectors/source-mssql
https://github.com/airbytehq/airbyte/actions/runs/4867134677


Connector Version Did it publish? Were definitions generated?
connectors/source-mssql 1.0.16 ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@akashkulk
Copy link
Contributor Author

akashkulk commented May 3, 2023 •
edited by github-actions bot
Loading

/publish connector=connectors/source-mssql-strict-encrypt run-tests=false

🕑 Publishing the following connectors:
connectors/source-mssql-strict-encrypt
https://github.com/airbytehq/airbyte/actions/runs/4867194871


Connector Version Did it publish? Were definitions generated?
connectors/source-mssql-strict-encrypt 1.0.16 ✅ ❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@akashkulk
Copy link
Contributor Author

akashkulk commented May 3, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4869030157
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/4869030157
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 22 passed, 6 skipped, 30 warnings in 20.25s ==================

@akashkulk
Copy link
Contributor Author

akashkulk commented May 3, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/4869030594
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/4869030594
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestConnection.test_check: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestDiscovery.test_discover: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestBasicRead.test_read: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestFullRefresh.test_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
================= 22 passed, 6 skipped, 30 warnings in 17.26s ==================

@akashkulk
Copy link
Contributor Author

akashkulk commented May 3, 2023 •
edited by github-actions bot
Loading

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4869031264
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/4869031264
No Python unittests run

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/plugin.py:63: Skipping TestIncremental.test_two_sequential_reads: not found in the config.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [2] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
=================== 68 passed, 5 skipped in 78.57s (0:01:18) ===================

@akashkulk akashkulk merged commit 931911d into master May 3, 2023
17 of 19 checks passed
@akashkulk akashkulk deleted the revert_logging branch May 3, 2023 15:44
marcosmarxm pushed a commit to natalia-miinto/airbyte that referenced this pull request Jun 8, 2023
@akashkulk @octavia-squidington-iii @marcosmarxm
🐛 Source MySQL/MsSQL Disable index logging for MySQL (airbytehq#25740)
0784c5b 
* Disable index logging for MySQL/MsSQL

* Update mysql.md

* Update mysql.md

* auto-bump connector version

* Disable for MsSQL as well

* auto-bump connector version

---------

Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@prateekmukhedkar prateekmukhedkar prateekmukhedkar approved these changes

Assignees
No one assigned
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/jdbc connectors/source/mssql connectors/source/mssql-strict-encrypt connectors/source/mysql connectors/source/mysql-strict-encrypt connectors/source/postgres
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Connector Issue: MySQL connector raises SELECT command denied error in version 2.0.18/2.0.19 but not in 2.0.17
3 participants
@akashkulk @prateekmukhedkar @octavia-squidington-iii