Skip to content

Comments

wip: Avoid SQLAchemy Query API usage in tests#37242

Closed
Taragolis wants to merge 7 commits intoapache:mainfrom
Taragolis:session-query-tests
Closed

wip: Avoid SQLAchemy Query API usage in tests#37242
Taragolis wants to merge 7 commits intoapache:mainfrom
Taragolis:session-query-tests

Conversation

@Taragolis
Copy link
Contributor

Attempt to reduce legacy SQLAchemy Query API usage in Airflow tests. Where it possible use common testing utils, like clear_db_* in the other cases use statements builder.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@Taragolis Taragolis added the full tests needed We need to run full set of tests for this PR to merge label Feb 8, 2024
@boring-cyborg boring-cyborg bot added area:API Airflow's REST/HTTP API area:CLI area:Scheduler including HA (high availability) scheduler area:Triggerer area:webserver Webserver related Issues labels Feb 8, 2024
@Taragolis Taragolis added the type:misc/internal Changelog: Misc changes that should appear in change log label Feb 8, 2024
@Taragolis
Copy link
Contributor Author

Interesting issue after switch to query builder AP, only related to the remote loggers

ERROR tests/providers/alibaba/cloud/log/test_oss_task_handler.py::TestOSSTaskHandler::test_oss_write_into_remote_existing_file_via_append - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/alibaba/cloud/log/test_oss_task_handler.py::TestOSSTaskHandler::test_oss_write_into_remote_non_existing_file_via_append - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/alibaba/cloud/log/test_oss_task_handler.py::TestOSSTaskHandler::test_oss_write_into_remote_existing_file_not_via_append - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/alibaba/cloud/log/test_oss_task_handler.py::TestOSSTaskHandler::test_oss_write_into_remote_non_existing_file_not_via_append - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_hook - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_hook_warns - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_wasb_log_exists - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_wasb_read_raises - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_write_log - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_write_on_existing_log - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_write_when_append_is_false - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/microsoft/azure/log/test_wasb_task_handler.py::TestWasbTaskHandler::test_write_raises - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_gcs_task_handler.py::TestGCSTaskHandler::test_client_conn_id_behavior[no-conn] - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_gcs_task_handler.py::TestGCSTaskHandler::test_client_conn_id_behavior[with-conn] - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_set_labels - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_append_labels - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_read_logs_for_all_try - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_read_logs_for_task_with_quote - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_read_logs_for_single_try - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_read_logs_with_pagination - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_read_logs_with_download - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_read_logs_with_custom_resources - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_use_credentials - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.
ERROR tests/providers/google/cloud/log/test_stackdriver_task_handler.py::TestStackdriverLoggingHandlerTask::test_should_return_valid_external_url - sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.

All errors are happen in teardown method when called clear_db_*

>                   raise orm_exc.StaleDataError(
                        "UPDATE statement on table '%s' expected to "
                        "update %d row(s); %d were matched."
                        % (table.description, len(records), rows)
                    )
E                   sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'task_instance' expected to update 1 row(s); 0 were matched.

I know how to fix the tests, but worthwhile check remote loggers, maybe session have uncommitted changes

@Taragolis Taragolis force-pushed the session-query-tests branch from f83cf3d to 86dc261 Compare February 8, 2024 14:59
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Mar 26, 2024
@Taragolis Taragolis closed this Mar 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:CLI area:Scheduler including HA (high availability) scheduler area:Triggerer area:webserver Webserver related Issues full tests needed We need to run full set of tests for this PR to merge stale Stale PRs per the .github/workflows/stale.yml policy file type:misc/internal Changelog: Misc changes that should appear in change log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant