Skip to content

Improve SnowflakeSqlApiOperator error message on query status check failure#66642

Open
justinpakzad wants to merge 1 commit into
apache:mainfrom
justinpakzad:fix/snowflake-missing-query-status-on-poll-error
Open

Improve SnowflakeSqlApiOperator error message on query status check failure#66642
justinpakzad wants to merge 1 commit into
apache:mainfrom
justinpakzad:fix/snowflake-missing-query-status-on-poll-error

Conversation

@justinpakzad
Copy link
Copy Markdown
Contributor

@justinpakzad justinpakzad commented May 10, 2026


Replace ValueError with RuntimeError on query status check failure in poll_on_queries, including the query id and original error for better debuggability. Added a new test to ensure a RuntimeError is raised when status check fails.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@justinpakzad justinpakzad requested a review from potiuk as a code owner May 10, 2026 01:00
@boring-cyborg boring-cyborg Bot added area:providers provider:snowflake Issues related to Snowflake provider labels May 10, 2026
Copy link
Copy Markdown
Collaborator

@jroachgolf84 jroachgolf84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an interesting one to think more about. So, the exception that is being thrown is from a failure when hitting the Snowflake API, not a failure of the query.

I'm not sure that it makes sense to remove the query and mark the statement as being failed. For example, what if your creds expire? This would cause the status check to fail even if the statement is running correctly. Your unit test captures this scenario (connection timing out) which doesn't mean that the query failed.

@jroachgolf84
Copy link
Copy Markdown
Collaborator

@justinpakzad - I think it might make sense to raise a more specific exception rather than just a ValueError. Thoughts?

@justinpakzad
Copy link
Copy Markdown
Contributor Author

justinpakzad commented May 10, 2026

@justinpakzad - I think it might make sense to raise a more specific exception rather than just a ValueError. Thoughts?

Hey, thanks for the review. To address your first comment - that's a good point, slipped over my head a bit that there is a distinction between a query failure and other types of errors that don't necessarily mean there was a failure on the query end (as you mentioned). Are you suggesting we just revert it back but maybe with something like:

try:
    statement_status = self._hook.get_sql_api_query_status(query_id)
except Exception as e:
    raise RuntimeError(f"Failed to get status for query {query_id}: {e}") from e

@justinpakzad justinpakzad marked this pull request as draft May 10, 2026 03:07
@jroachgolf84
Copy link
Copy Markdown
Collaborator

Yes, I think that would make sense!

@justinpakzad justinpakzad force-pushed the fix/snowflake-missing-query-status-on-poll-error branch from c03ce37 to 6348406 Compare May 10, 2026 04:55
@justinpakzad justinpakzad changed the title Fix SnowflakeSqlApiOperator losing query status on poll error Improve SnowflakeSqlApiOperator error message on query status check failure May 10, 2026
@justinpakzad justinpakzad marked this pull request as ready for review May 10, 2026 05:02
Copy link
Copy Markdown
Collaborator

@jroachgolf84 jroachgolf84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the change, this LGTM!

@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:snowflake Issues related to Snowflake provider ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants