Improve DbApi code readability#18660
Conversation
|
Please rebase. cc: @ashb |
|
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
|
Closing and reopening to trigger the CI |
|
THis change does a little bit more than just refactor - now also get_records and get_first will log at "info" level the "Running statemant" and number of rows affected as well: You could consider it as "good" for understanding what's going on, but this might be an undesireable side-effect of this change (especially that info is the default log level for most users). This also has a non-negligible performance impact - if your query has multiple parameters, all of them will be formatted and printed which might give a significant performance hit. Even worse (and that's disqualification IMHO), it has non-trivial security implicatons - I can easily imagine someone passing some sensitive data as parameters of the SQL query params. This might not be the best practice, but it might be unavoidable (and those parameters might not come from connections/secret variables so our secret masker might not mask it). This might be totally unexpected if it is changed without warning. I think the best solution would be to add optional "print_info_log" or something parameters with the right defaults for different methods and pass them down to keep backwards-compatible behaviour. Also that would add explicit parameter - it is not clear currently whether the info will be logged or not and having this parameter with explicit backwards-compatible default values should do much better in communicating it. I would not be such picky in other parts of code, but DBApi is pretty important and crucial part that is extended and used by a number of providers, and I think we should avold surprises here (also see the comment just above the DBApi class) - we already had a few cases where we got some backwards-incompatibility problems with this class so we have to be extra careful here. |
potiuk
left a comment
There was a problem hiding this comment.
Unless there are good reasons, I tink we should keep backwards-compatibility for the info messages (reasons explained in the comment).
Improve DbApi code readability
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.