fix: PrestoEngineSpec._show_columns return type #20757
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
The
PrestoEngineSpec._show_columns
method stated it returned aList[RowProxy]
type however it actually returned aResultProxy
, i.e., per the SQLAlchemy documentation,From a coding perspective this wasn't problematic as the output is later processed via,
and though the
ResultProxy
supports iteration (per the DB API), it does not support pickling (and thus caching) whereas theRowProxy
does (it's acts somewhat akin to a Pythontuple
). This was resulting in an issue at Airbnb where we actually override and cache thePrestoEngineSpec._show_columns
using a user lock to help reduce the number of queries we send to Presto.The fix is merely to augment the method to return a
List[RowProxy]
as promised rather than aResultProxy
.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TESTING INSTRUCTIONS
CI.
ADDITIONAL INFORMATION