Expose Parallel Fetch #787
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sfc-gh-kzaveri
force-pushed
the
parallel-fetch-prpr
branch
from
61c2fb2
to
75cc83f
Compare
sfc-gh-kzaveri
requested review from
sfc-gh-kwagner and
sfc-gh-mkeller
as code owners
This reverts commit c5466cf.
This reverts commit de8887d.
…nd `result_batch` (#733) * Revert "Revert "SNOW-296993 reworking internals for result_set and result_chunks (#688)"" This reverts commit c5c31fa. * Refactor resultset/batch, add to_arrow/pandas, add tests * Address comments
…s` (#735) * Revert "Revert "SNOW-296993 reworking internals for result_set and result_chunks (#688)"" This reverts commit c5c31fa. * Add schema to ResultBatch and add test * Update type hints * Use ResultMetadata named tuple in tests * Update formatting * Address comments, remove column_names * Keep column_converters, make code a little more readable * Address comments, add docstring * Add property for column_names * Fix rebase issues
* Add test_empty case for arrow * Simplify value string * Fetch arrow all test * Add fetch_arrow_batches test
* Rewrite result_set iterator * Fix rebase issue * Update test * Change logging to use f strings, update fetch behaviour
* Add proper retry and error handling * Changed to raise encapsulated request * Fix failing test * Update tests, add f-strings * Add retries to test
* Add using sessions functionality * Add tests * remove arrow from test * Address comments
* Replace generators with iterators * Make _parse return a list * Fix olddriver tests * Update _parse and cursor iterating logic * Update test to check iter function * Add type hints and result batch id * Make id an attribute
…ill keep verifying behaviors
* Fix prpr tests * Fix error msg formatting
sfc-gh-mkeller
force-pushed
the
parallel-fetch-prpr
branch
from
319706c
to
5559f9b
Compare
Codecov Report
@@ Coverage Diff @@
## master #787 +/- ##
==========================================
- Coverage 82.79% 82.63% -0.17%
==========================================
Files 58 58
Lines 7481 7680 +199
Branches 1127 1164 +37
==========================================
+ Hits 6194 6346 +152
- Misses 967 1011 +44
- Partials 320 323 +3
Continue to review full report at Codecov.
|
sfc-gh-kzaveri
force-pushed
the
parallel-fetch-prpr
branch
from
c5afad4
to
ff074b7
Compare
sfc-gh-mkeller
approved these changes
Aug 12, 2021
This was referenced Oct 5, 2021
|
Could it be that this leaves the https://github.com/snowflakedb/snowflake-connector-python/blob/master/src/snowflake/connector/gzip_decoder.py file unused? |
sfc-gh-aling
pushed a commit
that referenced
this pull request
Oct 17, 2022
* Revert "Revert "SNOW-296993 reworking internals for result_set and result_chunks (#688)"" This reverts commit 7401783. * Revert "Revert "SNOW-296993 fixing import statement for SnowSQL (#697)"" This reverts commit 4b7d612. * Revert "Revert "SNOW-296993 fixing a ResultBatch telemetry bug (#701)"" This reverts commit 6bab3dc. * Revert "Revert "SNOW-336959 Add telemetry for pandas fetch functions (#699)"" This reverts commit 03a2f85. * SNOW-177082 Adds `to_arrow` and `to_pandas`, refactors `result_set` and `result_batch` (#733) * Revert "Revert "SNOW-296993 reworking internals for result_set and result_chunks (#688)"" This reverts commit 7401783. * Refactor resultset/batch, add to_arrow/pandas, add tests * Address comments * SNOW-361357 Provide interface exposing `schema` info to `ResultBatches` (#735) * Revert "Revert "SNOW-296993 reworking internals for result_set and result_chunks (#688)"" This reverts commit 7401783. * Add schema to ResultBatch and add test * Update type hints * Use ResultMetadata named tuple in tests * Update formatting * Address comments, remove column_names * Keep column_converters, make code a little more readable * Address comments, add docstring * Add property for column_names * Fix rebase issues * SNOW-370785 Add Fetch Arrow Tests (#766) * Add test_empty case for arrow * Simplify value string * Fetch arrow all test * Add fetch_arrow_batches test * SNOW-361211 Adds Updated Fetching Logic (#775) * Rewrite result_set iterator * Fix rebase issue * Update test * Change logging to use f strings, update fetch behaviour * SNOW-361210 Adds Retry Logic and Tests (#780) * Add proper retry and error handling * Changed to raise encapsulated request * Fix failing test * Update tests, add f-strings * Add retries to test * SNOW-361211 Allow ResultBatches to use existing Sessions (#790) * Add using sessions functionality * Add tests * remove arrow from test * Address comments * SNOW-414845 Fix Pandas and JSON Parallel Fetch (#811) * Replace generators with iterators * Make _parse return a list * Fix olddriver tests * Update _parse and cursor iterating logic * Update test to check iter function * Add type hints and result batch id * Make id an attribute * reverting a behavior change before it happens and adding tests that will keep verifying behaviors * SNOW-421184 Parallel Fetch Clean Up (#821) * Misc: Fix prpr tests, error msg formatting, mark a new test skipolddriver, update logging Co-authored-by: Mark Keller <mark.keller@snowflake.com>
7 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR merges all parallel-fetch-prpr branch changes in to master.