-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cloud Fetch download handler #127
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mattdeekay
requested review from
susodapop,
arikfr,
yunbodeng-db and
andrefurlan-db
as code owners
May 16, 2023 21:42
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
May 16, 2023 21:45
60b036d
to
884bc0f
Compare
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
June 14, 2023 17:26
2ffb2ba
to
d0e0037
Compare
Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
June 14, 2023 17:44
d0e0037
to
d2ffe1e
Compare
yunbodeng-db
requested review from
rcypher-databricks
and removed request for
arikfr
June 20, 2023 21:45
…frame markers Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
susodapop
reviewed
Jun 21, 2023
susodapop
reviewed
Jun 21, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have integration tests for this part? Or are those in a subsequent PR?
- Linting - Type annotations - Use response.ok - Log exception - Remove semaphore and only use threading.event - reset() flags method - Fix tests after removing semaphore - Link expiry logic should be in secs - Decompress data static function - link_expiry_buffer and static public methods - Docstrings and comments Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
June 26, 2023 20:11
f92860e
to
17c813c
Compare
Nope, will be part of the full e2e PR |
Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
susodapop
approved these changes
Jun 27, 2023
Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
June 27, 2023 22:17
9872995
to
e8d2f77
Compare
susodapop
pushed a commit
to unj1m/databricks-sql-python
that referenced
this pull request
Sep 19, 2023
* Cloud Fetch download handler Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Issue fix: final result link compressed data has multiple LZ4 end-of-frame markers Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Addressing PR comments - Linting - Type annotations - Use response.ok - Log exception - Remove semaphore and only use threading.event - reset() flags method - Fix tests after removing semaphore - Link expiry logic should be in secs - Decompress data static function - link_expiry_buffer and static public methods - Docstrings and comments Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * Changing logger.debug to remove url Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * _reset() comment to docstring Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> * link_expiry_buffer -> link_expiry_buffer_secs Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com> --------- Signed-off-by: Matthew Kim <11141331+mattdeekay@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CloudFetch download handler. Each handler will run on a thread and a CloudFetch download manager will maintain a thread pool.