Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Async support for the queries in the SQL Lab. WIP. #946

Closed
wants to merge 1 commit into from

Conversation

bkyryliuk
Copy link
Member

@bkyryliuk bkyryliuk commented Aug 15, 2016

Added async support:

  • Refactored the tasks.py to return the query id and added ability to run the code async.
  • Changed the celery_tests to be more end to end (calling run_sql endpoint instead of celery function)
  • Added implementation of async run of the hive and presto queries
  • Added dependencies required by pyhive
  • Implemented progress bar for the async queries
  • Added extra fields to the Query object to improve traceability: limit_used, select_as_cta_used, executed_sql, select_sql

Tackles backend of the: #858, #746, #886

Implement progress bar for the presto / hive queries

Implement remote query execution

It is a preliminary PR and some more things needed to be done.
TODO:

  • Unit test async queries (presto and hive engines)
  • Add query_status / progress endpoint
  • Implement cancel_query endpoint
  • Add query_results endpoint
  • Find and implement a storage for the query results preview
  • Add ttl to the queries / tmp tables (optional)

@bkyryliuk bkyryliuk changed the title Bogdan/query status polling Async support for the queries in the SQL Lab. WIP. Aug 15, 2016
@bkyryliuk bkyryliuk force-pushed the bogdan/query_status_polling branch 8 times, most recently from e281390 to b78253b Compare August 15, 2016 22:09
@bkyryliuk bkyryliuk self-assigned this Aug 15, 2016
@coveralls
Copy link

coveralls commented Aug 16, 2016

Coverage Status

Coverage decreased (-2.1%) to 78.432% when pulling e1e3382 on bogdan/query_status_polling into 5688246 on carapal.

@bkyryliuk bkyryliuk closed this Aug 18, 2016
@mistercrunch mistercrunch deleted the bogdan/query_status_polling branch November 9, 2016 22:25
@dpgaspar dpgaspar mentioned this pull request Apr 18, 2019
3 tasks
zhaoyongjie pushed a commit to zhaoyongjie/incubator-superset that referenced this pull request Nov 17, 2021
zhaoyongjie pushed a commit to zhaoyongjie/incubator-superset that referenced this pull request Nov 24, 2021
zhaoyongjie pushed a commit to zhaoyongjie/incubator-superset that referenced this pull request Nov 25, 2021
zhaoyongjie pushed a commit to zhaoyongjie/incubator-superset that referenced this pull request Nov 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants