-
Notifications
You must be signed in to change notification settings - Fork 13.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[bugfix] Fixing regression introduced in #4396 #4500
Conversation
@@ -327,8 +322,9 @@ def get_df_payload(self, query_obj=None): | |||
if query_obj and not is_loaded: | |||
try: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The try/except block may no longer be required per the PR description. This would make the stacktrace
obsolete as well.
superset/viz.py
Outdated
if self.status != utils.QueryStatus.FAILED: | ||
if df is None or df.empty: | ||
raise Exception('No data') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note I'm not certain why we need to throw an exception here, as previously the error would be present in error_message
.
3408fd6
to
269754f
Compare
I feel like we need some unit tests here making sure errors bubble up somehow. It should be easy to generate a |
@mistercrunch for the later simply referencing an invalid metric doesn't actually cover the specific case we discovered as the compiler detects this, i.e., prior to this fix it wouldn't return |
269754f
to
11c9e8d
Compare
@mistercrunch I've updated the logic and added a couple of unit tests which test the two scenarios. |
@@ -151,9 +151,6 @@ def get_df(self, query_obj=None): | |||
# If the datetime format is unix, the parse will use the corresponding | |||
# parsing logic. | |||
if df is None or df.empty: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that the df.empty
check is only required by the test_get_df_returns_empty_df
unit test due to the mocking. Otherwise it would be safe to proceed with an empty pd.DataFrame
.
if self.status != utils.QueryStatus.FAILED: | ||
payload['data'] = self.get_data(df) | ||
if df is None or df.empty: | ||
payload['error'] = 'No data' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's more consistent to log the payload error as opposed to throwing an exception here.
@@ -612,7 +611,7 @@ def query_obj(self): | |||
return None | |||
|
|||
def get_df(self, query_obj=None): | |||
return None | |||
return pd.DataFrame() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this for consistency, i.e. get_df(...)
should return a pd.DataFrame()
.
11c9e8d
to
7440d34
Compare
@mistercrunch would you mind taking another look at this? I made a few small tweaks after adding the addition of a couple of unit tests. |
lgtm |
Sorry for the delay, was traveling last week with limited attention. Thanks @michellethomas for merging it! |
I think this broke the brittle |
Sorry @mistercrunch. I think I may have a fix and will add an additional unit test for the
and
|
Sounds about right, this area is pretty brittle around the no-query special cases... |
(cherry picked from commit ef4e5ec)
* Cherry pick apache#4581 * Add flask-compress cherry * Add shortner fix * Add Return __time in Druid scan apache#4504 * Picking cherry Fixing regression from apache#4500 (apache#4549) * [bugfix] SQL Lab 'MySQL has gone away' It appears the 'MySQL has gone away' is triggered by the line of code I wrapped in a try block here. This is a temporary fix, there will be another PR shortly getting to the bottom of this. Related: https://github.com/lyft/druidstream/issues/40
[bugfix] Fixing regression introduced in apache#4396
This PR fixes a regression introduced in #4396 which returned the error
No data
even when the query was unsuccessful for other reasons. This PR also refactors theNo data
logic (previously defined in two places) and also registered that the data was loaded if the query didn't fail. Note it seems likeBaseViz.get_df(...)
may never throw an exception give this (and thus I'm uncertain whether this logic is needed) hence the additional check that the query succeeded is necessary to confirm that the data was loaded.@mistercrunch from our offline conversation I found this difficult to write specific test cases, i.e., we discovered this regression in Presto when the underlying table metadata had changed either a column or the table no-longer existed. When I tried to mock this behavior, I was able to exercise the issue as in SQLite the inconsistency was discovered whilst the query was being compiled rather than during query execution.
to: @mistercrunch
cc: @graceguo-supercat @michellethomas @timifasubaa