-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Oracle backend: pandas Exception : KeyError #953
Comments
My caravel version is : 0.10.0 piovt table: |
subjet: # column_name ---> upper(column_name) cause exceptions 1.# test.py #!/usr/bin/python d={} df = pd.DataFrame(d) print('--------- 1 ----------') print('--------- 2 ----------') print('--------- 3 ----------') print('--------- 4 ----------')
--------- 1 ---------- During handling of the above exception, another exception occurred: Traceback (most recent call last): |
@PickLiDan that may or may be not be the cause :) To find out, just print the keys available in the dataframe so we can't see what's wrong here. |
@xrmx # i change my column to upper case,then i work well,so i am sure. lower case :2016-08-18 20:08:14,614:INFO:root: DT COUNT_DISTINCT__ORDER_ID During handling of the above exception, another exception occurred: upper case :2016-08-18 20:03:51,109:INFO:root: DT COUNT_DISTINCT__ORDER_ID |
@PickLiDan so to sum up, you have a field in your table called "DT" all uppercase, but on the select widget of group by you have something that has value "dt" lowercase? |
yeah . |
I don't see where we could be doing that in the code. Are you sure you just don't have a column in your table model with the wrong name? |
@PickLiDan - could you please provide the screenshots ? |
@PickLiDan - feel free to reopen is the issue persists. I cannot reproduce it. |
Column names become lower case regardless of forcing it to upper case in query. Oracle SQL database is used. If I make column names lowercase, then after selecting things in 'group by', the additional sql surround the original sql will not recognize that lowercase column name. |
Has anyone found a work-around for this yet? This is a blocker for me applying it at work because we have to deal with an Oracle DB. |
@TryHarder01 my current workaround is editing the column name in the UI after the table is created. |
@TryHarder01 @yileic I tried casting the column names to upper case and lower case in the UI but it still has the 'KeyError: [column_name]' issue The error doesn't come up when I choose 'Table View' but when I change the visualization type to any other view, it doesn't recognize the key |
Are all the ongoing issues here happening on an Oracle db? |
@mistercrunch Yes, I haven't had any issues on MySQL. Its just been Oracle so far. |
Would it be helpful to close and reopen as a bug specifically with Oracle DB? |
I renamed this issue to make it specific to Oracle. So it appeared to me like the SQLAlchemy dialect for Oracle returns lower case but then Oracle wants upper case at query time? Can someone with an Oracle database laying around confirm that the issue is with the SQLALchemy driver returning lower case column names? |
This seems very relevant: Damn Oracle! and don't get me started on the the 30 character limit on object names. Fully committed to bad decisions made in the 80s. |
@nikitathampi works with me. We can help test out some things w/ Oracle at the office tomorrow if it's helpful to you folks. If so, help me scope out exactly what you want to see as far as testing. |
@mistercrunch well that's fun wrinkle in their syntax. |
Perhaps it might be resolved by enclosing the name of the item containing uppercase and lowercase letters in double quotes in the column name definition. |
Adding more context on what I've found out. under the hood Superset renames the "min" , "max", etc.... metrics calls by querying the data and renaming the columns I traced this all the way up to the One idea I've thought about is all the calculated metrics could also be in all caps. For example, Right now, we'll need to apply something to make sure these calc'd metrics aren't made upper and that any future metrics don't become upper, but yet we also catch the names of any other columns from the database. This seems like a dangerous tightrope to walk. |
@TryHarder01 could you please reference a line in the file where you are adding these prints? |
@xrmx , sure. But I feel good about a solution I just tested in I'll be sending up a PR in a few hours. starting on line 615 (aprox) of core.py
|
Could you please check your user tablespace please? pandas-dev/pandas#8820 (comment) |
@xrmx I'm going to be connecting multiple Oracle DBs, but here's the tablespace for one that I'm testing. For all Oracle DBs I'll be connecting I won't have any admin authority over them to change any of the permission structure. |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue |
* feat(legacy-plugin-chart-heatmap): subject Add sort by metric * feat(legacy-plugin-chart-sunburst): subject Add sort by metric * feat(legacy-plugin-chart-force-directed): subject Add sort by metric * feat(legacy-plugin-chart-chord): subject Add sort by metric * feat(legacy-plugin-chart-country-map): subject Add sort by metric
* feat(legacy-plugin-chart-heatmap): subject Add sort by metric * feat(legacy-plugin-chart-sunburst): subject Add sort by metric * feat(legacy-plugin-chart-force-directed): subject Add sort by metric * feat(legacy-plugin-chart-chord): subject Add sort by metric * feat(legacy-plugin-chart-country-map): subject Add sort by metric
* feat(legacy-plugin-chart-heatmap): subject Add sort by metric * feat(legacy-plugin-chart-sunburst): subject Add sort by metric * feat(legacy-plugin-chart-force-directed): subject Add sort by metric * feat(legacy-plugin-chart-chord): subject Add sort by metric * feat(legacy-plugin-chart-country-map): subject Add sort by metric
* feat(legacy-plugin-chart-heatmap): subject Add sort by metric * feat(legacy-plugin-chart-sunburst): subject Add sort by metric * feat(legacy-plugin-chart-force-directed): subject Add sort by metric * feat(legacy-plugin-chart-chord): subject Add sort by metric * feat(legacy-plugin-chart-country-map): subject Add sort by metric
During handling of the above exception, another exception occurred:
The text was updated successfully, but these errors were encountered: