Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: add kylin physical datasets #20559

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zys2017
Copy link

@zys2017 zys2017 commented Jun 30, 2022

SUMMARY

Fix the error of kylin physical table add, as these two issues describe:
kylin table in the database cannot be added #19645
Unable to add table from kylin database #17870

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Congrats on making your first PR and thank you for contributing to Superset! 🎉 ❤️
We hope to see you in our Slack community too!

Copy link
Author

@zys2017 zys2017 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding engine.table_names() to judge if table_name exists.

@zhaoyongjie
Copy link
Member

Hi@zys2017, thanks for the contribution. There are a couple of database specs here. changing the specific database spec if a database has special behaviors.

@codecov
Copy link

codecov bot commented Jun 30, 2022

Codecov Report

Merging #20559 (ca5917a) into master (90d486a) will decrease coverage by 11.72%.
The diff coverage is 50.00%.

❗ Current head ca5917a differs from pull request most recent head c7c03fb. Consider uploading reports for the commit c7c03fb to get more accurate results

@@             Coverage Diff             @@
##           master   #20559       +/-   ##
===========================================
- Coverage   66.71%   54.98%   -11.73%     
===========================================
  Files        1752     1752               
  Lines       65478    65478               
  Branches     6916     6916               
===========================================
- Hits        43681    36004     -7677     
- Misses      20049    27726     +7677     
  Partials     1748     1748               
Flag Coverage Δ
hive 53.76% <50.00%> (?)
mysql ?
postgres ?
presto 53.62% <50.00%> (ø)
python 58.38% <50.00%> (-24.27%) ⬇️
sqlite ?
unit 50.66% <50.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/models/core.py 78.55% <50.00%> (-10.61%) ⬇️
superset/utils/dashboard_import_export.py 0.00% <0.00%> (-100.00%) ⬇️
superset/key_value/commands/upsert.py 0.00% <0.00%> (-89.59%) ⬇️
superset/key_value/commands/update.py 0.00% <0.00%> (-89.37%) ⬇️
superset/key_value/commands/delete.py 0.00% <0.00%> (-85.30%) ⬇️
superset/key_value/commands/delete_expired.py 0.00% <0.00%> (-80.77%) ⬇️
superset/dashboards/commands/importers/v0.py 15.62% <0.00%> (-76.25%) ⬇️
superset/datasets/commands/update.py 25.88% <0.00%> (-68.24%) ⬇️
superset/datasets/commands/create.py 30.18% <0.00%> (-67.93%) ⬇️
superset/datasets/commands/importers/v0.py 24.03% <0.00%> (-67.45%) ⬇️
... and 277 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 90d486a...c7c03fb. Read the comment docs.

@zys2017 zys2017 changed the title fix kylin datasets add fix: add kylin physical datasets. Jun 30, 2022
@@ -760,11 +760,11 @@ def get_perm(self) -> str:

def has_table(self, table: Table) -> bool:
engine = self.get_sqla_engine()
return engine.has_table(table.table_name, table.schema or None)
return engine.has_table(table.table_name, table.schema or None) or table.table_name in engine.table_names()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am the original Kylinpy maintainer, I just saw the source code of kylinpy. I think you should change here .

Copy link
Author

@zys2017 zys2017 Jun 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Table_names is the function of orm and seems effective to other db source,not only kylin. It can avoid that the engine does not implement relevant functions. kylin may also add the func.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The has_table is similar to table_names that should implement in dialect.

@zys2017 zys2017 changed the title fix: add kylin physical datasets. fix: add kylin physical datasets Jun 30, 2022
@albertojarabo
Copy link

albertojarabo commented Jan 17, 2023

Is there any plan to fix this issue in the short term?

@albertojarabo
Copy link

While this issue is being resolved somewhere I suggest everyone who has this problem to implement the has_table method in sqla_dialect.py from kylinpy plugin. You will find it in kylinpy/sqla_dialect.py, and the method is:

image

You should change that return to the next line:

return table_name in self.get_table_names(connection, schema)

And everything will be back to normal.

@rusackas
Copy link
Member

rusackas commented Feb 6, 2024

@zys2017 just checking in to see if you (and/or @zhaoyongjie ) have any plans to pick this up, rebase it, and get it across the finish line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants