Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glue statistics not showing up in stats tab #5156

Closed
Seojiwoong opened this issue Jun 14, 2022 · 2 comments
Closed

glue statistics not showing up in stats tab #5156

Seojiwoong opened this issue Jun 14, 2022 · 2 comments
Assignees
Labels
bug Bug report

Comments

@Seojiwoong
Copy link

Seojiwoong commented Jun 14, 2022

Describe the bug
glue statistics not showing up in stats tab

To Reproduce
Steps to reproduce the behavior:

  1. add statistics to my glue table through update-column-statistics-for-table ( https://docs.aws.amazon.com/cli/latest/reference/glue/update-column-statistics-for-table.html)
  2. checked statistics through get-column-statistics-for-table (https://docs.aws.amazon.com/cli/latest/reference/glue/get-column-statistics-for-table.html)
  3. Ingest glue table (add profile to config)
  4. nothing showing up in stats tab

Expected behavior
Statistical data must be exposed

Screenshots

  1. screenshot of stats tab

스크린샷 2022-06-13 오후 7 14 03

  1. screenshot of the get-column-statistics-for-table(aws cli) result

스크린샷 2022-06-13 오후 7 22 14

Desktop (please complete the following information):

  • OS: mac
  • Browser: chrome
  • Version: 0.8.36

Additional context
In datahub code, read statistics through get_table(https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_table) and get_partitions(https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_partition)
When I called the get_table api via boto3, there were no Parameters field in the Table field, and get_partitions too

Why didn't use get_column_statistics_for_partition(https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_column_statistics_for_partition) and get_column_statistics_for_table(https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_column_statistics_for_table)?
Is what I checked wrong? let me know if there is a right way

@Seojiwoong Seojiwoong added the bug Bug report label Jun 14, 2022
@Seojiwoong Seojiwoong changed the title A short description of the bug glue statistics not showing up in stats tab Jun 14, 2022
@Seojiwoong
Copy link
Author

I found that it should be used through update-table(https://docs.aws.amazon.com/cli/latest/reference/glue/update-table.html), not update-column-statistics-for-table ( https://docs.aws.amazon.com/cli/latest/reference/glue/update-column-statistics-for-table.html)

@keerthiis
Copy link

@Seojiwoong - Can you share details of what needs to be updated on the Glue table ? Are you on the Datahub Slack channel ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

No branches or pull requests

3 participants