Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Timestamp display arrow 13.0 #88

Closed
MCRE-BE opened this issue Aug 28, 2023 · 3 comments
Closed

[BUG] Timestamp display arrow 13.0 #88

MCRE-BE opened this issue Aug 28, 2023 · 3 comments
Labels
bug Something isn't working good first issue Good for newcomers waiting user response Just trying to visually identify which issues I need to get to vs. which ones I already answered

Comments

@MCRE-BE
Copy link
Contributor

MCRE-BE commented Aug 28, 2023

Parquet Viewer Version
What version of Parquet Viewer are you experiencing the issue with?
2.7.2.1

Where was the parquet file created?
Apache Spark, Hive, Java, C#, pyarrow, etc.

Python pyarrow

Sample File
Upload a sample file so the issue can be debugged!

Two sample dataframes with date features.
Features.zip

Describe the bug
A clear and concise description of what the bug is.

  • timestamp is displayed as integer instead of timestamp in arrow 13.0 vs 12.0.1
  • This while the metadata is the same

Screenshots

image image

Additional context
Add any other context about the problem here.

Note: This tool relies on the parquet-dotnet library for all the actual Parquet processing. So any issues where that library cannot process a parquet file will not be addressed by us. Please open a ticket on that library's repo to address such issues.

@MCRE-BE MCRE-BE added the bug Something isn't working label Aug 28, 2023
@mukunku mukunku added the good first issue Good for newcomers label Aug 28, 2023
@mukunku
Copy link
Owner

mukunku commented Aug 30, 2023

It looks like the Timestamp field is malformed in the Arrow 13 version as the ConvertedType is missing:
bug

I added a patch to the app nonetheless so we can still open such fields and process them as DateTimes in v2.7.2.2

I also replied to your message on the PR: #85 (comment)

@mukunku mukunku mentioned this issue Aug 30, 2023
@mukunku mukunku added the waiting user response Just trying to visually identify which issues I need to get to vs. which ones I already answered label Aug 30, 2023
@MCRE-BE
Copy link
Contributor Author

MCRE-BE commented Aug 30, 2023

Thanks. I'll test the fix tomorrow.

@MCRE-BE
Copy link
Contributor Author

MCRE-BE commented Aug 31, 2023

Working. Thanks for the quick fix

@MCRE-BE MCRE-BE closed this as completed Aug 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers waiting user response Just trying to visually identify which issues I need to get to vs. which ones I already answered
Projects
None yet
Development

No branches or pull requests

2 participants