Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Exception repeatedly thrown when widening window to include a column containing arrays of bytes #79

Closed
Dave-Kiwi opened this issue Jun 12, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@Dave-Kiwi
Copy link

Parquet Viewer Version
Version 2.6.0 and later. Works just fine on earlier versions.

Where was the parquet file created?
Locally using C#

Sample File
Contains PHI. I can make a sanitized one if you think it is needed. Let me know if you have trouble reproducing with a parquet file containing a byte[] column.

Describe the bug
When the window is widened to the column in question, the application repeatedly throws the exception in the attachment

Screenshots
image

Additional context
I suspect the app is having a problem with a column of type byte[]

Note: This tool relies on the parquet-dotnet library for all the actual Parquet processing. So any issues where that library cannot process a parquet file will not be addressed by us. Please open a ticket on that library's repo to address such issues.

@Dave-Kiwi Dave-Kiwi added the bug Something isn't working label Jun 12, 2023
@MachineParadox101
Copy link

MachineParadox101 commented Jul 5, 2023

Have the same issue with v2.6 throwing error for byte columns. 2.4.7834 also threw an error 'encoding RLE_Dictionary is not supported' for the same file.

Edit: also an issue in 2.7.0.3

@mukunku
Copy link
Owner

mukunku commented Aug 12, 2023

I haven't been able to replicate this issue on my end. I tried with a parquet file containing byte and byte[] fields.

image

Any chance you folks could provide a sample file?

@Dave-Kiwi
Copy link
Author

I haven't been able to replicate this issue on my end. I tried with a parquet file containing byte and byte[] fields.

image

Any chance you folks could provide a sample file?

Sure thing - the attached one fails pretty hard on Version 2.7.1.0. In 2.6.0.2 it shows just the first two columns, but not the last (byte[]) column. In 2.4.2.0 it works great (shows all three columns, including last byte[] column).
BytesFailsIn2.7.1.0.zip

@mukunku
Copy link
Owner

mukunku commented Aug 18, 2023

Thanks a lot for the sample file @Dave-Kiwi .

I added a temporary patch to support converting byte[] data to string so it can be displayed. This is available now in v2.7.2.1
image

Details

So this is an issue with the fact that we use AutoGenerateColumns = true when generating the gridview. And the internal .net framework logic thinks byte[] types are for images. So it tries to render an image with your byte data and fails. And it fails for each cell so that might be why it triggers so often.

I also put in a fix so any time a value can't be rendered it will be skipped like so:
image

A real proper fix would be to do our own column generation and handle all the different data types ourselves. This is something I'll have to pursue another time; if there's more desire to support different array types.

@mukunku mukunku added the waiting user response Just trying to visually identify which issues I need to get to vs. which ones I already answered label Aug 18, 2023
@Dave-Kiwi
Copy link
Author

Thanks @mukunku! Yes, the byte[] in the sample file is a 3D mask we use for our application that does not correspond to any well-known file type. I think the solution you have provided is excellent, and will serve our purposes very nicely.

@mukunku mukunku removed the waiting user response Just trying to visually identify which issues I need to get to vs. which ones I already answered label Aug 21, 2023
@mukunku mukunku closed this as completed Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants