[REVIEW] Update orc reader and writer fuzz tests - #7357
Conversation
| ) | ||
| if num_cols == 0: | ||
| """ | ||
| If a dataframe has no columns, then pyorc writer will throw |
There was a problem hiding this comment.
I wondered what the desired behavior is here, i.e. whether ORC as a format supports having no columns.
I think we also have some issues writing empty dataframes to ORC.
There was a problem hiding this comment.
I guess we need to write an empty struct. With pyorc we can do it this way:
>>> import pyorc
>>> import pandas as pd
>>> output = open("sample.orc", "wb")
>>> writer = pyorc.Writer(output, pyorc.Struct())
>>> writer.close()
>>> pd.read_orc('sample.orc')
Empty DataFrame
Columns: []
Index: []Looks like I also need to make some code-changes in this PR. I'll update this PR.
Updated the code-changes with comments, this is ready for a re-review. |
Codecov Report
@@ Coverage Diff @@
## branch-0.19 #7357 +/- ##
==============================================
Coverage ? 82.21%
==============================================
Files ? 100
Lines ? 16971
Branches ? 0
==============================================
Hits ? 13953
Misses ? 3018
Partials ? 0 Continue to review full report at Codecov.
|
Co-authored-by: Ram (Ramakrishna Prabhu) <42624703+rgsl888prabhu@users.noreply.github.com>
vuule
left a comment
There was a problem hiding this comment.
found a typo, LGTM otherwise
Co-authored-by: Vukasin Milovanovic <vukasin.milovanovic.87@gmail.com>
|
@gpucibot merge |
This PR introduces:
pyorcin usingpyorc.Struct.listdtype parameter changes introduced previously.