Skip to content

Commit

Permalink
ARROW-17583: [C++][Python] Changed datawidth of WrittenFile.size to i…
Browse files Browse the repository at this point in the history
…nt64 to match C++ code (#14032)

To fix an exception while writing large parquet files:
```
Traceback (most recent call last):
  File "pyarrow/_dataset_parquet.pyx", line 165, in pyarrow._dataset_parquet.ParquetFileFormat._finish_write
  File "pyarrow/dataset.pyx", line 2695, in pyarrow._dataset.WrittenFile.init_
OverflowError: value too large to convert to int
Exception ignored in: 'pyarrow._dataset._filesystemdataset_write_visitor'
```

Authored-by: Joost Hoozemans <joosthooz@msn.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
  • Loading branch information
joosthooz committed Sep 8, 2022
1 parent 6ff5224 commit 43670af
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion python/pyarrow/_dataset.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -161,4 +161,4 @@ cdef class WrittenFile(_Weakrefable):
# the written file.
cdef public object metadata
# The size of the file in bytes
cdef public int size
cdef public int64_t size

0 comments on commit 43670af

Please sign in to comment.