Use latest Parquet version for writing

[jonashaag]

Feature Request / Improvement

The current code always uses Parquet version 1.0 for writing:

iceberg-python/pyiceberg/io/pyarrow.py

Line 1729 in a4856bc

with pq.ParquetWriter(fos, schema=file_schema, version="1.0", metadata_collector=collected_metrics) as writer:

This version is already over 10 years old. All Parquet readers I know do support much more recent versions. With Python 3.8 you can get PyArrow 15 which supports the latest version of Parquet. IMO we should default to using a more recent version (maybe always use the most recent?) and offer an option to change the Parquet version.

[jonashaag]

@syun64 @Fokko @HonahX feedback?

[Fokko]

Hey @jonashaag Thanks for raising this. I think I made a mistake here. I thought it was referring to the data-page-version, but that one is set to 1.0. I think we should bump the version indeed since the UUID type was introduced in 2.3: https://github.com/apache/parquet-format/blob/master/CHANGES.md

[jonashaag]

Shall we then just stick to the PyArrow default?