Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-33972: [C++] Pass in metadata to ParquetReader #34015

Merged
merged 4 commits into from Feb 4, 2023
Merged

GH-33972: [C++] Pass in metadata to ParquetReader #34015

merged 4 commits into from Feb 4, 2023

Conversation

Fokko
Copy link
Contributor

@Fokko Fokko commented Feb 3, 2023 •
edited by github-actions bot

Closes #33972

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions
Copy link

github-actions bot commented Feb 3, 2023

@github-actions
Copy link

github-actions bot commented Feb 3, 2023

⚠️ GitHub issue #33972 has been automatically assigned in GitHub to PR creator.

@Fokko Fokko changed the title GH-33972: [C++] Pass in metadata GH-33972: [C++] Pass in metadata to ParquetReader Feb 3, 2023
@Fokko Fokko force-pushed the fd-pass-in-metadata-if-available branch 7 times, most recently from 916de4a to 4e9d4eb Compare February 3, 2023 16:10
@Fokko Fokko mentioned this pull request Feb 3, 2023
Closed
@Fokko Fokko force-pushed the fd-pass-in-metadata-if-available branch from 4e9d4eb to f1eb31d Compare February 3, 2023 19:07
@westonpace
Copy link
Member

This metadata behavior is kind of subtle and I'm hoping we have a lot of changes coming to file_parquet.cc (page indices, bloom filters, etc.) so let's add a unit test. I sent you a PR.

@Fokko
Copy link
Contributor Author

Fokko commented Feb 4, 2023

Thanks @westonpace for adding tests, much appreciated

@westonpace westonpace merged commit e39749d into apache:master Feb 4, 2023
@Fokko Fokko deleted the fd-pass-in-metadata-if-available branch February 4, 2023 13:09
@Fokko
Copy link
Contributor Author

Fokko commented Feb 4, 2023

Thanks @westonpace for merging and the support in this issue 👍🏻

@Fokko Fokko mentioned this pull request Feb 4, 2023
Closed
@ursabot
Copy link

ursabot commented Feb 4, 2023

Benchmark runs are scheduled for baseline = b56b91e and contender = e39749d. e39749d is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.34% ⬆️0.0%] test-mac-arm
[Finished ⬇️0.51% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.48% ⬆️0.0%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] e39749d1 ec2-t3-xlarge-us-east-2
[Failed] e39749d1 test-mac-arm
[Finished] e39749d1 ursa-i9-9960x
[Finished] e39749d1 ursa-thinkcentre-m75q
[Finished] b56b91e1 ec2-t3-xlarge-us-east-2
[Failed] b56b91e1 test-mac-arm
[Finished] b56b91e1 ursa-i9-9960x
[Finished] b56b91e1 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Feb 4, 2023

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

sjperkins pushed a commit to sjperkins/arrow that referenced this pull request Feb 10, 2023
@Fokko @westonpace @sjperkins
apacheGH-33972: [C++] Pass in metadata to ParquetReader (apache#34015)
7f1f738 
Closes apache#33972

### Rationale for this change

### What changes are included in this PR?

### Are these changes tested?

### Are there any user-facing changes?

* Closes: apache#33972

Lead-authored-by: Fokko Driesprong <fokko@apache.org>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Co-authored-by: Fokko Driesprong <fokko@tabular.io>
Signed-off-by: Weston Pace <weston.pace@gmail.com>
gringasalpastor pushed a commit to gringasalpastor/arrow that referenced this pull request Feb 17, 2023
@Fokko @westonpace
apacheGH-33972: [C++] Pass in metadata to ParquetReader (apache#34015)
e17ba2c 
Closes apache#33972

### Rationale for this change

### What changes are included in this PR?

### Are these changes tested?

### Are there any user-facing changes?

* Closes: apache#33972

Lead-authored-by: Fokko Driesprong <fokko@apache.org>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Co-authored-by: Fokko Driesprong <fokko@tabular.io>
Signed-off-by: Weston Pace <weston.pace@gmail.com>
fatemehp pushed a commit to fatemehp/arrow that referenced this pull request Feb 24, 2023
@Fokko @westonpace @fatemehp
apacheGH-33972: [C++] Pass in metadata to ParquetReader (apache#34015)
0e3c217 
Closes apache#33972

### Rationale for this change

### What changes are included in this PR?

### Are these changes tested?

### Are there any user-facing changes?

* Closes: apache#33972

Lead-authored-by: Fokko Driesprong <fokko@apache.org>
Co-authored-by: Weston Pace <weston.pace@gmail.com>
Co-authored-by: Fokko Driesprong <fokko@tabular.io>
Signed-off-by: Weston Pace <weston.pace@gmail.com>
Fokko added a commit to Fokko/iceberg that referenced this pull request May 11, 2023
@Fokko
Python: Bump PyArrow to 12.0.0
7c6da66 
This includes the fix apache/arrow#34015

Now:

```
[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:06:32.268 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        2.791ms      ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:06:32.558 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.379ms      ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.566 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.925ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:06:32.583 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        528µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        347µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        684µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        502µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        962µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.805ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.54ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.588 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.27ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.239ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.783ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        445µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        319µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        263µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        929µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        275µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.981ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        6.991ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        4.944ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        6.287ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        9.967ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.641 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        95.637ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        711.24ms     ↑ 159 B ↓ 15 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        767.182ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.694288s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.748438s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.895823s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        1.925349s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:06:32.644 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.944678s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:06:33.200 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.510093s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:33.972 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.326988s    ↑ 159 B ↓ 26 MiB
```

Before:
```
^C[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:19:09.894 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        897µs       ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:19:11.092 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.395ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.113 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.439ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.135ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        376µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        465µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        841µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.065ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        1.153ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.105ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.29ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        2.646ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.914ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        368µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        380µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        280µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        255µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        433µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        4.36ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.465ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        6.57ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        7.434ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        6.438ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        3.827ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.971ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.194 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        5.977ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.198 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        7.024ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        9.302ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        69.011ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        685.555ms    ↑ 159 B ↓ 15 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        886.157ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.711973s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.740533s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.754817s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:19:11.209 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.960224s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.066043s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:19:11.784 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.618623s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:12.526 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.407968s    ↑ 159 B ↓ 26 MiB
```
@Fokko Fokko mentioned this pull request May 11, 2023
Merged
Fokko added a commit to apache/iceberg that referenced this pull request May 12, 2023
@Fokko
Python: Bump PyArrow to 12.0.0 (#7588)
852c1bf 
This includes the fix apache/arrow#34015

Now:

```
[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:06:32.268 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        2.791ms      ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:06:32.558 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.379ms      ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.566 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.925ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:06:32.583 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        528µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        347µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        684µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        502µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        962µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.805ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.54ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.588 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.27ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.239ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.783ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        445µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        319µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        263µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        929µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        275µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.981ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        6.991ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        4.944ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        6.287ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        9.967ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.641 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        95.637ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        711.24ms     ↑ 159 B ↓ 15 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        767.182ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.694288s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.748438s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.895823s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        1.925349s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:06:32.644 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.944678s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:06:33.200 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.510093s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:33.972 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.326988s    ↑ 159 B ↓ 26 MiB
```

Before:
```
^C[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:19:09.894 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        897µs       ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:19:11.092 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.395ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.113 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.439ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.135ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        376µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        465µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        841µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.065ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        1.153ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.105ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.29ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        2.646ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.914ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        368µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        380µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        280µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        255µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        433µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        4.36ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.465ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        6.57ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        7.434ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        6.438ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        3.827ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.971ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.194 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        5.977ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.198 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        7.024ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        9.302ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        69.011ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        685.555ms    ↑ 159 B ↓ 15 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        886.157ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.711973s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.740533s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.754817s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:19:11.209 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.960224s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.066043s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:19:11.784 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.618623s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:12.526 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.407968s    ↑ 159 B ↓ 26 MiB
```
Fokko added a commit to apache/iceberg-python that referenced this pull request Sep 29, 2023
@Fokko
Python: Bump PyArrow to 12.0.0 (#7588)
91f2634 
This includes the fix apache/arrow#34015

Now:

```
[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:06:32.268 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        2.791ms      ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:06:32.558 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.379ms      ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.566 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.925ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:06:32.583 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        528µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        347µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        684µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        502µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        962µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.805ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.54ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.588 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.27ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.239ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.783ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        445µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        319µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        263µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        929µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        275µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.981ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        6.991ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        4.944ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        6.287ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        9.967ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.641 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        95.637ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        711.24ms     ↑ 159 B ↓ 15 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        767.182ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.694288s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.748438s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.895823s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        1.925349s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:06:32.644 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.944678s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:06:33.200 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.510093s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:33.972 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.326988s    ↑ 159 B ↓ 26 MiB
```

Before:
```
^C[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:19:09.894 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        897µs       ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:19:11.092 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.395ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.113 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.439ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.135ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        376µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        465µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        841µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.065ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        1.153ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.105ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.29ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        2.646ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.914ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        368µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        380µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        280µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        255µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        433µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        4.36ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.465ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        6.57ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        7.434ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        6.438ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        3.827ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.971ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.194 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        5.977ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.198 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        7.024ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        9.302ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        69.011ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        685.555ms    ↑ 159 B ↓ 15 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        886.157ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.711973s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.740533s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.754817s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:19:11.209 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.960224s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.066043s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:19:11.784 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.618623s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:12.526 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.407968s    ↑ 159 B ↓ 26 MiB
```
Fokko added a commit to apache/iceberg-python that referenced this pull request Sep 30, 2023
@Fokko
Python: Bump PyArrow to 12.0.0 (#7588)
d702ba6 
This includes the fix apache/arrow#34015

Now:

```
[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:06:32.268 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        2.791ms      ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:06:32.558 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.379ms      ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.566 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.925ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:06:32.583 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        528µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        347µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        684µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        502µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.584 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        962µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.805ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.54ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.588 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.27ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.239ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.589 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.783ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        445µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        319µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        263µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        929µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.618 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        275µs       ↑ 153 B ↓ 0 B
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.981ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        6.991ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        4.944ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.624 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        6.287ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.622 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        9.967ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:06:32.641 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        95.637ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        711.24ms     ↑ 159 B ↓ 15 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        767.182ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.694288s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.748438s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:32.640 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.895823s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:06:32.638 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        1.925349s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:06:32.644 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.944678s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:06:33.200 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.510093s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:06:33.972 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.326988s    ↑ 159 B ↓ 26 MiB
```

Before:
```
^C[root@fa7688db87e9 /]# mc admin trace --call s3 minio
2023-05-11T20:19:09.894 [206 Partial Content] s3.GetObject warehouse.minio:9000/nyc/taxis/metadata/00004-5eeae12f-6f3b-4c36-9413-2fbef26a94b7.metadata.json 172.22.0.2        897µs       ↑ 148 B ↓ 9.6 KiB
2023-05-11T20:19:11.092 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        2.395ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.113 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/snap-3831732930279655857-1-9f7fb22f-eb4a-42d2-9657-87cc8039004d.avro 172.22.0.1        1.439ms      ↑ 159 B ↓ 3.9 KiB
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        1.135ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        376µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        465µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.136 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        841µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.135 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        2.065ms      ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/9f7fb22f-eb4a-42d2-9657-87cc8039004d-m0.avro 172.22.0.1        1.153ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8e396fa9-dc45-4998-8d37-38665bf45724-m0.avro 172.22.0.1        1.105ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/8f666929-42ff-4198-a203-af5c85cfb233-m0.avro 172.22.0.1        2.29ms       ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.143 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/5f6429bd-1ae4-4a54-820e-f53387e04e2b-m0.avro 172.22.0.1        2.646ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.144 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/metadata/273e2d75-5d3a-4c2c-bebd-a6ef37cadbe2-m0.avro 172.22.0.1        1.914ms      ↑ 159 B ↓ 7.1 KiB
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        368µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        380µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.176 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        280µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        255µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.177 [200 OK] s3.HeadObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        433µs       ↑ 153 B ↓ 0 B
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        4.36ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.465ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        6.57ms       ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.180 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        7.434ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.182 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        6.438ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        3.827ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        3.971ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.194 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        5.977ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.198 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        7.024ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.196 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        9.302ms      ↑ 159 B ↓ 64 KiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        69.011ms     ↑ 159 B ↓ 6.6 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        685.555ms    ↑ 159 B ↓ 15 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        886.157ms    ↑ 159 B ↓ 16 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.711973s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.213 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1263-f568eb3b-a4d6-4a93-bcae-2b22f8baa1e7-00001.parquet 172.22.0.1        1.740533s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1272-519910f9-3e1b-4a64-8745-3c34d879233d-00001.parquet 172.22.0.1        1.754817s    ↑ 159 B ↓ 29 MiB
2023-05-11T20:19:11.209 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1281-d941a1d6-d041-4cf3-9e44-778191ae68fb-00001.parquet 172.22.0.1        1.960224s    ↑ 159 B ↓ 30 MiB
2023-05-11T20:19:11.211 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1290-df4bb385-1e5f-4841-960f-85600d3708e5-00001.parquet 172.22.0.1        2.066043s    ↑ 159 B ↓ 32 MiB
2023-05-11T20:19:11.784 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.618623s    ↑ 159 B ↓ 27 MiB
2023-05-11T20:19:12.526 [206 Partial Content] s3.GetObject 127.0.0.1:9000/warehouse/nyc/taxis/data/00003-1254-03dd06d8-f2da-4b9b-89ba-a2421e3f60fd-00001.parquet 172.22.0.1        1.407968s    ↑ 159 B ↓ 26 MiB
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@westonpace westonpace westonpace approved these changes

Assignees
No one assigned
Labels
Component: C++
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Python] Remove redundant S3 call
3 participants
@Fokko @westonpace @ursabot