ARROW-16294: [C++] Improve performance of parquet readahead

[westonpace]

• Turns out the batch size we were slicing internally for parquet (in the TableBatchReader) was not the batch size from the scanner. I added batch slicing in file_parquet to leave the parquet reader itself more or less unchanged here (and it's not clear it would make more sense to use the smaller batch size slicing inside the reader anyways).
• Changed parquet readahead so it reads ahead more than one row group. It now tries to keep batch_size * batch_readahead reads in flight. Users that want more parallel reads can increase batch readahead.

[github-actions]

https://issues.apache.org/jira/browse/ARROW-16294

[github-actions]

⚠️ Ticket has no components in JIRA, make sure you assign one.

[github-actions]

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

[lidavidm]

Looks like there's a lint failure, and MSVC is unhappy as usual:

D:/a/arrow/arrow/cpp/src/arrow/dataset/file_parquet.cc(465,1): error C2220: the following warning is treated as an error [D:\a\arrow\arrow\build\cpp\src\arrow\dataset\arrow_dataset_shared.vcxproj]
D:/a/arrow/arrow/cpp/src/arrow/dataset/file_parquet.cc(465,1): warning C4244: 'argument': conversion from 'int64_t' to 'int', possible loss of data [D:\a\arrow\arrow\build\cpp\src\arrow\dataset\arrow_dataset_shared.vcxproj]

[westonpace]

Looks like there's a lint failure, and MSVC is unhappy as usual:

The dangers of thinking I can fix an MSVC complaint using Github's edit feature 😆

I think this is good now. I changed the variable itself to int64_t instead of adding static_cast since it is now a "rows" variable and not a "batches" variable.

The behavior for 0 readahead is now "fetch batch when asked for" instead of defaulting to something like 1 row (which probably would have been the same thing anyways but I think this is more clear about our intentions).

[lidavidm]

Seems MSVC is still unhappy here:

C:/projects/arrow/cpp/src/parquet/arrow/reader.cc(1099): error C2220: warning treated as error - no 'object' file generated
C:/projects/arrow/cpp/src/parquet/arrow/reader.cc(1099): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data

[ursabot]

Benchmark runs are scheduled for baseline = f03f090 and contender = 24f3722. 24f3722 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.31% ⬆️0.04%] test-mac-arm
[Finished ⬇️0.36% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.2% ⬆️0.08%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 24f37229 ec2-t3-xlarge-us-east-2
[Finished] 24f37229 test-mac-arm
[Finished] 24f37229 ursa-i9-9960x
[Finished] 24f37229 ursa-thinkcentre-m75q
[Finished] f03f090d ec2-t3-xlarge-us-east-2
[Failed] f03f090d test-mac-arm
[Finished] f03f090d ursa-i9-9960x
[Finished] f03f090d ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

[ursabot]

['Python', 'R'] benchmarks have high level of regressions.
test-mac-arm