Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-16226: [C++] Add better coverage for filesystem tell. #14064

Merged
merged 5 commits into from
Sep 22, 2022
Merged

ARROW-16226: [C++] Add better coverage for filesystem tell. #14064

merged 5 commits into from
Sep 22, 2022

Conversation

benibus
Copy link
Collaborator

@benibus benibus commented Sep 7, 2022

Based on ARROW-16226.

Adds coverage to GenericFileSystemTest::TestOpenInput(Stream|File) for validating Tell() and reads after seeking.

@github-actions
Copy link

github-actions bot commented Sep 7, 2022

https://issues.apache.org/jira/browse/ARROW-16226

@benibus
Copy link
Collaborator Author

benibus commented Sep 8, 2022

Looks like some of the new assertions are firing for GcsFileSystem due to its objects' Tell methods [uniquely] returning -1 rather than the input size after reading past the end. Not sure whether this is a legitimate discrepancy or if the return value of FileInterface::Tell is meant to be implementation-defined - in which case, I can remove those checks.

@pitrou
Copy link
Member

pitrou commented Sep 14, 2022

Looks like some of the new assertions are firing for GcsFileSystem due to its objects' Tell methods [uniquely] returning -1 rather than the input size after reading past the end.

This sounds like a bug that deserves fixing.

pitrou
pitrou requested changes Sep 22, 2022
View reviewed changes
Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @benibus ! This looks good, just two minor suggestions.

cpp/src/arrow/filesystem/test_util.cc Outdated Show resolved Hide resolved
cpp/src/arrow/filesystem/test_util.cc Show resolved Hide resolved
@benibus benibus requested a review from pitrou September 22, 2022 14:19
pitrou
pitrou approved these changes Sep 22, 2022
View reviewed changes
Copy link
Member

@pitrou pitrou left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Could you please rebase on git master to fix the macOS CI failures?

@benibus benibus force-pushed the ARROW-16226-fs-seektell-tests branch from af17b3d to 5ef3e28 Compare September 22, 2022 14:25
@pitrou
Copy link
Member

pitrou commented Sep 22, 2022

I see CI tests have passed on your fork, so I'm gonna merge now.

@pitrou pitrou merged commit 44ae852 into apache:master Sep 22, 2022
@ursabot
Copy link

ursabot commented Sep 23, 2022

Benchmark runs are scheduled for baseline = 4cb7b50 and contender = 44ae852. 44ae852 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️0.27% ⬆️0.1%] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.46% ⬆️0.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 44ae8523 ec2-t3-xlarge-us-east-2
[Failed] 44ae8523 test-mac-arm
[Failed] 44ae8523 ursa-i9-9960x
[Finished] 44ae8523 ursa-thinkcentre-m75q
[Finished] 4cb7b500 ec2-t3-xlarge-us-east-2
[Failed] 4cb7b500 test-mac-arm
[Failed] 4cb7b500 ursa-i9-9960x
[Finished] 4cb7b500 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

zagto pushed a commit to zagto/arrow that referenced this pull request Oct 7, 2022
@benibus @zagto
ARROW-16226: [C++] Add better coverage for filesystem tell. (apache#1…
d21c776 
…4064)

Based on [ARROW-16226](https://issues.apache.org/jira/browse/ARROW-16226).

Adds coverage to GenericFileSystemTest::TestOpenInput(Stream|File) for validating Tell() and reads after seeking.

Authored-by: benibus <bpharks@gmx.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
fatemehp pushed a commit to fatemehp/arrow that referenced this pull request Oct 17, 2022
@benibus @fatemehp
ARROW-16226: [C++] Add better coverage for filesystem tell. (apache#1…
d4130ad 
…4064)

Based on [ARROW-16226](https://issues.apache.org/jira/browse/ARROW-16226).

Adds coverage to GenericFileSystemTest::TestOpenInput(Stream|File) for validating Tell() and reads after seeking.

Authored-by: benibus <bpharks@gmx.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@benibus benibus deleted the ARROW-16226-fs-seektell-tests branch February 17, 2023 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pitrou pitrou pitrou approved these changes

Assignees
No one assigned
Labels
Component: C++
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@benibus @pitrou @ursabot