Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-16247: [GLib] Add GArrowGCSFileSystem #12931

Closed
wants to merge 1 commit into from

Conversation

kou
Copy link
Member

@kou kou commented Apr 20, 2022

No description provided.

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@pitrou
Copy link
Member

pitrou commented Apr 21, 2022

Hmm... this looks quite empty, are you planning to add support for GcsOptions? Or are people expected to instantiate these filesystems using an URI?

@kou
Copy link
Member Author

kou commented Apr 22, 2022

this looks quite empty, are you planning to add support for GcsOptions?

I'll add do it eventually but it's usable without this by #12763 (or #12932).

Or are people expected to instantiate these filesystems using an URI?

Yes. We can load data by Arrow::Table.load("gcs://bucket/a.arrow") with this. https://github.com/apache/arrow/blob/master/ruby/red-arrow-dataset/lib/arrow-dataset/arrow-table-loadable.rb#L38 instantiates GArrowGCSFileSystem and it's used to load data.

@pitrou
Copy link
Member

pitrou commented Apr 22, 2022

Ok, so should there be a dedicated test?

@kou
Copy link
Member Author

kou commented Apr 22, 2022

We can't implement now because gcs://bucket/a.arrow doesn't work without #12763.

@pitrou
Copy link
Member

pitrou commented Apr 22, 2022

Ah, fair enough :-)

@kou kou closed this in 65e8863 Apr 22, 2022
@kou kou deleted the glib-gcs-file-system branch April 22, 2022 08:21
@ursabot
Copy link

ursabot commented Apr 25, 2022

Benchmark runs are scheduled for baseline = 24b4808 and contender = 65e8863. 65e8863 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Failed ⬇️1.13% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️1.22% ⬆️0.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/582| 65e88630 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/570| 65e88630 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/568| 65e88630 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/580| 65e88630 ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/581| 24b48083 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/569| 24b48083 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/567| 24b48083 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/579| 24b48083 ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@ursabot
Copy link

ursabot commented Apr 26, 2022

['Python', 'R'] benchmarks have high level of regressions.
ursa-i9-9960x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants