[Python] Windows wheel is built without GCS support #35193
Comments
westonpace
changed the title
|
When we originally added GCS support it was not enabled for Windows wheels. This is the original PR: #12763 I have tried to find if there's an issue opened for it but I can't find anything. I am unsure if there's a reason for it as I am no expert on GCS but at the moment we don't build |
|
Alright - we are using linux in prod containers, and I can use WSL for Py interpreter for dev, however, it would be good to post a disclaimer in the doc about what's not included in the Windows wheels. Feel free to close. |
|
I'm not sure the rationale either but I don't think we should close this. Let's leave it open to add support for GCS on Windows wheels. We do build GCS for our windows C++ CI tests (appveyor) so I'm not sure why it was excluded from wheels. It's possible it was just an oversight. |
raulcd
changed the title
|
I agree with @westonpace . I've modified the title so it is easier to find in the future. Thanks for opening the issue! |
raulcd
added a commit
to raulcd/arrow
that referenced
this issue
Apr 20, 2023
raulcd
added a commit
to raulcd/arrow
that referenced
this issue
Apr 27, 2023
raulcd
added a commit
to raulcd/arrow
that referenced
this issue
May 10, 2023
assignUser
pushed a commit
that referenced
this issue
May 16, 2023
### Rationale for this change GCS could be enabled on Windows wheels. ### What changes are included in this PR? Enabling GCS on Windows wheel ### Are these changes tested? Crossbow jobs for wheels run tests for GCS now. I have tested locally that I can install the built wheel and I can import `GcsFileSystem`: ``` Python 3.9.12 (main, Apr 4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32 Warning: This Python interpreter is in a conda environment, but the environment has not been activated. Libraries may fail to load. To activate this environment please see https://conda.io/activation Type "help", "copyright", "credits" or "license" for more information. >>> from pyarrow.fs import GcsFileSystem >>> fs = GcsFileSystem(access_token='abc',target_service_account='service_account@ apache',credential_token_expiration=datetime.now(),default_bucket_location='us-west2',scheme='https', endpoint_override='localhost:8999') >>> fs.default_bucket_location 'us-west2' >>> fs.create_dir('hello') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "pyarrow\_fs.pyx", line 593, in pyarrow._fs.FileSystem.create_dir File "pyarrow\error.pxi", line 113, in pyarrow.lib.check_status PermissionError: [Errno 13] google::cloud::Status(UNAUTHENTICATED: Permanent error GetBucketMetadata: Could not create a OAuth2 access token to authenticate the request. The request was not sent, as such an access token is required to complete the request successfully. Learn more about Google Cloud authentication at https://cloud.google.com/docs/authentication. The underlying error message was: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.). Detail: [errno 13] Permission denied ``` ### Are there any user-facing changes? No but Windows wheels should contain `ARROW_GCS` * Closes: #35193 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
rtpsw
pushed a commit
to rtpsw/arrow
that referenced
this issue
May 16, 2023
…che#35255) ### Rationale for this change GCS could be enabled on Windows wheels. ### What changes are included in this PR? Enabling GCS on Windows wheel ### Are these changes tested? Crossbow jobs for wheels run tests for GCS now. I have tested locally that I can install the built wheel and I can import `GcsFileSystem`: ``` Python 3.9.12 (main, Apr 4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32 Warning: This Python interpreter is in a conda environment, but the environment has not been activated. Libraries may fail to load. To activate this environment please see https://conda.io/activation Type "help", "copyright", "credits" or "license" for more information. >>> from pyarrow.fs import GcsFileSystem >>> fs = GcsFileSystem(access_token='abc',target_service_account='service_account@ apache',credential_token_expiration=datetime.now(),default_bucket_location='us-west2',scheme='https', endpoint_override='localhost:8999') >>> fs.default_bucket_location 'us-west2' >>> fs.create_dir('hello') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "pyarrow\_fs.pyx", line 593, in pyarrow._fs.FileSystem.create_dir File "pyarrow\error.pxi", line 113, in pyarrow.lib.check_status PermissionError: [Errno 13] google::cloud::Status(UNAUTHENTICATED: Permanent error GetBucketMetadata: Could not create a OAuth2 access token to authenticate the request. The request was not sent, as such an access token is required to complete the request successfully. Learn more about Google Cloud authentication at https://cloud.google.com/docs/authentication. The underlying error message was: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.). Detail: [errno 13] Permission denied ``` ### Are there any user-facing changes? No but Windows wheels should contain `ARROW_GCS` * Closes: apache#35193 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug, including details regarding any error messages, version, and platform.
Error using latest 11.0 on Windows.
I suspect
pyarrow-11.0.0-cp311-cp311-win_amd64.whlhas no GCS support, and maybe others.Component(s)
Python
The text was updated successfully, but these errors were encountered: