Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-signed urls problem with GCS #4138

Open
mevis89 opened this issue May 3, 2023 · 8 comments
Open

Pre-signed urls problem with GCS #4138

mevis89 opened this issue May 3, 2023 · 8 comments
Assignees
Labels
import question just a question storages External / Cloud storage connections

Comments

@mevis89
Copy link

mevis89 commented May 3, 2023

Describe the bug
When i try to connect the GCS, with "Use pre-signed URLs" enabled, the response is a list of obj (in my case) with the link gs:// and not http..

To Reproduce
Steps to reproduce the behavior:

  1. Go to Settings
  2. Click on Cloud Storage
  3. Set Google Cloud Storage
  4. set all fields and enable "Use pre-signed URLs"
  5. Save
  6. Sync Storage

Expected behavior
I expect to receive links to images located within my bucket so that i can use them within the image tag.

Environment (please complete the following information):

  • OS: Ubuntu 20.04.6 LTS
  • Label Studio Version 1.7.3
@AbubakarSaad
Copy link
Collaborator

Hello mevis89,

I'm not sure whats happening here. Is it possible to give example? Are you using the uri when importing your data?

@AbubakarSaad AbubakarSaad added storages External / Cloud storage connections import question just a question labels May 10, 2023
@ncgovindans
Copy link

ncgovindans commented Jul 3, 2023

+1 - V1.8.0 - the URI for GCS isn't resolving to a https:// pre-signed url - it's just using the GS:// uri to try and locate the image

Update:
@makseq - turned the feature flag off via a docker build and feature_flag.json. Seemingly didn't work, but might have be a caching on the server causing this. Subsequently, updated credentials to the GCS bucket. That seemed to kick start it. Deleting the credentials from the storage settings, and just using ADC it all seems to be working.

@philipkopylov
Copy link

Same issue here, cannot get labelstudio to load objects from GCS.

@ncgovindans
Copy link

@philipkopylov Version, and build info? - would love to try and help if I'm working from the same settings.

@Dr-Desty-Nova
Copy link

Dr-Desty-Nova commented Oct 24, 2023

Same issue, here's the error from logs

[2023-10-24 19:39:37,899] [data_import.api::get::661] [ERROR] 
Failed to resolve storage uri gs://<bucket>/<path-to-image>.png for task 1: you need a private key to sign credentials.
the credentials you are currently using <class 'google.oauth2.credentials.Credentials'> just contains a token. 
see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.

@gbmarc1
Copy link

gbmarc1 commented Oct 26, 2023

I get the same above error

[2023-10-26 19:08:36,533] [data_import.api::get::699] [ERROR] Failed to resolve storage uri gs://.../A2908_02_003.jpg for task 549: you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.

V 1.9.1

@remoyson
Copy link

I'am having the same issue with loading images from GCS. Anybody that has been able to fix this or has a workaround? Would really like to use the newer versions from Labelstudio as it includes some performance improvements but with the later versions I have this blocking GCS loading problem :(

@rjanovski
Copy link

Not sure if this is the same issue, but in our case using the default credentials (leaving the field blank) was causing the permission issue.

Solution: Don't use the default google-application-credentials

Instead:

  1. create a service-account (iam)
  2. add AIM permissions for storage and service-account-token-creator to the service-account
  3. create a key for the service account (json)
  4. use the key (entire json) as application-credentials:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
import question just a question storages External / Cloud storage connections
Projects
None yet
Development

No branches or pull requests

9 participants