Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fetching empty GCS storage bucket of type dir fails #646

Closed
bobcatfish opened this issue Mar 20, 2019 · 1 comment · Fixed by #737
Closed

Fetching empty GCS storage bucket of type dir fails #646

bobcatfish opened this issue Mar 20, 2019 · 1 comment · Fixed by #737
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. meaty-juicy-coding-work This task is mostly about implementation!!! And docs and tests of course but that's a given

Comments

@bobcatfish
Copy link
Collaborator

Expected Behavior

If a user is invoking a Task that needs a GCS bucket, sometimes that bucket will be empty and will be populated by the Task.

It should be possible to fetch this empty bucket and use it in the Task.

Actual Behavior

If you try to use a GCS storage resource with the dir flag set as an input, and the bucket is empty, the container which fetches the contents of the bucket fails with an error like:

Error executing command "gsutil cp -r gs://christies-empty-bucket/* /workspace/bucket" ; error exit status 1; cmd_output CommandException: No URLs matched: gs://christies-empty-bucket/*
"   
  stacktrace:  "main.main
	/Users/christiewilson/Code/go/src/github.com/tektoncd/pipeline/cmd/gsutil/main.go:94
runtime.main
	/usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198"   
  ts:  1553120763.622061   

Steps to Reproduce the Problem

  1. Make an empty GCS bucket (e.g. create a new one or delete everything from an existing one)
  2. Crate a GCS bucket resource, e.g.:
---
apiVersion: tekton.dev/v1alpha1
kind: PipelineResource
metadata:
  name: tekton-bucket
spec:
  type: storage
  params:
   - name: type
     value: gcs
   - name: location
     value: gs://christies-empty-bucket # REPLACE with your own bucket
   - name: dir
     value: "y
  1. Try to use that resource as an input to a task via a taskrun

Additional Info

In #632 we create a Task that operates on a bucket. If the bucket is empty, the intention is that it will create the folders it needs, and in subsequent calls it won't be empty.

@bobcatfish bobcatfish added kind/bug Categorizes issue or PR as related to a bug. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. meaty-juicy-coding-work This task is mostly about implementation!!! And docs and tests of course but that's a given labels Mar 20, 2019
@blrsn
Copy link

blrsn commented Mar 29, 2019

This seems to be an issue with gsutil. This is the same issue, but for rm command.
GoogleCloudPlatform/gsutil#417

gsutil fails for both
gsutil cp -r gs://empty-bucket /workspace/bucket
and
gsutil cp -r gs://empty-bucket/* /workspace/bucket

chmouel pushed a commit to chmouel/tektoncd-pipeline that referenced this issue Apr 13, 2021
…o-switch

Replace docker.io images with a mirror from public cloud provider
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Denotes an issue ready for a new contributor, according to the "help wanted" guidelines. help wanted Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines. kind/bug Categorizes issue or PR as related to a bug. meaty-juicy-coding-work This task is mostly about implementation!!! And docs and tests of course but that's a given
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants