Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Directly use bucket instead of Walking through containers #234

Open
himilshahvcc opened this issue Jun 12, 2020 · 3 comments
Open

Directly use bucket instead of Walking through containers #234

himilshahvcc opened this issue Jun 12, 2020 · 3 comments

Comments

@himilshahvcc
Copy link

Hi, I want to use this package to satisfy the requirement of connecting my container with various clouds. I have done the poc for uploading file to s3 bucket but it fails and gives below error.
Our infrastructure does not permit to iterate through the buckets but can access directly a particular bucket by providing - Region, KeyID & Secret.

Throws error from below configuration
err = stow.WalkContainers(location, stow.NoPrefix, 100, func(c stow.Container, err error) error {

Error
Error walking through container : Containers, listing the buckets: AccessDenied: Access Denied
status code: 403,

Kindly suggest if there is any workaround.

@jasonsattler
Copy link
Contributor

The stow.WalkContainers will iterate through all of you buckets which is why it need the "s3:ListAllMyBuckets" permission.

If you know the bucket name you can use location.Container to connect directly to that bucket.
For example:

package main

import (
	"fmt"
	"log"

	"github.com/graymeta/stow"
	"github.com/graymeta/stow/s3"
)

func main() {
	kind := "s3"
	config := stow.ConfigMap{
		s3.ConfigAccessKeyID: "xxxxxxxxxx",
		s3.ConfigSecretKey:   "xxxxxxxxxxxx",
		s3.ConfigRegion:      "us-east-1",
	}
	location, err := stow.Dial(kind, config)
	if err != nil {
		log.Fatal("unable to Dial location")
	}
	defer location.Close()

        // Pull Container info
	container, err := location.Container("example-bucket-useast1")
	if err != nil {
		log.Fatal("Unable to find container")
	}

        // Walk the entire container
	err = stow.Walk(container, stow.NoPrefix, 100, func(item stow.Item, err error) error {
		if err != nil {
			return err
		}

		fmt.Println("Name: ", item.Name())
		return nil
	})
	if err != nil {
		log.Fatal("Unable to walk container")
	}
}

@marbergq
Copy link
Contributor

I think the first problem lies in the fact that the stow s3 tries to make a location lookup and a region lookup, which will fail, since these are made anonymously. Is that expected?

@jasonsattler
Copy link
Contributor

In that case you can create a new stow kind and check all the api calls to make sure they are supported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants