S3 persistency possible not working?

[speier]

I've tried a really basic example, but doesn't see anything persisted to the specified S3 bucket.

var Gun = require('gun');

var gun = Gun({
    s3: {
        bucket: 'foo-bar' // my aws keys are set in env variables
    }
});

gun.set({ hello: 'world' }).key('my/first/data');

What's wrong, am I miss something here?
Many thanks, keep up the good work!

[amark]

What are the names of the ENV variables? They have to be specific. Could you also maybe try passing the S3 opt with the explicitly referencing process.env? See https://github.com/alexlafroscia/stickies/blob/master/server/app.js as an example.

Also, if you don't already know this... S3 can be fickle, you have to create the bucket before hand and you have to create an Access Control Policy for your token that allows it read/write to that bucket.

If you're not familiar with S3 let me know and I'll send you a sample.

[speier]

Using S3 regularly, so my env keys are fine. It turns out, I have to specify the region though.

$ aws s3 cp test.txt s3://foo-bar/test.txt

upload failed: [...] You are attempting to operate on a bucket in a region that requires Signature Version 4. You can fix this issue by explicitly providing the correct region location [...]

So I've specified the region and it wokrs like charm.

var gun = Gun({
    s3: {
        bucket: 'foo-bar', // my aws keys are set in env variables
        region: 'eu-central-1'
    }
});

Btw, thanks for the quick response.

ps: it would be nice if the S3 plugin could provide error details, if possible..

[amark]

Errors propagate back up through the plugins, for instance if:

gun.set({foo: 'bar'}, function(err){
  if(err){ console.log(err) }
});

Fails at the S3 level, you'll get that error back in that callback.

[speier]

There is no callback in this case. Gun keeps trying to create the key..
https://github.com/amark/gun/blob/master/lib/s3.js#L76

[amark]

ooooh, yeah. That counts as a bug, because it should auto-retry up to an exponential retry limit where it finally does halt, and report the error. This needs to be fixed. Know any easy/simple inline exponential retry tricks?

[speier]

Here is a really simple exponential backoff algorithm: http://blog.johnryding.com/post/78544969349/how-to-reconnect-web-sockets-in-a-realtime-web-app

OR here is a more sopisthicated package: https://github.com/unshiftio/recovery

[speier]

I was interested implementing this, so came up with a simple solution, please find here: speier/bakeoff

[amark]

Thanks, the johnryding article is actually really nice and helpful. Nice bakeoff library, just starred it.

While, yes, this thread should be closed, a bug still remains on the infinite loop of S3 retries. Not very high on my priority list, but it is still on my mind.

You should chat with me in the https://gitter.im/amark/gun!