Skip to content

Loading…

Write JSON output to S3 without fs.readFile #33

Closed
intool opened this Issue · 7 comments

5 participants

@intool

Hi TJ,
Don't know the right place to ask howto questions for Knox [couldn't find a wiki/doc beyond the Readme.md...]
Simple question:
I want to write the output of a db query directly to S3 to use Cloudfront as my database cache. This does not involve an fs.readFile rather a db.find() query which I want to put the result to S3 as a JSON object.
Is this possible? or do I need to write it as a temporary file on my fs and then read/put it to S3?
Thanks.

@tj
tj commented

if you look at the source (or i think in the readme as well) you can stream arbitrary data to s3, the putFile stuff etc are just sugar on top of this

@jockster

Did you manage to get this working? I have checked out client.putStream but can't get it working as throws TypeError: Bad argument at Object.stat (fs.js:402:11) when I feed the method a JSON object

@standingwave

Any update on how to do this? I have the exact same need.

@jockster

I figured out that I could do new Buffer(someJSON, 'binary'), feed it to client.put and get it working.

@standingwave

Thanks! I thought I had tried that already but apparently not.

@jockster

Well. It would be nice if this could be mentioned in the docs though. It took me some time to figure it out.

@domenic

I'll use this bug to track the fact we need to better document how to put buffers, text, etc. directly into S3.

@domenic domenic added a commit that closed this issue
@domenic domenic Update README with string-PUTing example. Closes #33.
This is also great because I hated showing PUTing a file via buffering it into memory, since that's a bad idea. Now we have a legit example for PUT.
114088e
@domenic domenic closed this in 114088e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.