request image + r.pipe(fs.createWriteStream(...)) = bad file :( #47

Closed
NHQ opened this Issue Jul 21, 2011 · 8 comments

Comments

Projects
None yet
3 participants
@NHQ

NHQ commented Jul 21, 2011

I'm trying to

var r = request({uri:'http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif', method:'GET'});
r.pipe(fs.createWriteStream('public/images/new.gif'))

but the file that is being written is no good, for some reason. I have tried setting encoding for the response, with no luck. I also tried puling a profile pic from facebook, but with no better luck.

Sometimes the file seems to write completely blank. Yet when I request an html document, I have no problem writing that stream to a new file.

The simplicity of request makes this my failure all the more consternating. Help pls.

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Jul 21, 2011

Owner

Are you using master or the latest release?

-Mikeal

On Jul 20, 2011, at July 20, 20115:58 PM, NHQ wrote:

I'm trying to

var r = request({uri:'http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif', method:'GET'});
r.pipe(fs.createWriteStream('public/images/new.gif'))

but the file that is being written is no good, for some reason. I have tried setting encoding for the response, with no luck. I also tried puling a profile pic from facebook, but with no better luck.

The simplicity of request makes this my failure all the more consternating. Help pls.

Reply to this email directly or view it on GitHub:
mikeal#47

Owner

mikeal commented Jul 21, 2011

Are you using master or the latest release?

-Mikeal

On Jul 20, 2011, at July 20, 20115:58 PM, NHQ wrote:

I'm trying to

var r = request({uri:'http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif', method:'GET'});
r.pipe(fs.createWriteStream('public/images/new.gif'))

but the file that is being written is no good, for some reason. I have tried setting encoding for the response, with no luck. I also tried puling a profile pic from facebook, but with no better luck.

The simplicity of request makes this my failure all the more consternating. Help pls.

Reply to this email directly or view it on GitHub:
mikeal#47

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Jul 21, 2011

Owner

I just published a new version and I wrote a test for this functionality to make sure it's working .

-Mikeal

On Jul 20, 2011, at July 20, 20115:58 PM, NHQ wrote:

I'm trying to

var r = request({uri:'http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif', method:'GET'});
r.pipe(fs.createWriteStream('public/images/new.gif'))

but the file that is being written is no good, for some reason. I have tried setting encoding for the response, with no luck. I also tried puling a profile pic from facebook, but with no better luck.

The simplicity of request makes this my failure all the more consternating. Help pls.

Reply to this email directly or view it on GitHub:
mikeal#47

Owner

mikeal commented Jul 21, 2011

I just published a new version and I wrote a test for this functionality to make sure it's working .

-Mikeal

On Jul 20, 2011, at July 20, 20115:58 PM, NHQ wrote:

I'm trying to

var r = request({uri:'http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif', method:'GET'});
r.pipe(fs.createWriteStream('public/images/new.gif'))

but the file that is being written is no good, for some reason. I have tried setting encoding for the response, with no luck. I also tried puling a profile pic from facebook, but with no better luck.

The simplicity of request makes this my failure all the more consternating. Help pls.

Reply to this email directly or view it on GitHub:
mikeal#47

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Jul 21, 2011

Owner

btw, you can shorten your code to

request.get('http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif').pipe(fs.createWriteStream('public/images/new.gif'))
Owner

mikeal commented Jul 21, 2011

btw, you can shorten your code to

request.get('http://my-dev-ip/images/f68def8251d58fb9c9833168d8a7833a.gif').pipe(fs.createWriteStream('public/images/new.gif'))
@NHQ

This comment has been minimized.

Show comment Hide comment
@NHQ

NHQ Jul 21, 2011

I was using request request@1.9.5 with node-v0.4.10-pre

I updated to request@1.9.9 and will try it out tomorrow morning first thing.

Thx

NHQ commented Jul 21, 2011

I was using request request@1.9.5 with node-v0.4.10-pre

I updated to request@1.9.9 and will try it out tomorrow morning first thing.

Thx

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Jul 21, 2011

Owner

does this work now? i want to release 2.0 soon and i'm trying to get rid of all the pending bugs.

Owner

mikeal commented Jul 21, 2011

does this work now? i want to release 2.0 soon and i'm trying to get rid of all the pending bugs.

@NHQ

This comment has been minimized.

Show comment Hide comment
@NHQ

NHQ Jul 21, 2011

Sorry for the delay. (I thought I clicked to submit comment hours ago!)

Yes! Seems to be working fine, and with the shortened code as you provided. Thnx very much.

I have another question: What if I don't know the image file type I am requesting? That information will usually be in the response.headers. Is that object available to the writeStream?

NHQ commented Jul 21, 2011

Sorry for the delay. (I thought I clicked to submit comment hours ago!)

Yes! Seems to be working fine, and with the shortened code as you provided. Thnx very much.

I have another question: What if I don't know the image file type I am requesting? That information will usually be in the response.headers. Is that object available to the writeStream?

@mikeal

This comment has been minimized.

Show comment Hide comment
@mikeal

mikeal Jul 21, 2011

Owner

you can still pass a callback to request that will be executed when the headers are received and you can check the content-type.

Owner

mikeal commented Jul 21, 2011

you can still pass a callback to request that will be executed when the headers are received and you can check the content-type.

@mikeal mikeal closed this Jul 21, 2011

@maicss

This comment has been minimized.

Show comment Hide comment
@maicss

maicss Sep 15, 2017

Same problem.

I'm trying donwnload 50 images, about 10 images were damaged.

gist

Same code in Python is fine:

for img in imagesDate:
    r = requests.get(img["url"])
    if r.status_code == 200:
        with open('img/' + str(img["id"]) + '.' + img["format"], 'wb') as f:
            f.write(r.content)

Any ideas about what's wrong? Thanks.

maicss commented Sep 15, 2017

Same problem.

I'm trying donwnload 50 images, about 10 images were damaged.

gist

Same code in Python is fine:

for img in imagesDate:
    r = requests.get(img["url"])
    if r.status_code == 200:
        with open('img/' + str(img["id"]) + '.' + img["format"], 'wb') as f:
            f.write(r.content)

Any ideas about what's wrong? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment