Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Displaying images to a client... #80

johnwards opened this Issue Sep 19, 2011 · 9 comments


None yet
4 participants

So I am writing a simple proxy.

The following code works for html and text fine, however for images I just get an error

The image “http://localhost:3000/foo.com/image.jpg” cannot be displayed because it contains errors.

I have upgraded to the latest version on github and confirmed it's using that code by waking in some console logs in where the chunks are being added.

Not sure what I'm doing wrong!

app.get(/\/([^\/]+)(\/.*)/, function(req,res){

      var target_host = req.params[0],
      target_path = req.params[1];

      request({uri:"http://" + target_host + target_path}, function(error, response, content) {

mikeal commented Sep 20, 2011

the headers aren't getting set that you want, most notably content-type. also, you should never buffer the whole body in to memory like this when you're proxying. all these issues are fixed if you use streaming.

request("http://" + target_host + target_path).pipe(res)

If you also want to respect the incoming headers

var x = request("http://" + target_host + target_path)

@mikeal mikeal closed this Sep 20, 2011


Marsup commented Sep 20, 2011

This may also be due to this line response.body = body.toString(). I had similar problems and switched to a pipe because the content, when binary, was converted to some utf8 string, which I haven't been able to convert back to binary properly.

Hmm problem is I need to modify text/html content, before piping to the client.

Off to have a look to see if there are suitable events I guess :(


Marsup commented Sep 20, 2011

I've used the module bufferstream to circumvent the problem. I've set it to flexible mode so that I get a buffer that I can manipulate myself. Hope this helps.

dscape commented Sep 20, 2011

@johnwards What @mikeal said applies fine, if you want to see a little blog post introducing the topic you can look at:

See Hello Pipe

Also node-http-proxy (https://github.com/nodejitsu/node-http-proxy) might be interesting for you


Marsup commented Sep 20, 2011

I don't think if fits if you want to modify what's transmitted in the pipe, or am I missing something ?

dscape commented Sep 20, 2011

All requests are readable/writeable streams so you can always look at the data chunks and modify them. You can do this by not providing a callback function:

request.on('data', /* do something */);
request.on('end', /* finished */);

Now questions is why would you do that on a binary file? Piping seems like a better approach, plus as @mikeal stated you avoid buffering eventually large binary files :) You can always request.on('error') for error handling code.

As for the modifying an html file on the run you can use saxjs. @mikeal has a good example on https://github.com/mikeal/planet/blob/master/feedstream.js

As for proxying using node-http-proxy might be a better idea than implementing a simple but naive proxy. Source code is open and if you are curious you can look at it's implementation. This is an important piece of nodejitsu architecture so you can probably feel safer than rolling your own at first (if you don't like something you can always send a pull request).

Sample proxy code:

var proxy   = require('http-proxy')
  , www     = require('cfg/www').port
  , api     = require('cfg/api').port
  , options = { router: { 'api.localhost': '' + api
                        , 'localhost':     '' + www
  , proxy_server = proxy.createServer(options);

console.log("Listening on 80 with options " + JSON.stringify(options));

I'm afraid I can't help further cause I can't see how request is a limitation here, and how this would be an issue with request. Maybe I am missing something? :)

Yeah I need to modify the pipe.

@Marsup I can't find an example of how to use bufferstream with request. :( Any quick code sample to get me started?

Missed the @dscape comment. Will need to digest that first.

I'm trying to modify html/text streams and not binary streams.

Looks like the request "data" and "end" events are what I need...sorry for cluttering up this issue. I do hope all this will be useful for someone in the future ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment