Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream the http response #213

Closed
3 tasks done
bajtos opened this issue Mar 18, 2014 · 13 comments
Closed
3 tasks done

Stream the http response #213

bajtos opened this issue Mar 18, 2014 · 13 comments
Labels

Comments

@bajtos
Copy link
Member

bajtos commented Mar 18, 2014

Consider a remotable method that returns a stream as the result, for example "download file" in loopback-storage-service (source).

The current implementation takes the response object as an argument and does not call the provided callback. This means that "after" hooks are never called. (remoting metadata).

File.download = function(file, res, cb) {
  var reader = fs.createReadStream('storage/' + file);
  reader.on('error', function (err) {
    res.type('application/json');
    res.send(500, { error: err });
  });
  reader.pipe(res);
  // callback is intentionally not invoked
};

File.remoteMethod('download', {
  accepts: [
    {arg: 'file', type: 'string', 'http': {source: 'path'}},
    {arg: 'res', type: 'object', 'http': {source: 'res'}}
  ],
  http: {
    verb: 'get',
    path: '/download/:file'
  }
});

We should make streams a first-class citizen in strong-remoting.

HttpContext.prototype.invoke seems to already support piping a stream provided by the remotable method to the HTTP response (source).


Tasks

A note for loopback-swagger support. Swagger spec says: http://swagger.io/specification/#responseObject

As an extension to the Schema Object, its root type value may also be "file". This SHOULD be accompanied by a relevant produces mime-type.

I don't remember whether loopback-swagger supports per-method produces option. If it does not, then we should implement it as part of this work.

Related: strongloop/loopback-component-explorer#103

@raymondfeng
Copy link
Member

There are a couple of issues related to our strong-remoting stream support.

  • It expects the stream is returned from the method call instead of the callback
  • It only handles the stream, not headers such as Content-Type.

Maybe we should plug in an adapter into strong-remoting to handle multipart/form-data requests and have the storage-service upload function to be configured to the 'stream' binding.

@bajtos
Copy link
Member Author

bajtos commented Mar 18, 2014

👍

This is my take on the API for stream responses (not a final version, just an idea):

function streaming(cb) {
  var bodyStream = // create response stream from file
  var contentType = // set from file extension
  cb(null, bodyStream, contentType);
}

loopback.remoteMethod(streaming, {
  returns: [
    { arg: 'bodyStream', type: 'stream', root: true },
    { arg: 'content-type', type: 'string', http: { target: 'header' } }
  ]
});

Not sure what API to use for request streams and multi-part requests.

@ritch
Copy link
Member

ritch commented Mar 18, 2014

+1

@bajtos
Copy link
Member Author

bajtos commented May 16, 2014

Similar discussion: strongloop/strong-remoting#35

@haio
Copy link
Contributor

haio commented Jul 20, 2014

Hi, any update on this issue? is there a example of how to implement download file functionality with loopback?
Thanks!

@bajtos
Copy link
Member Author

bajtos commented Jul 20, 2014

@haio We don't have a proper support for streams implemented yet. The workaround of passing the HTTP response object to the remoted function should work though - see the links to loopback-storage-services in the issue description for more details.

File.download = function(file, res, cb) {
  var reader = fs.createReadStream('storage/' + file);
  reader.on('error', function (err) {
    res.type('application/json');
    res.send(500, { error: err });
  });
  reader.pipe(res);
  // callback is intentionally not invoked
};

File.remoteMethod('download', {
  accepts: [
    {arg: 'file', type: 'string', 'http': {source: 'path'}},
    {arg: 'res', type: 'object', 'http': {source: 'res'}}
  ],
  http: {
    verb: 'get',
    path: '/download/:file'
  }
});

@bajtos bajtos added feature and removed question labels Jul 20, 2014
@bajtos bajtos changed the title How to stream the http response Stream the http response Jul 20, 2014
@haio
Copy link
Contributor

haio commented Jul 25, 2014

@bajtos I follow your way and see it works, thanks!

@tchambard
Copy link

Hi, I followed your server side implementation example to implement my own streaming from mongoDb gridfs storage, but I would be very interesting by the client side solution.
What I do not understand is how to pass the http response object as second argument...
I'm using Angular SDK and lbServices... but even with $http, I don't know how to do it...
Thanks for your help.

@tchambard
Copy link

I found the issue for my concern...
I needed to use res.send(500).send({ error: err }) instead of res.send(500, { error: err });

@benawarez
Copy link

Could you share how you implemented gridfs storage? Thanks.

@bajtos
Copy link
Member Author

bajtos commented Apr 7, 2016

Closing in favour of follow-up issues: strongloop/loopback-swagger#34 and #2209

@bajtos bajtos closed this as completed Apr 7, 2016
@web-hollywood
Copy link

Hi bajtos

I'm currently trying to download a file from the backend API(loopback3).
So I integrated it like this.

UserProfile.remoteMethod (
'totalDeductions',
{
description: 'Total deductions for exporting',
accepts: [
{arg: 'sDate', type: 'date', description: 'Start date for the report.', required: true},
{arg: 'eDate', type: 'date', description: 'End date for the report.', required: true},
{arg: 'res', type: 'object', 'http': {source: 'res'}}
],
isStatic: true,
returns: [
{ arg: 'body', type: 'file', root: true },
{ arg: 'Content-Type', type: 'string', http: { target: 'header' } },
{ arg: 'Content-Disposition', type: 'string', http: { target: 'header' } },
],
http: {verb: 'get', path: '/total-deductions'}
}
);

UserProfile.totalDeductions = function(sDate, eDate, res, cb) {
var DeductionModel = app.models.Deduction;
var filter = {
where: {
dateCreated: {between: [sDate,eDate]}
}
};

DeductionModel.find(filter, function(err, deductions) {
  var conf = {};
  conf.name = "mysheet";
  conf.cols = [{
    caption:'string',
        type:'string',
        beforeCellWrite:function(row, cellData){
       return cellData.toUpperCase();
    },
        width:28.7109375
  },{
    caption:'date',
    type:'date',
    beforeCellWrite:function(){
      var originDate = new Date(Date.UTC(1899,11,30));
      return function(row, cellData, eOpt){
              if (eOpt.rowNum%2){
                eOpt.styleIndex = 1;
              }  
              else{
                eOpt.styleIndex = 2;
              }
                if (cellData === null){
                  eOpt.cellType = 'string';
                  return 'N/A';
                } else
                  return (cellData - originDate) / (24 * 60 * 60 * 1000);
      } 
    }()
  },{
    caption:'bool',
    type:'bool'
  },{
    caption:'number',
     type:'number'        
  }];
  conf.rows = [
    ['pi', new Date(Date.UTC(2013, 4, 1)), true, 3.14],
    ["e", new Date(2012, 4, 1), false, 2.7182],
        ["M&M<>'", new Date(Date.UTC(2013, 6, 9)), false, 1.61803],
        ["null date", null, true, 1.414]  
    ];
  var result = nodeExcel.execute(conf);
  if (err) {
    cb(err);
    return;
  }
  var contentType =  'application/force-download';
  var contentDisposition = 'attachment; filename=xrt.pdf';
  cb(null, result,contentType, contentDisposition);
});

But I'm not able to download a file. It still returns response as a contect, not a file.
Can you please help me?

@bajtos
Copy link
Member Author

bajtos commented Dec 14, 2017

@web-hollywood I don't see any obvious problem. Please create a small app that I can use to reproduce the issue on my computer (see http://loopback.io/doc/en/contrib/Reporting-issues.html#bug-report) and open a new issue. Thanks!

@strongloop strongloop locked and limited conversation to collaborators Dec 14, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

8 participants