Setting the POST size limit in sails 0.10.5

[sheoak]

I can’t find how to configure the POST size limit in sails 0.10.5.

When I post to a simple blueprint url like /mymodel with a lot of data, I get this, due to the default limit:

error: Unable to parse HTTP body- error occurred :: { [Error: request entity too large]
type: 'entity.too.large',
message: 'request entity too large',
statusCode: 413,
status: 413,
expected: 1647840,
length: 1647840,
limit: 1048576 }

I tried configuring bodyParser in http.js but none of the methods I found seems to work. The error message just disappear and blueprint don’t see the data anymore.

I’m not sure if it’s a bug or if I’m just doing it wrong. Any idea?

[MBrouns]

Could you post how you set up your http.js?

[sheoak]

Sure. With this version the API doesn’t get the POST data anymore in blueprints. I tried skipper too. I’m not sure I understand properly the role of bodyParser yet.

module.exports.http = {
    middleware: {
         order: [
             'startRequestTimer',
             'cookieParser',
             'session',
             'myRequestLogger',
             'bodyParser',
             'handleBodyParserError',
             'compress',
             'methodOverride',
             '$custom',
             'router',
             'www',
             'favicon',
             '404',
             '500'
        ],
        compress: require('compression')(),
        bodyParser: {
            fn: require('connect-busboy'),
            options: {
                limit: '30M'
            }
        }
    },
};

[sheoak]

I also tried with a limit like 1000000000

[MBrouns]

Could you try incorporating

module.exports.http = {
  bodyParser: (function () {
    var opts = {limit:'50mb'};
    var fn;

    // Default to built-in bodyParser:
    fn = require('skipper');
    return fn(opts);

  })
};

as mentioned in balderdashy/skipper#22

[sheoak]

I tried that too (I tried again). The API returns no response and the page keeps loading until I get a 502 error (no data received). I got no error from the API.

[sheoak]

Ah! But I tried the version that outputs an error and it works:

  bodyParser: require('skipper')({
      limit: 52428800
  })

I just need to get rid of the error now… I wonder why your version doesn’t work, it’s supposed to be the fix.

[sheoak]

After looking at the code in https://github.com/balderdashy/sails/blob/master/lib/hooks/http/middleware/defaults.js I think it might be a bug. The syntax I was using is what is suggested in the error message, yet it seems to break the bodyParser. I will make some test about that tomorrow.

Thanks for your help @MBrouns!

[jdcauley]

I just read through this and am trying to understand what goes where for this?

[jdcauley]

is that limit in bits or bytes?

[sheoak]

@jdcauley I’m not sure I understood your question. This is in config/http.js, under middleware key. Not sure it’s in bytes or bits but right now I don’t really care, it just doesn’t work properly (getting an error in sails log, but the limit does work)

[johnferro]

@sheoak Not sure if you're still working on this, but I came across it and figured I'd respond because I was getting similar issues to you. You mentioned a timeout error/no response when using this code from balderdashy/skipper#22:

bodyParser: (function () {
  var opts = {limit:'50mb'};
  var fn;

  // Default to built-in bodyParser:
  fn = require('skipper');
  return fn(opts);
})

I think this problem might have come around because you were putting this under the middleware key of http when it should be directly under http (so it would be http.bodyParser instead of http.middleware.bodyParser). When putting it there I was able to get skipper to increase its limit without the timeout issue you mentioned, and with no error message being logged

[sheoak]

I am not working on it anymore and have no access to a computer at the moment. I remember trying that option but sails updates may have change something, or I did it wrong. I will notify the person in charge now!

[hybrisCole]

@johnferro yo forgot to add () at the end of bodyParser's IIFE ;), thx for the snippet amigo

[johnferro]

@hybrisCole Actually I get an error when lifting sails if I try to call it like that, but the extra parentheses I had wrapping the function are unnecessary.

if (req.method.toLowerCase() === 'get' || req.method.toLowerCase() === 'op
               ^
TypeError: Cannot call method 'toLowerCase' of undefined

[hybrisCole]

@johnferro Here's the snippet that's working for me in case you need it, I'm using 0.11.0 not sure if that changes somethig

bodyParser: (function() {
  var opts = {limit:'50mb'},
      fn;

  // Default to built-in bodyParser:
  fn = require('skipper');
  return fn(opts);
})(),

[golojads]

@hybrisCole can you please post your whole http.js file ?
I'm also on 0.11.0 but POST limit is not working.

[hybrisCole]

@golojads At the end I decided to come up with a nginx based solution. I'm reverse proxing my sails and handling that limit on nginx instead

[hybrisCole]

Lol @golojads I actually have it here, I confused the projects I guess

/**

• HTTP Server Settings
• (sails.config.http)
  *
• Configuration for the underlying HTTP server in Sails.
• Only applies to HTTP requests (not WebSockets)
  *
• For more information on configuration, check out:
• http://sailsjs.org/#/documentation/reference/sails.config/sails.config.http.html
  */
  'use strict';

var lusca = require('lusca');

module.exports.http = {

/****************************************************************************

•                                                                       *
  

• Express middleware to use for every Sails request. To add custom *

• middleware to the mix, add a function to the middleware config object and *

• add its key to the "order" array. The $custom key is reserved for *

• backwards-compatibility with Sails v0.9.x apps that use the *

• customMiddleware config option. *

•                                                                       *
  

  ****************************************************************************/

  middleware: {
  csp: lusca.csp({
  policy: {
  'default-src': ''self''
  }
  }),
  hsts: lusca.hsts({
  maxAge: 15768000,
  includeSubDomains: true
  }),
  xframe: lusca.xframe('DENY'),
  xss: lusca.xssProtection(),
  contentLenght: function(req, res, next) {
  var contentLength = req.headers['content-length'];
  if (contentLength) {
  if (parseInt(contentLength, 10) > 30000000) {
  return res.json(413, {err: 'Request Entity Too Large'});
  }
  }
  return next();

  },
  removeHeaders: function(req, res, next) {
  res.set('X-Powered-By', 'XXXX');
  return next();
  },
  tooBusy: function(req, res, next) {
  if (require('toobusy-js')()) {
  return res.json(503, {err: 'I'm busy right now, sorry.'});
  }else {
  next();
  }
  },
  /***************************************************************************

 *                                                                          *
 * The order in which middleware should be run for HTTP request. (the Sails *
 * router is invoked by the "router" middleware below.)                     *
 *                                                                          *
 ***************************************************************************/

order: [
  'tooBusy',
  'contentLenght',
  'csp',
  'hsts',
  'xframe',
  'xss',
  'removeHeaders',
  'startRequestTimer',
  'cookieParser',
  //'session',
  //'myRequestLogger',
  'bodyParser',
  'handleBodyParserError',
  'compress',
  'methodOverride',
  '$custom',
  'router',
  'www',
  'favicon',
  '404',
  '500'
],

/***************************************************************************
 *                                                                          *
 * The body parser that will handle incoming multipart HTTP requests. By    *
 * default as of v0.10, Sails uses                                          *
 * [skipper](http://github.com/balderdashy/skipper). See                    *
 * http://www.senchalabs.org/connect/multipart.html for other options.      *
 *                                                                          *
 ***************************************************************************/
bodyParser: (function() {
  var opts = {limit:'50mb'},
      fn;

  // Default to built-in bodyParser:
  fn = require('skipper');
  return fn(opts);
})(),
// bodyParser: require('skipper')

// },

/***************************************************************************
 *                                                                          *
 * The number of seconds to cache flat files on disk being served by        *
 * Express static middleware (by default, these files are in `.tmp/public`) *
 *                                                                          *
 * The HTTP static cache is only active in a 'production' environment,      *
 * since that's the only time Express will cache flat-files.                *
 *                                                                          *
 ***************************************************************************/

cache: 31557600000

}
};

[nickewansmith]

This still is not working properly... +1

[nickewansmith]

Update, this is what worked for my situation

/**
 * HTTP Server Settings
 * (sails.config.http)
 *
 * Configuration for the underlying HTTP server in Sails.
 * Only applies to HTTP requests (not WebSockets)
 *
 * For more information on configuration, check out:
 * http://sailsjs.org/#/documentation/reference/sails.config/sails.config.http.html
 */

module.exports.http = {

    /****************************************************************************
     *                                                                           *
     * Express middleware to use for every Sails request. To add custom          *
     * middleware to the mix, add a function to the middleware config object and *
     * add its key to the "order" array. The $custom key is reserved for         *
     * backwards-compatibility with Sails v0.9.x apps that use the               *
     * `customMiddleware` config option.                                         *
     *                                                                           *
     ****************************************************************************/

    bodyParser: function () {
        var opts = {limit:'50mb'};
        var fn;

        // Default to built-in bodyParser:
        fn = require('skipper');
        return fn(opts);
    },

    middleware: {

        /***************************************************************************
         *                                                                          *
         * The order in which middleware should be run for HTTP request. (the Sails *
         * router is invoked by the "router" middleware below.)                     *
         *                                                                          *
         ***************************************************************************/

        order: [
            'startRequestTimer',
            'cookieParser',
            'session',
            'myRequestLogger',
            'bodyParser',
            'handleBodyParserError',
            'compress',
            'methodOverride',
            'poweredBy',
            '$custom',
            'router',
            'www',
            'favicon',
            '404',
            '500'
        ],

        /****************************************************************************
         *                                                                           *
         * Example custom middleware; logs each request to the console.              *
         *                                                                           *
         ****************************************************************************/

        myRequestLogger: function (req, res, next) {
            sails.log("Requested :: ", req.method, req.url);
            return next();
        }






        /***************************************************************************
         *                                                                          *
         * The body parser that will handle incoming multipart HTTP requests. By    *
         * default as of v0.10, Sails uses                                          *
         * [skipper](http://github.com/balderdashy/skipper). See                    *
         * http://www.senchalabs.org/connect/multipart.html for other options.      *
         *                                                                          *
         ***************************************************************************/

         //bodyParser: require('skipper')


    }

    /***************************************************************************
     *                                                                          *
     * The number of seconds to cache flat files on disk being served by        *
     * Express static middleware (by default, these files are in `.tmp/public`) *
     *                                                                          *
     * The HTTP static cache is only active in a 'production' environment,      *
     * since that's the only time Express will cache flat-files.                *
     *                                                                          *
     ***************************************************************************/

    // cache: 31557600000
};