request entity too large #1856

BenjaminHorn · 2015-12-03T18:29:51Z

If I try to send a json to the server, which is about 220kb. (It contains a base64 coded image as a long string). I am getting the following error:

{"errors":[{"status":413,"source":"","title":"Error","code":"","detail":"request entity too large"}]}

server/confing.json

{
  "restApiRoot": "/api",
  "host": "0.0.0.0",
  "port": 3001,
  "remoting": {
    "context": {
      "enableHttpContext": false
    },
    "rest": {
      "normalizeHttpPath": false,
      "xml": false
    },
    "json": {
      "strict": false,
      "limit": "50mb"
    },
    "urlencoded": {
      "extended": true,
      "limit": "50mb"
    },
    "cors": false,
    "errorHandler": {
      "disableStackTrace": false
    }
  },
  "legacyExplorer": false
}

raymondfeng · 2015-12-03T19:55:29Z

Do you have a json parser set up in the middleware.json or server.js?

Thanks,

Raymond Feng (rfeng@us.ibm.com)
Co-Founder and Architect @ StrongLoop, Inc. (An IBM Company)

StrongLoop http://strongloop.com/ makes it easy to develop APIs http://strongloop.com/mobile-application-development/loopback/ in Node, plus get DevOps capabilities http://strongloop.com/node-js-performance/strongops/ like monitoring, debugging and clustering.

On Dec 3, 2015, at 10:29 AM, BenjaminHorn notifications@github.com wrote:

If I try to send a json to the server, which is about 220kb. (It contains a base64 coded image as a long string). I am getting the following error:

{"errors":[{"status":413,"source":"","title":"Error","code":"","detail":"request entity too large"}]}

server/confing.json

{
"restApiRoot": "/api",
"host": "0.0.0.0",
"port": 3001,
"remoting": {
"context": {
"enableHttpContext": false
},
"rest": {
"normalizeHttpPath": false,
"xml": false
},
"json": {
"strict": false,
"limit": "50mb"
},
"urlencoded": {
"extended": true,
"limit": "50mb"
},
"cors": false,
"errorHandler": {
"disableStackTrace": false
}
},
"legacyExplorer": false
}
—
Reply to this email directly or view it on GitHub #1856.

BenjaminHorn · 2015-12-03T20:34:19Z

I did not alter any of those files. They look like this:

server.js

var loopback = require('loopback');
var boot = require('loopback-boot');

var app = module.exports = loopback();

app.start = function() {
  // start the web server
  return app.listen(function() {
    app.emit('started');
    var baseUrl = app.get('url').replace(/\/$/, '');
    console.log('Web server listening at: %s', baseUrl);
    if (app.get('loopback-component-explorer')) {
      var explorerPath = app.get('loopback-component-explorer').mountPath;
      console.log('Browse your REST API at %s%s', baseUrl, explorerPath);
    }
  });
};

// Bootstrap the application, configure models, datasources and middleware.
// Sub-apps like REST API are mounted via boot scripts.
boot(app, __dirname, function(err) {
  if (err) throw err;

  // start the server if `$ node server.js`
  if (require.main === module)
    app.start();
});

middleware.json

{
  "initial:before": {
    "loopback#favicon": {}
  },
  "initial": {
    "compression": {},
    "cors": {
      "params": {
        "origin": true,
        "credentials": true,
        "maxAge": 86400
      }
    }
  },
  "session": {},
  "auth": {},
  "parse": {},
  "routes": {
    "loopback#rest": {
      "paths": [
        "${restApiRoot}"
      ]
    }
  },
  "files": {},
  "final": {
    "loopback#urlNotFound": {}
  },
  "final:after": {
    "loopback#errorHandler": {}
  }
}

BenjaminHorn · 2015-12-07T21:07:47Z

@raymondfeng is there any news on this?

richardpringle · 2015-12-08T18:14:26Z

Hey @BenjaminHorn , do you have an sample repo from which I can reproduce the issue?
You can follow this guide which basically says to fork the loopback-sandbox and reproduce your issue in the fork, but any sample repo will do.

If I am not mistaken, you should be able to use a Node.js buffer object to transfer binary data instead of a string. I don't know your particular use case though.

BenjaminHorn · 2016-01-07T15:06:43Z

@richardpringle @raymondfeng After some investigation I found that the problem is the content type.
If I try to Post with

Content-Type:application/x-www-form-urlencoded; charset=UTF-8

then it works. My client (ember) app sends

contentType: 'application/vnd.api+json'

then the new limit has no effect.

Setting the type wont fix it. I know that is not in the strongloop documentation, only in https://www.npmjs.com/package/body-parser#type-2
server/config.json

...
  "remoting": {
    "context": {
      "enableHttpContext": false
    },
    "rest": {
      "normalizeHttpPath": false,
      "xml": false
    },
    "json": {
      "strict": false,
      "limit": "50mb"
      "type": "application/vnd.api+json"
    },
...

digitalsadhu · 2016-01-08T10:19:00Z

I believe this is an issue with loopback-component-jsonapi. Needs a quick fix.
https://github.com/digitalsadhu/loopback-component-jsonapi/blob/master/lib/headers.js#L33
Needs to not overwrite but rather merge with options defined elsewhere (link in config.json)
Does that sound like its the issue @BenjaminHorn ?

BenjaminHorn · 2016-01-08T18:45:20Z

thanks @digitalsadhu that was the issue indeed. I made a PR.

tborja · 2016-11-15T10:51:39Z

What's the fix on this? I am running loopback on Bluemix. I am getting the Request too large error using swagger js client.

roboflank · 2016-11-16T16:43:13Z

Also running into the same error

tamitutor · 2016-11-17T12:29:39Z

+1
As am I

raymondfeng · 2016-11-17T16:29:22Z

Have you adjusted the limit per https://loopback.io/doc/en/lb2/config.json.html?

tborja · 2016-11-18T02:31:08Z

Hello,

This is my config.json

{
"restApiRoot": "/api",
"host": "0.0.0.0",
"port": 3000,
"remoting": {
"context": false,
"rest": {
"normalizeHttpPath": false,
"xml": false
},
"json": {
"strict": false,
"limit": "80mb"
},
"urlencoded": {
"extended": true,
"limit": "50mb"
},
"cors": false,
"handleErrors": false
},
"legacyExplorer": false
}

tborja · 2016-11-18T03:14:57Z

I checked where the error message Request Entity Too Large occurs and it is in here:
from /node-modules/raw-body/index.js

function readStream (stream, encoding, length, limit, callback) {
var complete = false
var sync = true

// check the length and limit options.
// note: we intentionally leave the stream paused,
// so users should handle the stream themselves.
if (limit !== null && length !== null && length > limit) {
return done(createError(413, 'request entity too large', 'entity.too.large', {
expected: length,
length: length,
limit: limit
}))
}

Does anybody know which configuration file this picks up the options for limit and length from?

serpentinefire · 2017-01-17T17:01:14Z

I found that the limit value is picked up from server.js

tborja · 2017-04-07T05:20:50Z

Serpentinefire,

Can you supply the code on server.js to fix this issue?

vinimoret · 2017-04-10T19:38:45Z

If you're using bodyParser,
Try put that where you configure your express in the application.
app.use(bodyParser({ limit: '10mb' }));

tborja · 2017-04-11T06:25:56Z

I already have this on my server.js and I still get the error.

`var bodyParser = require('body-parser');

app.use( bodyParser.json({limit: '50mb'}) );
app.use(bodyParser.urlencoded({
limit: '50mb',
extended: true,
parameterLimit:50000
}));`

ketansp · 2017-04-11T07:13:53Z

I got it working by modifying values in middleware.json file. You can set the actual request limit there. Apparently loopback picks it up from middleware.json, not config.json.

{ ..., "parse": { "body-parser#json": { "params" : { "limit" : "50mb"}}, "body-parser#urlencoded": {"params": { "limit" : "50mb", "extended": true }} }, ... }

vinimoret · 2017-04-11T12:42:18Z

@tborja
Inside the folder node_modules you can find this folder.
try change this value limit (i.e limit: '50mb') for the type of data that you send, and see if it works for you.
If don't work the problem is not with body parser.
If work your body-parse configuration is in the wrong place

tborja · 2017-04-12T03:03:36Z

Hello mnvini,

I did as you suggested and changed the limit parameter for the types on body-parser, but I am still getting the same entity too large error.

My loopback database is Cloudand on Bluemix and I am using this connector:
"loopback-connector-cloudant": "^1.2.2"

It seems that it is the cloudant server that is sending out the entity too large error.

"couch returned 413"

I am sending to Cloudant webGL as part of the json msg since the loopback connector does not support cloudant attachments. I guess I will have to find out where to increase the entity limit in cloudant.

Thanks for all your help guys.

tborja · 2017-04-17T04:02:34Z

It seems that there is a 1MB limit in the Cloudant request.

cloudant/java-cloudant#345

It is just weird that this limit is there without a way to change it.

nirmal25990 · 2017-06-02T10:52:29Z

"parse": { "body-parser#json": { "params": { "strict": false, "limit": "50mb" } }, "body-parser#urlencoded":{ "params": { "extended": true, "limit": "50mb" } } }

Changes in config.json doesn't working for me. But making changes in middleware.json resolve my issue.

tborja · 2017-06-02T19:18:55Z

Thanks nirmal25990, but IBM confirmed that middleware.json fixes won't fix the issue because it is Cloudant that has a 1MB limit on it's entities. They suggest either using an attachment in Cloudant or use an ObjectStore.

javahaxxor · 2017-10-27T07:44:43Z

Thanks ketansp, your solution was enough for me

through-a-haze · 2018-10-16T16:02:33Z

I got it working by modifying values in middleware.json file. You can set the actual request limit there. Apparently loopback picks it up from middleware.json, not config.json.

{ ..., "parse": { "body-parser#json": { "params" : { "limit" : "50mb"}}, "body-parser#urlencoded": {"params": { "limit" : "50mb", "extended": true }} }, ... }

Are there any caveats for increasing this limit?

youngtk96 · 2019-07-02T14:24:59Z

If you are hosting the server on NGINX try the following

Modify NGINX Configuration File

sudo nano /etc/nginx/nginx.conf
Search for this variable: client_max_body_size. If you find it, just increase its size to 100M, for example. If it doesn’t exist, then you can add it inside and at the end of http

client_max_body_size 100M;
Restart nginx to apply the changes.

sudo service nginx restart

natuan62 · 2020-02-14T09:13:55Z

@youngtk96 its works. tks so much

anirudhr95 · 2020-06-11T09:40:24Z

@nirmal25990 your fix worked for me too :)

vivek-kandhvar · 2020-07-15T04:49:12Z

"parse": { "body-parser#json": { "params": { "strict": false, "limit": "50mb" } }, "body-parser#urlencoded":{ "params": { "extended": true, "limit": "50mb" } } }

Changes in config.json doesn't working for me. But making changes in middleware.json resolve my issue.

Unless you override parse param in middleware.json, by using a body-parser, setting it in config.js will work. It does work for us.

richardpringle added the triaging label Dec 8, 2015

richardpringle self-assigned this Dec 8, 2015

richardpringle added the blocked label Dec 8, 2015

BenjaminHorn mentioned this issue Jan 8, 2016

in server/config.js remote.json settings gets overwritten digitalsadhu/loopback-component-jsonapi#92

Closed

BenjaminHorn closed this as completed Jan 8, 2016

tborja unassigned richardpringle Nov 15, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

request entity too large #1856

request entity too large #1856

BenjaminHorn commented Dec 3, 2015

raymondfeng commented Dec 3, 2015

BenjaminHorn commented Dec 3, 2015

BenjaminHorn commented Dec 7, 2015

richardpringle commented Dec 8, 2015

BenjaminHorn commented Jan 7, 2016

digitalsadhu commented Jan 8, 2016

BenjaminHorn commented Jan 8, 2016

tborja commented Nov 15, 2016

roboflank commented Nov 16, 2016

tamitutor commented Nov 17, 2016 •

edited

raymondfeng commented Nov 17, 2016

tborja commented Nov 18, 2016

tborja commented Nov 18, 2016

serpentinefire commented Jan 17, 2017

tborja commented Apr 7, 2017

vinimoret commented Apr 10, 2017

tborja commented Apr 11, 2017

ketansp commented Apr 11, 2017

vinimoret commented Apr 11, 2017 •

edited

tborja commented Apr 12, 2017

tborja commented Apr 17, 2017

nirmal25990 commented Jun 2, 2017 •

edited

tborja commented Jun 2, 2017

javahaxxor commented Oct 27, 2017

through-a-haze commented Oct 16, 2018

youngtk96 commented Jul 2, 2019

natuan62 commented Feb 14, 2020

anirudhr95 commented Jun 11, 2020

vivek-kandhvar commented Jul 15, 2020

request entity too large #1856

request entity too large #1856

Comments

BenjaminHorn commented Dec 3, 2015

raymondfeng commented Dec 3, 2015

BenjaminHorn commented Dec 3, 2015

BenjaminHorn commented Dec 7, 2015

richardpringle commented Dec 8, 2015

BenjaminHorn commented Jan 7, 2016

digitalsadhu commented Jan 8, 2016

BenjaminHorn commented Jan 8, 2016

tborja commented Nov 15, 2016

roboflank commented Nov 16, 2016

tamitutor commented Nov 17, 2016 • edited

raymondfeng commented Nov 17, 2016

tborja commented Nov 18, 2016

tborja commented Nov 18, 2016

serpentinefire commented Jan 17, 2017

tborja commented Apr 7, 2017

vinimoret commented Apr 10, 2017

tborja commented Apr 11, 2017

ketansp commented Apr 11, 2017

vinimoret commented Apr 11, 2017 • edited

tborja commented Apr 12, 2017

tborja commented Apr 17, 2017

nirmal25990 commented Jun 2, 2017 • edited

tborja commented Jun 2, 2017

javahaxxor commented Oct 27, 2017

through-a-haze commented Oct 16, 2018

youngtk96 commented Jul 2, 2019

natuan62 commented Feb 14, 2020

anirudhr95 commented Jun 11, 2020

vivek-kandhvar commented Jul 15, 2020

tamitutor commented Nov 17, 2016 •

edited

vinimoret commented Apr 11, 2017 •

edited

nirmal25990 commented Jun 2, 2017 •

edited