Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@middy/http-json-body-parser perf improvement #643

Merged
merged 12 commits into from
Apr 9, 2021
Merged

Conversation

willfarrell
Copy link
Member

@willfarrell willfarrell commented Apr 8, 2021

@nponeccop This should be much faster, let me know if this is what you had in mind.

Related to #641

@nponeccop
Copy link
Contributor

nponeccop commented Apr 8, 2021

Here is my unscientific bench:

  • @middy/http-json-body-parser: 30 32 34
  • this PR: 26 30 30
  • my body-parser.js: 15 19 16

Also, note that the content-type parser allows everything permitted by https://tools.ietf.org/html/rfc7231#section-3.1.1.1 and then something more. I don't think we can ever fail on an RFC compliant input. However, the degree to which we can handle non-compliant input is up to us. The tests of the content-type package seemingly allow more whitespace than the RFC, and simultaneously reject non-compliant headers if they use something disallowed after the ;.

Moreover, my parser doesn't allow arbitrary application/foo+json content types. But if a user wants them, he can list them individually. It's very rare that you really need to support all JSON-related content types instead of a finite subset of them. Neither it is clear whether it's the regexp there that slows it down. It may be the HTTP-errors package. So more research is needed.

Upd: content-parser isn't actually slow. body-parser.js using it shows 19 18 17 which is only marginally worse.

@willfarrell
Copy link
Member Author

Yeah, it's the regex. I think supporting application/foo+json has value for those that use predefined standards. Let me play around with a few other ideas I have.

@nponeccop
Copy link
Contributor

My opinion is that the predefined standards also predefine the types. So it's only one or a few fixed standard-compliant content-types.

BTW http-errors also introduce a delay. I conditionally required it, so they only affect the "bad" path and the "good" path remains fast:

nponeccop/serverless-openapi-templates@e820eb4#diff-dd3eee162448fc7700f528048fe3d0d2b7b4b850685cdd3c716a295bf0f3beef

@willfarrell
Copy link
Member Author

After a bunch of jspref testing, I think this is a fast as it's going to get for now.

I did try passing in a static value and checking for an exact match but found it performed the same. Once we have a proper benchmarking tool in place we can revisit.

@nponeccop
Copy link
Contributor

After a bunch of jsperf testing, I think this is a fast as it's going to get for now.

The current version is as fast as my best code. Congratulations!

I did try passing in a static value and checking for an exact match but found it performed the same.

I think it's because the RegExp is still compiled. Also I wonder if there's measurable difference between // and '':

const pattern = new RegExp('^application/(.+\\+)?json(;.*)?$')

BTW the PR has a typo: it's iMprovement.

@willfarrell willfarrell changed the title @middy/http-json-body-parser perf inprovement @middy/http-json-body-parser perf improvement Apr 9, 2021
@nponeccop
Copy link
Contributor

Formatting is wrong

@willfarrell
Copy link
Member Author

Good idea. The big gain was from only compiling the RegExp once.

const slowest = new RegExp( '^application\/(.+\\+)?json(;.*)?$')
const fast = new RegExp( /^application\/(.+\\+)?json(;.*)?$/)
const fastest = /^application\/(.+\\+)?json(;.*)?$/

@willfarrell willfarrell merged commit 096a547 into main Apr 9, 2021
@willfarrell willfarrell deleted the feature/body-parse branch April 9, 2021 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants