Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error that gets handled by middleware is still passed on to next moddleware #485

Closed
jacsamell opened this issue Feb 24, 2020 · 10 comments
Closed

Comments

@jacsamell
Copy link

When you have a chain of middlewares that implement onError, when the first middleware handles the error and calls next() without passing in the error as a parameter the next one should not be invoked according to the documentation.

However all onErrors in the chain are called regardless of whether the error has been passed into the next function.

@philprime
Copy link
Contributor

philprime commented Mar 23, 2020

I noticed the same issue when handling errors and setting them as a response:

export function RestErrorMapperMiddleware<T, R>(): middy.MiddlewareObject<T, R> {
  return ({
    onError: (handler: middy.HandlerLambda): Promise<void> => {
      const error = handler.error;
      let statusCode = INTERNAL_SERVER_ERROR;
      let errorMessage = error.message;

      if (error instanceof BadCredentialsException) {
        statusCode = UNAUTHORIZED;
        errorMessage = "Bad credentials."
      } 
      handler.response = new LambdaResult(statusCode, {
        statusCode: statusCode,
        body: JSON.stringify({
          message: errorMessage,
        })
      });
      return Promise.resolve();
    }
  });
}

All middlewares used afterwards are still called.

@lmammino
Copy link
Member

lmammino commented Mar 25, 2020

Hello @jacsamell and @philprime, thanks a lot for reporting this.

This seems indeed a bug on how the error propagation is handled when using promises (or async-await).

Unfortunately, I currently have very little time to investigate this, but if you have some time to invest on this, this part of the code might be a good starting point.

Sorry if I can't be more helpful at this time.

@philprime
Copy link
Contributor

Hi @lmammino, I just took a look at the code and if I change line 113 to the following, it does not execute the remaining error middlewares anymore:

if (nextMiddleware) {

if (nextMiddleware && !instance.__handledError) {

Now I expected it to call the remaining onAfter middlewares outside of the one that handled the error, but I couldn't find anything to that topic.

If I extend the sample code of your documentation by another onAfter handler in middleware2, should that be called?

middleware1 = {
  onError: (handler) => {
    handler.response = { error: handler.error };
    return Promise.resolve();
    // Resolves to a falsy value
  }
}
middleware2 = {
  onError: (handler) => {
    return Promise.resolve(handler.error)
  },
  onAfter: (handler) => {
    // Should this be called?
  }
}
handler.use(middleware1).use(middleware2);

@lmammino
Copy link
Member

Thanks a lot for spending some time on this issue @philprime! It might be worth to revisit the test suite and maybe submit a PR if you think that change is enough to stop the bug and does not have side effects on other behaviours.

Regarding your question, middleware2's onAfter should not be invoked by design. The idea is that once you are on the onError track you derailed outside the normal request/response flow and only onError handlers will be invoked (in order) giving them a chance to handle the error as needed.

If you want to have an additional step of error handling you should implement a onError also in middleware2 and keep propagating the error from middleware1 by doing return Promise.reject(handler.error).

@philprime
Copy link
Contributor

I just created a pull request, only one test is failing:

If theres an error and one error middleware handles the error, the next error middlewares is executed

Is this test case actually doing what is intended?
Either I misunderstand its objective or it is contradicting to the design/documentation

@philprime
Copy link
Contributor

I changed the test case as mentioned before and added correct handling for callbacks. See PR #497

@lmammino
Copy link
Member

Thanks a lot @philprime and sorry again for the delay in reviewing this! I should be able to find some time during this Easter break.

@lmammino lmammino reopened this Apr 11, 2020
@lmammino
Copy link
Member

lmammino commented Apr 14, 2020

Sorry for all the confusion here. I haven't touched this in a while and I did not fully remember what was the intended flow and why.

Today I finally had some time to deep dive on this and get some more clarity (after messing up a release on master 😢 )

As mentioned in #497

As per the docs:

When a middleware handles the error and creates a response, the execution is still propagated to all the other error middlewares and they have a chance to update or replace the response as needed. At the end of the error middlewares sequence, the response is returned to the user.

This is needed for instance in the case of the HTTP error handler which will format the error in a correct response for the lambda HTTP integration. If this is combined with the response serializer it must be possible to convert this response to XML or other formats.

There are two ways in your custom middleware that you can decide not to perform any extra action in your error middleware if the error has been already handled:

  1. Check the handler.__handledError property. If true another middleware has handled the error and produced a response. so you can wrap your middleware in an if statement and skip any additional action. Similarly, you can check if there's already a response in handler.response

Example:

middleware1 = {
  onError: (handler) => {
    handler.response = { error: handler.error };
    return Promise.resolve();
    // Resolves to a falsy value and will set instance.__handledError = true
  }
}

middleware2 = {
  onError: (handler) => {
    // will be called but you can check handler.response or handler.__handledError to 
    // decided whether to perform some additional action or not
  },
  onAfter: (handler) => {
    // Will NOT be called
  }
}

handler.use(middleware1).use(middleware2);

Now, it is true that the documentation section is not accurate (it is indeed wrong):

If onError promise resolves to a falsy value (null, undefined, false etc.), the error handling pipeline exits early and the response is returned without an error.

This needs to be fixed as for consistency with the callback version the code won't exit early.

I will make sure to update the documentation to:

If onError promise resolves to a falsy value (null, undefined, false etc.), the error handling pipeline continues and eventually the response is returned without an error.

This part is instead correct:

If onError promise rejects, the error handling pipeline exits early and the lambda execution fails.

So, this is a valid way of short-circuiting the error handling If needed when using promises or Async/Await.

Sorry again for all the mess and confusion... One of those days where I don't feel great about having enough time and focus to keep leading this project as good as needed 😞

@lmammino
Copy link
Member

Closed by:

lmammino added a commit that referenced this issue Apr 25, 2020
* Updated codecov and jest to remove security warning on GitHub

* Updated circleci to work with lerna

* Added README files

* Updated CI and readmes

* Made lerna publish non interactive

* Improved lerna publishing

* make packages public

* Properly publishing scoped packages as public

* Updated version badges

* Do not publish packages as next

* Ported changes on ssm middleware from 0.x by @tommy5dollar

* Http header normaliser to lowercase (#185)

* change middleware so the request headers can be lowercase or normalised

* Normalize headers to lowercase rather than camelCase

* Fix eslint (removed extra semicolon)

* Change option for http-header-normalizer from camelCase to canonical

* version bump

* Renamed cors package to http-cors (#181)

* Backport changes to ssm in 0.x by @theburningmonk (#191)

* updated AWS SDK and running tests for both node6 and node8

* Multi Origin Cors (#195)

* feat: multi origin cors

* fix: accounting for header case

* updated AWS SDK and running tests for both node6 and node8

* fix: made origins optional in typescript definitions

* Merged with current master and version bump

* Adds GitHub PR and Issue templates (#206)

* Adds GitHub PR and Issue templates

* Added todo list and version bumped

* Add input output logger middleware (#205)

* Add input output logger middleware

* JSON stringify logs and wrap into an object

* Update unit tests

* version bump

* Ported changes for making sure that handlers run only once. Closes #175

* Added error-logger middleware

* Fixed error logger and unstable tests in core

* backported SSM caching fix from 0.16.2 (#225)

* backported SSM caching fix from 0.16.2

* Ported http-security-headers middlewares by @willfarrell to 1.x

* add secrets-manager middleware (#228)

backported SSM caching fix from 0.16.2

* update secrets manager with throwOnFailedCall flag (#236)

* Initial work for 1.0.0 with secrets-manager

* bump typings

* bump package-locks

* bump version to 17

* fix package locks

* refactor: folder name to package name for http-security-header (#240)

* refactor: folder name to package name for http-security-header

* version bump

* build: add a pre commit hook to run lint checks on all changed files (#242)

* build: add a pre commit hook to run lint checks on all changed files

* version bump

* added FunctionShield middleware

* Fix json body parse example

(cherry picked from commit c03bdb0)

* chore: bump version

- Fix json body parse example #254
- Add functionShield middleware link to the main readme #248

* FunctionShield updated to 1.2.2

* Updates S3 event version to allow event versions of 2.x (thanks @kenleytomlin)

* Integrated LGTM badge

* Added default export to ssm middleware

* Added default export to secrets-manager middleware

* Allow use() to receive multiple middlewares (#288)

* Allow use() to receive multiple middlewares

* Updated typings and added a CHANGELOG

* Version bump

* Made middlewares optional in typings

* Updated dependencies

* Add community middleware to README (#299)

* Add community middleware to README

* Version bump and merge with current 1.0.0-alpha

* Added http-response-serializer middleware (#304)

* Added http-response-serializer middleware

* Updated deps

* Version bump

* add: required properties list (#307)

* add: required properties list

* version bump

* return promise from instance if no callback is present (#314)

* return callback from instance if no callback is present
* Version bump

* port cors error swallow fix (#313)

* port cors error swallow fix

* added cors error swallow test

* cors doc sample fix import

* version bump

* Make ICorsOptions fields optional (#319)

* Make ICorsOptions fields optional

* Port #326  (#328)

* added test for multiple ssm names with same path

* stop inverting userParams object; allow multiple SSM names with same path

* remove Object.entries for node 6 support

* only fetch duplicated SSM params once

* version bump

* Support reviver parameter in for http-json-body-parser (#331)

* add reviver support for http-json-body-parser
* version bump
* Updated dependencies

* fix: content type request bug in http serializer (#335)

* fix: tests not waiting for callbacks

* fix: content-type skip using request headers not response headers

* version bump and handling Cannot read property headers of undefined

* fix: httpErrorHandler minification type checking (#341)

* fix: httpErrorHandler minification type checking
* version bump

* Port waitForEmptyEventLoop in Warmup middlware (#345) to v.1.0.0 branch (#354)

* Port waitForEmptyEventLoop in Warmup middlware (#345) to v.1.0.0 branch

* version bump

* Make greenkeeper ignore updates of aws-sdk to keep parity w/ actual Lambda runtime (#369)

* feat: make greenkeeper ignore updates of aws-sdk to keep parity w/ actual Lambda runtime

* version bump

* Port documentation changes from #371 (#376)

* http-cors: expose getOrigin in options (#385)

* feat: exposes http-cors getOrigin to options

* Version bump

* test: make tests succeed via wallaby

* feat: normalize multiValueQueryStringParameters property

* docs: update readme by multiValueQueryStringParameters property support

* chore: increment version in package-lock.json

* fix: propagate error when "runOnError" is true in doNotWaitForEmptyEventLoop

closes #318

* chore: bump versions

* chore: run unit tests on node 10 as well

* chore: don't install lerna globally in ci

* chore: bump versions

* chore: bump versions [ci skip]

* chore(commit-hook): fix eslint errors and stage files [ci skip]

* chore: bump versions

* feat: version bump

* chore: update all dev deps to their latest versions

* chore: bump versions

* chore: run lint as own job on node 10 and tests on different node versions

* chore: run test:packages correctly

* chore: don't repeat typings-check in each node version

* Fix relative imports (#383)

* Add @middy/core to devDependencies instead of using relative import

* version bump

* Rewrite tests to use async/await (fix #270) (#391)

* test: add expect.hasAssertions before each test

* test: add invoke test helper which promisifies handler

* test: rewrite "packages/cache" tests to async/await

fixes #270

* test: rewrite "packages/do-not-wait-for-empty-event-loop" tests to async/await

fixes #270

* test: rewrite "packages/error-logger" tests to async/await

fixes #270

* test: rewrite "packages/function-shield" tests to async/await

fixes #270

* test: rewrite "packages/http-content-negotiation" tests to async/await

fixes #270

* test: rewrite "packages/http-cors" tests to async/await

fixes #270, fixes 2 tests which were incorrect due callbacks

* test: rewrite "packages/http-error-handler" tests to async/await

fixes #270

* test: rewrite "packages/http-event-normalizer" tests to async/await

fixes #270

* test: rewrite "packages/http-header-normalizer" tests to async/await

fixes #270

* test: rewrite "packages/http-json-body-parser" tests to async/await

fixes #270

* test: rewrite "packages/http-partial-response" tests to async/await

fixes #270

* test: rewrite "packages/http-response-serializer" tests to async/await

fixes #270

* test: rewrite "packages/http-security-header" tests to async/await

fixes #270

* test: rewrite "packages/http-urlencode-body-parser" tests to async/await

fixes #270

* test: rewrite "packages/input-output-logger" tests to async/await

fixes #270

* test: rewrite "packages/s3-key-normalizer" tests to async/await

fixes #270

* test: rewrite "packages/secrets-manager" tests to async/await

fixes #270

* test: rewrite "packages/ssm" tests to async/await

fixes #270

* test: rewrite "packages/validator" tests to async/await

fixes #270

* test: rewrite "packages/warmup" tests to async/await

fixes #270

* Changes to fix tests

* Using es6-promisify for Node6

* Properly added es6-promise as dev dependency to all packages

* importing es6-promisify correctly

* One moar attempt :(

* Version bump

* Fix security with npm audit

* eslint and typescript improvements

* move @types/aws-lambda to a peerDependency (#412)

* move @types/aws-lambda to a peerDependency

* version bump

* [typings] Make middy.use accept array of middlewares as well (#413)

* feat(typings): make middy.use accept array of middlewares

* refactor: rename "input" to "middlewares"

* chore: version bump

* Typo & grammar fixes in README (#416)

* Typo & grammar fixes in README

* version bump

* Context specified in middy constructor should be passed forward to the Middy interface (#427)

* Context specified in middy constructor should be passed forward to the Middy interface

FIx for 1.0.0-alpha branch

* version bump

* dbManager middleware @ monorepo (#407)

* dbManager middleware @ monorepo

* just some whitespaces

* downgrade knex for node@6 support

* remove typescript from db manager

* remove redundant ignores

* version bump

* Renamed dbManager package to db-manager (#430)

* Fixed http-security-header name discrepancy. Fixes #372 (#431)

* Fixed http-security-header name discrepancy. Fixes #372

* Version bump

* Fix IURLEncodeBodyParserOptions definition (#439)

* fix: syntax error, false is not a type

* Merge with current 1.0.0-alpha and version bump

* urlEncodePathParser (#438)

* feat: add in decoder for path parameters

* docs: update main readme to include new middleware

* docs: fix merge conflict

* Version bump and updated typescript definitions

* fix: requested changes

#438

* ci: deprecate old runtimes (#441)

* ci: deprecate old runtimes

* version bump

* fixed build test deploy pipeline definition

* [warmup] add in info about provisioned concurrency (#447)

* docs: add in info about provisioned concurrency

* version bump

* Dependency Performance Improvements (#442)

* fix: move dep include out of middleware functions

* fix: only include relevent clients

* version bump

* HttpSecurityHeader Improvements (#443)

* feat: add in missing features

- Add in X-Permitted-Cross-Domain-Policies
- Add in catch for thrown errors

Closes #426 #434

* version bump

* Port cache middleware chages from #359 to 1.0.0-alpha (#444)

* fix: add in missing catch

Closes #377

* version bump

* version bump

* Port httpMultipartBodyParser to v1.0.x (#445)

* feat: add in httpMultipartBodyParser

Closes #321

* Version bump and other small fixes

* readded package-lock

* restoring package-lock in every package

* feat: add in RDS Auth Token option

Closes #428

* fix: mock out RDS sdk

* add custom context to 1.0.0 typings (#454)

* fix: add custom context to 1.0.0

* version bump

Co-authored-by: Luciano Mammino <lucianomammino@gmail.com>

* Update dependencies (#456)

* fix: remove async due to causing multiple `next()` calls

* fix: bring back async without next()

* fix: remove unused next

* Added update instructions (#457)

* Added update instructions

* Mandatory emoji

* made link to UPDATE relative

* Fixes after review

* 1.x in upgrade guide

* version bump

* version 1.0.0-beta

* Fix syntax highlighting in Readme

* FucntionShield: add warning and bump dep version (#464)

* fix: add warning and bump dep version

Addresses #460

* Merged current 1.0.0-beta in

* version bump

Co-authored-by: Luciano Mammino <lucianomammino@gmail.com>

* Add in RDS Auth Token option to dbManager (#462)

* feat: add in RDS Auth Token option

Closes #428

* fix: mock out RDS sdk

* fix: remove async due to causing multiple `next()` calls

* fix: bring back async without next()

* fix: remove unused next

* version bump

Co-authored-by: Luciano Mammino <lucianomammino@gmail.com>

* fix: secretsManager regression

* Put required next to validator body (#469)

* Put required next to validator body

* Version bump

Co-authored-by: Luciano Mammino <lucianomammino@gmail.com>

* fix: variable clean up

* fix: update typescript interface

* version bump

* ci: fix how RDS service is mocked out

* feat: add sqs-batch middleware (#471)

* feat: sqs-partial-batch-failure

* Add maxAge and cacheControl options to @middy/http-cors (#473)

* Add maxAge and cacheControl options to @middy/http-cors

* version bump

Co-authored-by: Luciano Mammino <lucianomammino@gmail.com>

* Fix: Http-response-serializer returns {} when body is falsey (#477)

* fix: http response serializer returns {} when body is falsey

* Add FAQ item about context.done called twice warning (#481)

Co-authored-by: Pedro Resch <pedro@agenciaatom.com.br> [ci skip]

* Update index.d.ts (#487)

* Update index.d.ts

* version bump

Co-authored-by: Luciano <lucianomammino@gmail.com>

* [feature] sqs json body parser (#491)

Added sqs-json-body-parser middleware. Thanks @thematrimix!

* Move to GitHub actions and improve release process (#504)

* Moves CI to GitHub Actions and improves release process

* Version bump and updated dependencies

* Updated release workflow

* Release triggers on release publish

* release workflow named properly

* must install dependencies to be able to release -.-

* Add middy-mongoose-connector 3rd-party middleware (#496)

* update `accept` to `@hapi/accept` (#503)

* version bump and updated changelog

* Clarifies the behaviour of error handling when using promises as requested in #485

* Updated test name

* Run tests also on PRs

* Ability to omit fields from logged events and responses in input-output-logger (#507)

* adds the ability to omit fields from logged events and responses

* Version 1.0.0-beta.11

* Fix import middleware name (#509)

Co-authored-by: Miki <1433184+miki79@users.noreply.github.com>
Co-authored-by: Adam Mills <adam@chambills.com>
Co-authored-by: Kevin Rambaud <kevin.rambaud@gmail.com>
Co-authored-by: Yan Cui <theburningmonk@gmail.com>
Co-authored-by: Sebastian Domagała <sdomagala@users.noreply.github.com>
Co-authored-by: Christian Murphy <christian.murphy.42@gmail.com>
Co-authored-by: Yuri Shapira <yuri@puresec.io>
Co-authored-by: Rafael Renan Pacheco <rafael.renan.pacheco@gmail.com>
Co-authored-by: Vlad Holubiev <golubev.vld@gmail.com>
Co-authored-by: Luciano Mammino <lmammino@vectra.ai>
Co-authored-by: Daniel Bartholomae <daniel@bartholomae.name>
Co-authored-by: Alex DeBrie <alexdebrie1@gmail.com>
Co-authored-by: Alex Joyce <im@alex-joyce.com>
Co-authored-by: Natan Deitch <natan.luz.deitch@gmail.com>
Co-authored-by: Andrej Novikov <jazzaiman@gmail.com>
Co-authored-by: Gurarpit Singh <gsingh.ldn@gmail.com>
Co-authored-by: Markus Olsson <j.markus.olsson@gmail.com>
Co-authored-by: Gary Holland <garyhollandxyz@users.noreply.github.com>
Co-authored-by: Roni Frantchi <roni-frantchi@users.noreply.github.com>
Co-authored-by: Vlad Holubiev <vladgolubev@users.noreply.github.com>
Co-authored-by: Vitalii Sikora <vitalii.sikora@gmail.com>
Co-authored-by: Chris Shepherd <chris@chrisshepherd.me>
Co-authored-by: Adam Duro <adam@duromedia.com>
Co-authored-by: Cathy Casey <16669450+CathyC93@users.noreply.github.com>
Co-authored-by: Jakub Chrzanowski <jakub@chrzanowski.info>
Co-authored-by: will Farrell <willfarrell@users.noreply.github.com>
Co-authored-by: will Farrell <will.farrell@gmail.com>
Co-authored-by: Paweł Kowalski <pavelloz@Gmail.com>
Co-authored-by: Ali Clark <ali@clark.gb.net>
Co-authored-by: Brett Andrews <brett.j.andrews@gmail.com>
Co-authored-by: Marcelo Luiz Onhate <onhate@users.noreply.github.com>
Co-authored-by: rschpdr <34459903+rschpdr@users.noreply.github.com>
Co-authored-by: Mohamed Gamal <Mohamed.abugalala@gmail.com>
Co-authored-by: Joshua Vaughn <thematrimix@yahoo.com>
Co-authored-by: Bailey Tincher <btincher99@gmail.com>
Co-authored-by: Jarrod Davis <developer@jarrodldavis.com>
Co-authored-by: J.A.C.C <abdala+github@e.email>
benjifs pushed a commit to benjifs/middy that referenced this issue May 21, 2020
@thejuan
Copy link
Contributor

thejuan commented Jun 1, 2020

I think the docs are still incorrect on this?

Here, only middleware1.onError will be called. The rest of the error handlers will be skipped, and the lambda will finish normally and return the response. middleware2.onError will not be called.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

4 participants