Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[static-website] fix: stop silent fail on s3 deleteObjects #115

Merged
merged 1 commit into from
Oct 19, 2021

Conversation

fargito
Copy link
Contributor

@fargito fargito commented Oct 8, 2021

Hello and thanks a lot for this awesome project!

When using the static-website construct on my project, I realized that I had forgotten the s3:deleteObject IAM permission of my deployment policy. Therefore, the s3 files synced with lift were never deleted. However, no error was thrown and the deployement failed silently.

This silent fail is due to a particularity in the S3 deleteObjects SDK and the corresponding HTTP API. Since this API can encounter partial failures, the call never fails but returns the errors in a Errors key.

I think that Lift should explicitely fail when the Errors key is defined and the list of errors is not empty. In this case, the proposed error is the following:

Deleting index.html
{
  Key: 'index.html',
  Code: 'AccessDenied',
  Message: 'Access Denied'
}
 
 Serverless Error ----------------------------------------
 
  Unable to delete files
 
  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com
 
  Your Environment Information ---------------------------
     Operating System:          linux
     Node Version:              14.16.1
     Framework Version:         2.61.0 (local)
     Plugin Version:            5.4.5
     SDK Version:               4.3.0
     Components Version:        3.17.1
 

Copy link
Contributor

@t-richard t-richard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing I'm worried about is: what are the other cases a delete might fail ?

The current state is not ideal but is it better than a randomly failing deploy ? Would a warning be enough (probably not) ?

Would love to hear about what others think.

src/utils/s3-sync.ts Outdated Show resolved Hide resolved
@t-richard
Copy link
Contributor

Also kinda linked to #111 because this originated from misconfigured deploy credentials

@fargito
Copy link
Contributor Author

fargito commented Oct 11, 2021

Hello @t-richard and thanks for the review.

The s3Sync function is used in the serverSideWebsite and staticWebsite constructs, so this change would only affect them.

Maybe a warning could be enough, but I would argue that the deployment failure would only happen once (most probably at the second deploy). The alternative with the silent fail is that the list of files to delete grows at each deploy and the deployment fails at one point because the XML body sent to the s3 api is too large.

Concerning the explicit error, I logged the error returned by the S3 api (as in the description of the PR).

I saw that the tests were failing, I will try and fix them :)

@t-richard
Copy link
Contributor

@fargito the test failure is unrelated to your change 🙂
See #116

@fargito
Copy link
Contributor Author

fargito commented Oct 11, 2021

Ah ok ! Nonetheless I think my change broke the s3 sync test, so I fixed it

@t-richard
Copy link
Contributor

I also think failing is somewhat better than a warning because it could lead to unnecessary objects being stored and billed which could become a significant cost over time.

Concerning the explicit error, I logged the error returned by the S3 api (as in the description of the PR).

I know you logged the errors but what I was suggesting is to add instructions on how to solve the issue

Trying to give users some guidance on what may be wrong will help to get a good developer experience and reduce frustration (and IAM is a very good frustration generator 😅).

See some examples:

@fargito
Copy link
Contributor Author

fargito commented Oct 11, 2021

Great ! I'll do that !

@fargito fargito force-pushed the fix/s3-deleteObjects-silent-fail branch from 2d9eaea to 0c59547 Compare October 11, 2021 09:48
@fargito
Copy link
Contributor Author

fargito commented Oct 11, 2021

@t-richard done :)

Copy link
Member

@mnapoli mnapoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I added minor suggestions for wording.

src/utils/s3-sync.ts Outdated Show resolved Hide resolved
src/utils/s3-sync.ts Outdated Show resolved Hide resolved
@fargito fargito force-pushed the fix/s3-deleteObjects-silent-fail branch from 0c59547 to 9bf219f Compare October 11, 2021 13:24
@fargito
Copy link
Contributor Author

fargito commented Oct 11, 2021

@mnapoli thanks a lot, I applied your suggestions

Copy link
Contributor

@t-richard t-richard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mnapoli
Copy link
Member

mnapoli commented Oct 14, 2021

master is green again, you should be able to rebase or merge it into your branch, and then I think we'll be good to merge 💪

@fargito fargito force-pushed the fix/s3-deleteObjects-silent-fail branch from 9bf219f to 78def6e Compare October 15, 2021 07:56
@fargito
Copy link
Contributor Author

fargito commented Oct 15, 2021

Hello @mnapoli I think you can relaunch the workflow

@fargito fargito force-pushed the fix/s3-deleteObjects-silent-fail branch from 78def6e to 0d16f8a Compare October 15, 2021 08:44
@fargito fargito force-pushed the fix/s3-deleteObjects-silent-fail branch from 0d16f8a to d588e18 Compare October 15, 2021 08:47
@fargito
Copy link
Contributor Author

fargito commented Oct 15, 2021

@mnapoli actually it is also used by the "server-side-website" contruct so I updated the tests and added it to the error message

@mnapoli mnapoli added the enhancement New feature or request label Oct 19, 2021
@mnapoli
Copy link
Member

mnapoli commented Oct 19, 2021

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants