Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy Checklist: New Expensify 2022-09-01 #10748

Closed
24 tasks done
OSBotify opened this issue Sep 1, 2022 · 33 comments
Closed
24 tasks done

Deploy Checklist: New Expensify 2022-09-01 #10748

OSBotify opened this issue Sep 1, 2022 · 33 comments
Assignees
Labels
🔐 LockCashDeploys 🔐 Prevent new code from being deployed to staging Monthly KSv2 StagingDeployCash

Comments

@OSBotify
Copy link
Contributor

OSBotify commented Sep 1, 2022

Release Version: 1.1.96-5
Compare Changes: production...staging

This release contains changes from the following pull requests:

Deploy Blockers:

Deployer verifications:

  • I checked the App Timing Dashboard and verified this release does not cause a noticeable performance regression.
  • I checked Firebase Crashlytics and verified that this release does not introduce any new crashes.

cc @Expensify/applauseleads

@mvtglobally mvtglobally added the 🔐 LockCashDeploys 🔐 Prevent new code from being deployed to staging label Sep 1, 2022
@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 1, 2022

🚀 All staging deploys are complete, @Expensify/applauseleads please begin QA on version https://github.com/Expensify/App/releases/tag/1.1.96-0 🚀

@mvtglobally
Copy link

Regression is completed

Deploy Blockers
#10442
#10751
#10753
#10754
#10756
#10757
#10759

PR #9532 is failing due to #10751 & #10766
PR #10556 is failing due to #10758.
PR #10582 & #10717 are failing due to #10751

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

👏 Heads up @Expensify/applauseleads 👏
A new pull request has been 🍒 cherry-picked 🍒 to staging, and will be deployed to staging in version 1.1.96-1 🚀

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

🎉 All set?…You bet! @Expensify/applauseleads https://github.com/Expensify/App/releases/tag/1.1.96-1 has been deployed to staging 🎉

@roryabraham
Copy link
Contributor

Tried reverting #10582 and was able to reproduce the crash on that branch, so I don't think it's that

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

👏 Heads up @Expensify/applauseleads 👏
A new pull request has been 🍒 cherry-picked 🍒 to staging, and will be deployed to staging in version 1.1.96-2 🚀

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

🎉 All set?…You bet! @Expensify/applauseleads https://github.com/Expensify/App/releases/tag/1.1.96-2 has been deployed to staging 🎉

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

👏 Heads up @Expensify/applauseleads 👏
A new pull request has been 🍒 cherry-picked 🍒 to staging, and will be deployed to staging in version 1.1.96-4 🚀

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

🎉 All set?…You bet! @Expensify/applauseleads https://github.com/Expensify/App/releases/tag/1.1.96-4 has been deployed to staging 🎉

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

👏 Heads up @Expensify/applauseleads 👏
A new pull request has been 🍒 cherry-picked 🍒 to staging, and will be deployed to staging in version 1.1.96-5 🚀

@luacmartins
Copy link
Contributor

@mvtglobally @kbecciv can you still reproduce the following issues?

We are having issues reproducing these locally and on staging.

@mvtglobally
Copy link

@luacmartins we will check and confirm in a bit

@OSBotify
Copy link
Contributor Author

OSBotify commented Sep 2, 2022

🎉 All set?…You bet! @Expensify/applauseleads https://github.com/Expensify/App/releases/tag/1.1.96-5 has been deployed to staging 🎉

@mvtglobally
Copy link

We are re-testing PR #9532, #10582 & #10717 which were failing due to #10751
Checking CP as well

@AndrewGable
Copy link
Contributor

AndrewGable commented Sep 2, 2022

Thank you @mvtglobally, we have completed all blockers so please let us know when you are finished!

@Julesssss
Copy link
Contributor

This Firebase Exception is possibly a new regression. 12 unique users and only occuring on this version.

The alternative is that this was one of the reported blockers, is now fixed, and can be closed.

@AndrewGable
Copy link
Contributor

Exported here and fixed already! 🎉

@mvtglobally
Copy link

Checking off #9532 & #10717

CP #10780 is failing due to #10799. Its a new feature, Ok to checkoff or mark as Blocker?
I dont think we can test #10764. Ok to check off?

@AndrewGable @roryabraham Can you confirm if #10582 was reverted? Should we test it or can just check off?

@AndrewGable
Copy link
Contributor

#10582 was not reverted and should be tested still. Looking into others!

@AndrewGable
Copy link
Contributor

Checked off #10764 as it was reverted shortly after.

@PauloGasparSv
Copy link
Contributor

I think we can check #10780 off as @mvtglobally said and address that on the follow up issue if that's ok! Because the offline-first and delete functionality are working despite the message quickly popping up before being deleted when toggling!

Is that ok?

@mvtglobally
Copy link

PR #10556 is failing due to #10758. Still seems to be broken

Recording.1064.mp4
Screen_Recording_20220902-111516_Chrome.mp4

@luacmartins
Copy link
Contributor

@mvtglobally can you test with an account that hasn't attempted to send the link more than 3 times? I think the reason the message doesn't show up is because we never send the link since that account is throttled.

For now, the error message won't show up the first time you click Forgot? since we will fix the error message in a follow up PR in our API layer.

@mvtglobally
Copy link

Checking off #10582 - retest pass.
#10556 is the last one left.

@bondydaa
Copy link
Contributor

bondydaa commented Sep 2, 2022

yep agreed can you confirm that you're not actually throttled in these #10748 (comment)?

Users are throttled after on the 4th attempt within 5 minutes of each other: so you should be able to click it 3 times and get magic links sent and then a 4th should give you that error.

@mvtglobally
Copy link

New account seem ok. When you use re-set password it is showing automatically the message,

But can you guys re-validate this last one internally as well, to make sure we dont miss anything here.

Recording.1065.mp4

@bondydaa
Copy link
Contributor

bondydaa commented Sep 2, 2022

When you use re-set password it is showing automatically the message

yep that's expected for now will be fixed on #10785.

I just tried this myself (hours after those initial attempts) and got the same message for those accounts which is unexpected
image

logs https://www.expensify.com/_devportal/tools/logSearch/#query=request_id:(%22744792dfda233839-SAN%22)+AND+timestamp:[2022-09-02T15:20:33.984Z+TO+2022-09-02T17:20:33.984Z]&index=logs_expensify-018393

744792dfda233839-SAN | 2022-09-02 16:20:33 987 | mnata.utest+sr@gmail.com | undefined | db1.sjc | [audit] User just got throttled because they reached 3 number of calls for resendValidateCode
dbondy@db2.rno:~$ sudo readdb.sh -line "SELECT name, value FROM nameValuePairs WHERE name like 'private_throttleAttempts_resendValidateCode%' and accountid=12451070;"
 name = private_throttleAttempts_resendValidateCode
value = [1662135599059757,1662135601949561,1662135632575400,1662135633987843]

 name = private_throttleAttempts_resendValidateCode-212.231.184.140
value = [1662135599059930,1662135601949716,1662135632575547]

 name = private_throttleAttempts_resendValidateCode-99.235.190.147
value = [1662050591612159]

But if anything this is something broken in auth not this code - new dot is actually handling this well and showing us the error now 🎉

@mvtglobally
Copy link

@bondydaa Are we ok to check it off then?

@bondydaa
Copy link
Contributor

bondydaa commented Sep 2, 2022

hm strange and I just tried again and it worked fine for mnata.utest+sr@gmail.com
image

looking at the db again, the throttle attempts had been cleared out and only 2 show now

dbondy@db2.rno:~$ sudo readdb.sh -line "SELECT name, value FROM nameValuePairs WHERE name like 'private_throttleAttempts_resendValidateCode%' and accountid=12451070;"
 name = private_throttleAttempts_resendValidateCode
value = [1662138028209658,1662138031477769]

 name = private_throttleAttempts_resendValidateCode-212.231.184.140
value = [1662138028209808,1662138031477953]

 name = private_throttleAttempts_resendValidateCode-99.235.190.147
value = [1662050591612159]
dbondy@db2.rno:~$

Just copying over from Slack I tested with a brand new account

  1. created a new account, clicked resend validation link 3 times and got this image (1)
  2. on the 4th click of resend validation link got the error as expected: image (2)
  3. And got 4 emails as expected (when you first create the account we send you a validation link and then the 3 from the repeated clicking of resend validation link). image (3)

So yep I think 👍 to check it off.

@mvtglobally
Copy link

We are good to 🚀 then

@melvin-bot melvin-bot bot added the Monthly KSv2 label Sep 5, 2022
@Julesssss
Copy link
Contributor

Awesome, we're good to deploy first thing this morning.

  • I rechecked the timing dash and there are no noticeable timing regressions.
  • I see one new exception here. I haven't been able to reproduce yet as the logs are a bit vague. Posting in #deployer before I move forward.

@Julesssss
Copy link
Contributor

From testing the app I'm still unable to find any exception. Going to move forward as Applause's checklist would have picked up on any noticable regression.

@Julesssss
Copy link
Contributor

:shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🔐 LockCashDeploys 🔐 Prevent new code from being deployed to staging Monthly KSv2 StagingDeployCash
Projects
None yet
Development

No branches or pull requests

9 participants