New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add telemetry to capture MFA login durations #3376
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3376 +/- ##
=======================================
Coverage 98.58% 98.59%
=======================================
Files 113 113
Lines 3399 3408 +9
=======================================
+ Hits 3351 3360 +9
Misses 48 48
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
8cac74d
to
c09afbc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One question but otherwise LGTM.
0de9c58
to
8d7648f
Compare
Last commit is |
Thanks for following up @simi π . I was gonna review whether I need to add any additional tests before I fix up the commit. There's also one other metric we realized would be beneficial to capture (number of times WebAuthn is set up). I'll likely be working on this later today. Although, on second thought, it may make more sense to create a separate PR for adding that metric. Either way, I'll be making final amendments later today and will definitely squash the wip commit then π π . Should be good for another round of reviews at that point. |
8c60488
to
ec03967
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Just have a few nits regarding the tests but not blocking.
I think we should edit the current login integration test to check if statsd is incremented (mfa login, webauthn login <- this test is moved to webauthn_sign_in_test.rb
or something similar if you rebase from current master).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
be1b1b4
to
6f8ea8e
Compare
Add telementry to record how long it takes for a user to login with webauthn as their 2nd factor. Also, refactor some tests within #webauthn_create. Extract helper methods login_to_session and verify_challenge. Co-authored-by: Jenny Shen <42748004+jenshenny@users.noreply.github.com> Co-authored-by: bettymakes <makewithbetty@gmail.com>
Add telementry to record how long it takes for a user to login with otp as their 2nd factor. Co-authored-by: bettymakes <makewithbetty@gmail.com> Co-authored-by: Jenny Shen <42748004+jenshenny@users.noreply.github.com>
Since the Time objects are converted to Strings on the client anyway, we decided it'd be best to represent the data consistently with the client. So, once we retrieve the session values from the client later, we will always parse the string value back to a Time object. Co-authored-by: bettymakes <makewithbetty@gmail.com>
6f8ea8e
to
5c0d66c
Compare
π€ Squash and merge or is it ok to merge as is now? |
Yup! I think we're good to go. @simi, if you can help me merge, that'd be great π . But also, I can ping Jenny or Ashley to help with that tomorrow. We plan on checking it out on staging first to make sure it's rigged up correctly π. |
π€ What problem are you solving?
Closes https://github.com/Shopify/ruby-dependency-security/issues/146
We hypothesize that WebAuthn improves the MFA user experience by enabling users to authenticate more quickly compared to TOTP. To verify this assumption, we'll capture login durations for Webauthn and TOTP to monitor and compare the difference.
π οΈ What approach did you choose and why?
1οΈβ£ Starting the timer for capturing duration
In order to capture the duration of the login flow for both WebAuthn or TOTP, we need a starting point. We store the starting time on the browser's session once the user has successfully logged in with their username and password.
This makes sense as the starting point because the user will then be shown the 2nd factor authentication view (either prompted for TOTP code or WebAuthn).
2οΈβ£ Calculate and capture duration
Upon successfully authenticating with TOTP or WebAuthn, we'll calculate the duration of time passed (in seconds) and record it.
π What should reviewers focus on?
I'm open to renaming the helper methods created in the tests (
login_to_session
andverify_challenge
).@jenshenny & I had also thought about other metrics we may want to track (e.g. tracking failed MFA attempts, potentially its duration to failure?). But we decided it was best to start with the metric we're most interested in first.
π§ͺ Acceptance testing
We can only test this the PR is shipped and we begin capturing data.
πΆ π After PR is merged
We'll need to create a dashboard in Datadog in order to review the metrics for
login.mfa.webauthn.duration
andlogin.mfa.otp.duration
.