Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metric to help assess awareness of key binding customizability #94

Merged
merged 4 commits into from Jul 5, 2018

Conversation

Projects
None yet
2 participants
@jasonrudolph
Copy link
Member

jasonrudolph commented Jul 3, 2018

Description of the Change

Building on the effort kicked off in #90, this pull request continues our quest to better understand the awareness and approachability of Atom's "hackability."

Atom's hackability comes in multiple forms, but for many users, custom keyboard shortcuts will be the first thing they look for. The Flight Manual describes the steps for defining custom keyboard shortcuts, but we don't currently have a way of knowing whether that translates into users actually setting up custom keyboard shortcuts. To assess the awareness and approachability of defining custom keyboard shortcuts, this pull request adds a metric to report the number of user-defined key bindings that are present.

Equipped with this information, we'll be able to assess the effectiveness of our efforts to increase the awareness and approachability of keyboard shortcut customization. For example, does describing custom keyboard shortcuts in the welcome guide increase or decrease the percentage of users that define custom keyboard shortcuts?

Implementation-wise, this pull request sends an event that reports the quantity of user-defined key bindings that get loaded at Atom startup.

Alternate Designs

Instead of reporting this metric at startup, we could report it each time a user adds, removes, or changes a user-defined key binding. Doing so would require that we subscribe to the did-reload-keymap event, in addition to the did-load-user-keymap event currently in use in this pull request. It's do-able, but the additional data doesn't seem worthwhile to justify the additional overhead and maintenance. And, we could always make this change later if we decide that it would be sufficiently useful. 馃槆

Verification Process

  • Using an Atom installation that has no user-defined key bindings, verify that an event is recorded at startup reflecting that the instance has zero user-defined key bindings
  • Using an Atom installation that has some user-defined key bindings, verify that an event is recorded at startup reflecting the number of user-defined key bindings

jasonrudolph added some commits Jul 3, 2018

Avoid leaking subscriptions for keymap paths
Resolves the following warnings seen when running specs:

```
spec-helper.coffee:140 WARNING: Leaking subscriptions for paths: ~/github/metrics/spec/fixtures/keymaps/custom-keymap.cson
spec-helper.coffee:140 WARNING: Leaking subscriptions for paths: ~/github/metrics/spec/fixtures/keymaps/default-keymap.cson
```

@jasonrudolph jasonrudolph requested a review from annthurium Jul 3, 2018

atom.keymaps.loadUserKeymap()

await conditionPromise(() => {
return Reporter.request.calls.find((call) => {

This comment has been minimized.

@annthurium

annthurium Jul 3, 2018

Contributor

do you think it's worth having a helper method to do this kind of assertion, seeing as how we have > 2 tests now that are following this same pattern?

This comment has been minimized.

@jasonrudolph

jasonrudolph Jul 5, 2018

Author Member

@annthurium: Thanks for that suggestion. 鈿★笍 Implemented in 2f43713.

@@ -125,6 +126,21 @@ module.exports = {
}))
},

watchLoadingOfUserDefinedKeyBindings () {

This comment has been minimized.

@annthurium

annthurium Jul 3, 2018

Contributor

this is a very descriptive method name.

bitmoji

@jasonrudolph

This comment has been minimized.

Copy link
Member Author

jasonrudolph commented Jul 3, 2018

The screenshot below shows the metrics resulting from my testing.

You can view metrics by various dimensions, and this particular view shows the average number of user-defined key bindings loaded by Atom version. In a VM running Atom 1.30.dev-2f827bf, you can see that I have no user-defined key bindings. And on my main dev machine running Atom 1.30.0-nightly2, you can see that I have 29 user-defined key bindings.

analytics_360

jasonrudolph added some commits Jul 5, 2018

@jasonrudolph jasonrudolph merged commit 0143e61 into master Jul 5, 2018

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@jasonrudolph jasonrudolph deleted the understand-keybinding-customizability-awareness branch Jul 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can鈥檛 perform that action at this time.