Add recordTracksEvent for Editor NUX modal #46796

autumnfjeld · 2020-10-27T07:09:36Z

Changes proposed in this Pull Request

track NUX modal opening: calypso_editor_wpcom_nux_open
track NUX modal dismiss: calypso_editor_wpcom_nux_dismiss
track each NUX modal screen the user sees: calypso_editor_wpcom_nux_slide_view
track is_last_slide

Testing instructions

sync Editing Toolkit to your sandbox
create new user & new site -or- see below for instructions to reset nux status for exisiting user
View NUX model opening in the Editor
Verify tracking network request (see screenshot) for each of the 3 tracking events. The last event will fire for each of the 4 slides with data to track slide number.
- calypso_editor_wpcom_nux_open
- calypso_editor_wpcom_nux_dismiss
- calypso_editor_wpcom_nux_slide_view (fired on each slide)
  - note the data passed for each slide: slide_number is_last_slide.

Hover over the network request to show the full tracking event label

🚧 This didn't work reliably every time for me. To try reset your nux status to `enabled` so that NUX dialog shows on existing account:

reset nux status in your sandbox

wpsh
update_user_attribute( your_user_id, 'wpcom-block-editor-preferences', [ 'nux-status' => 'enabled' ] );
update_user_meta( your_user_id, 'wpcom_block_editor_nux_status', 'enabled' )
clear local storage: both WP_DATA_USER AND WP_LAUNCH as shown in screenshot

Fixed #47257

matticbot · 2020-10-27T07:09:37Z

Test live: https://calypso.live/?branch=add/track-editing-toolkit-nux-modal

matticbot · 2020-10-27T07:21:18Z

Caution: This PR affects files in the Editing Toolkit Plugin on WordPress.com
Please ensure your changes work on WordPress.com before merging.

D51826-code has been created so you can easily test it on your sandbox. See this FieldGuide page about developing the Editing Toolkit Plugin for more info: PCYsg-ly5-p2

matticbot · 2020-10-27T07:25:21Z

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

ramonjd · 2020-10-29T00:42:05Z

apps/editing-toolkit/editing-toolkit-plugin/wpcom-block-editor-nux/src/wpcom-nux.js

@@ -52,6 +53,9 @@ function WpcomNux() {
 	if ( ! isWpcomNuxEnabled || isSPTOpen ) {
 		return null;
 	}
+	recordTracksEvent( 'calypso_newsite_wpcomnux_view', {
+		is_new_flow: isGutenboarding,


I think isGutenboarding is defined a few lines beneath, so this will be undefined

autumnfjeld · 2020-10-30T06:32:57Z

@ramonjd I left instructions to reset nux stats in this PR description, but really that didn’t work for me most of the time. Do you see an error I made? Is there somewhere else where the nux status needs to be cleared? Or cache cleared? (I'll delete that part of the PR description if it is useless.)

Going another route to fake the nux status: I set the relevant api endpoints to return hard coded values:
returned true here

wp-calypso/apps/editing-toolkit/editing-toolkit-plugin/wpcom-block-editor-nux/class-wp-rest-wpcom-block-editor-nux-status-controller.php

Line 60 in 720b5da

return 'enabled' === $nux_status;

and returned enabled in class-wpcom-block-editor-nux.php#53

But still I was seeing the redux store return “false”

wp-calypso/apps/editing-toolkit/editing-toolkit-plugin/wpcom-block-editor-nux/src/store.js

Line 27 in bf02435

isWpcomNuxEnabled: ( state ) => state.isNuxEnabled,

In the end, I did other hacky things to make dev-ing easier. Final testing should be done on new user.

ramonjd · 2020-11-01T21:52:42Z

calypso_editor_wpcom_nux_1 ( and _2 , _3 , _4)

Rather than creating a new event for each page, we could fire one event and sent different data along with it. Similar to the example PR from @razvanpapadopol.

For example

recordTracksEvent( 'calypso_editor_wpcom_nux_slide', {
   slide_number: pageNumber,
   is_last_slide: isLastPage,
} )

Not only is it tidier, but it makes data analysis easier if we're looking at and filtering data for one event, and not n.

The is_last_slide is a great idea by the way! That way, we'll know if the user reached the end regardless of how many slides there are.

I left instructions to reset nux stats in this PR description, but really that didn’t work for me most of the time. Do you see an error I made?

🤔 I think what you've done it fine. A wild guess is that something is cached. Not sure where though... maybe even in your browser if it has cached the response.

In the end, I did other hacky things to make dev-ing easier. Final testing should be done on new user.

Don't fear the power of the hacky side!

Thanks for writing up what you did. It will make testing a lot easier! 🙇

autumnfjeld · 2020-11-01T22:21:56Z

@ramonjd

Rather than creating a new event for each page, we could fire one event and sent different data along with it. Similar to the example PR from @razvanpapadopol.

Yes! Actually I used Razvan's code! I just forgot to update the PR description to reflect that slides are tracked by passing the slide number and is_last_slide as data.

Credit for is_last_slide tracking goes to @razvanpapadopol. 😃

Thanks everyone for helping me with this!

ramonjd · 2020-11-01T22:29:11Z

Yes! Actually I used Razvan's code! I just forgot to update the PR description to reflect that slides are tracked by passing the slide number and is_last_slide as data.

Oh! I will look at the code next time too. Sorry! 😇

…ast_slide' props.

…x_slide_view

ramonjd · 2020-11-02T01:28:23Z

✅ Event fire when wpcom Nux loads and first slide is shown:

calypso_editor_wpcom_nux_open
is_gutenboarding: true

and

calypso_editor_wpcom_nux_slide_view
slide_number: 1
is_last_slide: false
is_gutenboarding: true

✅ Navigating through the slide fires calypso_editor_wpcom_nux_slide_view with the correct property values , e.g.,

calypso_editor_wpcom_nux_slide_view
slide_number: 2
is_last_slide: false
is_gutenboarding: true

✅ Navigating to last slide sets is_last_slide to true

slide_number: 4
is_last_slide: true
is_gutenboarding: true
_en: calypso_editor_wpcom_nux_slide_view

✅ Closing Nux modal by either clicking on "Get started" or on the page body triggers calypso_editor_wpcom_nux_dismiss

calypso_editor_wpcom_nux_dismiss
is_gutenboarding: true

By the way, I think I found a way to test without worrying about user meta or attributes:

clear WP_DATA_USER AND WP_LAUNCH as you mentions, then to set a default value of bypassApi = true in actions.setWpcomNuxStatus

ramonjd

Nice work!

LGTM

ramonjd · 2020-11-02T01:41:17Z

It's not related to this PR since it's happening on production too, but the modal isn't translated even though most of the strings exist, e.g.,
https://translate.wordpress.org/projects/wp-plugins/full-site-editing/stable/de/default/?filters%5Bterm%5D=Welcome+to+your+website&filters%5Bterm_scope%5D=scope_any&filters%5Bstatus%5D=current_or_waiting_or_fuzzy_or_untranslated&filters%5Buser_login%5D=&filter=Apply+Filters&sort%5Bby%5D=priority&sort%5Bhow%5D=desc

Just adding a note here because I noticed while testing.

Have put it on the TODO list over at #37665

noahtallen · 2020-11-02T22:57:22Z

apps/editing-toolkit/editing-toolkit-plugin/wpcom-block-editor-nux/src/wpcom-nux.js

@@ -11,6 +11,7 @@ import { useDispatch, useSelect } from '@wordpress/data';
 import { useEffect } from '@wordpress/element';
 import { __ } from '@wordpress/i18n';
 import { registerPlugin } from '@wordpress/plugins';
+import { recordTracksEvent } from '@automattic/calypso-analytics';


Does this work on Atomic sites? I have no idea, just thought it would be worth bringing up just in case something is weird there

Hiya @noahtallen Why do you think this wouldn't work on Atomic sites? You probably have some valuable insight that I haven't yet mastered. 😄 Is there a known issue with analytics tracking on Atomic sites?

Haha, I don’t have a specific reason. I’m just used to things not working exactly the same on atomic so it’s always worth checking! I mentioned this because it seems like the sort of library that could theoretically rely on a connection to wpcom or mission control, which is one of the areas that can be different between the two platforms!

autumnfjeld · 2020-11-09T02:57:48Z

@ramonjd and/or @noahtallen Looking for input on something I just learned:

I just learned that the NUX modal can be triggered from the three vertical dot menu. Should we pass additional paramaters to indicate when/how the NUX modal is being viewed? This would only apply to the NuxPage tracking and we could track isWpcomNuxEnabled to know if user has accessed the NUX modal through the editor's three-dot-menu.
If my knowledge is complete it can be viewed:

as brand new user in the editor for the first time
by clicking the three-vertical-dot-menu (does this UI feature have a name?)

I'll create a new issue if we need to be tracking additional data for the NuXPage tracking.

ramonjd · 2020-11-09T04:04:59Z

I just learned that the NUX modal can be triggered from the three vertical dot menu.

TIL 🎉

Should we pass additional paramaters to indicate when/how the NUX modal is being viewed?

@autumnfjeld Great point!

I think it definitely makes sense to pass a parameter to tell us that it hasn't popped up after site creation.

We already have is_gutenboarding which indicates that the user has come from the onboarding flow, but perhaps isWpcomNuxEnabled is more reliable, especially since is_gutenboarding sounds like it doesn't have a long shelf life if it becomes THE signup flow.

This is just a suggestion, but we could add is_nux to all the calls (A.K.A is new user experience) to distinguish one from the other?

			recordTracksEvent( 'calypso_editor_wpcom_nux_open', {
				is_gutenboarding: window.calypsoifyGutenberg?.isGutenboarding,
				is_nux: isWpcomNuxEnabled,

			} );

noahtallen · 2020-11-09T21:15:23Z

Agreed on the tracking! Some more notes about it:

I think if you open it from the 3-dot menu, it re-enables NUX for your account. So (assuming you don't close it), it would show it the next time you see it. (At least, this used to be the behavior, which is very handy for testing PRs!)
I think Atomic sites might be buggy because I keep seeing the NUX modal there for every new site I make even though it should be off for my account.

ramonjd · 2020-11-09T23:34:22Z

Added an issue #47257

I think if you open it from the 3-dot menu, it re-enables NUX for your account

Hmm, that sounds like it's a convenient bug? 😆 Takes the "N" out of NUX...

noahtallen · 2020-11-10T19:56:23Z

Hmm, that sounds like it's a convenient bug

I think it's deliberate, and it works out since you would probably close it right away, which disables it again :P

autumnfjeld · 2020-11-17T01:55:33Z

@Automattic/ganon
Tracks Registration PRs:
386-gh-Automattic/tracks-events-registration
387-gh-Automattic/tracks-events-registration
388-gh-Automattic/tracks-events-registration

autumnfjeld changed the title ~~🚧 🚧 WIP: first attemp at recordTracksEvent for discussion~~ 🚧 🚧 WIP: add recordTracksEvent for Editor NUX modal Oct 27, 2020

ramonjd reviewed Oct 29, 2020

View reviewed changes

ramonjd mentioned this pull request Oct 29, 2020

Editing Toolkit: Adding WPCOM NUX basic tracking #46762

Closed

razvanpapadopol mentioned this pull request Oct 29, 2020

Add tracking to Editing Toolkit NUX #46705

Closed

autumnfjeld changed the title ~~🚧 🚧 WIP: add recordTracksEvent for Editor NUX modal~~ Add recordTracksEvent for Editor NUX modal Oct 30, 2020

autumnfjeld added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Oct 30, 2020

autumnfjeld requested review from a team and razvanpapadopol October 30, 2020 06:33

autumnfjeld and others added 7 commits November 2, 2020 11:07

WIP: first attemp at recordTracksEvent for discussion

f3c0e8e

NUX modal tracking: add calypso_editor_wpcom_nux_open event

8535be7

NUX modal tracking: add calypso_editor_wpcom_nux_dismiss event

d806fa8

fire Tracks event on each NuxPage mount with 'slide_number' and 'is_l…

f86d098

…ast_slide' props.

Update var pages to nuxPages

a5ed74b

Nux page tracking: update track event name to calypso_editor_wpcom_nu…

bb6c67d

…x_slide_view

Nux page tracking: pass is_gutenboarding boolean

9adc40b

autumnfjeld force-pushed the add/track-editing-toolkit-nux-modal branch from 75a160f to 9adc40b Compare November 2, 2020 00:07

ramonjd assigned autumnfjeld Nov 2, 2020

ramonjd approved these changes Nov 2, 2020

View reviewed changes

noahtallen reviewed Nov 2, 2020

View reviewed changes

autumnfjeld merged commit 2ec2714 into master Nov 3, 2020

autumnfjeld deleted the add/track-editing-toolkit-nux-modal branch November 3, 2020 06:26

matticbot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label Nov 3, 2020

razvanpapadopol mentioned this pull request Nov 3, 2020

Persistent Launch Button: enable in all environments #46817

Merged

ciampo mentioned this pull request Nov 6, 2020

Editing Toolkit: Update to 2.8.4 #47186

Merged

ramonjd mentioned this pull request Nov 9, 2020

Editing Toolkit NUX #47257

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add recordTracksEvent for Editor NUX modal #46796

Add recordTracksEvent for Editor NUX modal #46796

autumnfjeld commented Oct 27, 2020 •

edited by ramonjd

matticbot commented Oct 27, 2020

matticbot commented Oct 27, 2020

matticbot commented Oct 27, 2020

ramonjd Oct 29, 2020

autumnfjeld commented Oct 30, 2020 •

edited

ramonjd commented Nov 1, 2020

autumnfjeld commented Nov 1, 2020

ramonjd commented Nov 1, 2020

ramonjd commented Nov 2, 2020

ramonjd left a comment

ramonjd commented Nov 2, 2020

noahtallen Nov 2, 2020

autumnfjeld Nov 3, 2020

noahtallen Nov 3, 2020

autumnfjeld commented Nov 9, 2020

ramonjd commented Nov 9, 2020

noahtallen commented Nov 9, 2020

ramonjd commented Nov 9, 2020

noahtallen commented Nov 10, 2020

autumnfjeld commented Nov 17, 2020

Add recordTracksEvent for Editor NUX modal #46796

Add recordTracksEvent for Editor NUX modal #46796

Conversation

autumnfjeld commented Oct 27, 2020 • edited by ramonjd

Changes proposed in this Pull Request

Testing instructions

🚧 This didn't work reliably every time for me. To try reset your nux status to enabled so that NUX dialog shows on existing account:

matticbot commented Oct 27, 2020

matticbot commented Oct 27, 2020

matticbot commented Oct 27, 2020

ramonjd Oct 29, 2020

Choose a reason for hiding this comment

autumnfjeld commented Oct 30, 2020 • edited

ramonjd commented Nov 1, 2020

autumnfjeld commented Nov 1, 2020

ramonjd commented Nov 1, 2020

ramonjd commented Nov 2, 2020

ramonjd left a comment

Choose a reason for hiding this comment

ramonjd commented Nov 2, 2020

noahtallen Nov 2, 2020

Choose a reason for hiding this comment

autumnfjeld Nov 3, 2020

Choose a reason for hiding this comment

noahtallen Nov 3, 2020

Choose a reason for hiding this comment

autumnfjeld commented Nov 9, 2020

ramonjd commented Nov 9, 2020

noahtallen commented Nov 9, 2020

ramonjd commented Nov 9, 2020

noahtallen commented Nov 10, 2020

autumnfjeld commented Nov 17, 2020

autumnfjeld commented Oct 27, 2020 •

edited by ramonjd

🚧 This didn't work reliably every time for me. To try reset your nux status to `enabled` so that NUX dialog shows on existing account:

autumnfjeld commented Oct 30, 2020 •

edited