Follow-up email-only registration test

[donlair]

Changes proposed in this Pull Request

This is a follow-up test to the email-only test described and discussed at pbxNRc-144-p2. That flow auto-generates a site name based on the user's email address and is likely leading to a drop in purchase rate for this flow (pbxNRc-144-p2#comment-2770).

This PR changes the generated site name to one based on random words and adds a note to the checkout page. The note informs the user that they can change the site name at any point.

Here's a screenshot of the updated checkout page:

The new Explat test names are:
registration_email_only_mobile_random_usernames
registration_email_only_desktop_random_usernames

Testing instructions

• Patch your sandbox with D71395 (this updates the api that generates the username)
• Sandbox public-api.wordpress.com
• Checkout this branch and start Calypso
• Add the Abacus bookmarklets for both tests listed above.
• Navigate to [LOCALCALYPSO]/start/new in an incognito browser
• Repeat the following steps in both mobile and desktop viewports:
• Assign yourself to the treatment variation for the relevant viewport (it may take a minute or two after clicking the bookmarklet for your variation assignment to propagate).
• Go through the control and treatment variations for each viewport.
• Test the flow through a non-onboarding signup flow (such as /start/personal)
• Check non-signup checkout flows and confirm that the user isn't assigned to either test, regardless of viewport. (This check is handled in Abacus).

Related to #58404

[github-actions]

Link to Calypso live: https://calypso.live?image=registry.a8c.com/calypso/app:build-22557
Link to Jetpack Cloud live: https://calypso.live?image=registry.a8c.com/calypso/app:build-22557&env=jetpack

[matticbot]

Here is how your PR affects size of JS and CSS bundles shipped to the user's browser:

Sections (~551 bytes added 📈 [gzipped])

name      parsed_size           gzip_size
checkout      +1718 B  (+0.1%)     +551 B  (+0.1%)
signup          +20 B  (+0.0%)       +0 B

Sections contain code specific for a given set of routes. Is downloaded and parsed only when a particular route is navigated to.

Async-loaded Components (~298 bytes added 📈 [gzipped])

name                                             parsed_size           gzip_size
async-load-calypso-blocks-editor-checkout-modal       +599 B  (+0.1%)     +221 B  (+0.1%)
async-load-signup-steps-user                          +187 B  (+0.1%)      +77 B  (+0.1%)

React components that are loaded lazily, when a certain part of UI is displayed for the first time.

Legend

What is parsed and gzip size?

Parsed Size: Uncompressed size of the JS and CSS files. This much code needs to be parsed and stored in memory.
Gzip Size: Compressed size of the JS and CSS files. This much data needs to be downloaded over network.

Generated by performance advisor bot at iscalypsofastyet.com.

[niranjan-uma-shankar]

@donlair There seems to be a git conflict, this will require a rebase :). Also, a unit test seems to be failing.

[donlair]

I fixed that rebase conflict. Not sure why that unit test is failing - looks like it's having trouble connecting to the logstash api, but I don't see why these changes would break that. Rerunning those tests to see if it resolves itself.

[sirbrillig]

We should probably add isDesktop to the dependencies of useEffect here, or add a // eslint-ignore-next-line react-hooks/exhaustive-deps above it to make it explicit that we only want to run this once.

[sirbrillig]

looks like it's having trouble connecting to the logstash api, but I don't see why these changes would break that. Rerunning those tests to see if it resolves itself.

That's weird. I tried running the failing tests manually (yarn test-client client/my-sites/checkout/composite-checkout/test/composite-checkout.js) and I get errors too,

Nock: No match for request {
      "method": "POST",
      "url": "https://public-api.wordpress.com/rest/v1.1/logstash",
      "headers": {
        "accept-encoding": "gzip, deflate",
        "user-agent": "node-superagent/3.8.3",
        "content-type": "application/json",
        "accept": "*/json,*/*",
        "content-length": 8025
      },

Also this error: Error: Uncaught [TypeError: window.matchMedia is not a function] (which I think is coming from adding useViewportMatch).

And those errors don't show up in trunk... 🤔

I wonder if loadExperimentAssignment is somehow calling logstash? In any case, we can probably fix it by adding mocking for that endpoint in that test file.

[sirbrillig]

Probably we can run this line on all the tests in the file instead of just one:

wp-calypso/client/my-sites/checkout/composite-checkout/test/composite-checkout.js

Line 496 in d1175b3

nock( 'https://public-api.wordpress.com' ).post( '/rest/v1.1/logstash' ).reply( 200 );

[donlair]

@sirbrillig @niranjan-uma-shankar getting those tests to work sent me down a number of rabbit holes, but they're passing now. This PR is ready for another look whenever you get a chance. Thanks!

[sirbrillig]

Looks good to me!

[a8ci18n]

This Pull Request is now available for translation here: https://translate.wordpress.com/deliverables/7045063

Thank you @donlair for including a screenshot in the description! This is really helpful for our translators.

[tyxla]

Noting that .includes( 'wordpress.com' ) will be positive also for domains like:

• maliciouswordpress.com
• wordpress.com.somemalicioussite.com
• wordpress.comalicioussite.xyz

Should we be checking if the domain ends on .wordpress.com instead?

[niranjan-uma-shankar]

Good point @tyxla, however I don't think it's possible to map or transfer a domain that contains "wordpress" as it's a blacklisted string, and this is checked in the domain suggestions API 2b0a4-pb.

[tyxla]

Glad to hear! I guess my comment was mostly about precision here. Having a precedent for an imprecise check somewhere can easily lead to introducing more imprecision in the future, and it could happen in a place where we don't perform a special verification.

[niranjan-uma-shankar]

Yes indeed, and it makes sense to be specific in code to avoid unintended consequences down the line. Since this is an experiment, I don't foresee an immediate issue, but if the variant wins and we decide to launch it, we should definitely make the change you suggested. Appreciate your eyes on this @tyxla , thanks!

[a8ci18n]

Translation for this Pull Request has now been finished.