Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a/b test for new user segment step #20369

Merged
merged 3 commits into from
Dec 5, 2017
Merged

Conversation

fditrapani
Copy link
Contributor

@fditrapani fditrapani commented Nov 30, 2017

This PR creates an A/B test for our new site segment screen outlined here: p5XAZ9-1Cn-p2.

Testing new user

image

variant

  • Visit /start either logged out or in an incognito window
  • Ensure a/b test signupSiteSegmentStep is set to variant
  • You should see the new sign up screen pictured above with the last experience question.

control

  • Visit /start either logged out or in an incognito window
  • Ensure a/b test signupSiteSegmentStep is set to control
  • You should see this existing signup screen.

Testing existing user

image

  • Visit /start logged in to an existing account
  • Ensure a/b test signupSiteSegmentStep is set to variant
  • You should see the new sign up screen pictured above without the last experience question.

@markryall and @taggon can you take a look?

@matticbot
Copy link
Contributor

Copy link
Contributor

@markryall markryall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one small recommendation regarding the designType being provided as a dependency from the about step.

@@ -109,7 +109,7 @@ export default {

about: {
stepName: 'about',
providesDependencies: [ 'themeSlugWithRepo', 'siteTitle' ],
providesDependencies: [ 'designType', 'themeSlugWithRepo', 'siteTitle', 'surveyQuestion' ],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you might also need to add designType to the call to SignupActions.submitSignupStep in the about step component.

It's pretty confusing that we pass designType around in two different ways - something we should cleanup later across all flows I think.

@fditrapani
Copy link
Contributor Author

Good catch. Thanks @markryall. I've updated it and tested it out. Looks good except for a couple errors being thrown at the end of signup. Not sure if it's related to my code or not. Can you take a look again?

@designsimply designsimply added Testing [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. labels Dec 4, 2017
@markryall
Copy link
Contributor

I didn't see any errors - just a warning from 'client/signup/processing-screen/index.jsx' about assigning directly to state instead of calling setState (which wasn't anything that your PR touches).

Looks good 👍

@fditrapani fditrapani merged commit 05b0c04 into master Dec 5, 2017
@fditrapani fditrapani deleted the add/site-segment-ab-test branch December 5, 2017 13:57
@matticbot matticbot removed [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. [Status] Ready to Merge labels Dec 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants