Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AB test “Report a problem” design #583

Merged
merged 7 commits into from Apr 29, 2015
Merged

Conversation

@fofr
Copy link
Contributor

@fofr fofr commented Apr 28, 2015

On specified pages, create a multivariate test showing a new report a problem design.

The design:

  • collects quantitative data about the page's usefulness which gets sent to Google Analytics
  • analytics also tracks opening and closing of original form on test pages, and successfully submitted feedback for both variants
  • hides the yes/no links and asks for more details about problems
  • forms are still submitted to the same endpoint, but the "what doing" field is pre-filled and hidden

screen shot 2015-04-28 at 11 18 43

screen shot 2015-04-28 at 11 18 51

cc @binaryberry @benilovj

@binaryberry
Copy link
Contributor

@binaryberry binaryberry commented Apr 28, 2015

👍

@benilovj
Copy link
Contributor

@benilovj benilovj commented Apr 28, 2015

When the test is finished, we'd ideally like to be able to revert this change wholesale.

This branch has some good improvements to the report-a-problem form that we'd like to keep... is it possible to spin those out into a separate PR, leaving just the stuff that we can remove later?

],
slug = ReportAProblem.getCurrentSlug();

return slugsBeingTested.indexOf(slug) > -1;

This comment has been minimized.

@benilovj

benilovj Apr 28, 2015
Contributor

I don't think that this will work for subpages of guides and smart answers (eg /check-uk-visa/y/afghanistan)

This comment has been minimized.

@fofr

fofr Apr 28, 2015
Author Contributor

Update to match any URL that begins with the specified slug. See 1d6e520

fofr added 6 commits Apr 24, 2015
If a page is being tested (currently set to all pages), then create a
standard test named `report-a-problem-redesign-ab-test`, with two
variants: the original design and the new design.

Refactor tests to ensure that the expected behaviours works in all
three variants: no test, control and new design. Move standard tests
into a helper that can be called in different contexts.
Add a new design which includes a Yes/No link that when clicked opens a
form containing a textarea.

Re-uses the existing end point to avoid significant refactoring, which
is unnecessary while we are only testing.

* Continue to submit to the existing end point
* Coerce Yes/No values into a hidden input field previously used for
“What you were doing”
* Add analytics tracking to Yes/No and original variant toggling
Use a list of slugs to denote that the test should run
Denote to analysts that this is a temporary event.
eg `/in-test` and `/in-test/also/in/test`
@benilovj benilovj force-pushed the report-a-problem-design-test branch from 1d6e520 to 8e1a6c7 Apr 28, 2015
@binaryberry
Copy link
Contributor

@binaryberry binaryberry commented Apr 28, 2015

A couple of styling points:

  • In the original version, the toggle font size is 16px, vs. 19px in the variant version, which I don't think was part of the design
  • Also in variant version, the form text area seems really small, cf. screenshot

screen shot 2015-04-28 at 15 51 17

@fofr
Copy link
Contributor Author

@fofr fofr commented Apr 28, 2015

@binaryberry Are your styles cached?

@binaryberry
Copy link
Contributor

@binaryberry binaryberry commented Apr 28, 2015

@fofr Urgh, yes. Hard refresh solved the form size issue, though not the font size difference between original and variant toggles.

To keep test variants as similar as possible:
* Make text light grey
* Reduce text to 16px
@fofr
Copy link
Contributor Author

@fofr fofr commented Apr 28, 2015

Altered "Was this useful" text to be a light grey and smaller, to match existing toggle text in 1c83e91

screen shot 2015-04-28 at 16 18 16

@binaryberry
Copy link
Contributor

@binaryberry binaryberry commented Apr 28, 2015

👍

benilovj added a commit that referenced this pull request Apr 29, 2015
@benilovj benilovj merged commit bcde41c into master Apr 29, 2015
1 check passed
1 check passed
default "Build #481 succeeded on Jenkins"
Details
@benilovj benilovj deleted the report-a-problem-design-test branch Apr 29, 2015
benilovj added a commit that referenced this pull request May 26, 2015
…-test"

This reverts commit bcde41c, reversing
changes made to decc161.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.