Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add block of Google Analytics dimensions for A/B tests #890

Merged
merged 1 commit into from
Feb 10, 2017

Conversation

suzannehamilton
Copy link
Contributor

@suzannehamilton suzannehamilton commented Feb 2, 2017

The GA dimensions from 40-49 have been allocated to A/B tests. These may be reserved for any A/B test for the duration of the test.

This commit identifies which dimension to use for the A/B test based on an extra data attribute added to the meta tag. Only dimensions in the allowed range are permitted. Any others are ignored and are not reported to GA.

It would be good to have some feedback on this approach. I'm aiming to make it as simple as possible for new teams to add an A/B test. With this proposal, they would have to do the following:

  • Request one of the dimensions in the 40-49 block from the performance analysts for the duration of the A/B test. They will assign one which isn't currently being used.
  • Set a meta tag in the header like <meta name="govuk:ab-test" content="NameOfTest:#{bucket}" data-analytics-dimension="43">.
    • Ideally, this would be done through the AB test gem. The application code would just need to pass the experiment name and dimension to the gem.

https://trello.com/c/g9OyC7Pn/310-tell-google-analytics-about-a-b-tests

Before this can be merged:

dimension = parseInt($meta.data('analytics-dimension')),
testNameAndBucket = $meta.attr('content');

if (dimension >= minAbTestDimension && dimension <= maxAbTestDimension) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it OK to silently ignore invalid or out-of-range dimensions? I'd prefer to throw an error here to alert the developer to the bad configuration, but I don't want people to accidentally break all analytics and not notice.

@tijmenb
Copy link
Contributor

tijmenb commented Feb 7, 2017

Closing this temporarily to focus on the education A/B testing. We'll revisit after that's live.

Trello: https://trello.com/c/QoJsEjgw/381-better-a-b-testing

@tijmenb tijmenb closed this Feb 7, 2017
@suzannehamilton
Copy link
Contributor Author

@tijmenb - I'd like to reopen this because we need to add another dimension for the Education Navigation A/B test, and I think this change is the easiest way to do that. Is that OK with you?

The alternative would be to give the Example and EducationNavigation meta tags different names, but that won't be compatible with more general A/B test analytics which will make the switchover harder when we release this change.

@suzannehamilton suzannehamilton changed the base branch from analytics-ab-tests to master February 8, 2017 16:52
@36degrees
Copy link
Contributor

I've changed the required status checks for this repo. You'll need to rebase this branch against master so that this branch gets a Jenkinsfile can be built against new CI.

The GA dimensions from 40-49 have been allocated to A/B tests. These may
be reserved for any A/B test for the duration of the test.

This commit identifies which dimension to use for the A/B test based on
an extra `data` attribute added to the `meta` tag. Only dimensions in
the allowed range are permitted. Any others are ignored and are not
reported to GA.
@tijmenb tijmenb changed the title [DO NOT MERGE] Add block of Google Analytics dimensions for A/B tests Add block of Google Analytics dimensions for A/B tests Feb 10, 2017
@tijmenb tijmenb merged commit 1575113 into master Feb 10, 2017
@tijmenb tijmenb deleted the multiple-ab-tests branch February 10, 2017 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants