Same goal, multiple experiments

[noctivityinc]

I find myself having to do something like this for my conversion page:

finished('product_info_partial')
finished('mobile_get_it_now_copy_cheat_sheet')
finished('mobile_get_it_now_copy_practice_test')
finished('cs_catch_all_landing_pages')
finished('cs_landing_pages')
finished('pt_landing_pages')
finished('bundle_landing_pages')
finished('dmvcs_cheat_sheets_index')
finished("#{@site.subdomain}_homepage_copy")
finished("desktop_practice_test_link")
finished("dmvcs_mobile_practice_test_url")

which is, IMO, a bit crazy. I would prefer to just do something like this:

finished(goal: 'purchase')

and then when defining an experiment I could do something like:

ab_test({goal: 'purchase', label: 'mobile_get_it_now_copy_cheat_sheet'}, 'alt1', 'alt2', 'alt3')

This seems to be a much DRYer way of handing experiments when they are all related to the same goal.

[andrew]

Sounds like a handy feature, would you like to have a go at implementing it and sending a pull request?

[iangreenleaf]

This would be awesome. it's exactly what I suggested in the comments on #109, but I think it never got implemented then.

[noctivityinc]

@andrew quick question. From looking at the yml defined experiments it appears that there is a :metric option that can be defined for any experiment and then used in finish. I believe, if I am reading the notes correctly, that the same metric could be used across experiments right? So if I defined a metric: :purchase then if I did finished(:purchase) any experiment with the purchase metric would be marked complete, hence allowing for a single completion for multiple experiments.

Is that correct or am I not understanding metrics correctly? If seems that if that is the case the only thing that needs to be done is enhancing the split_helper file to create a new metric if specified via the ab_test helper.

Let me know if I am right about the above and I can give it a shot.

[andrew]

I'm not 100% sure on that but I believe so, might be worth checking with @coolzilj who worked on this stuff last year: #109

[coolzilj]

I believe :metric is what you are looking for. @noctivityinc

[noctivityinc]

Great. So I don't duplicate effort, can you confirm that there is no way to currently specify a metric in the ab_test helper? Thanks

Sent from my  iPhone

On Jan 14, 2014, at 10:29 AM, Liu Jin notifications@github.com wrote:

I believe :metric is what you are looking for. @noctivityinc

—
Reply to this email directly or view it on GitHub.

[andrew]

Yeah I can confirm that, looking forward to seeing the results, thanks @noctivityinc

[stevenou]

I think the next step would be to allow multiple metrics defined for each experiment instead of just one - same way you can define multiple goals.

:metrics => [:add_to_cart, :purchase]

[andrew]

It's worth noting that @stevenou has submitted a related PR if anyone's interested: #260