Unable to call finished(experiment_name) on two different experiments

[rscherf]

My config which the Dashboard confirms both experiments are running without issue at the correct weights:

  config.experiments = {
    "homepage_redirect" => {
        :alternatives => [
            {
                :name           => 'NoShopRedirect', 
                :percent    => 20
            },
            { 
                :name           => 'ShopRedirect', 
                :percent    => 80
            }
        ]
    },

    "shop_v1_vs_v2b" => {
        :alternatives => [
            { 
                :name           => 'Aros', 
                :percent    => 25
            },
        { 
          :name     => 'BackToSchool', 
          :percent  => 25
        },
            { :name         => 'v2b',
                :percent    => 50
            }
        ],
        :resettable     => false
    }
  }

Being called with: ab_test('shop_v1_vs_v2b') and ab_test('homepage_redirect') - these happen in different controllers
And using finished as: finished('shop_v1_vs_v2b') and finished('homepage_redirect') - these happen in the same controller in the same def.

It's possible I should be making use of goals and metrics more, but I would have expected this to work. And here is the Split Dashboard showing the different between shop_v1_vs_v2b working and homepage_redirect not.

[rscherf]

FWIW, I also tried configuring a metric and finishing the metric instead of the experiment name, and that also didn't work.

[rscherf]

Maybe this relates to #172

[shoshi]

+1. I have an identical issue. I would expect either:

With config.allow_multiple_experiments = false, I would expect only one experiment to run and log a completion. Instead, it runs both and logs completions for only one.

With config.allow_multiple_experiments = true, I would expect both experiments to run and log completions. Instead, I've observed a couple behaviors:

• If both experiments start in the same controller, both run and log completions. (This is great.)
• If one experiment starts before the other in a flow, it runs multiple times, while the later one runs once. Both log completions, though the first experiment only logs completions for the last time it was run (giving you 2 bogus non-finished data points).

[karmakaze]

@shoshi You might be interested in Issue #405 being addressed in PR #421

[shoshi]

@karmakaze Wonderful! I like the "control" solution. Thank you!

[andrew]

Closing as this issue it's over a year old, please reopen if it's still affecting you.