Split::ExperimentCatalog.all returning nil values in array

[danielschwartz]

We are seeing a weird behavior where when calling Split::ExperimentCatalog.all the array returned includes a nil value. Here is a sample array that was returned to us this morning:

[nil, #<Split::Experiment:0x007f5f53ac44a0 @name="20140422_homepage_testimonials", @alternatives=[#<Split::Alternative:0x007f5f53ad7410 @experiment_name="20140422_homepage_testimonials", @name="no", @weight=1>, #<Split::Alternative:0x007f5f53ad73e8 @experiment_name="20140422_homepage_testimonials", @name="yes", @weight=1>], @goals=[], @resettable=false, @algorithm=Split::Algorithms::WeightedSample>, #<Split::Experiment:0x007f5f53ad5318 @name="engraving_option", @alternatives=[#<Split::Alternative:0x007f5f53ae8a58 @experiment_name="engraving_option", @name="separate", @weight=1>, #<Split::Alternative:0x007f5f53ae8a30 @experiment_name="engraving_option", @name="inline", @weight=1>], @goals=[], @resettable=false, @algorithm=Split::Algorithms::WeightedSample>]

This breaks a bunch of expectations we have when running code like this Split::ExperimentCatalog.all.select {|e| !e.has_winner?}

[knoopx]

Happened to me too, basically it is because there's a non-existing experiment in the catalog. Deleting non-existing experiments solves the problem.

[andrew]

Do either of you have a way I can repeat the error you're both running into?

[knoopx]

@andrew I'm not sure about the origin but it is probably because of upgrading the split gem on our production servers, some experiment data may have been lost or corrupted.

[andrew]

I wonder if we can just call .compact within #all as an easy way to avoid the upgrade pain.

[danielschwartz]

That's what we did to get rid of this issue, seems like it would work generally.

[andrew]

Do either of you fancy sending that fix as a pull request with a comment noting why it's in there?

[danielschwartz]

Will do.