Participants stop showing up on dashboard when I run the field_test_events migration

[kyleschmolze]

If I run rails g field_test:events and run the migration, then add some goals and track conversions with them, the dashboard stops reporting any new participants. All numbers show up as 0, even though I can see that the FieldTest::Memberships are being created. Once I rollback the migration, the dashboard shows the proper numbers again. Don't have time to try to debug this right now, but I can later, especially if you have any ideas where to look!

BTW thanks for this great gem!

[ankane]

Hey @kyleschmolze, different code paths are run after the field_test_events table is created. Here's a good place to start debugging:

field_test/lib/field_test/experiment.rb

Line 169 in 05ea051

def use_events?

[pedrorio]

@kyleschmolze , did you have any progress on this? I am having the same issue.

[kyleschmolze]

Yeah, I haven't had time to finish, yet, but I have some code along the right path. The problem is with the way the "participated" number is calculated here. The memberships-events join prevents any memberships from being returned that don't also have an event. And since events are only recorded for converted users, this means "participated" will always be equal to "converted" in this calculation, and so it will say 100% conversion rate, and ignore all un-converted users (returning 0 for that).

Instead, I just split it into 2 queries. One to calculate the number of total participants, and one to get the number of converted participants (using the same join). My code looks like this, starting on line 99:

relation = memberships
relation = relation.where("created_at >= ?", started_at) if started_at
relation = relation.where("created_at <= ?", ended_at) if ended_at

if use_events?
  participants = relation.group(:variant).count

  data = {}
  data = relation.joins(:events).where(field_test_events: {name: goal}).uniq.group_by(&:variant)
  data.keys.each do |variant|
    data[[variant, true]] = data[variant].length
    data[[variant, false]] = participants[variant].to_i - data[variant].length
    data.delete variant
  end

  # if no conversions, make sure to still put the participants
  participants.keys.each do |variant|
    if data[[variant, true]].blank?
      data[[variant, true]] = 0
      data[[variant, false]] = participants[variant].to_i
    end
  end
else
  data = relation.group(:variation, :converted).count
end

[pedrorio]

I also had this behaviour on my project.
I am trying to implement a recommendation system but will come back to take a look at this.

[pedrorio]

PS: @ankane Thanks for your awesome gems