Silent Failures When Batching Tracking Events

[willisplummer]

Hi, I've tried a number of versions from 2.0.0 through 2.2.5 and consistently experience silent failures when firing a number of events at once

client = Segment::Analytics.new({
  write_key: ENV['SEGMENT_KEY'],
  on_error: proc { |_status, msg| print }
})

20.times do
  puts client.track(user_id: 12, event: '2121', properties: {})
end

client.flush

the loop returns true 20 times

when i look in the segment debugger, i only see 10-15 events actually reaching segment. i've tried without the flush and get basically the same result.

at this point it seems like i'll need to switch to simple_segment and enqueue the tracking events in background jobs?

(running rails 5.1.0 and ruby 2.3.1)

thanks in advance!

edit:

tried with 2.2.6.pre and it's closer to working but i still inconsistently get 19 instead of 20 events even when running

20.times do
  puts client.track(user_id: 12, event: '2121', properties: {})
  client.flush
end

[rohitpaulk]

Thanks for reporting this, @willisplummer.

You're right, there were changes in 2.2.6.pre that fixed an issue related to silent dropping of track calls.

I'll try this out soon. To help us debug this better, could you share your Gemfile and Gemfile.lock please? (Email us at friends@segment.com and cc prateek@segment.com if you can't share publicly)

[willisplummer]

Thanks for getting back to me - just emailed the Gemfile and Gemfile.lock

[rohitpaulk]

@willisplummer I used the script you shared, and can confirm that some events were being dropped in the debugger. This is because the debugger attempts to sample requests for display in the UI. This shouldn't happen with actual destinations - I'll try this out with a webhook destination and confirm soon.

Also, based on the Gemfile you shared - I don't think this as a case of #161 (which only happens with a specific combination gems).

[rohitpaulk]

Hey @willisplummer, I got a chance to try this out today and don't see an issue other than the debugger dropping events due to sampling.

Here's what I used for the test:

1. A script to generate events (based on the one in the description of this ticket)

# Usage: SEGMENT_KEY=xxx ruby main.rb <event_name>
require "bundler"
Bundler.require(:default)

client = Segment::Analytics.new({
  write_key: ENV['SEGMENT_KEY'],
  # Edited based on @tcollier's comment 
  on_error: proc { |_status, msg| puts msg }
})

puts "Segment version: #{Segment::Analytics::VERSION}"

event_key = ARGV.first

20.times do |x|
  count = x+1
  client.track(user_id: 12, event: event_key, properties: {
    count: count
  })
end

client.flush
puts "done"

2. A server to receive events via webhooks

require "sinatra"

event_counts = {}

post "/from_segment" do
  event = JSON.parse(request.body.read)["event"]

  if event_counts.has_key?(event)
    event_counts[event] += 1
  else
    event_counts[event]  = 1
  end

  puts "Counts:"
  puts "-------"
  to_print = event_counts.to_a.last(5).to_h
  to_print.each do |k, v|
    puts "#{v} events - #{k}"
  end
  puts "-------"
  puts ""

  "200 OK"
end

I tried sending events with different event names, and consistently got 20 events via webhooks.

Here's what the count looked like soon after sending out events:

After a minute or so, all events were received

(due to the at-least once nature of the webhook destination, one event was sent 21 times instead of 20)

---

I'm going to close this issue, as it looks like the confusion arised from the segment debugger's sampling. Please feel free to re-open if you're able to replicate this otherwise 👍

[tcollier]

The issue is likely with your proc, the body of which is a no-op

proc { |_status, msg| print }

You should pass the msg block argument to print. I'd actually suggest puts msg over print msg as the former adds a new line and the latter does not, which will make logs easier to read.

[rohitpaulk]

Nice catch, @tcollier!