extreme slowness with large number of events

[afrozl]

We consistently process several thousand events (~10,000/day) every day and at any point in time there might be a thousand active events in the console. Sensu-admin seems to render these events individualy and it can take up to 2 minutes to render the events page:

  Rendered events/_output.html.haml (0.5ms)
  Rendered events/_actions.html.haml (3.5ms)
  Rendered events/_issued.html.haml (0.8ms)
  Rendered events/_status.html.haml (0.3ms)
  Rendered events/_output.html.haml (0.5ms)
  Rendered events/_actions.html.haml (3.0ms)
  Rendered events/_issued.html.haml (0.9ms)
Completed 200 OK in 61683ms (Views: 868.0ms | ActiveRecord: 3.5ms)

Any suggestions on changes we can make to speed this up somewhat?

Thanks!

[afrozl]

the api call for events takes about half a second for ~1000 events. However, running the api service in debug shows that sensu-admin is making about 2000 api calls for info and stashes. That seems to be what slows everyting down. Where could these be coming from?

[afrozl]

Problem is semi-resolved by making the following change to events_controller.rb

changed:

    events_datatable = []
    @events.each_with_index do |event, i|
      client_silenced = event.client_silenced
      check_silenced = event.check_silenced

to

    events_datatable = []
    stashes = Stash.stashes
    @events.each_with_index do |event, i|
      client_silenced =  stashes.detect {|stash| stash['path'] == "silence/#{event.client}"}
      check_silenced =  stashes.detect {|stash| stash['path'] == "silence/#{event.check}"}

This lets us call the api once to check the stashes.