Skip to content

Performance issues #94

Closed
Linuus opened this Issue Jul 2, 2013 · 13 comments

4 participants

@Linuus
Linuus commented Jul 2, 2013

Hi

I have had impressionist running for a while and the impressions table is currently at about 220.000 rows. This has made my site pretty slow.

Do you have any tips on how to increase the performance? I have all the default indexes as well as the counter cache.
I guess it could be that I'm only recording unique impressions so it has to look through the database records before creating a new impression...

How do you handle these issues?

I'm running on Heroku using Postgres.

For instance, queries like this isn't very funny:

 (13684.8ms)  SELECT DISTINCT COUNT(DISTINCT "impressions"."request_hash") FROM "impressions" WHERE "impressions"."impressionable_id" = $1 AND "impressions"."impressionable_type" = $2  [["impressionable_id", 60], ["impressionable_type", "Artist"]]

This was on my local machine (which is a bit slow) but anyway....

@Linuus
Linuus commented Jul 3, 2013

I will try to log all impressions instead and counter cache unique impressions. Perhaps a better idea.

@acnalesso
Charlotte Ruby Group member

Hi @Linuus ,
Thanks for pointing this out.
I've done some research and found what I think would be a good solution.
My thought is that we could make impressionist to use ActiveSupport::Cache and then everytime an impression is saved it just writes this impression to the cached array of impressions.
I'll be working on it tonight.

If you have any ideas please post them here so we can get this sorted.

@acnalesso
Charlotte Ruby Group member

Also the DISTINCT SQL Clause makes this very slow.

@acnalesso
Charlotte Ruby Group member

What I am going to do is to use Threads while updating or creating a record.
Any thoughts?

@Linuus
Linuus commented Jul 8, 2013

Not sure if it's a viable thing to do, but try it and see if it holds up for some performance testing :)

I had to stop impressionist from logging anything for now.

@acnalesso
Charlotte Ruby Group member

We could use threads and then cache the results with Memcache, Then tell impressionist to search for distinct data in memory(cached results).
I don't know but I think it is not the simplest solution.
I'm still doing some research to implement this.

@acnalesso
Charlotte Ruby Group member

@Linuus ,
It seams you've reached a scalability issue.
Due to some research I've done upon this issue, I thought Threads would be suitable for the job.
However I've changed my mind due to the way Ruby Threads work and the GIL etc...
Therefore a reliable solution would be to use EventMachine.?
There's a PG driver which you could have a look.

WDYT?

@acnalesso acnalesso closed this Jul 19, 2013
@Linuus
Linuus commented Aug 9, 2013

Sorry for the late reply.

I have never used EventMachine so I'm not sure if it is appropriate.

I am using Sidekiq already, would it be possible to log the impression in a Sidekiq job or do I have to do it from the specific controller action that I want to track?

@Linuus
Linuus commented Aug 14, 2013

Bump :coffee:

@dylancopeland

I'm running into some serious performance issues, too. For example, getting the impressionist_count on a model that has around 250,000 impressions is taking around 20 seconds.

@tam-vo
tam-vo commented Sep 6, 2014

@dylancopeland I'm thinking about that too, it's better to store tracking into separate database, like Redis. RelationalDb is not wise way to store all impressions. How did you resolve your performance issues?

@dylancopeland

@tam-vo I wound up ditching impressionist and just rolling out my own solution.

@tam-vo
tam-vo commented Sep 7, 2014

One of solution I used before https://github.com/soveran/ohm, it's very nice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.