Caching of data

[cesswairimu]

Implement weekly caching of data to address the performance issues that may result from collating so much data.

[vaarigupta]

hello @cesswairimu I am an outreachy applicant and excited to work on this issue. How can I start contributing ? what is it all about?

[cesswairimu]

Hello @vaarigupta.
It is about caching data in various places in the site to fasten page load esp on this page which loads a lot of dynamic data http://publiclab.org/stats
Also I want to optimize the queries in http://github.com/publiclab/plots2/blob/master/app/controllers/stats_controller.rb on this issue too. If you are interested in doing any of these we can break down this to minit-tasks and work on it together. Thanks.

[ananya]

If @vaarigupta is not working on the issue then I would like to work on this. I am new to the organsation and would be happy to contribute.

[vaarigupta]

Hello @vaarigupta.
It is about caching data in various places in the site to fasten page load esp on this page which loads a lot of dynamic data http://publiclab.org/stats
Also I want to optimize the queries in http://github.com/publiclab/plots2/blob/master/app/controllers/stats_controller.rb on this issue too. If you are interested in doing any of these we can break down this to minit-tasks and work on it together. Thanks.

ya I can do. Please guide me how to start working on it?

[jywarren]

Hi, all! @cesswairimu is this perhaps already done? If not, what would be the next step here? Thanks a lot!

[cesswairimu]

Hey @jywarren, I didn't really get around to doing this and from #5524 it seems like its still an issue and we could do some caching of the data esp for the stats page, per yearly maybe so that retrieval could be faster. Not sure how to go about this yet but would like to investigate more on this when I get some time.Thanks

[skilfullycurled]

Hey everyone, wanted to ask a follow up question: suppose you want to view (not download) a years worth of data. Does 365 daily caches help the issue of wanting to view a years worth of data? Or, would easing the load of a years worth of data require a cache for that year? I ask because I'm wondering to what extent this will address a part of the issue @cesswairimu referenced in #5524 regarding stats overload which I think includes the problem larger viewings.

[cesswairimu]

@skilfullycurled No, I do think so, what happens is if you query a year's worth of data, say 01/2012-12/2012 it stores the results in a cache, and if you execute the same range after say two hours it will return the cached results instead of performing the same query again.

[cesswairimu]

I think we have cached most parts of the stats. Closing this. Thanks all