New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raw data from stats page #963

Open
sdosemagen opened this Issue Nov 7, 2016 · 24 comments

Comments

@sdosemagen
Copy link

sdosemagen commented Nov 7, 2016

For the https://publiclab.org/stats page I'd like to either be able to download the raw data as a .csv (privacy considerations here?) and/or to potentially hover over a graph and be able to see the numbers per week.

@jywarren

This comment has been minimized.

Copy link
Contributor

jywarren commented Nov 17, 2016

OK -- i'll begin breaking this out into sub-issues that'd have to be done before we could achieve this; is there a timeline you're hoping for, or can we enter this in the software-outreach queue for newcomers to tackle?


Currently, data is presented inline in https://github.com/publiclab/plots2/blob/master/app/views/notes/stats.html.erb#L84-L86, like:

    var notes = [[0,16],[1,16],[2,12],[3,25],[4,10],[5,8],[6,9],[7,9],[8,10],[9,22],[10,14],[11,12],[12,16],[13,29],[14,8],[15,14],[16,13],[17,13],[18,11],[19,12],[20,21],[21,22],[22,19],[23,21],[24,21],[25,16],[26,9],[27,9],[28,14],[29,13],[30,21],[31,4],[32,12],[33,16],[34,9],[35,7],[36,11],[37,10],[38,14],[39,19],[40,13],[41,15],[42,13],[43,19],[44,21],[45,17],[46,20],[47,57],[48,22],[49,11],[50,14],[51,11],[52,0]]
    var wikis = [[0,9],[1,1],[2,7],[3,1],[4,0],[5,0],[6,7],[7,4],[8,2],[9,8],[10,9],[11,8],[12,8],[13,2],[14,2],[15,0],[16,0],[17,1],[18,0],[19,1],[20,2],[21,2],[22,0],[23,0],[24,1],[25,2],[26,0],[27,2],[28,5],[29,3],[30,2],[31,1],[32,0],[33,0],[34,2],[35,1],[36,1],[37,2],[38,2],[39,4],[40,2],[41,2],[42,1],[43,0],[44,5],[45,2],[46,0],[47,4],[48,2],[49,2],[50,0],[51,0],[52,0]]
    var comments = [[0,21],[1,65],[2,69],[3,62],[4,46],[5,19],[6,70],[7,43],[8,23],[9,72],[10,39],[11,107],[12,80],[13,92],[14,36],[15,69],[16,113],[17,54],[18,35],[19,54],[20,100],[21,127],[22,96],[23,78],[24,48],[25,51],[26,35],[27,20],[28,35],[29,28],[30,34],[31,30],[32,41],[33,63],[34,56],[35,49],[36,33],[37,21],[38,64],[39,40],[40,55],[41,59],[42,46],[43,77],[44,53],[45,37],[46,63],[47,45],[48,57],[49,25],[50,20],[51,50],[52,0]]

Steps would be:

  • move the collection of data into its own controller method such as def compile_stats using these three methods: Solved in #1103
  • then make a new method which displays those as a .json file at, for example, https://publiclab.org/stats.json to which you pass a time parameter: stats.json?time=xxxxxx.
  • consider breaking it up into three separate methods, such as /stats/notes.json, /stats/wikis.json in its own stats_controller.rb
  • add a link to the stats.html.erb template that goes directly to the stats.json file so it can be downloaded.
  • figure out a CSV presentation of this data. This could be manually, as adapted here: https://github.com/publiclab/spectral-workbench/blob/master/app/helpers/spectrums_helper.rb#L3-L17 or maybe there's a more standard ruby way to do this, with a gem
@jywarren

This comment has been minimized.

Copy link
Contributor

jywarren commented Nov 30, 2016

@ebarry - can you offer some input on prioritization here, so we can consider bumping this up or putting it on the back burner? Thanks! 🎈

jywarren added a commit to jywarren/plots2 that referenced this issue Dec 16, 2016

@jywarren

This comment has been minimized.

Copy link
Contributor

jywarren commented Dec 16, 2016

Made an initial stab at this, but it'd need specifics on exactly what data would need to be downloadable: https://github.com/jywarren/plots2/tree/stats-export

@ebarry - we could alternatively seek to replace the graphs here with something from Chart.js, which would include hovers; http://www.chartjs.org/

image

@ebarry

This comment has been minimized.

Copy link
Member

ebarry commented Jan 13, 2017

hi! @sdosemagen and I wanted to comment that this issue is not currently a priority, but it might be nice in the future to have easy access to the raw data.

@jywarren jywarren added this to the API and search improvements milestone Jan 17, 2018

@ebarry

This comment has been minimized.

Copy link
Member

ebarry commented Mar 7, 2018

This is something still relevant to the evaluation project but not urgent for 2018 (in relation to other more urgent priorities)

@ebarry ebarry modified the milestones: Search improvements, API Mar 7, 2018

@ebarry ebarry added the metrics label Mar 7, 2018

@jywarren jywarren referenced this issue Sep 26, 2018

Open

Planning for expanded community stats system #3498

3 of 16 tasks complete
@Kanikashrivastav

This comment has been minimized.

Copy link
Contributor

Kanikashrivastav commented Oct 2, 2018

Hi @jywarren ! I would like to work on this issue as to participate in this outreachy round. I think It would take me some time to figure out the solution as I'm very new to open source and in this project. It would be great if you guide me where I should start :)

@jywarren

This comment has been minimized.

Copy link
Contributor

jywarren commented Oct 3, 2018

@Radhikadua123

This comment has been minimized.

Copy link
Contributor

Radhikadua123 commented Oct 25, 2018

Hi @jywarren and @Kanikashrivastav

Can I take this task ? I have given thought to it about how to implement it.

  get 'stats' => 'stats#index'
  get 'stats/:type(/raw)' => 'stats#type'
  get 'stats/range/:start/:end(/raw)' => 'stats#range'
  get 'stats/:type/range/:start/:end(/raw)' => 'stats#range_with_type'

I think these routes should cover up everything. Optional parameter raw would give the exact same data in raw format which has been used in the charts. Though I'm not sure if it should be csv or json ?

:type can be subscriptions, people, notes, wikis, questions, comments, contributors etc - pretty much anything, making it generic enough.

What do you think @jywarren ?

@Kanikashrivastav

This comment has been minimized.

Copy link
Contributor

Kanikashrivastav commented Oct 25, 2018

@Radhikadua123 thanks for your interest, sorry but i am working on it , can you please look on any other issue available.

Thanks
Kanika

@Radhikadua123

This comment has been minimized.

Copy link
Contributor

Radhikadua123 commented Oct 25, 2018

Sure, no worries! :)

@gauravano

This comment has been minimized.

Copy link
Member

gauravano commented Nov 3, 2018

As mentioned in the issue by @sdosemagen that privacy needs to be considered so whoever is/will be solving this issue, please take note of it. Although, I think only moderators and admin should be given privileges for downloading data.Thanks!

cc: @publiclab/community-reps

@Radhikadua123

This comment has been minimized.

Copy link
Contributor

Radhikadua123 commented Nov 3, 2018

Hi @gauravano,

Even though I'm not working on this issue, it's part of outreachy project. So, thanks for reminding us about privacy concerns. I'll think about it how we can address this issue and change my proposal.

Thanks!

@cesswairimu

This comment has been minimized.

Copy link
Collaborator

cesswairimu commented Dec 4, 2018

@gauravano @SidharthBansal @jywarren @sagarpreet-chadha I am currently working on this.

@sagarpreet-chadha

This comment has been minimized.

Copy link
Contributor

sagarpreet-chadha commented Dec 4, 2018

Awesome 😄 !

@jywarren

This comment has been minimized.

Copy link
Contributor

jywarren commented Dec 4, 2018

Awesome!!

@SidharthBansal

This comment has been minimized.

Copy link
Member

SidharthBansal commented Dec 5, 2018

@cesswairimu

This comment has been minimized.

Copy link
Collaborator

cesswairimu commented Dec 5, 2018

Thanks @SidharthBansal, on it.

@SidharthBansal

This comment has been minimized.

Copy link
Member

SidharthBansal commented Dec 6, 2018

@cesswairimu as per Outreachy rules you need to write Blog post each week throughout the Outreachy period on the Public lab official website to tell us your progress.
Please mention what you have till now achieved, what are you currently working on and thinking to work upon.
Be thorough in writing the blogs. Be descriptive so that new folks next summer can read it and plan for Outreachy.
I hope this makes sense for you.

@SidharthBansal

This comment has been minimized.

Copy link
Member

SidharthBansal commented Dec 6, 2018

Can you please tell me the link to your Milestone so that I can have a look?
Thanks

@cesswairimu

This comment has been minimized.

Copy link
Collaborator

cesswairimu commented Dec 6, 2018

Yes it does thanks. Here is the link https://github.com/publiclab/plots2/milestone/20 @SidharthBansal, I am still adding issues

@SidharthBansal

This comment has been minimized.

Copy link
Member

SidharthBansal commented Dec 6, 2018

Please also add the work related to Outreachy which is merged.

Once you will create the blog please tag the mentors so that we can review.
Thanks

@SidharthBansal

This comment has been minimized.

Copy link
Member

SidharthBansal commented Dec 6, 2018

Thanks for creating the Milestone.
Looks good to me!

@jywarren

This comment has been minimized.

Copy link
Contributor

jywarren commented Dec 6, 2018

@cesswairimu

This comment has been minimized.

Copy link
Collaborator

cesswairimu commented Dec 6, 2018

@jywarren sounds good. Thanks.

@cesswairimu cesswairimu referenced a pull request that will close this issue Jan 18, 2019

Open

(WIP)Raw data from stats page #4654

0 of 5 tasks complete

@cesswairimu cesswairimu self-assigned this Jan 18, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment