New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feed of basic statistics in tidy data format #1038

Open
jduckles opened this Issue Oct 3, 2016 · 5 comments

Comments

Projects
None yet
4 participants
@jduckles
Contributor

jduckles commented Oct 3, 2016

This is a medium-term enhancement request, but @tracykteal and I have tried at times to answer specific questions using the wealth of data available in AMY. Short of querying the normalized web of tables, this can be challenging.

It would be nice to create some DB views that are made available using tidy data(PDF) best practices. I'm imagining this as a few views pertaining to instructor, workshop, partner, learner segments of the data. I think this can likely be done in a way that no personally identifying information is made available if we heavily aggregate. Ideally I'd like the feed to be open in order to improve transparency and involve the community in conducting analysis of it.

Example datasets I'd like to see for starters:

country, airport, organization, count of instructors
country, airport, organization, count of workshops
country, airport, organization, count of learners
country, airport, organization, count of host orgs
country, airport, count of partners
month, year, count workshops
month, year, count instructors
month, year, count learners
month, year, count active partnerships

where organization allows aggregations by swc/dc or both if ignored.

Then we can write analysis scripts that keep our relevant slides and pages that draw from AMY data up-to-date. This minimum set above would be so helpful to answer routine questions I get from prospective partners.

@tracykteal

This comment has been minimized.

Collaborator

tracykteal commented Oct 3, 2016

+1

@chrismedrela

This comment has been minimized.

Collaborator

chrismedrela commented Oct 8, 2016

month, year, count workshops
month, year, count instructors
month, year, count learners

These plots are already available in AMY in "reports" menu (i.e. here; the plots are not displayed due to a bug, but this issue will be fixed in the next release) and raw data is available as REST endpoint, although both plots and raw data are not open to public.

month, year, count active partnerships

This is easy to add (~10 lines of code).

country, airport, organization, count of instructors
country, airport, organization, count of workshops
country, airport, organization, count of learners
country, airport, organization, count of host orgs
country, airport, count of partners

These are a bit more tough to add, but I think these can be implemented as new REST endpoints, so that you can download it as csv/json/yaml. @pbanaszkiewicz any comments about implementation? @jduckles do you want somehow to visualize these datasets? If yes, what kind of plots do you want?

Opening raw data of all these reports to public is one-line change. @jduckles do you want to open the plots too or is it enough if only AMY users can view the plot?

@gvwilson

This comment has been minimized.

Member

gvwilson commented Oct 13, 2016

@jduckles @tracykteal is it OK to open this up or should it be accessible only to AMY Users?

@jduckles

This comment has been minimized.

Contributor

jduckles commented Oct 13, 2016

Fine by me, I'd like it open so we can have JS that depends on it without API keys or anything like that. I think it is sufficiently aggregated that there are minimal identifiability risks.

@jduckles

This comment has been minimized.

Contributor

jduckles commented Oct 13, 2016

@chrismedrela I don't think we need to open the plots right now. I'll probably generate some new static plots that I use with presentations and try to find a way to have the website build hook into rebuilding those, so no worries about that for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment