This is a repo for collaborating on the Open Source Day Project of Cloudera in collaboration with the Bay Area Discovery Museum.
Cloudera will be working on at OSD 2015 on the project: Data Insights for the Bay Area Discovery Museum. Located in Sausalito, CA, the Bay Area Discovery Museum annually welcomes 300,000 children ages 0 to 8, parents, and teachers to its early childhood education campus. The mission of the Museum is to ignite and advance creative thinking for all children. The Museum’s goal is to reach 1 million children a year – onsite at the Museum, in the community, and nationally through its research-based Center for Childhood Creativity – and give them a strong start to life. For more information about the Bay Area Discovery Museum, please visit www.baykidsmuseum.org. In partnership with Cloudera Cares, we will be analyzing Museum visitor data to help the Museum maximize visitor experience and reach additional families.
Technologies/Skills include: Familiarity with data wrangling and visualization would help. But, participants will be free to pick languages and tools of their choice. We are more interested in a willingness to learn and contribute.
Cloudera is revolutionizing enterprise data management by offering the first unified Platform for big data, an enterprise data hub built on Apache Hadoop. Cloudera offers enterprises one place to store, access, process, secure, and analyze all their data, empowering them to extend the value of existing investments while enabling fundamental new ways to derive value from their data. Cloudera’s open source big data platform is the most widely adopted in the world, and Cloudera is the most prolific contributor to the open source Hadoop ecosystem. Only Cloudera provides proactive and predictive support to run an enterprise data hub with confidence.
We have the dataset from BADM which contains the following details:\ **Membership level| Zip code| Visitation date | Entry time ** \ You can get the data from here We should likely use this data in conjunction with publicly available data sets. See below
Main goal of our project is to help BADM see patterns in their data especially on how visiting patterns are influenced by socio economic backgrounds of the visitors. And hopefully, they should be able to use these insights to better tailor their open hours and programs to reach more families from diverse socio economic backgrounds.
- What correlations can we see between single visitors versus repeat visitors
- Patterns in income levels and number of times people visited or when they visited
- What zipcode to target for outreach?
- What medium of advertising works might work best to target the zipcode?
===== Tools which can be used (but not limited to )=====
Analyzing data:
- SQL (mySQL, postgres)
- Weka
- [[https://www.google.com/trends/correlate#|Google Correlate]]
- [[https://code.google.com/p/google-refine/wiki/Downloads|Google refine]]
- Google sheets / Excel
- python
- R
Analyze/Visualize:
- [[https://support.google.com/fusiontables/answer/2566732|Google fusion tables]]
- [[http://www.tableau.com/public|Tableau public]]. Tutorials [[https://public.tableau.com/s/resources?edition=public&version=9.1&video=I_9nMSvY1FE|here]]
Visualization tools:
- [[https://developers.google.com/chart/?hl=en|Google Charts]]
- [[https://developers.google.com/maps/|Google map api]]
- [[http://www.freemaptools.com/|FreeMapTools]]
===== Datasets which can complement organizations data (but not limited to ) =====
- Census Data (Ex: Median income by zip)
- School Sets Data
- [[https://research.google.com/tables?ei=IvAbVtPNFoSGogStzJHwAg|Google tables]]
- Museum Calendar
- Museum “open hours” calendar and other information
- Other similar museums with possibly public released data * Smithsonians? * Other national museums
- [[https://www.groupon.com/deals/bay-area-discovery-museum-5-14-san-jose|Groupon data]]
===== Other inspiration ===== Google research data management: http://research.google.com/pubs/DataManagement.html
Papers:\ Disaster monitoring with wikipedia and online social networks sites:\ http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44015.pdf \ Big data story telling with interactive maps: \ http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/39959.pdf \ Finding related tables: \ http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/38124.pdf \
Articles: \ 10 great non profits making a difference in the tech sector: \ http://www.searchenginejournal.com/10-great-non-profits-making-difference-tech-sector/136757/? utm_content=buffer17907&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer \ Data science, a critical tool in the non profit world in 2015: \ http://www.marketsforgood.org/data-science-a-critical-tool-in-the-nonprofit-world-in-2015/ \
Other tools for hackathons:\ http://www.tokbox.com/blog/having-fun-at-startup-weekend/ \
Google map using spreadsheet:\ http://www.makeuseof.com/tag/7-ways-to-make-a-google-map-using-google-spreadsheet-data/ \
Markets for good: \ http://www.marketsforgood.org \