README

John Little 2022-04-22

Thes are my slides, supporting materials for DSVIL 2018, the Data Science & Visualization Institute hosted by NCSU Libraries. June 6, 2018

Who / What

Data Science And Visualization Institute for Librarians. I am teaching modules on Data Cleaning, Web Scraping, HTML and JSON parsing, and Twitter Stream Gathering

When

https://www.lib.ncsu.edu/data-science-and-visualization-institute/schedule

Where

NCSU Libraries

Why

Because data transformations are crazy-fun awesome

Slides

https://is.gd/dsvil2018

Slides are divided into the following sections

0. Index
1. Introduction
2. Web Scraping
3. OpenRefine: Data Cleaning Basics
4. OpenRefine: Reconciliation
5. Capturing Twitter Data
6. APIs & JSON Parsing
7. More HTML Parsing

Exercises

Web Scraping

1. Web Scraping

   • http://v.gd/webscraping1111

Data Cleaning

1. Data Cleaning – Basic Transformation with OpenRefine (Exercise 1)
2. Data Cleaning – GREL (Exercise 2)
3. Reconciliation with OpenRefine

Social Media

1. Social Media – Twitter gathering with TAGS app (Exercise 1)
2. Social Media – Twitter: TAGS visualization and tools

API & JSON Parsing

1. APIs & JSON parsing – OpenRefine (exercise 1)
2. APIs – using API Keys (exercise 2)

HTML Parsing

1. Intro HTML Parsing: Steps 1 -6 (exercise 1)
2. More OpenRefine – Looping Control: Steps 7-end (exercise 2 – This section will introduced more advanced features of OpenRefine using HTML parsing as the example exercise)

Datasets

https://github.com/libjohn/openrefine/tree/master/data

Shareable / CC-BY-NC license

Data, presentation, and handouts are shareable under CC BY-NC license