St. Olaf Course Data
This repository holds the data for Gobbldygook in both raw (XML) and proccessed (JSON) forms. You do not need this repository if your are only interested in using Gobbldygook.
Usage
If you are interested in using the course data in another tool, it's hosted through Github on the gh-pages
branch.
You can download it however you'd like, such as via curl
. Assuming you want the 2016-Fall data in csv
format (aka the file 20161.csv
), you'd do:
curl https://stolaf.dev/course-data/terms/20161.json
We have data from 1994-Fall through this year, whetever this year is, automatically updated from the SIS's Class & Lab page once a day.
Changes
We make some changes to the original XML data we get from the SIS, as detailed in the table below.
from key |
to key |
change details |
---|---|---|
clbid |
String to int | |
coursename |
name |
Unescape HTML entities |
coursenumber |
level |
Extract the "level" (number from 1-3) from the course number |
coursenumber |
number |
String to int |
coursesection |
section |
None |
coursestatus |
status |
None |
coursesubtype |
type |
Expand the letter to a word; e.g., R to Research |
credits |
String to float | |
crsid |
String to int | |
deptname |
departments |
Put the departments into a list |
description |
If the course description (in the SIS detail window) is different from the notes , it's embedded here |
|
gereqs |
Extract text from the HTML provided, and turn them into a list | |
groupid |
String to int | |
instructors |
Extract text from the HTML provided, turn them into a list, and flip them to be "First Last", instead of "Last, First" | |
meetinglocations |
locations |
Extract text from the HTML provided, and turn them into a list |
meetingtimes |
times |
Extract text from the HTML provided, and turn them into a list |
notes |
Remove <br> tags and extraneous whitespace |
|
pn |
Y/N to boolean |
|
prerequisites |
Simple implementation: look for the string Prereq in notes and description , then taken everything from "Prereq" to the end of the string and put it in prerequisites |
|
revisions |
A recorded history of revisions of the course | |
term |
term , year , semester |
Take the five-digit term string, turn it into a number, and split it into year and semester for simplicity |
title |
If the course title (in the SIS detail window) is different from the name , it's embedded here |
|
varcredits |
Remove it entirely. I have no idea what it does; it's set to N for every single course in the database. |
Revisions
A revision
records what changed in a course. Every revision will have an _updated
property, which records when the revision was recorded¹, as well as what changed. The other key/value pairs in the revision are the old values for those keys; null
indicates that there was no old value.