Skip to content
Tooling to maintain our copy of the course data for St. Olaf College
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
lib
playground
.editorconfig
.gitattributes
.gitignore
.travis.yml
LICENSE
Pipfile
Pipfile.lock
README.md
bundle.py
download.py
maintain-datafiles.py

README.md

St. Olaf Course Data Tools

Here lie the tools for extracting course data from the St. Olaf SIS.

There are three scripts: download.py, maintain-datafiles.py, and bundle.py.

All of these tools expect the course data to be one folder up from the CWD, in ../course-data.

These scripts require python3 >= 3.4, as well as beautifulsoup4, requests, xmltodict.

The libraries are also specified in the Pipfile file, so a pip3 install pipenv and pipenv run $command

download.py

Usage:
./download.py
./download.py 2016
./download.py 1994 1995 20141

You can pass a mix of years and terms to download.py. A term is a year followed by 1-5 – 1 for fall, 5 for summer session 2.

Arguments:
  • -w, --workers — how many processes to spawn
  • --force-terms — force downloading the terms from the SIS
  • --force-details — force downloading the course details from the SIS
  • --no-revisions — don't check for revisions
  • --ignore-revisions $PROP — don't check for revisions in these properties
  • -q, --quiet — be quieter

bundle.py

Usage:
./bundle.py
./bundle.py 2016
./bundle.py 1994 1995 20141

You can pass a mix of years and terms to bundle.py. A term is a year followed by 1-5 – 1 for fall, 5 for summer session 2.

bundle.py outputs bundles into ../course-data/terms.

Arguments:
  • -w — how many processes to spawn
  • --format (json|csv|xml) — how to output the bundle. can be given multiple times to generate multiple formats
  • --legacy — create files in the legacy gobbldygook format
  • --out-dir — path to a folder to contain the output

maintain-datafiles.py

Usage:
./maintain-datafiles.py

maintain-datafiles.py takes no arguments and does one thing: it loads every course in ../course-data, and it collects lists of departments, gereqs, instructors, locations, times, and types into ../course-data/data-lists.

You can’t perform that action at this time.