A Python tool for scraping class schedules from the Mines Self-Service Banner system into a database. Produced as a final project for CSCI-403 by Jack Rosenthal, Ariel Shlosberg, and Nicholas Lantz (aka. That Rubik's Cube Team).
Install these using
pip (or however you like). You should install them for
SQLAlchemy psycopg2 beautifulsoup4 lxml PyYAML
You're going to need to setup
config.yaml with the connection to the
database. There's a sample
config.yaml to give you an idea how this looks.
You can even use a different database connector (SQLite for example).
To import data, use the
import_tool.py, providing the term (as banner likes
it in its post requests) as the only argument, like so:
$ python3 import_tool.py 201710
201710 is the way that banner represents terms, the format is relatively
trivial, the first four digits are the year, and then the month of the start of
the term follows as the next digit, followed by a zero to fill.
We included some cool examples of how to use the data with SQLAlchemy in