MinervaScraper

Because it takes effort to log onto Minerva and navigate through the different webpages to see my grades only to realize that they haven't been uploaded yet.

Usage

Requirements

This program currently supports four browsers: Chrome, Edge, Firefox, and Brave . Your browser of interest should be indicated in the .env file. This script now supports webdriver-manager functionalities, and so the drivers' version is automatically checked and updated when the script is called. All drivers are saved in the folder Drivers.

To download all required dependencies, navigate to Minerva's parent directory and pip install requirements.txt.

    pip install -r requirements.txt

.env file

Login credentials are retrieved from a .env file, stored in the same directory as the script. Create a .env file following the format below:

• Set LOGIN=1 for ID login or LOGIN=0 for email login
• Set BROWSER to indicate your preferred browser

    # .env

    LOGIN={0, 1}

    BROWSER={CHROME, EDGE, FIREFOX, BRAVE}

    MCGILLID={insert ID here} # can exclude if LOGIN=0
    MINERVAPIN={insert PIN here} # can exclude if LOGIN=0

    MCGILLUSERNAME={insert McGill email here} # can exclude if LOGIN=1
    MCGILLPASSWORD={insert password here} # can exclude if LOGIN=1
    
    EMAIL={insert sender email here}
    PASS={insert sender email password here}

    MYEMAIL={insert recipient email here}
    
    NAME={Your name, so it feels a bit more personalized when email addresses you :D}

For security reasons, it's recommended to create a throwaway email account and use its credentials for EMAIL and PASS. This script was tested using GMAIL. As Google no longer allows access from insecure sources (valid), an App Password must be generated under 2FA .

MinervaScrape

To use from the command-line:

• Navigate to the directory Minerva (assuming currently in MinervaScraper)

    cd Minerva

• Run the command python minervascraper.py with no arguments to scrape for all terms

    python minervascraper.py

• Run the command python minervascraper.py ARG1 ARG2 etc... to scrape for specified terms

    python minervascraper.py f2019 w2020 s2020 F2020

Arguments are case-insensitive and can be passed in any order (time-independent). They take the form TERM + YEAR, where TERM is represented by a single letter, and YEAR has the form yyyy.

• S/s --> Summer
• F/f --> Fall
• W/w --> Winter
• Example:
• f2019 and F2019 are equivalent for Fall term, 2019

Output is written to an output file named Scraped_Transcript_TERM1_TERM2.json if terms are specified, or to Scraped_Transcript_All_Terms.json if none are specified.

GradeUpdate

There are two ways to check for transcript updates. The only reason why two ways exist is because I was bored, so really, there's only one (useful) way.

(1) Flag for a one-time update with minervascrape.py

python minervascraper.py -u

python minervascraper.py --update

(2) Periodically call gradeupdate.py

By making use of external schedulers, gradeupdate.py will run the scraper at prescribed intervals and send an email when a change (grade, course average) is present on Minerva's unofficial transcript, excluding changes related to add/drop (because you probably don't care about that). The email will contain a formatted table that describes these changes.

• MacOS, Linux
• Schedule through crontab
• Windows
• Windows task scheduler
• PROGRAM : path to python.exe
• ARGUMENTS : path to gradeupdate.py
• START IN : path to folder holding gradeupdate.py (Minerva)

When the scraper is enabled on a schedule, it saves all output into a logfile ('minerva_log.txt').

To-Do:

• ¯\_(ツ)_/¯
• Find a better system to send periodic updates.