Home

Tech setup/General docs

NOTE: Below is background information/general docs on the topic of this project. If you only want to be able to deploy the project then please read Installing the dependencies and deploying the project

The server we use to host our website is rented from Digital Ocean. The operating system is CentOS 7 and its IP address is 138.197.6.26 but all you have to remember is our domain name: conucourseplanner.online. If you can remember the name but need to know the IP address, you can always run nslookup conucourseplanner.online. Unfortunately all devs currently sign onto the server as the same user: david. This quirk may change in the future.

We have three processes related to this project which are constantly running on the server:

1. Our main Tomcat server

• Installed at /opt/tomcat
• Started via sudo systemctl start tomcat

This serves the frontend to all clients as well as a REST API. Our REST API provides a number of functions callable by the client via AJAX requests in the front end. These functions include but are not limited to:

POST validate - send a sequence to the server and it responds with an indication of whether the sequence is valid or not. If invalid, it provides the list of issues which caused validation to fail.

POST courseinfo - send a course code to the server (e.g. COMP 248, COMP 249) and it will respond with the info related to that course. (See courseInfo database below).

POST export - send a sequence to the server and it responds with downloadable URL which provides a file containing the sequence in question in a human readable format (either PDF or Markdown).

GET allsequences - request this function to the server (no params) and it responds with a list of all the recommended course sequences we support (See courseSequences database below).

2. Our MongoDB database server

• Started via sudo mongod --dbpath=/opt/cp-course-db &
• View contents of db via mongo (this command starts the mongo shell)

The backend of our website (the REST API) communicates with this server in order to perform the aforementioned functions.

We have two databases courseplannerdb (used only by prod version of website) and courseplannerdb-dev (shared by all development versions of website). Each database contains documents of two important data types related to the functioning of the website:

courseInfo which represents information related to individual courses such as name, credits, pre-requisites, etc.

courseSequences which represents the sequence of courses recommended to students by Concordia that they should take from their first to last semester.

3. Our webscraper cron job

• Crontab entry: 0 0 * * * sh /opt/cp-webscraping/scrapeTheWeb.sh --prod

This is a cron job which periodically (currently every 24 hours) runs a shell script which performs all of our data acquisition (scraping) by downloading various public concordia webpages, pulling and formatting relevant data out of the pages, then storing the results in the MongoDB database. The fact that it runs periodically ensures that changes to the concordia site will be continuously integrated into our database.