Plan your courses
Getting up and running
The fastest way to get running is to use Docker. This will let you run Flow inside of a virtual machine to avoid dealing with package installation problems and to avoid polluting your development environment. It should also take less time.
To get started, install Docker.
Once you have docker installed, run:
$ make shell_in_docker
This will download and run a Docker image with all of Flow's dependencies already installed. Don't worry if you don't know what a Docker image is.
It will only need to download the image once, so booting up in the future should be much faster.
Once the image is downloaded and running, you should find yourself in a
/bin/bash shell. This is running inside a Docker container (effectively a
virtual machine). Inside this container, the
rmc repository can be found in
/rmc, which is where you should be right after running
To start running Flow locally, run the following inside this new shell.
$ make local
If you point your browser at http://localhost:5000/, you should now see the Flow homepage running on your computer!
Congratulations! You now have Flow running locally.
Getting seed data
To do anything interesting in Flow, you need data in your database. This is where information about courses, professors, and scheduling information is stored.
To get started, open a new terminal (different from the one running
make local), and start a new shell inside Docker by running
$ make shell_in_docker
Inside the Docker container, run the following to get some basic course data into the DB.
$ make init_data
If you're eager to dive into the code, you might want to read this first. This isn't exhaustive, but it should be enough to get you started if you want to contribute.
config/: Configuration for frameworks, databases, or anything that might vary between the development environment and production.
data/: This is where we collect data and load it into the database
crawler.pydownloads data by scraping pages and hitting APIs
processor.pyprocesses the data grabbed by
crawler.pyand loads it into the DB
aggregator.pyis run on a regular schedule (daily for the most part) to keep our data up to date
models/: "Schema" definitions for our models backed by MongoEngine
server/: Request handlers, static assets, and templates
templates/: Jinja2 templates
- Files in here ending with
course_page.html) are rendered directly by the Flask server with
render_templatecalls, with the exception of the
base_*_page.htmlfiles which other
_page.htmltemplates inherit from.
- Most of the other files (e.g.
course.html) contain Underscore templates used to render stuff on the client-side
- Files in here ending with
static: Static assets eventually ending up as files served directly by nginx when on production
server.py: The majority of the request handlers for the application, written in Flask
Using the REPL
If you need a REPL to fool around with the database or test out some code, check
It automatically loads some imports and connects to the database for you. This
setup code can be found in
Here's what an example session might look like:
$ tools/devshell.py Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) Type "copyright", "credits" or "license" for more information. IPython 0.13.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. In : m.User.objects(first_name__in=['Jamie', 'David', 'Sandy', 'Mack'], last_name__in=['Wong', 'Hu', 'Duan', 'Wu']) Out: [<User: David Hu>, <User: Mack Duan>, <User: Sandy Wu>, <User: Jamie Wong>]
It might seem funny that this repository and a bunch of the code references
RMC stands for "Rate My Courses", which was the prototype name for this project before it was given the (slightly) better name of Flow.
Because of the profileration of this 3 letter prefix throughout the code, and the unfortunate coupling of the repository name and our python namespace, we decided to leave it be.
To run all the tests in the entire system:
To run all the tests except the really slow ones (namely Selenium tests):
To run all the tests under a specific directory tree or in a specific file:
PYTHONPATH=.. nosetests server/api PYTHONPATH=.. nosetests server/api/v1_test.py
Setting up without Docker
If you'd prefer to avoid the docker route, you can install the dependencies directly on your own machine.
To set up your dev environment, run
We work inside a virtualenv, so remember to
source ~/.virtualenv/rmc/bin/activate whenever you're working within the repo.
You should now be ready to boot the local server, with
Once it starts running, point your browser to http://localhost:5000/
MongoDB error on Linux
If you are getting a connection refused error when trying to run
make local and are on Linux, this is
most likely due to MongoDB taking too long to start the first time it's run. To fix this, run
mongod --config config/mongodb_local.conf
and let it warm up for about 30 seconds to 1 minute. Then kill the process, and run
make local again. It should work now.