Web scraping application developed by students at UGA for the AADM Freedom Fund.
Here’s a link to the project description. Overall, the idea is to build a password protected web application that Athens Anti-Discrimination Movement (AADM) can use internally to identify individuals eligible for their Freedom Fund program.
- Scrape publicly available data from this website every morning (on a timer)
- Process the scraped data into a Python Dictionary and import to an SQL database
- We will have two schema’s: one for user login and storing scraped data
- Create a Login Page and Home Page
- Login Page.- We can keep it super basic for now, just email and password
- Homepage with a form to filter out persons experiencing incarceration from the DB based on certain criteria:
- Whether or not they meet the exclusion criteria listed out in the Google Docs linked above
- Demographic Characteristics
- Agency
- Backend Using Python Flask
- Pretty light weight backend frameworks, simpler syntax, and faster deployment
- Frontend: HTML/CSS and Bootstrap (Responsive Design ftw!)
- Backend: Python Flask
- Web Scrape: Python BeautifulSoup
- Database: SQL - likely SQLLite3 with Flask SQLAlchemy because it fits in really well with Flask
In order to push updates to the master branch, a pull request and an approving review from another contributor are required. This avoids conflicts and helps with overall repository organization.
To do this, follow these instructions:
git checkout -b new-branch-name
git add --all
git commit -m "Your commit message"
git push -u origin new-branch-name
- Go to the repository on the GitHub website
- Click on the Pull requests tab
- Click the New pull request button
- Set
base
tomaster
, andcompare
tonew-branch-name
- Click the Create pull request button
- Leave a comment about what you did, then click Create pull request
Now wait for another contributor to review and merge it.
- Go to the repository on the GitHub website
- Click on the Pull requests tab
- Click on an open pull request
- In the first box with a red X, click Add your review
- Click on the Review changes button
- If everything looks good, click Approve request then Submit review
- Click the Merge pull request button
- Click the Confirm merge button
- Click the Delete branch button
- Setting up Virtual Environment: virtualenv env
- Launching Virtual Environment: source env/bin/activate
- Setting Up Flask: pip3 install flask flask-sqlalchemy flask-login
- To run the App: Run main.py and open your browswer to http://127.0.0.1:5000/