Fish Fry Form
Fish Fry Form is a web application the includes a data management GUI and ReST API for cataloging and accessing the rich array of Lenten Fish Fry events that occur every spring in Western Pennsylvania. It is the foundation on which the Fish Fry Map, the brainchild of Hollen Barmer, is built.
The Fish Fry Form is the entry point for editing and accessing the Fish Fry Map's data. It is both an actual graphic user interface (GUI) for accessing and editing Fish Fry map data (as a simple little web application), and an application programming interface (API) for doing the same. It enables a consistent method for soliciting input and curating data for the Fish Fry Map. It has also been designed to ensure is the map and data entry interfaces can be database-agnostic.
Fish Fry Curation Primer
In setting this project up, we wanted to accomplish something that hadn't been done in the past: put the power of adding Fish Frys to the map into the hands of everyone.
We also wanted to make sure the data is reliable, since hopefully people will be using it to select which Fish Fry they might attend. We don't, for instance, want to inadvertently send people to a Fish Fry where they think they'll be able to enjoy an ice cold beer in addition to homemade pierogies at lunch on a Friday afternoon, but in fact find out that not only does the venue not serve alcohol, but it's closed for lunch.
So, we've created a process layer between open submission and formal publication; a QA/QC or data curation process. This is how that works:
- An anonymous user of the Fish Fry Form submits an entry. In the database, two fields,
publishedare set to
- Someone on the Fish Fry Team responsible for scrubbing the data looks at the database, makes sure things at the very least are not a mess, and/or cleans up any mistakes or misinformation, and switches
- Someone (maybe the same person, but maybe not) looks at all records where
TRUE. Since this is all supposed to be for a map, this person may then only be responsible for checking the address/geocoding and make sure it all jives. Then they switch
For us, the Fish Fry people, the point of having this two-step validation-publication check is practical: we're going to be starting with last year's Fish Fry venues, which we need to first update (validate) for this year, and then, since timelines for this are tight, need to quickly yay/nay for display to the map (publish). Then, presumably next year we will do the same.
Fish Fry API Usage
The Fish Fry API provides some basic
GET access to the Fish Fry database, and full CRUD access for administrators.
See the complete API documentation here.
This a pretty rote Python Flask application, with (a bunch of) plugins to make it punch above its weight.
Flask-Admin provides a quick way to manage users and roles. For this application, users are stored in a
SQLAlchemy talks to that database for
API and API UI
For ReST-ful access to the data, we're using
Swagger UI generator for Flask). The former simplifies API design; the latter provides a way to automatically generate API documentation (among other things).
Fish Fry data is stored as the features array of a
geojson FeatureCollection in AWS DynamoDB. Schema and validation is managed at the application-level with
Marshmallow, with further management provided for the form itself by
To talk to DynamoDB, we've put some calls made using the AWS
boto3 library behind a simple CRUD layer (
core/api/db_interface). This makes CRUD requests easy; it also keeps our ReST-ful API logic separate from the database API. If for some reason we need to migrate to another database (e.g., PostgreSQL), this CRUD layer is the only part of the code that knows about the database, and the only part that would need to be changed.
So you want to clone or fork this thing and get it running yourself? Have at it. The steps below assume you:
- have a Python 3.6+ installation on your machine
- can run
pipcommands (or whatever aliases you've configured for for Python 3) from the command line
gitinstalled and can run some basic
gulpinstalled, for running tasks. This isn't strictly required for development, unless you plan to work with the client side
This was built with Python 3.6 on Windows 10. Since Python on Windows can sometimes be a pain, we recommend pairing the official Python 3 installer from python.org with a Windows console emulator like Cmder w/Git-for-Windows
To work with notebooks in the
/notebooks folder, you'll also need to have Jupyter installed.
1. Clone this repository
git clone https://github.com/CodeForPittsburgh/fishfryform.git cd fishfryform
It's a Python package that combines package management and virtual environment management in one place.
pip install pipenv
3. Install dependencies in a virtual environment
Get a Python
pipenv virtual environment running with required dependencies:
$ pipenv install
This command finds the
requirements.txt file and installs dependencies.
4. Turn on the database
Unzip the local Dynamo DB database provided in
startup.bat file found in the unzipped folder. That will spin up a test database with some old Fish Fry data loaded up.
5a. For development related to Python-Flask only:
Run just the Python Flask application:
$ pipenv run python application.py
Navigate to http://localhost:5000 to see the site.
Skip Step 5a. Run:
$ npm run dev
This will work some magic on the files in the
src folder, put them where the Flask application expects them to be, fire up
application.py, watches for changes to
css files in
src, and starts
browser-sync. It should automatically open the browser for you; if not, navigate to http://localhost:3000 to see the site.
It doesn't fire up the database (Step 4), so make sure that's running in the background (in another command line console) before you start.
6. Building the client side code for production
$ npm run build
Some basic Python unit tests are in the
tests folder (
test_*.py). Coverage is low at the moment; it's a WIP.