In this project we are designing a FLASK based Web App to maintain Leaderboards for SphinxCAPT.org. The app is based on the REST design with inherent security infrastructure. This project is being maintained as part of the Google Summer of Code 2017.
This project was envisaged in order to provide developers and volunteers with a one-stop- solution for managing and tracking donations and details of prospective donors for the SphinxCapt project at CMU. The project included a leaderboard of Donor, which is our special way of thanking donors who are supporting the project. The project also provides a workflow to the project members and volunteers to log telephonic and email correspondence with all donors.
- Backend
- Python 3
- Flask
- Pickle
- Frontend
- Web Browser with HTML5
- Javascript
-
Install Python 2.7 - In case Python 2.7 is not already installed on your computer you need to install it. You can refer to these tutorial videos for help.
- Windows Install
-
Ubuntu Install
Python 2.7 is installed by default in case of Ubuntu. If you are using some other flavour of Debian or a minimal install version then you need to install Python 2.7 seperately. You can also confirm the default Python version using the following command
python -V
In case you cannot find Python 2.7 on your PC, you can use the following video tutorial to install.
-
Virtual Environment Install - In case you wish to execute the code in a Python Virtual Environment (VirtualEnv) you can use the tutorial below.
-
Package Install - The required Python packages can be installed using the following commands
pip install -r requirements.txt
In order to connect the MySQL database the user must provide the config file containing details of SQLALCHEMY_DATABASE_URI. An example config file is given below
SQLALCHEMY_DATABASE_URI='mysql://username:password@localhost/databasename'
The Donor Management Workflow Flask application can be executed by the following command
CONFIG=databaseconfigfile python leaderboard.py
The databaseconfigfile file contains the SQLALCHEMY_DATABASE_URI as mentioned above.
The python interpreter will respond with the IP and Port on which the Flask app is running
Eg
Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
The user can then use this web address to open the default page or index.html
The Donor Leaderboard gives the details of all donors who have helped support this project through donations. These donations are used to buy Amazon Mechanical Turk Credits for the project. The Donor Leaderboard lists donors as per their donated amount. It also provides the facility to maintain anonymity of donors (if they wish to) as well as search for donors using various search criteria. In future we wish to make this leaderboard more interesting.
This module will provide the various types of users to access sub modules which will help them in managing donors across various stages of the donation process. The various stages are mentioned in details below.
- Administrator - This type of user is basically for overall management of the user group and the backend functionality. The Administration module of the index page is only visible to this kind of user
- Supervisor - This type of user has rights to add and modify donor details and contacts and also freeze the donation amount and payment method and generate reciept for the same. Additional funcionalities can be added at a later stage.
- Volunteer - This type of user has access to details of various prospective donors with contact details but can only perform mailing and phone contact logging. Other roles can also be added to this account type at a later stage.
The Donor Management workflow System has the following sub modules which are accessible to specific user types who can use these modules to manage the prospective donors at various stages of the donation process
-
Administration
- User Registration Sub Module - Allows new users to register for various roles subject to approval by a administrator account.This module is accessible by default.
- User Login - Allows registered and approved users to login.
- Approve User - Allows Administrator to approve a new user for a specific role. Without explicit approval from an Administrator a new user cannot Login or perform any functions.
- Delete User - Allows Administrator to Suspend or Delete an existing user account.(Currently under development)
- Email Setup - Allows Administrator to Add/Delete or Modify Email Templates which would be used by Volunteers to send Emails to Donors.These templates provide the bare structure on which the volunteers can build their mail and thus gives a standard interface to volunteers for quick email. These Email Templates are divided into the following parts
- Salutation - Eg Dear Mr XXX
- Main Body - The Bare Content of the Email
- Closing - Eg With Warm Regards
- Signature - Signature of the logged user
- Allot Volunteer to Donor - Allows Administrator to allot a dedicated Volunteer to every prospective donor who is responsible for 1contacting the donor and persuading him/her to donate money for the project. The volunteer would generally act as a Single Point of Contact to the donor.
-
Donor Leader Board - This module is visible by default and allows everyone to track the various donors who have credited confirmed donations to the project . Only certain details of the donor are visible. In case the donor wished to remain anonymous, this facility could be added while committing the donation amount.
-
Donor Management - This sub modules allows the various authorized users to perform the following roles
- Add Prospective Donors - This module allows all users to add credentials of a new prospective user into the Workflow Application.
- Update Donor Contact - This module allows all users to modify contact information (including Phone and Email) of Donors who's details already exist in the Application.
- Donor Phone Contact Logging - This module allows all users to Log details of telephonic conversation with Donors. Details of previous Phone and Email Conversations are available to the Volunteer for reference.
- Donor Mailing (Including Single Mail,Bulk Email and Bulk Email with Mail Merge) - This module allows all users to send Emails to Donors using either a Saved Template or by using Manual Composition option. Once the volunteer selects a template they are provided with a semi-filled structure of the Email which can be modified before sending. Details pertaining the donor and the volunteer are automatically entered by the application in the respective fields
- Donation Commit - This module allows Volunteers to enter details of donations which have been committed by a donor. These details include the committed amount as well as the mode of transfer of the donation. These details are used by the Supervisors to track such users in order to facilitate easy transfer of the donation to the project account.
- Donation Credit and Receipt Generation - This module allows all users to enter details of donation amount which has been credited to the project account by committed donors. Once the details are entered in the application it allows automatic generation of Receipt in form of a PDF file which is automatically sent to the Donor on his email and in case if the Donor wishes it is also sent to him by Post.
The project follows the standard Flask Directory Structure. The detailed directory structure is being updated and would be posted soon.
- root
- static
- css - Contains CSS files used by HTML files in templates folder.
- fonts - Contains Fonts files used by HTML files in templates folder.
- images - Contains images used by HTML files in templates folder.
- js - Contains Javascript files used by HTML files in templates folder.
- templates - Contains HTML templates used by Flask for various Endpoints.
- tests - Contains Unit Test modules for testing the application.
- receipts - Contains copies of receipts generated by the application for credited donations for record keeping.
- static
The project is divided into various sub modules as per Flask convention. The details of important Python files are given below
- config.py - Contains certain environment configuration variables which gives flexibility during migration of code to a different platform or directory.
- email_service.py - This file contains methods required to authenticate using OAUTH 2.0 protocol with _Gmail and to send email using Google Python API.
- forms.py - This file contains wtforms structure which are used to process data entered by users using various endpoints.
- exception.py - This file contains code for Exception Handling used by Flask.
- generate_receipt.py - This file is used to generate Donation Receipts in Pdf Format which are further sent to Donors using Email/Post.
- leaderboard.py - The core Flask Application file which initializes the application. This file also contains code to connect to the MySQL Database.
- models.py - This file contains database models which are used to connect to the MySQL Database. It uses sqlalchemy library to connect to the database.
- sub_process.py - This is one of the most important files in the application and contains methods which are regularly called by other files in order to retrieve/process data.
- upload_receipt_gdrive.py - This file is used to upload the donation receipts generated by generate_receipt.py file to Google Drive using Google Python API.
- views.py - This file contains various Endpoints which are used in the application. The details of the endpoints are given below.
The project used a MySQL database to store details of Donors,Communication (Telephonic & Email), User Details as well as Accounting transactions. The Current Scheme for the databse can be founded in the Schema.png file in the repo.
The project uses a large number of SQLAlchemy Models to connect to the application database. The details of these models are given below.
- User - Contains model description of registered users of the application. This model connects to the workflow_users table of the database.
- Donor - Contains model description of Donors who have been registered in the application and connects to the donor_details table.
- DonorPhoneLog - Contains model description of Telephone Logs entered by Volunteers with Donors and connects to the donor_phone_logs table.
- DonorEmailLog - Contains model description of Email Logs entered by Volunteers with Donors and connects to the donor_email_logs table.
- CommittedDonation - Contains model description of committed donation details which have been given to the volunteer by the donors. This model connects to the committed_donation_details table.
- CreditDonation - Contains model description of credited donation details which have been transferred to the project accounts by the donors. This model connects to the credit_donation_details table.
- EmailTemplate - Contains model description of Email templates which have been created by administrators to be used to send emails to donors.This model connects to the email_templates table.
- BulkEmailList - Contains model description of Bulk Email List which contains details of donors who are addresses in the bulk email module.This model connects to the bulk_email_list table.
This section provides an overview of all the endpoints available to users in the Donor Management Workflow
-
/ - This module will launch index.html and give the default view of the application
-
/default - This endpoint is only for testing purpose to check redirecting of webpages
-
/donorleaderboard - This endpoint will launch the donor_leaderboard.html webpage which displays the current Donor Leaderboard to all users
-
User Management Endpoints
- /user_register - This endpoint is used for registering new users and launches register_new_user.html
- /process_new_user_register - This endpoint collects the POST response from /user_register endpoint and extracts user credentials like email address, phone, name, username, password etc and stores it in the User Account Database. After this the user is redirected to index.html
- /user_login - This endpoint is used for logging existing and approved users and launches user_login.html
- /user_login_process - This endpoint collects the POST response from /user_login endpoint and extracts user credentials like username and password and matches it with credentials stored in the User Account Database. On matching the user status is changed to Logged and user is redirected to index.html. In case of failed login the user is directed to the Login Error Page i.e. invalid_login.html
- /logout - This endpoint deletes the user login session variables and updates the status to Not Logged. The user is then redirected to index.html
- /approve_new_user - This endpoint is only available to Administrator User Group and allows these users to approve pending users who have freshly registered. This endpoint launches approve_user.html. In case these is no user account pending for approval, the user is redirected to index.html
- /approve_new_user_process - This endpoint collects the POST response from /approve_new_user endpoint and extracts user credentials like email address, phone, name, username etc. In case the user is approved by the Administrator the account status is changed to Active. In case the administrator rejects the user account, the same is removed from the list of pending users.
-
Donor Management Endpoints
- /register_new_donor - This endpoint is used by authorized users to add details of prospective donors to the Donor Details Database which would be further used in the workflow during the whole donor management process
- /process_donor_form - This endpoint collects the POST response from /register_new_donor endpoint and extracts Donor user credentials like email address, phone, name, organisation etc and stores it in the Donor Details Database
- /donor_contact_update - This endpoint launches donor_contact_update.html and used to update contact details of registered donors. The user can choose the specific donor by searching for the donor from the list and update the contact information
- /donor_contact_update_process - This endpoint extract the Donor ID selected by the user for updating contact information in the /donor_contact_update endpoint and passes this ID to donor_contact_update_form.html where it is read by Jinja for displaying the available contact info about the donor
- /donor_contact_update_form_process - This endpoint collects the POST response from /donor_contact_update_process endpoint and extracts the updated Donor contact info like email address and phone and updates it in the Donor Details Database
- /donor_phone_contact - This endpoint lauches donor_phone_contact.html which is used to select the donor with whom the user wants to log the telephonic conversation. The user can select the target donor from the Donor List and using form submission this information is passed to the /donor_phone_contact_process endpoint
- /donor_phone_contact_process - This endpoint collects the POST response from /donor_phone_contact endpoint and extracts Donor ID. Then is searches the Donor ID in the Donor Details Database to extract other credential of the user. Details of previous conversations with the donor are also retrieved. The endpoint further passes this information to donor_phone_contact_logging.html
- /donor_phone_log_process - This endpoint collects POST response from donor_phone_contact_logging.html. These details include summary of the telephonic conversartion with the donor as well as details shared with the donor. These details are updated in the DONOR_CONTACT_LOGS_FILE_
- Index Page and Menu Items
- User Login Module
- User Registration Module
- Prospective Donor Registration
- Update Donor Contacts
- Phone Contact Logging ☎️
- Email Contact 📨
- Bulk Emailing 📨
- Bulk Emailing using Mail Merge 📨
- Track Donor Communication
- Commit Donation and Receipt
- Accounting
- Logging
- Data Validation (Using JQuery Form Validator)
This project contains certain additional files which are used to perform various functions which are described below.
- mail_merge_tags.json - This JSON file contains details of Mail Merge tags which will be parsed while sending bulk emails using merging technique. All tags which are not mentioned in this file are discarded during processing.
This project is sponsored by Google Summer of Code 2017 and maintained by Saurabh Singh under the mentorship of James Salsman and Tom Hartung with support from the whole CMU Sphinx Team.
Completed Receipt Generation & Accounting 💳
Updating of README file for new endpoints and database needed to be done.
📣