decide on what recommendations from python package audit to implement

[jason-upchurch]

Recommendations from #3782

Suggest creating a separate issue for the following issues as appropriate.

1. Upgrade to python@3.7.4:

• update python image in config.yml to circleci/python:3.7.4-buster
• update requirements.txt with gevent==1.4.0
• update requirements-dev.txt to remove pandas
  • update manage.py to remove functions (or comment) that rely on pandas (these functions are not currently used elsewhere)
• update runtime.txt with python-3.7.4

2. Consider replacements for marshmallow-pagination as its requirements are not routinely maintained for our purposes and has implications for generating transitive dependencies

• an fecgov/marshmallow-pagination fork has been created with requirements updated (we can at low cost maintain this fork, or incorporate it into our codebase)

3. At a minimum, upgrade all libraries with breaking changes (apispec is of particular interest as it is likely necessary to modify the codebase to mitigate backwards incompatibility--there is an issue open for this)
4. remove unused libraries
5. specify requirements in requirements.in (and, requirements-dev.in, for example) and use pip-compile --output-file requirements.txt requirements.in so that transitive dependencies are then specified in requirements.txt. This allows for deterministic builds and avoids the potential for build fails due to problems in dependent packages, but will need to be incorporated into the workflow. This approach is analogous to committing package-lock.json, pinning the transitive Node dependencies.
6. consider removing lightly used libraries by replacing sections of codebase with functions with custom or built-in equivalents, e.g., pandas, marshmallow-pagination, rdbms-subsetter
7. Stay ahead of library vulnerabilities by incorporating semi-annual library reviews and upgrading packages routinely (and identifying and resolving those with backwards incompatibilities)
8. update documentation where appropriate

See reference implementation in PR #3931

Completion critera

• hold engineering sync to have devs decide on what if any issues above to implement.
• make additional tickets as needed.

[jason-upchurch]

marshmallow-pagination: Latest commit 626a6a9 on Sep 15, 2015

[jason-upchurch]

Working on composing invite list

[jason-upchurch]

engineering sync today at 1:00 p.m.

[jason-upchurch]

Had engineering sync and opened related issues to conduct the work.