refactor(ci, build): Simplify build process and developer workflow for running tests and container builds

[m3mike]

What Changed

PR adds some automation and shortcuts to simplify common developer/user usage scenarios and best practices.

• Release Process

  • When a release on the github repository is created, the container will automatically be built and pushed to the Github Container Registry with the version set to the tag, in addition to a latest tag.

• Linting

  • Adds automatic linting locally (via pre-commit), in CI (via super-linter).
  • Adds pre-commit configuration for local running of standard linting tasks (see .pre-commit-config.yaml for details)
  • CI will report on each individual linter separately to easily identify any linting failures (Dockerfiles, shell scripts, python, etc.)

• Makefile

  • Adds Makefile to simplify/standardize setup, build, and usage of product
  • Targets for the following
    • container build (make build-container, automatically adds OCI label with git hash to built image),
    • venv creation (make venv),
    • test execution (make test, via tox),
    • list all make targets and description (either make help or make with no arguments will print summary of make targets available)

• Container

  • Adds check for existing user from entrypoint script, to avoid cycle of overwriting user credentials on subsequent container restarts
  • Django superuser is created on first start
  • Uses module syntax of downloading nltk data (no functional changes)
  • Moves nltk download to dockerfile (from entrypoint)
  • Adds industry-standard OCI labels to Dockerfile, includes reference URL for info, repo URL, description, license, and git hash when built via Makefile
  • Adds .dockerignore
  • Updates Dockerfile to use buildkit
  • Use cache volumes in Dockerfile for apt/pip to speed up subsequent builds

• Docs

  • Adds Add Makefile usage notes to README
  • Update CONTRIBUTING.md doc to reference master branch (develop branch does not exist on repo)
  • Updates docker-compose.yml link in README to point to specific release within git (ties to specific release of file, and makes file location independent from repository structure and branch)

• Misc

  • add tox to test-requirements file
  • alphabetize python requirements, a sort a day keeps the ocd away
  • pin version of django-rest-framework to allow scanning by safety
  • Update ignore files to add coverage files and other standard git ignores

Limitations

• Training data still duplicated on subsequent startups. If not desired, may want to consider ome of the following to resolve:
  • Bundle attackdata loading and initial train into Dockerfile, or
  • Modify pipeline command to add an option/alternative command to only load data if not previously loaded
  • Add standalone database to Docker compose configuration, and do training in separate container instance, so training can be controlled/scaled/modified independent of the web application

[codecov]

Codecov Report

Merging #125 (298266e) into master (435e501) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #125   +/-   ##
=======================================
  Coverage   94.43%   94.43%           
=======================================
  Files          20       20           
  Lines         827      827           
=======================================
  Hits          781      781           
  Misses         46       46           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 435e501...298266e. Read the comment docs.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication