Fix CI Pipelines. Drop Python3.6. Bring CI pipelines into harmony with local dev experience. Lint and format entire code base. Accelerate CI pipelines. Update setup.py to correctly define extras packages.

[coltonbh]

Description

Fix CI pipelines. Update code to have reliable builds, linting, formatting, etc...

Changelog description

• Dropped support for dead Python 3.6. Minimum supported Python is now 3.7.
• Updated CONTRIBUTING.md to contain detailed instructions for developers on how to contribute.
• Fixed broken code that failed to prepend the "v" to version numbers.
• Updated CI to run without conda and using only packages defined in setup.py. CI is now much faster and runs the same way for local developers and GitHub Actions.
• Added test.sh and format.sh to devtools/scripts for easy execution of formatting and testing.
• Formatted all code with black. Sorted imports with isort.
• Added pre-commit to repo so code formatting, linting, and testing will all run as part of regular git workflow.

More Details

• Bumped min python version from 3.6->3.7 This is the reason why CI is failing on lint jobs. If you develop locally using python3.6 (which you should, since qcel supports 3.6+) and then push, Lint.yaml github action runs on python3.7. A newer version of black is available on 3.7 that updates some defaults for formatting. This causes CI Lint to automatically fail on the code directly from master that passes lint checks on your local machine. We should drop support for non-supported python versions (python3.6 end of life'd 2021-12-23). Locally you can never install black v23.1.0 because it doesn't support python3.6. Using supported versions of python avoids these types of tricky version conflicts. We need to move with the whole python community on this--we only shoot ourselves in the foot by supporting dead python versions :) This is why other PRs like this one that should be one-liners come with 25 files changed.
• Moved CONTRIBUTING.md to root. Nobody will find it in .github ;P Added detailed instructions for setting up the repo on a local machine for development.
• Added test.sh and format.sh scripts to devtools/scripts This makes it easy for new devs to quickly and idiomatically run tests and format/lint their code. Also quicker for experienced devs too :)
• Added pre-commit hooks. Will automatically lint code and run test as part of a regular git commit/push workflow. Saves devs the time of having to wait for CI to check correctness of code.
• Formatted code with formatting tools using Python3.7 This formats a lot of files, unfortunately, but it's necessary to get CI passing and in harmony with the local dev experience. We need everyone using supported versions of python (3.7+) so install supported versions of black (v23.10++) to avoid these discrepancies between local lint passing and CI failing. I have updated this dependency in setup.py.
• Added ipykernel" to viz extras /tests/test_molecule.test_show() did not work because ipykernel is not installed and needs to be to work with nglview
• Added jsonschema, msgpack to setup.py extras[tests]. Also needed so that all tests can run and pass locally.
• Updated test_pubchem_multiout_g() so it passes PubChem was no longer returning the 193687 molecule due to changing its name. Added a new molecule instead.
• Fixed Codecov bugs. Updated GH Action from v1->v3. Fixed codecov.yaml to be valid yaml and renamed (removed .) so it works with actions.
• Dropped all the conda stuff from GitHub actions in CI. There's no reason for the conda stuff. It's slow, heavy, etc. QCElemental is pure python and needs no non-python packages. Running it in a python env is great. CI is much faster now and easier to understand.

Did not do

• Wanted to add flake8 for much stricter code quality. Repo has too many violations for now. That should be a separate PR.
• Wanted to run mypy to improve code quality and remove errors but currently too many issues. This will be a separate PR.

Future Work

• How does the team feel about switching to poetry for package dev? We can dump setup.py, setup.cfg, and move to fully modern pyproject.yaml. Also encapsulates venv management--easier for new devs, I think. This will also avoid issues like different linter versions causing linting to pass on some machines, fail on others, and conflict with CI.
  • This will also have the added benefit of dropping all the version code that has many, many layers and caused testing issues. We just need one, simple version designation for this package, I think :)
• In some dream world where you guys allow me I'd love to drop a bunch of cruft from qcel--Makefiles, all the layered versioning/versioneer.py stuff, LGTM (which no longer exits), so we have a leaner package that's easier to reason about and easier to maintain. Let me know your thoughts :)

Status

• [x ] Code base linted
• [ x] Ready to go

[codecov]

Codecov Report

Merging #308 (4fddd91) into master (292350f) will decrease coverage by 7.64%.
The diff coverage is 100.00%.

❗ Current head 4fddd91 differs from pull request most recent head 35e43d1. Consider uploading reports for the commit 35e43d1 to get more accurate results

Additional details and impacted files

[loriab]

lgtm, thanks very much for the maintenance! This is a bit more pypi-tools than conda-tools focused, but the repo is pure python, so why not.

I've a couple typos and queries, but merge at will.

[coltonbh]

@loriab will merge in tonight later when I get home. Thanks for accepting and looking forward to getting the repo a bit leaner on the next rev too :)

[coltonbh]

@loriab I still can't merge because of the "required" status that have to pass. I removed theses names from the CI suite but cannot change their "required" status--this is probably an admin-level task. Can you remove these as "required" or update the required tests to the new names I've used? Thanks!

[loriab]

"required" status has been transferred to new names, so you should be good to merge!