HARMONY-388: Better consistency across Python repos

[vinnyinverso]

What follows are the acceptance criteria and what I found or did to address them. (I also spoke with Chris prior to submitting this PR.)

• Consistent python environment setup and use across the 3 repos
  • For all but the harmony-service example, the READMEs were suggesting to use some combination of pyenv, pyenv-virtualenv, or venv. I tried to update the READMEs so that it was clear to the reader that all of these were acceptable tools.
• Either build scripts or make across the 3 repos
  • All of the regular (non-Docker service) Python module/lib repos (harmony-service-lib, harmony-py) use make files. All of the Docker/service-oriented repos (harmony-netcdf-to-zarr, harmony-service-example) use build scripts because they seem more conducive to the scripting logic needed to satisfy some of the more complex build logic. This seemed like a reasonably consistent solution (makefiles for pure libraries and build scripts for docker services).
• Consistent way of installing deps across the 3 repos
  • All except one repo are using pip for installing dependencies in a fairly consistent manner, with one requirements file for dev, and one for core requirements. The only repo not following this pattern is the harmony-service-example, which uses conda. However, conda is a recommended way of installing one of the main dependencies of this repo (gdal) whereas pip is not (and is difficult to use in this instance from my experience). The use of conda, is also still compatible with the aforementioned recommended Python version manager and virtual env tools.
• Consistent linting across the 3 repos
  • All repos (except for harmony-service-example) use flake8. We could add flake8 to harmony-service-example if desirable. I'm guessing it wasn't added due to the simplicity of the example.
• Consistent unit testing framework and execution across the 3 repos
  • I believe Patrick said there was another ticket addressing this.
• Consistent CI/CD across the 3 repos
  • Much of this seems to be adequately addressed by the recent transition to GitHub and related work.
• Update READMEs across the 3 repos to document the consistent tools, packages, and configuration for setting up a dev environment; dev environment does not presume editor
  • I updated 3 READMEs to give more consistent suggestions on virtual environment setup. None of the dev environment setups assume the user has a specific editor.

[vinnyinverso]

Had to update faker to fix an issue with the github actions test failures in newer versions of Python: joke2k/faker#1439

[chris-durbin]

Since this is a recommendation - what about we just recommend venv built-in with Python rather than pyenv-virtualenv as well?

[vinnyinverso]

The reason I didn't do that is because in a similar discussion a month or so ago, Patrick seemed to allude that we should not be too opinionated on which virtual environment tool is used, as long as there's a virtual environment. So here I was just trying to make the reader aware that they can use either. Also, in the description of this ticket, there was a line about trying to make things "more similar to harmony-py" -- which offers both venv and pyenv-virtualenv as viable tools. Although this alone doesn't make this repo identical to harmony-py, it seemed like a step in that direction, without being too disruptive. I'm ok with either route and think you have a good point -- just explaining my thought process.

[bilts]

I was mostly suggesting that to avoid complicating the end-user documentation, i.e. not mentioning or requiring one as a setup step. It feels akin to telling people which editor to use (though I think we may also do that in other docs...)

[bilts]

Small nit (that I'm not even sure I'm right about): I'm on the fence about whether we should link across repos like this, but if we do, we should probably link in the direction of our dependency arrows. harmony-service-example depends on this repo, so the ENVHELP.md should go in this repo with harmony-service-example pointing to it.

[vinnyinverso]

Yeah, it does feel kinda weird. I think I'll remove it.