EP10 config

[SpacemanPaul]

Reason for this pull request

Ground-up backwards-incompatible rewrite of config engine, as per ODC-EP10 Replace Configuration Layer

Proposed changes

• New Config engine implemented and tested, as per EP.

• Remove old config engine.

• API documentation (in docstrings)

• User documentation (for readthedocs)

• Tests added / passed

• Fully documented, including docs/about/whats_new.rst for all changes

[codecov]

Codecov Report

Attention: 27 lines in your changes are missing coverage. Please review.

Comparison is base (41c33cf) 85.19% compared to head (580c6c6) 85.37%.

Files	Patch %	Lines
datacube/scripts/system.py	59.25%	11 Missing ⚠️
datacube/cfg/opt.py	95.79%	5 Missing ⚠️
datacube/ui/click.py	77.27%	5 Missing ⚠️
datacube/cfg/api.py	97.34%	3 Missing ⚠️
datacube/drivers/postgis/_connections.py	88.88%	1 Missing ⚠️
datacube/drivers/postgres/_connections.py	90.90%	1 Missing ⚠️
datacube/index/_api.py	87.50%	1 Missing ⚠️

Additional details and impacted files

@@               Coverage Diff               @@
##           develop-1.9    #1505      +/-   ##
===============================================
+ Coverage        85.19%   85.37%   +0.18%     
===============================================
  Files              131      137       +6     
  Lines            14551    14784     +233     
===============================================
+ Hits             12396    12622     +226     
- Misses            2155     2162       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[whatnick]

Looks like a decent config update should merge and let errors in deployments pop out.

[omad]

This all looks generally fine to me, thank you Paul! Sorry it's taken me so long to respond.

The documentation is very comprehensive. I think there's room for improvement it making it easy to understand, but that's not at all a blocker for merging.

I made a bunch of documentation fixes already, but there's a few more sphinx niceties I'll add, remove the need for the numbered headings.

Overall, my remaining concern is that this is still more complicated than we need, in terms of how configuration is resolved between searching for files in multiple places, as well as environment variables. But it is at least a bit simpler to think about than before, and the implementation looks a lot better.