Add custom class for handling configuration dictionaries

[brews]

Idea from Rising to have implactlab_tools.utils.files.get_allargv_config() return a specialized dictionary for run configuration data. This new feature is also an onboarding project for @brews.

The rules for these configs for the projection system are (paraphrasing):

1. A configuration file may contain multiple levels of configuration: dictionaries within other dictionaries, and so on.

2. Different parts of the code are passed different parts of the configuration. This has two use cases: (1) because they are meant to use a specific sub-level of the configuration (e.g., a function may just want the sub-dictionary that contains weather variable definitions), or (2) when different parts of the config. initialize different parts of the calculation, so the same function is called with different parts at different times. The config. dictionary is usually included in a function call as an optional config argument.

3. Dictionary keys are "inherited" into all lower levels. So, when we access a sub-level of the dictionary, what we actually use as the configuration dictionary at that level contains all of the keys of that sub-level, plus any keys of the parent level(s) that were not over-written by the sub-level. The code that currently creates this combined config. is merge(parent, child) in https://bitbucket.org/ClimateImpactLab/impact-calculations/src/master/interpret/configs.py.

4. Key access should be case-insensitive and underscore-vs.-dash insensitive. This is not fully implemented in the current system.

Specifically looking for these features in the new class:

1. It looks just like a dict object, although I think only the getitem, setitem, and iter methods need to be defined.

2. It will always be initialized by a dictionary.

3. It silently implements (3) and (4) above. I suggest that (3) is implemented by defining a parent attribute for each ConfigurationDictionary, which is accessed if the key is not found in the present dictionary. This will be useful for the next item.

4. Whenever a key is accessed, the inspect.stack() is stored, associated with that key. Only the most recent is needed.

5. A merge method, like the one from the code above, which can be used to merge two different configurations. Probably this can be implemented using by just making them both ConfigurationDictionarys, and setting the parent attribute on one of them. Make sure that a merge does not cause all keys to be accessed in the sense used in 4.

6. A method on the object, "assert_completelyused" would assert that every key in the config. tree has been accessed.

Also need to write documentation and tests.

[brews]

Yesterday I wrote an implementation for this that combines link-lists with dictionaries. It ticks all of the boxes above but I feel there this could be a really elegant usecase for collections.ChainMap. I got something implemented with ChainMap but it doesn't handle the key logging/inspect.trace storage without doing weird things. Might be something worth revisiting if we move away from that key/trace feature...

[brews]

Here is a brief summary of the solution I have so far in PR #437. This is a first pass so please speak up if something is broken here or if something is FUBAR.

Key functions in impactlab_tools.utils.files now return a ConfigDict instance with the appropriate parent/child relationship between nested dictionaries. Dicts keeps track of accessed which keys have been accessed along with a inspect.stack() snapshot when it's accessed. This is kept in self.key_access_stack. It only keeps track of local keys, so if a child config pulls a key from its parent, that access is noted in the parent.

self.accessed_all_keys() will return a bool indicating whether all keys have been accessed. This recursively checks all parent dictionaries -- checking the full config tree up to self.

self.merge() lets you merge in other ConfigDict instances. Use the optional xparent arg to decide which parent to use for the resulting merged dict. This is really more of an "update" than a "merge" but whatever...