pick up the salt shaker

[nesnoj]

Parts of Dingo are not deterministic - for different runs with same input data, different grids are generated. So let's get back control of the salt shaker to gain explicit instead of implicit salt!

1. Review results step by step to determine reasons
2. Probably node order has impact - eventually sort it prior to mv_connect

[nesnoj]

I found it!
..cleaned up the salt in 68c54c5.
closed.

[gplssm]

This solution is valid for a static input data set. But, if something changes there resulting MV grids generated by Dingo may be different.

On the example of changed IDs in the load_area data set:

• It is probable that load_area IDs will change if for example new MV stations are added
• This may lead to different results obtained from the routing in Dingo

Either clearly state that MV grids generated by Dingo belong to one particular input data (or are a part of this data set) or try to keep all IDs constant in the datasets prior to Dingo.
Alternative or: try to iterate over a different column than id within Dingo.

[nesnoj]

Current status:

1. Topology of MV grid including strings diverting from main ring is deterministic.
2. The major part of the MV topology within LV load areas seems to be deterministic (checked with >10 runs). However, in some larger LV load areas there are small parts that differ for several runs.
   Example:

Run 1:

Run 2:

This behaviour is related to the methods in mv_connect_satellites() and mv_connect_stations() (called in MVGridDingo.routing() for every MV grid). If one of the two functions mentioned above is omitted during execution the grid seems to be fully deterministic hence the combination of both functions seem to spill some salt..
Detail from diving into mv_connect_stations(): During connection of stations within a load area, the nearest connection objects for every station are determined using find_nearest_conn_objects(). The returned stack changes for different runs.

Conducted measures:

• The topology of a grid strongly depends on the order in which all single steps are conducted. One important measure is to avoid implicit "salt" (seemingly random results), e.g. by sorting the results of a generator prior to further use. Therefore, in (hopefully) all necessary parts of the code this basic idea was considered. Example: e0c410d
• To compare the result (MV grid's graph) of two different runs there's a basic function compare_graphs(). It can save a graph (using mode='write') as gpickle and compare (using mode='compare') the graph of a second run with the saved one. The comparison is done in a very basic manner and does only include nodes and edges. NetworkX provides some functions to check if two graphs are isomorphic but it fails to hande Dingo's classes (repr()).

Further steps:

• I surmise that all mentioned issues regarding non-determinism (grid topology within load areas as well as branch id (branch ids are not deterministic #97), number of nodes (Salted number of node/buses #98)) are related to the behaviour described above
• Further investigations should start at find_nearest_conn_objects()
• To speed up identifying the "salt" a library could be used to track differences between runs (which one?)

The milestone "Salt shaker" contains all related issues.

Summary: Most wanted Christmas present: A dustpan for all the salt

[mltja]

Fixed or obsolete, add new issues of not deterministic behavior to the salt shaker milestone.