package version compatibility dingo <-> eDisGo

[nesnoj]

A current example
In dingo we require pandas >= 0.17.0, <= 0.19.2 in setup.py, thus with latest dingo the DF in exported pickles have format of v0.19.2. I installed dingo v0.1.3 manually (pip3 install -e ...) and pandas v0.19.2 was installed accordingly.
When importing these pickles in eDisGo with the same version of pandas it throws

ImportError: No module named 'pandas.core.indexes'

What, what?
Surprisingly, with the latest pandas (v0.20.3) it works!
@gplssm, I assume you exported the pickles with pandas >v0.19.2 ? If so, please update the setup.py.

General issue
This example raises the general question of how we can cope with package compatibility. There's a pyPSA-specific discussion in #17. How can we deal with non-overlapping "transitive dependencies" (see article here)? According to this article, this can be fixed by "modifying one or both of your internal dependencies to be less specific in its install_requires" (-> specify a broader version range). @gplssm Great, you already took this into account..
Note: with ongoing development of eDisGo it will probably getting harder to make these ranges overlap...

[birgits]

I propose to update dingo every time eDisGo is updated. We might draft a workflow on "how to update pyPSA version" (including regression test of one grid in dingo to approve it).

I agree with what you said in #17 @nesnoj since we want to stay up to date with pypsa.

As for other dependencies we should always check for cases 1 and 2 and follow the suggestion in case 4 to keep track of why we require a certain version by commenting on the requirements constraints. E.g. in the case of the pandas dependency - why do we set an upper version limit?

[nesnoj]

'cause we cannot make sure that our piece of software will run on a new version unless we've tested it. We cannot rely on backward compatibility.

[birgits]

True, but then you'd have to do this for every package you import. Maybe we could rather test with the newest version before a release and then note somewhere with which versions it was tested? Do you test the lower version limits before a release?

[nesnoj]

1. Yes, testing with the recent versions of all packages is preferable (being the upper limit in setup.py).
2. No we don't. But this is how it should be done (Christmas wish :).

I'm happy to have the regression test in dingo which we can use for testing compatibility. We should something similar in eDisGo...but this is currently a nice-to-have feature.

[gplssm]

Thanks for discussing this meta-topic so much! In general, I agree with your ideas and I like them.

On aspect I don't understand: how does the constraints file work in detail? Maybe someone can explain it during next lunch break....

[gplssm]

Package versions are aligned