[core only] Development/dependencies dsl

[WilliamHPNielsen]

This is some internal core dev thing.

Introduction

Here is my proposal for a first iteration DSL to describe dependencies.

The idea is simple: let's use what we already have (and then build on that later). What is it that we have? The ParamSpecs. The proposal is therefore to simply write down the ParamSpec in a yaml json file. That is, in all its glory, the DSL.

This PR introduces one two objects and two functions, an InterDependencies object consuming ParamSpecs and RunDescriber object and the functions interdeps_to_yaml and yaml_to_interdeps with methods to_json/from_json. Note that yaml is also supported.

Regarding implementation (to be done in following PRs):

If we want to pursue this, the proposal then includes moving all information from the dependencies and layouts table into a yaml file stored in a column in the runs table (just like snapshot). I'd further propose that we

• Update the table schema
• Move all validation logic of a set of ParamSpec (are all setpoints present, are there cyclic dependencies, etc.) into the InterDependencies object. The Measurement class can then use that.
• Refactor the following methods to no longer query dependencies and layout but read and parse the yaml file instead:
  • get_setpoints
  • get_layout
  • get_dependents
  • get_dependencies
  • get_paramspec
• Extend the ParamSpec as extensions are needed. One extension comes to mind:
  • ~~Give the ParamSpec a run_id (preferably the global one) ~~

@QCoDeS/core, let us see if this can work for us.

[codecov]

Codecov Report

Merging #1227 into master will increase coverage by 0.62%.
The diff coverage is 92.61%.

@@            Coverage Diff             @@
##           master    #1227      +/-   ##
==========================================
+ Coverage   72.84%   73.47%   +0.62%     
==========================================
  Files          74       76       +2     
  Lines        8588     8866     +278     
==========================================
+ Hits         6256     6514     +258     
- Misses       2332     2352      +20

[WilliamHPNielsen]

There is now a little notebook showing the basic idea of object -> yaml and yaml -> object.

[astafan8]

does this module support the case when ParamSpec class changes but older yaml file is read?

[WilliamHPNielsen]

Good point.

[WilliamHPNielsen]

It does now, via versioning. The code contains an example of a somewhat contrived upgrade, but it establishes the principle.

[astafan8]

nice. it looks neat. are the "bullet points" going to be implemented in this PR?

[WilliamHPNielsen]

@astafan8, yes, the idea is to make this PR give us an MVP. Sorry, I think I forgot write that. Let me do it now: we have decided to implement the DSL into the database, meaning that the four bullet points will be covered.

[astafan8]

perhaps a useful note: I was looking at ruamel, and on their pypi page they state that

For production systems you should pin the version being used with ruamel.yaml<=0.15

Does it impact us in any way?

[astafan8]

Sorry that I haven't reviewed the queries and the upgrade algorithm before. The most important comments are about them.

[astafan8]

I think it's good to go!