Add compute_mapped

[SimonHeybrock]

This fixes a big UX hurdle when computing results that use mapped nodes.

We will later on consider if this should be integrated more tightly with Pipeline.compute, for now this should serve as a solution that is 80% there an can be used to gather more insights/experience.

[jl-wynen]

Did you consider how you could integrate this functionality into compute? E.g., check whether a given key is mapped and then do the equivalent of compute_series?

[SimonHeybrock]

Something like that, yes.

[jl-wynen]

Ok for a prototype. But in the long run, I would prefer merging this with compute. And I would also prefer not using pandas if possible because users might not use pandas either.

[SimonHeybrock]

Ok for a prototype.

This is not meant to be a prototype.

And I would also prefer not using pandas if possible because users might not use pandas either.

We could use a plain dict, but then we do not have index names (aka dims), so one would need to implement a custom data structure for this, including ultimately something like a multi-index (or, simpler something N-D like Scipp, but that won't work any more as soon as Cyclebane supports groupby). I don't think users would prefer using a custom class from an unknown library over a very well known and package in widespread use. Do you have something else in mind?

But in the long run, I would prefer merging this with compute.

It is not yet clear to me that this is a good idea, so I refrained from expressing a preference at this point. Pipeline.compute can compute multiple keys (returning a dict, instead of a single value). So we would need to return a dict containing one or more pandas.Series (instead of a single Series). Would it be too confusing to get a dict, a Series, or a dict containing Series, depending on the args? It is starting too look like a bad interface. Maybe the methods for one target (or a single mapped target) should be separate from the one computing multiple targets? Edit: Furthermore, I had to add an optional index_names argument, in case there are multiple mapped nodes with the same name, which would be cumbersome to integrate into compute with multiple targets.

[jl-wynen]

This link and the link below should be relative links.

[jl-wynen]

I don't like that this uses a protected attribute of graph. Should get_mapped_node_names be a method of Pipeline?

[SimonHeybrock]

This is definitely something that we need to consider for a lot of other functionality (in particular around planner graph operations) that we intend to implement. Either we need to make those properties accessible on the public interface, or potentially add a lot more methods to Pipeline. Right now I cannot say which is better? So unless you can clearly say which solution should be chosen, I'd like to keep this as it is for now.

[jl-wynen]

I don't know what operations you have in mind that need direct access to the graph. But I would say that we should expose a minimal set of primitive operations that can be composed in, e.g., methods of workflow classes.

[jl-wynen]

Is it a graph or a pipeline? We should be consistent with names.

[jl-wynen]

'name' or 'key'? What is the difference?

[jl-wynen]

Should this check be before filtering by index names? It seems that key refers to a mapped node but that is filtered out by the index names.