Remaining design issues re: Components

[rjplevin]

I'll try to explain these here, but it might require meeting to flesh it out adequately.

A. External params

External params are created when a parameter is set using set_param!. Following the legacy code, the external param is named the same as the "internal" param defined in @defcomp. This creates a namespace conflict, as multiple leaf components can define a parameter with the same name. With non-component ("flat") models, we simply avoided this issue since the modeler controlled all the components. This is no longer adequate for "third-party" components.

In the old structure, external params were stored in the ModelDef. The current component implementation stores them in the "parent" composite object, which may be a ModelDef or any CompositeComponentDef. This, however, duplicates the namespace clash.

Similarly, backups are indicated only by a single Symbol, with potential namespace conflicts.

A few solutions occur to me:

1. Use as a key to reference parameters and backups a ParameterDefReference (an immutable struct with a ComponentPath and Symbol rather than just a Symbol) and keep the current structure. Use ParameterDefReference to identify backups as well.

2. Move the dict of external params into the leaf component in which the param is defined, rather than in the "parent" composite. Within the component, a name::Symbol is unique.

3. Drop the external_params dict altogether and add (i) a flag to ParameterDef to indicate which params are "external", and (ii) a ModelParameter` slot to store the parameter value.

The last 2 still require addressing backups, but I like the idea of dropping the external_params dict and using the existing structure instead.

B. Visibility of vars and params

In the current composite design, exporting vars/params serves two functions:

1. Limiting visibility to the exported data items
2. Renaming items referenced in a composite to handle namespace collisions

We discussed limiting visibility to exported vars/params, but it's not clear to me when this would occur. The current composite implementation allows setting params and connecting vars & params anywhere in the composite structure, if you know the full "path" to a component.

We could treat composites as essentially opaque, with only explicitly exported vars/pars visible to the "consumer" of the component. This seems potentially annoying though, if a needed data item is not exported by the component developer.

We might consider a different approach, where a composite developer or end user can define an "alias" for any var/param occurring anywhere in the composite structure, and attach these at any composite level.

This would allow an alias to a parameter :foo in the component at ComponentPath("/top/A/B/C") to be stored as, say, :foo_C in any other composite. This is similar to the "export" idea except that end users can use them to reference data in pre-existing components. The user would have the option of referencing data using the full path, or via an alias.

Where exports exposed data elements only to the next composite level, aliases would allow a developer of a multi-level component to add aliases to the top level referencing data anywhere in the structure. We could add creation of aliases to @defcomposite, allowing them to refer to anything data items below the composite, using a relative component path.

[corakingdon]

Hi Rich, I've started looking at this a bit more. I don't have very strong thoughts on either of these points that you've outlined, so hopefully we will get to discuss them more tomorrow!

I started trying to make a composite version of MimiDICE2010 as an exercise for me to further understand the new composite capabilities. One thing that jumps out at me is that I think it might be useful to allow syntax for setting internal connections within the @defcomposite macro. I started trying to outline what that could like like in this PR here: anthofflab/MimiDICE2010.jl#24
I define a "top" composite component that holds "Socioeconomics," "Climate," and "Damages" composite components (all defined in "src/modules/"), and I include potential syntax for making internal connections, which slightly simplifies the actual model construction in get_model_composite in "src/MimiDICE2010.jl".

The possible syntax for making connections between components within a Composite definition that I was thinking of are:

# option 1
radiativeforcing.MAT = co2cycle.MAT

# option 2
connect(radiativeforcing => MAT, co2cycle => MAT)

I think this would make the pre-defined composite components much more useful "out of the box" and could simplify the function calls needed for constructing a model. However, I really have no sense of how feasible the implementation would be, so maybe this is something we could also discuss tomorrow!

This definitely would bring up questions about parameter/variable name exporting related to what you were discussing here.

[rjplevin]

I agree, and I like option 1 a lot. I sometimes forget the direction of connection in "connect", i.e., whether connect(arg1, arg2) mean arg1 = arg2 or arg2 = arg1? The assignment operation makes it abundantly clear.

[davidanthoff]

Each comp defines a namespace. For leaf comps, the namespace has parameters and vars. In composit comps the namespace has child comps, pars and vars.

All unbound pars of child comps in a composite comp are auto-imported into the namespace of the composite comp.

Any child comp is automatically in the namespace of the parent/composite comp.

When you add a comp to a composite comp, you can give it a name in the parent namespace.

One can import variables from child components (including nested child components), into the namespace of the parent composite comp, with a rename:

EXP_VAR = CHILD_COMP1.VAR2
EXP_VAR2 = CHILD_COMP1.COMP2.VAR4

One can override the auto-import for a parameter of a child component, by specifying an alternative name in the parent namespace, like this:

EXP_PAR1 = CHILD_COMP1,PAR2

If two child comps have parameters with the same name, auto import will error.

One can import n params of child comps to a single name in the parent namespace:

EXP_PAR1 = CHILD_COMP1, PAR2, CHILD_COMP2.PAR2, CHILD_COMP3.PAR5, CHILD_COMP3.PAR6

For connections in a composite component, we use this syntax:

COMP1.PAR2 = COMP2.VAR5
COMP1.PAR2 = COMP2.COMP5.VAR2

[rjplevin]

One can import n params of child comps to a single name in the parent namespace:
EXP_PAR1 = CHILD_COMP1, PAR2, CHILD_COMP2.PAR2, CHILD_COMP3.PAR5, CHILD_COMP3.PAR6

I don't understand the semantics of this form.

• Should the first two elements on the right side be dot-delimited (CHILD_COMP1.PAR2) rather than comma-delimited?

• What happens when the user sets a value for EXP_PAR1? Do we propagate the value to all the referenced parameters?

• What if the user tries to set a value in one of the referenced parameters? Do we try to propagate this? If not, the semantics aren't clear since the assumed relationship is broken. Handling this would rather more complicated.

[corakingdon]

• The first one should be dot-delimited: CHILD_COMP1.PAR2
• I think what you said is correct. Setting a value for EXP_PAR1 in this composite component (either from an external data source or from another component) should feed that value to all parameters on the right hand side, even when they have different names
• I'm not sure I know what your asking in this case, can you elaborate an example if I have this wrong? If your asking what happens if later the user tries to set a value for say PAR5, I think that should error, because PAR5 won't be "bubbled up" by this composite, because it is being taken care of by EXP_PAR1. I think the idea of this form is that all of the right hand side parameters are now bound to EXP_PAR1, so there should be no other way of setting their values besides changing the value of EXP_PAR1 in this composite.

[rjplevin]

If your asking what happens if later the user tries to set a value for say PAR5, I think that should error

Yes, that's it.

In the composite version, components have a parent reference to the level above. So when setting a parameter, we could ascend the tree to see whether the parameter in question has been "linked", directly or indirectly (i.e., by linking an imported reference to the leaf component's param), to other parameters, and then error.

That seems reasonable. Any other opinions?

[corakingdon]

That seems reasonable to me!

Although now I'm realizing I'm not sure if we fleshed out what our new design means in terms of setting parameters in child components. In your current composite design, if you have a model with one composite component with the following structure:

@defcomp child1 begin
      some_param = Parameter()
end

@defcomp top begin
     Component(child1)
     # no explicit renaming or connecting of `some_param` from child1, so it should be bubbled up and now be an external parameter of this composite
end

m = Model()
set_dimension!(m, :time, 2000:2100)
add_comp!(m, :top)

are you trying to allow the user to do the following next:

set_param!(m, :child1, :some_param, 5)
#OR 
set_param!(m, "top/child1", :some_param, 5)

I thought based on our design discussion that we would no longer allow either of those. And that the only way to set the value in the child component is through the external param in the top component:

set_param!(m, :top, :some_param, 5)

Not sure if this is what you were asking about or not! I realize this will probably mean some pretty big internal changes on your branch if this is the case. Also I'm not sure if I've interpreted our design plan for this case correctly, @davidanthoff should probably comment about intended functionality here as well

[rjplevin]

This wouldn't work in the current code:

set_param!(m, :child1, :some_param, 5)

You would have to first get a reference to the top that was added to the model (recalling that the variable top now refers to a "template" component that is copied into the model.

model_top = find_comp(m, "/top")
# or 
model_top = m.md[:top]

set_param!(model_top, :child1, :some_param, 5)

This would work, though:

set_param!(m, "top/child1", :some_param, 5)

And that the only way to set the value in the child component is through the external param in the top component:
set_param!(m, :top, :some_param, 5)

The root of the ComponentDef is actually ModelDef, which subclasses CompositeComponentDef. So :some_param would percolate up to m.md, and with delegation, would be directly accessible at the model level as:

set_param!(m, :some_param, 5)

though this is not yet one of the 18 (!) set_param! methods. Should be though given our design decisions.

[corakingdon]

So what I'm trying to say is that I think we don't want to allow:

set_param!(m, "top/child1", :some_param, 5)

Because the modeler shouldn't have access to that parameter in the child1 component anymore, especially because of the renaming possibility (but also in the case when it isn't renamed):

@defcomp child1 begin
      some_param = Parameter()
end

@defcomp top begin
     Component(child1)
     par1 = child1.some_param
end

m = Model()
set_dimension!(m, :time, 2000:2100)
add_comp!(m, :top)

# shouldn't allow this, because :some_param is bound to the :par1 parameter of :top
set_param!(m, "top/child1", :some_param, 5)

[rjplevin]

Given that everything percolates up, either with the original name, or a new one, is there any reason to support set_param!(m, "top/child1", :some_param, 5) at all?

I think we allow only setting parameters at the top of any hierarchy, i.e., when obj.parent === nothing. This may be a ModelDef or a non-ModelDef composite "template".

Perhaps the only user-facing method we need is:

set_param!(obj::AbstractCompositeComponentDef, param_name::Symbol, value, dims=nothing)

The other methods are either deprecated or could become internal _set_param!(...) methods.