Feat/composition/probes (#2236)

* • component.py
  docstring mod to **size**

* • optimizationcontrolmechanism.py:
  added feature_input_ports attribute and num_feature_input_ports property

* • optimizationcontrolmechanism.py:
  added feature_input_ports attribute and num_feature_input_ports property

• parameterestimationcomposition.py: fixed misplacement of its Parameters() attribute

* • optimizationcontrolmechanism.py:
  added feature_input_ports attribute and num_feature_input_ports property

• parameterestimationcomposition.py: fixed misplacement of its Parameters() attribute

• optimizationfunctions.py: made num_estimates a Parameter

* - modified test_mode_based_num_estimates

* -

* -

* • optimizationcontrolmechanism.py:
  - _instantiate_control_signals: random_seeds -> random_seed_mod_values

* • composition.py
  - _add_controller:  modifying to instantiate feature_input_ports if none are specified

* • composition.py:
  - add_controller:  now adds feature_input_ports for Compostion INPUT nodes if not state_features not specified

* -

* • composition.py
  - _add_controller:  modifying to instantiate feature_input_ports if none are specified

* • composition.py:
  - add_controller:  assign simulation_input_ports

* -

* • optimizationcontrolmechanism.py:
  - feature_input_ports -> state_input_ports
  - _instantiate_input_ports():
      state_features only allowed to specifying state_input_ports if
      agent_rep is a CompositionFunctionApproximator (i.e., model-free optimization)

• composition.py:
  - add_controller: adds state_input_ports to shadow INPUT Nodes of Composition
    if controller.agent_rep is Composition (model-based optimziation)
    or state_features have not been specified (for model-free optimizaton)

* -

* • optimizationcontrolmechanism.py:
  _instantiate_input_ports:
     reinstate allowance of state_features specification if agent_rep is a Composition
     (i.e., model-based optimization) as long as they are all INPUT Nodes of agent_rep

* -

* -

* • optimizationcontrolmechanism.py
  - _gen_llvm_evaluate_function: num_estimates -> num_estimates_per_trial

* -

* • optimizationcontrolmechanism.py
  - _gen_llvm_evaluate_function: num_estimates -> num_trial_per_estimate

* • optimizationcontrolmechanism.py
  - _gen_llvm_evaluate_function: num_estimates -> num_trials_per_estimate

* -

* -

* -

* -

* • composition.py
  - __init__: moved controller instantiation until after nodes, projections and pathways

* • composition.py
  - __init__: restored add_controller position

* llvm/struct generation: Make sure num_estimats per trial is always integer

Signed-off-by: Jan Vesely <jan.vesely@rutgers.edu>

* -

* • composition.py:
  - _update_controller: added
  - add_controller and _analyze_graph(): call _update_controller

* -

* • composition.py
  _update_controller:  fixed to loop through all input_ports of comp INPUT nodes

* • test_control.py
  - test_agent_rep_assignement_as_controller_and_replacement:
      updated to test that shadowing projections to state_input_ports
      are properly added and deleted

* • optimizationfunctions.py:
  - _function: refactored to put use aggregation_function at end
  - _grid_evaluate:  still needs to return all_samples

* -

* • composition.py
  - added call to _update_controller to add_node
  - moved test for projections to controller.state_input_ports to run()

* -

* • composition.py:
  moved calls to _update_controller to _complete_init_of_partially_initialized_nodes
  moved _update_controller to ocm._update_state_input_ports

• optimizationcontrolmechanism.py:
  added _update_state_input_ports [**still needed work**]

* • composition.py:
  moved calls to _update_controller to _complete_init_of_partially_initialized_nodes
  moved _update_controller to ocm._update_state_input_ports
  _instantiate_controller_shadow_projections [still needs to be implemented]

• optimizationcontrolmechanism.py:
  added _update_state_input_ports [**still needed work**]

* • composition.py
  added needs_update_controller

* -

* • composition.py:
  - implemented self.needs_update_controller
  - moved implementation of controlsignal projections
    from add_controller to _instantiate_control_projections
    that is called in _complete_init_of_partially_initialized_nodes

Note: still need to set  self.needs_update_controller to False
      after instantiating state_input_ports and projections to them

* -

* -

* -

* -

* -

* -

* -

* -

* • Passing all test_control tests except test_mode_based_num_estimates

* • Passing all test_control tests

* -

* • optimizationcontrolmechanism.py
  - _update_state_input_ports_for_controller: handle nested input nodes

* -

* • optimizationcontrolmechanism.py
  _update_state_input_ports_for_controller:  fixed bug with > 1 INPUT node in Composition

* • test_show_graph.py: passes all tests

* -

* • test_report.py:  passing all tests

* • Passes all tests!

* -

* -

* • composition.py: reorganize with #region and #enregions

* • composition.py: reorganize with #region and #enregions

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* -

* -

* -

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* • controlmechanism.py, optimizationcontrolmechanism.py:
  - _instantiate_monitor_for_control_input_ports -> _parse_monitor_control_input_ports
  - refactored to support allow_probes option on ocm

* -

* • composition.py:
  __init__: move controller to after add_nodes and add_linear_pathway

* -

* - test_control: only test_hanging_control_spec_outer_controller not passing

* -

* -

* -

* -

* -

* -

* • composition.py:
  _instantiate_control_projections:
     weird requirement for double-call to controller._instantiate_control_signal

* • test_paremtercomposition.py:
  restored parameter spec that causes crash ('threshold',Decision2)

* ª Attempt to fix problem with partially overlapping local and ocm control specs

- composition.py
  - _get_control_signals_for_composition:  (see 11/20/21)
      - added (but commented out change) to "if node.controller" to "if not node.controller"
      - changed append to extend
  - _instantiation_control_projection:
      - got rid of try and except double-call to controller._instantiate_control_signals
      -  outdented call to self.controller._activate_projections_for_composition at end

- controlmechanism.py:
    - _check_for_duplicates:  add warning and return duplicates

- optimizationcontrolmechanism._instantiate_control_signals:
    - add call to self.agent_rep._get_control_signals_for_composition() to get local control specs (on mechs in comp)
    - eliminate duplicates with control_signal specs on OCM
    - instantiate local + ocm control_signals

- parameterestimationcomposition.py
  - added context to various calls

* see later commit

* see later commit

* see later commit

* see later commit

* - This branch passes all tests except:
   - test_parameterestimationcomposition
   - test_composition/test_partially_overlapping_control_specs (ADDED IN THIS COMMINT)

- All relevant changes to this branch are marked as "11/21/21."
  However, most are commented out as they break other things.

- The tests above both involve local control specifications (on mechanism within a nested comp)
  and on the OCM for the outer composition, some of which are for the same nested mechs

- Both tests fail with:
   "AttributeError: 'NoneType' object has no attribute '_get_by_time_scale'" (in component.py LINE 3276)
   This may be due to a problem with context setting, since the error is because the modulation Parameter
   of the ControlProjection is returning "None" rather than "multiplicative_param" (when called with get(context)),
   whereas "multiplicative_param" is returned with a call to get() (i.e., with no context specified)

- Most of test_partially_overlapping_control_specs is passed if
   changes marked "11/21/21 NEW" in optimizationcontrolmechanism.py (LINE 1390) are implemented,
   but it does not properly route ControlProjections through parameter_CIMS (see last assert in test).
   Furthermore, test_parameterestimationcompsition fails with the mod param error, even though the
   model has similar structure (i.e., outer composition -- in this case a ParameterEstimationComposition)
   with an OCM that is given control specs that overlap with ones in a nested composition.

- There are also several other things in composition I found puzzling and tried modifying, but that cuased failures:
  - _get_control_signals_for_composition():
      - seems "if node.controller" should be "if **not** node.controller" (emphasis added just for comment)
      - "append" should be "extend"
  - _instantiate_control_projection():
      -  call to self.controller._activate_projections_for_composition (at end of method) should not be indented

* - small mods; don't impact anything relevant to prior commit message

* - small mods; don't impact anything relevant to prior commit message

* - small mods; don't impact anything relevant to prior commit message

* - finished adding formatting regions to composition.py

* -

* • composition.py:
  - rename _check_projection_initialization_status -> _check_controller_initialization_status
  - add _check_nodes_initialization_status(context=context)
    (and calls it with _check_controller_initialization_status)

* • show_graph.py:  addressed bug associated with ocm.allow_direct_probe

* • show_graph.py:  addressed bug associated with ocm.allow_direct_probe

* -

* Composition: add_controller: set METHOD as context source early

* -

* • composition.py
  retore append of control_signals in _instantiate_control_projections()

* • composition.py
  restore append of control_signals in _instantiate_control_projections()

• test_composition.py:
  add test_partially_overlapping_local_and_control_mech_control_specs_in_unnested_and_nested_comp

* • test_partially_overlapping_local_and_control_mech_control_specs_in_unnested_and_nested_comp():
  - added clear_registry() to allow names to be reused in both runs of test

* • composition.py
  docstring:  added projections entry to list of attributes
  - add_controller:  added call to _add_node_aux_components() for controller

* • composition.py
  _add_node_aux_components(): added deletion of item from aux_components if instantiated

* • composition.py
  - comment out _add_node_aux_components() (causing new failures)
  - move _instantiate_control_projections to be with _instantiate_control_projections,
       after self.add_node(self.controller.objective_mechanism (to be more orderly)

* -

* - confirm that it passes all tests exception test_composition/test_partially_overlapping...
  (with addition of _add_aux_components in add_controller commented out)

* • composition.py:  some more fixed to add_controller that now fail only one test:
    - test_agent_rep_assignement_as_controller_and_replacement

* • Passes *all* current tests

* • composition.py:
  - add_controller:  few more minor mods;
  still passes all tests

* -

* -

* -

* • controlmechanism.py:
  - __init__: resrict specification to only one of control, modulatory_signals, or control_signals (synonyms)

* -

* • composition.py:  in progress fix of bug in instantiating shadow projections for ocm.state_input_ports

* • composition.py:
  - _get_original_senders():  added support for nested composition
    needs to be checked for more than one level
    needs to be refactored to be recursive

* • optimizationcontrolmechanism.py
  - _update_state_input_ports_for_controller:  fix invalid_state_features to allow input_CIM of nested comp in agent_rep

* -

* • composition.py
 - _get_original_senders: made recursive

* • test_show_graph.py: update for fixes

* -

* • tests:  passes all in test_show_graph.py and test_report.py

* Passes all tests

* - comment clean-up

* • composition.py
 - add_controller and _get_nested_node_CIM_port:
   added support for forced assignment of NodeRole.OUTPUT for nodes specified in OCM.monitor_for_control,
   but referenced 'allow_probes' attribute still needs to be implemented

* • composition.py, optimizationcontrolmechanism.py:
  allow_probes fully implemented

* • show_graph.py:  fixed bug causing extra projections to OCM

* • composition.py:
  - _update_shadow_projections(): fix handling of deep nesting

* • optimizationcontrolmechanism.py: add agent_rep_type property

* • optimizationcontrolmechanism.py:
  - state_feature_function -> state_feature_functions

* • optimizationcontrolmechanism.py:
  - _validate_params:  validate state_feature_functions
  - _update_state_input_ports_for_controller: implement assignment of state_feature_functions

* -

* -

* • Passes all tests except test_json with 'model_with_control'

* -

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)
  - remove calls to _update_shadows_dict

* • composition.py
  - add_projection:  delete instantiation of shadow projections (handled by _update_shadow_projections)
  - remove calls to _update_shadows_dict

* -

* • test_two_origins_two_input_ports:  crashes on failure of C->B to update

* -

* • composition.py
  - added property shadowing_dict that has shadowing ports as keys and the ports they shadow as values
  - refactored _update_shadowing_projections to use shadowing_dict

* • optimizationcontrolmechanism.py
  - _update_state_input_ports:  modified validations for nested nodes;  still failing some tests

* • optimizationcontrolmechanism.py
  - _update_state_input_ports:
     more careful and informative validation that state_input_ports are in comp or nested comp and are INPUT nodes thereof;
     passes all tests except test_two_origins_two_input_ports as before

* • composition.py
  _get_invalid_aux_components():  defer all shadow projections until _update_shadow_projections

* • composition.py
  _get_invalid_aux_components():  bug fix in test for shadow projections

* Port: _remove_projection_to_port: don't reduce variable below length 1

even ports with no incoming projections have variable at least length 1

* • composition.py
  add_node(): marked (but haven't removed) code block instantiating shadow_projections
   that seems now to be redundant with _update_shadow_projection

* • show_graph.py
  - _assign_cim_components: supress showing projections not in composition

* • composition.py:
  _analyze_graph():  add extra call to _determine_node_roles after _update_shadow_projections
  _run():  moved block of code at beginning initializing scheduler to after
           _complete_init_of_partially_initialized_nodes and _analyze_graph()

• show_graph.py
  - add test to all loops on projections:  "if proj in composition.projection"

* • show_graph.py
  - add show_projections_not_in_composition option for debugging

* • composition.py
  _update_shadow_projections(): delete unused shadow projections and corresponding ports

* • composition.py
  _update_shadow_projections(): fix bug in deletion of unused shadow projections and ports

• test_show_graph:  tests failing, need mods to accomodate changes

* • composition.py:
  _analyze_graph():  add extra call to _determine_node_roles after _update_shadow_projections
  _run():  moved block of code at beginning initializing scheduler to after
           _complete_init_of_partially_initialized_nodes and _analyze_graph()

• show_graph.py
  - add test to all loops on projections:  "if proj in composition.projection"

* • show_graph.py
  fixes; now passes all show_graph tests

* -

* • composition.py
  _update_shadow_projections:  raise error for attempt to shadow INTERNAL Node of nested comp

* -

* -

* • test_composition.py
  implemented test_shadow_nested_nodes that tests shadowing of nested nodes

* -

* -

* -

* -

* • optimizationcontrolmechanism.py: docstring mods

* • composition.py:
  - add allow_probes and exclude_probes_from_output

* • composition.py:
  - docstring mods re: allow_probes

• optimizationcontrolmechanism.py:
  - allow_probes:  eliminate DIRECT setting
  - remove _parse_monitor_for_control_input_ports (no longer needed without allow_probes=DIRECT)

* • composition.py:
  - change "exclude_probes_from_output" -> "include_probes_in_output"

* • composition.py:
  - docstring mods re: allow_probes and include_probes_in_output

* • composition.py:
  - docstring mods re: allow_probes and include_probes_in_output

* • controlmechanism.py:
  - add allow_probes handling (moved from OCM)

• optimizationcontrolmechanism.py:
  - move allow_probes to controlmechanism.py

• composition.py:
  - refactor handling of allow_probes to permit for any ControlMechanism

• objectivemechanism.py:
  - add modulatory_mechanism attribute

* • controlmechanism.py:
  - add allow_probes handling (moved from OCM)

• optimizationcontrolmechanism.py:
  - move allow_probes to controlmechanism.py

• composition.py:
  - refactor handling of allow_probes to permit for any ControlMechanism
  - add _handle_allow_probes_for_control() to reconcile setting on Composition and ControlMechanism

• objectivemechanism.py:
  - add modulatory_mechanism attribute

* • composition.py
  add assignment of learning_mechanism to objective_mechanism.modulatory_mechanism for add_learning methods

* • docstring mods

* -

* -

* • optimizationcontrolmechanism.py: docstring revs

* -

* -

* • test_composition.py:
  - add test_unnested_PROBE
  - add test_nested_PROBES
    TBD: test include_probes_in_output

* -

* • composition.py
  - add_node():  support tuple with required_role

* -

* • composition.py:
  - _determine_node_roles:
     fix bug in which nested comp was prevented from being an OUTPUT Node if,
     in addition to Nodes that qualifed as OUTPUT, it also had nodes that projected
     to Nodes in an outer comp (making it look like it was INTERNAL)

* -

* • composition.py:
  - add_node(): enforce include_probes_in_output = True for nested Compositions
  - execute():
    - replace return of output_value with get_output_value()

* -

Co-authored-by: jdcpni <pniintel55>
Co-authored-by: Jan Vesely <jan.vesely@rutgers.edu>
Co-authored-by: Katherine Mantel <kmantel@princeton.edu>