Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/ocm/state features #2292

Merged
merged 253 commits into from
Jan 16, 2022
Merged

Feat/ocm/state features #2292

merged 253 commits into from
Jan 16, 2022

Conversation

jdcpni
Copy link
Collaborator

@jdcpni jdcpni commented Jan 16, 2022

• composition.py:

  • _build_predicted_inputs_dict():
    • get input_keys directly from agent_rep INPUT nodes (rather than use state_features)

• controlmechanism.py:

  • Parameters: implement outcome_input_ports

• optimizationcontrolmechanism.py:

  • Parameters: implement state_input_ports
  • self.state_featues: implement as property
  • init():
    • state_features assigned to self.state_feature_specs
  • _validate_state_features(): implement
  • _update_state_input_ports_for_controller():
    • skip if ContextFlags.METHOD -> skip if ContextFlags.PROCESSING
    • validation of state_features moved to _validate_state_features()
    • implement assignment of automatically determined state_features to self.state_features

• test_ocm_state_feature_specs_and_warnings_and_errors():

  • add test for automatic assignment of state_features
  • add test for self.state_features

jdcpni added 30 commits May 5, 2021 07:10
  added feature_input_ports attribute and num_feature_input_ports property
  added feature_input_ports attribute and num_feature_input_ports property

• parameterestimationcomposition.py: fixed misplacement of its Parameters() attribute
  added feature_input_ports attribute and num_feature_input_ports property

• parameterestimationcomposition.py: fixed misplacement of its Parameters() attribute

• optimizationfunctions.py: made num_estimates a Parameter
  - _instantiate_control_signals: random_seeds -> random_seed_mod_values
  - _add_controller:  modifying to instantiate feature_input_ports if none are specified
  - add_controller:  now adds feature_input_ports for Compostion INPUT nodes if not state_features not specified
  - _add_controller:  modifying to instantiate feature_input_ports if none are specified
  - add_controller:  assign simulation_input_ports
  - feature_input_ports -> state_input_ports
  - _instantiate_input_ports():
      state_features only allowed to specifying state_input_ports if
      agent_rep is a CompositionFunctionApproximator (i.e., model-free optimization)

• composition.py:
  - add_controller: adds state_input_ports to shadow INPUT Nodes of Composition
    if controller.agent_rep is Composition (model-based optimziation)
    or state_features have not been specified (for model-free optimizaton)
  _instantiate_input_ports:
     reinstate allowance of state_features specification if agent_rep is a Composition
     (i.e., model-based optimization) as long as they are all INPUT Nodes of agent_rep
  - _gen_llvm_evaluate_function: num_estimates -> num_estimates_per_trial
jdcpni and others added 26 commits December 12, 2021 15:07
  - _determine_node_roles:
     fix bug in which nested comp was prevented from being an OUTPUT Node if,
     in addition to Nodes that qualifed as OUTPUT, it also had nodes that projected
     to Nodes in an outer comp (making it look like it was INTERNAL)
…nUniversity/PsyNeuLink into feat/composition/probes

� Conflicts:
�	tests/composition/test_composition.py
  - add_node(): enforce include_probes_in_output = True for nested Compositions
  - execute():
    - replace return of output_value with get_output_value()
  - correct path ref

• compositioninterfacemechanism.py:
  - docstring fixes
…Link into devel

� Conflicts:
�	psyneulink/core/components/mechanisms/processing/compositioninterfacemechanism.py
  - _update_state_input_ports_for_controller:
    - skip if ContextFlags.METHOD -> ContextFlags.PROCESSING
  - add _validate_state_features
  - _update_state_input_ports_for_controller:
     validation of state_features moved to _validate_state_features
  - Parameters:  add outcome_input_ports and state_input_ports
  - move outcome_input_ports Parameter from ocm to controlmechanism
  - state_features: set to make Parameter on ocm, but waiting to deal with LLVM
  - move outcome_input_ports Parameter from ocm to controlmechanism
  - state_features: set to make Parameter on ocm, but waiting to deal with LLVM
  - correct output for fix of typo

• test_greedy_agent.py:
  - test_predator_prey():  reorder args in state_features to fix failure
@github-actions
Copy link

This PR causes the following changes to the html docs (ubuntu-latest-3.7-x64):

diff -r docs-base/OptimizationControlMechanism.html docs-head/OptimizationControlMechanism.html
880,881c880,882
< <li><p><strong>state_features</strong> (<em>Mechanism</em><em>, </em><a class="reference internal" href="InputPort.html#psyneulink.core.components.ports.inputport.InputPort" title="psyneulink.core.components.ports.inputport.InputPort"><em>InputPort</em></a><em>, </em><a class="reference internal" href="OutputPort.html#psyneulink.core.components.ports.outputport.OutputPort" title="psyneulink.core.components.ports.outputport.OutputPort"><em>OutputPort</em></a><em>, </em><em>Projection</em><em>, </em><em>dict</em><em>, or </em><em>list containing any of these</em>) – specifies the Components from which <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a>
< receive their inputs, the <a class="reference internal" href="InputPort.html#psyneulink.core.components.ports.inputport.InputPort.value" title="psyneulink.core.components.ports.inputport.InputPort.value"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">values</span></code></a> of which are assigned to <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span></code></a> and provided as input to the <code class="xref any docutils literal notranslate"><span class="pre">agent_rep</span></code> method.  See <a class="reference internal" href="#optimizationcontrolmechanism-state-features-arg"><span class="std std-ref">state_features</span></a> for details of specification.</p></li>
---
> <li><p><strong>state_features</strong> (<em>Mechanism</em><em>, </em><a class="reference internal" href="InputPort.html#psyneulink.core.components.ports.inputport.InputPort" title="psyneulink.core.components.ports.inputport.InputPort"><em>InputPort</em></a><em>, </em><a class="reference internal" href="OutputPort.html#psyneulink.core.components.ports.outputport.OutputPort" title="psyneulink.core.components.ports.outputport.OutputPort"><em>OutputPort</em></a><em>, </em><em>Projection</em><em>, </em><em>numeric value</em><em>, </em><em>dict</em><em>, or </em><em>list containing any of these</em>) – specifies the Components from which <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a>
> receive their inputs, the <a class="reference internal" href="InputPort.html#psyneulink.core.components.ports.inputport.InputPort.value" title="psyneulink.core.components.ports.inputport.InputPort.value"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">values</span></code></a> of which are assigned to <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span></code></a> and provided as input to the <code class="xref any docutils literal notranslate"><span class="pre">agent_rep</span></code> method whent it is executed.
> See <a class="reference internal" href="#optimizationcontrolmechanism-state-features-arg"><span class="std std-ref">state_features</span></a> for details of specification.</p></li>
962,963c963,964
< <dd><p>dictionary listing the <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.NodeRole.INPUT" title="psyneulink.core.compositions.composition.NodeRole.INPUT"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">INPUT</span></code></a> <a class="reference internal" href="Composition.html#composition-nodes"><span class="std std-ref">Nodes</span></a> of <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a> (keys) and the specifications in the <strong>state_features</strong>
< argument (values) of the sources of their inputs used to construct <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a>, the values of which are assigned to
---
> <dd><p>dictionary listing the <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.NodeRole.INPUT" title="psyneulink.core.compositions.composition.NodeRole.INPUT"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">INPUT</span></code></a> <a class="reference internal" href="Composition.html#composition-nodes"><span class="std std-ref">Nodes</span></a> of <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.agent_rep"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">agent_rep</span></code></a> (keys) and the source of their inputs (values)
> as specified in <strong>state_features</strong> (or determined automatically), and used to construct <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_input_ports"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_input_ports</span></code></a>, the values of which are assigned to
970c971
< <dd class="field-odd"><p>Dict[Node:Port]</p>
---
> <dd class="field-odd"><p>Dict[Node:source]</p>
1043,1047c1044,1049
< <dd><p>dictionary containing information about the Components corresponding to the values in <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state</span></code></a>.  Keys are (<a class="reference internal" href="Port.html"><span class="doc">Port</span></a>, <a class="reference internal" href="Mechanism.html"><span class="doc">Mechanism</span></a>, <a class="reference internal" href="Composition.html"><span class="doc">Composition</span></a>) tuples, and values are
< the corresponding values in <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state</span></code></a>. The initial entries are for the
< OptimizationControlMechanism’s <a class="reference internal" href="#optimizationcontrolmechanism-state-features"><span class="std std-ref">state features</span></a>, that are the
< sources of its <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span></code></a>;  they are
< followed by entries for the parameters modulated by the OptimizationControlMechanism’s <a class="reference internal" href="#optimizationcontrolmechanism-output"><span class="std std-ref">control_signals</span></a> using the corresponding values of its <a class="reference internal" href="ControlMechanism.html#psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation" title="psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">control_allocations</span></code></a>.</p>
---
> <dd><p>dictionary containing information about the Components corresponding to the values in <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state</span></code></a>.  Keys are (<a class="reference internal" href="Port.html"><span class="doc">Port</span></a>, <a class="reference internal" href="Mechanism.html"><span class="doc">Mechanism</span></a>, <a class="reference internal" href="Composition.html"><span class="doc">Composition</span></a>, index) tuples,
> identifying the source of the value for each item at the corresponding index in
> <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state</span></code></a>, and values are its value in <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state</span></code></a>. The initial entries are for the OptimizationControlMechanism’s
> <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_features" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_features"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state</span> <span class="pre">features</span></code></a>, that are the sources of its
> <a class="reference internal" href="#psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values" title="psyneulink.core.components.mechanisms.modulatory.control.optimizationcontrolmechanism.OptimizationControlMechanism.state_feature_values"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">state_feature_values</span></code></a>;  they are followed
> by entries for the parameters modulated by the OptimizationControlMechanism’s <a class="reference internal" href="#optimizationcontrolmechanism-output"><span class="std std-ref">control_signals</span></a> with the corresponding <a class="reference internal" href="ControlMechanism.html#psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation" title="psyneulink.core.components.mechanisms.modulatory.control.controlmechanism.ControlMechanism.control_allocation"><code class="xref any py py-attr docutils literal notranslate"><span class="pre">control_allocation</span></code></a> values.</p>
1050c1052
< <dd class="field-odd"><p>Dict[node:value]</p>
---
> <dd class="field-odd"><p>Dict[(Port, Mechanism, <a class="reference internal" href="Composition.html#psyneulink.core.compositions.composition.Composition" title="psyneulink.core.compositions.composition.Composition">Composition</a>, index)):value]</p>
diff -r docs-base/searchindex.js docs-head/searchindex.js
1c1
< Search.setIndex({docnames:["AGT","AGTControlMechanism","AutoAssociativeLearningMechanism","AutoAssociativeProjection","AutodiffComposition","BasicsAndPrimer","BotvinickConflictMonitoringModel","BustamanteStroopXORLVOCModel","Cohen_HustonModel","CombinationFunctions","ComparatorMechanism","Compilation","Component","Components","Composition","CompositionFunctionApproximator","CompositionInterfaceMechanism","Compositions","Condition","Context","ContrastiveHebbianMechanism","ContributorsGuide","ControlMechanism","ControlMechanisms","ControlProjection","ControlProjections","ControlSignal","ConventionsAndDefinitions","Core","DDM","DefaultControlMechanism","Defaults","DistributionFunctions","EVC","EVCAuxiliary","EVCControlMechanism","EpisodicMemoryMechanism","Function","Functions","GatingMechanism","GatingMechanisms","GatingProjection","GatingProjections","GatingSignal","GilzenratModel","InputPort","IntegratorFunctions","IntegratorMechanism","IntegratorMechanisms","KWTAMechanism","Keywords","KohonenMechanism","LCAMechanism","LCControlMechanism","LCMechanism","LeabraMechanism","LearningFunctions","LearningMechanism","LearningMechanisms","LearningProjection","LearningProjections","LearningSignal","Library","Log","MappingProjection","MaskedMappingProjection","Mechanism","Mechanisms","MemoryFunctions","Models","ModulatoryMechanism","ModulatoryMechanisms","ModulatoryProjection","ModulatoryProjections","ModulatorySignal","MontagueModel","NieuwenhuisModel","NonStatefulFunctions","ObjectiveFunctions","ObjectiveMechanism","ObjectiveMechanisms","OptimizationControlMechanism","OptimizationFunctions","OutputPort","PCTC_model","ParameterEstimationComposition","ParameterPort","Parameters","Pathway","PathwayProjection","PathwayProjections","Port","PredictionErrorMechanism","Preferences","ProcessingMechanism","ProcessingMechanisms","Projection","Projections","QuickReference","RecurrentTransferMechanism","RefactoredLearningGuide","Registry","RegressionCFA","Report","Scheduler","Scheduling","SelectionFunctions","Services","StatefulFunction","StatefulFunctions","Subystems","Time","TransferFunctions","TransferMechanism","TransferMechanisms","UserDefinedFunction","UserGuide","UserGuide_TBD","Visualization","globals","index","index_logo_with_text","json"],envversion:{"sphinx.domains.c":2,"sphinx.domains.changeset":1,"sphinx.domains.citation":1,"sphinx.domains.cpp":4,"sphinx.domains.index":1,"sphinx.domains.javascript":2,"sphinx.domains.math":2,"sphinx.domains.python":3,"sphinx.domains.rst":2,"sphinx.domains.std":2,sphinx:56},filenames:["AGT.rst","AGTControlMechanism.rst","AutoAssociativeLearningMechanism.rst","AutoAsso
...

See CI logs for the full diff.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.005%) to 84.537% when pulling ae3efed on feat/ocm/state_features into 1f5a70d on devel.

@jdcpni jdcpni merged commit c91a359 into devel Jan 16, 2022
@jdcpni jdcpni deleted the feat/ocm/state_features branch January 16, 2022 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants