diff --git a/examples/06_Combining Models.ipynb b/examples/06_Combining Models.ipynb index a568c54d..272855da 100644 --- a/examples/06_Combining Models.ipynb +++ b/examples/06_Combining Models.ipynb @@ -4,21 +4,33 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Combining Prognostic Models" + "# 6. Combining Prognostic Models" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "This section demonstrates how prognostic models can be combined. There are two times in which this is useful: \n", + "This section demonstrates how prognostic models can be combined. There are two instances in which this is useful: \n", "\n", - "1. When combining multiple models of different inter-related systems into one system-of-system model (i.e., [Composite Models](https://nasa.github.io/progpy/api_ref/prog_models/CompositeModel.html)), or\n", + "1. Combining multiple models of different inter-related systems into one system-of-system model (i.e., [Composite Models](https://nasa.github.io/progpy/api_ref/prog_models/CompositeModel.html)), or\n", "2. Combining multiple models of the same system to be simulated together and aggregated (i.e., [Ensemble Models](https://nasa.github.io/progpy/api_ref/prog_models/EnsembleModel.html) or [Mixture of Expert Models](https://nasa.github.io/progpy/api_ref/progpy/MixtureOfExperts.html)). This is generally done to improve the accuracy of prediction when you have multiple models that each represent part of the behavior or represent a distribution of different behaviors. \n", "\n", "These two methods for combining models are described in the following sections." ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Table of Contents\n", + "\n", + "* [Composite Model](#Composite-Model)\n", + "* [Ensemble Model](#Ensemble-Model)\n", + "* [Mixture of Experts (MoE)](#Mixture-of-Experts-(MoE))\n", + "* [Conclusion](#Conclusion)" + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -30,11 +42,11 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "A CompositeModel is a PrognosticsModel that is composed of multiple PrognosticsModels. This is a tool for modeling system-of-systems. i.e., interconnected systems, where the behavior and state of one system affects the state of another system. The composite prognostics models are connected using defined connections between the output or state of one model, and the input of another model. The resulting CompositeModel behaves as a single model.\n", + "A `CompositeModel` is a `PrognosticsModel` that is composed of multiple `PrognosticsModels`. This is a tool for modeling system-of-systems. (i.e., interconnected systems), where the behavior and state of one system affects the state of another system. The composite prognostics models are connected using defined connections between the output or state of one model, and the input of another model. The resulting `CompositeModel` behaves as a single model.\n", "\n", - "To illustrate this, we will create a composite model of an aircraft's electric powertrain, combining the DCMotor, ESC, and PropellerLoad models. The Electronic Speed Controller (ESC) converts a commanded duty (i.e., throttle) to signals to the motor. The motor then acts on the signals from the ESC to spin the load, which enacts a torque on the motor (in this case from air resistence).\n", + "To illustrate this, we will create a composite model of an aircraft's electric powertrain, combining the `DCMotor`, `ESC`, and `PropellerLoad` models. The Electronic Speed Controller (`ESC`) converts a commanded duty (i.e., throttle) to signals to the motor. The motor then acts on the signals from the ESC to spin the load, which enacts a torque on the motor (in this case from air resistence).\n", "\n", - "First we will import the used models, and the CompositeModel class" + "First we will import the used models, and the `CompositeModel` class." ] }, { @@ -69,7 +81,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next we have to define the connections between the systems. Let's first define the connections from the DCMotor to the propeller load. For this, we'll need to look at the DCMotor states and understand how they influence the PropellerLoad inputs." + "Next we have to define the connections between the systems. Let's first define the connections from the `DCMotor` to the propeller load. For this, we'll need to look at the `DCMotor` states and understand how they influence the `PropellerLoad` inputs." ] }, { @@ -86,9 +98,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Each of the states and inputs are described in the model documentation at [DC Motor Docs](https://nasa.github.io/progpy/api_ref/prog_models/IncludedModels.html#dc-motor) and [Propeller Docs](https://nasa.github.io/progpy/api_ref/prog_models/IncludedModels.html#propellerload)\n", - "\n", - "From reading the documentation we understand that the propeller's velocity is from the motor, so we can define the first connection:" + "Each of the states and inputs are described in the model documentation at [DC Motor Docs](https://nasa.github.io/progpy/api_ref/prog_models/IncludedModels.html#dc-motor) and [Propeller Docs](https://nasa.github.io/progpy/api_ref/prog_models/IncludedModels.html#propellerload). From reading the documentation we understand that the propeller's velocity is from the motor, so we can define the first connection:" ] }, { @@ -106,7 +116,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Connections are defined as couples where the first value is the input for the second value. The connection above tells the composite model to feed the DCMotor's v_rot into the PropellerLoad's input v_rot.\n", + "Connections are defined as couples where the first value is the input for the second value. The connection above tells the composite model to feed the `DCMotor`'s `v_rot` into the `PropellerLoad`'s input `v_rot`.\n", "\n", "Next, let's look at the connections the other direction, from the load to the motor." ] @@ -141,7 +151,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now we will repeat the exercise with the DCMotor and ESC." + "Now we will repeat the exercise with the `DCMotor` and `ESC`." ] }, { @@ -183,7 +193,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The resulting model includes two inputs, ESC voltage (from the battery) and duty (i.e., commanded throttle). These are the only two inputs not connected internally from the original three models. The states are a combination of all the states of every system. Finally, the outputs are a combination of all the outputs from each of the individual systems. " + "The resulting model includes two inputs, `ESC` voltage (from the battery) and duty (i.e., commanded throttle). These are the only two inputs not connected internally from the original three models. The states are a combination of all the states of every system. Finally, the outputs are a combination of all the outputs from each of the individual systems. " ] }, { @@ -214,6 +224,7 @@ " (m_esc, m_load, m_motor), \n", " connections=connections,\n", " outputs={'DCMotor.v_rot', 'DCMotor.theta'})\n", + "\n", "print('outputs: ', m_powertrain.outputs)" ] }, @@ -221,7 +232,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now the outputs are only DCMotor angle and velocity.\n", + "Now the outputs are only `DCMotor` angle and velocity.\n", "\n", "The resulting model can be used in simulation, state estimation, and prediction the same way any other model would be, as demonstrated below:" ] @@ -236,12 +247,14 @@ " 'ESC.duty': 1, # 100% Throttle\n", " 'ESC.v': 23\n", " })\n", + "\n", "def future_loading(t, x=None):\n", " return load\n", "\n", "simulated_results = m_powertrain.simulate_to(1, future_loading, dt=2.5e-5, save_freq=2e-2)\n", - "fig = simulated_results.outputs.plot(compact=False, keys=['DCMotor.v_rot'], ylabel='Velocity')\n", - "fig = simulated_results.states.plot(keys=['DCMotor.i_b', 'DCMotor.i_c', 'DCMotor.i_a'], ylabel='ESC Currents')" + "\n", + "fig = simulated_results.outputs.plot(keys=['DCMotor.v_rot'], ylabel='velocity (rad/sec)', xlabel='time (s)', title='Composite model output')\n", + "fig = simulated_results.states.plot(keys=['DCMotor.i_b', 'DCMotor.i_c', 'DCMotor.i_a'], ylabel='ESC currents', xlabel='time (s)', title='Composite model states')" ] }, { @@ -274,15 +287,16 @@ "outputs": [], "source": [ "simulated_results = m_powertrain.simulate_to(1, future_loading, dt=2.5e-5, save_freq=2e-2)\n", - "fig = simulated_results.outputs.plot(compact=False, keys=['DCMotor.v_rot'], ylabel='Velocity')\n", - "fig = simulated_results.states.plot(keys=['DCMotor.i_b', 'DCMotor.i_c', 'DCMotor.i_a'], ylabel='ESC Currents')" + "\n", + "fig = simulated_results.outputs.plot(keys=['DCMotor.v_rot'], ylabel='velocity (rad/sec)', xlabel='time (s)', title='Composite model output with increased load')\n", + "fig = simulated_results.states.plot(keys=['DCMotor.i_b', 'DCMotor.i_c', 'DCMotor.i_a'], ylabel='ESC Currents', xlabel='time (s)', title='Composite model with increased load states')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note: A function can be used to perform simple transitions between models. For example, if you wanted to multiply the torque by 1.1 to represent some gearing or additional load, that could be done by defining a function, as follows" + "Note that a function can be used to perform simple transitions between models. For example, if you wanted to multiply the torque by 1.1 to represent some gearing or additional load, that could be done by defining a function, as follows:" ] }, { @@ -318,7 +332,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's add back in the other connections and build the composite model" + "Now let's add back in the other connections and build the composite model." ] }, { @@ -334,13 +348,16 @@ " ('DCMotor.theta', 'ESC.theta'),\n", " ('DCMotor.v_rot', 'PropellerLoad.v_rot')\n", "])\n", + "\n", "m_powertrain = CompositeModel(\n", " (m_esc, m_load, m_motor, torque_multiplier), \n", " connections=connections,\n", " outputs={'DCMotor.v_rot', 'DCMotor.theta'})\n", + "\n", "simulated_results = m_powertrain.simulate_to(1, future_loading, dt=2.5e-5, save_freq=2e-2)\n", - "fig = simulated_results.outputs.plot(compact=False, keys=['DCMotor.v_rot'], ylabel='Velocity')\n", - "fig = simulated_results.states.plot(keys=['DCMotor.i_b', 'DCMotor.i_c', 'DCMotor.i_a'], ylabel='ESC Currents')" + "\n", + "fig = simulated_results.outputs.plot(keys=['DCMotor.v_rot'], ylabel='velocity (rad/sec)', xlabel='time (s)', title='Complete composite model output')\n", + "fig = simulated_results.states.plot(keys=['DCMotor.i_b', 'DCMotor.i_c', 'DCMotor.i_a'], ylabel='ESC currents', xlabel='time (s)', title='Complete composite model states')" ] }, { @@ -377,6 +394,7 @@ "outputs": [], "source": [ "from progpy.models import BatteryCircuit\n", + "\n", "m_circuit = BatteryCircuit()\n", "m_circuit_2 = BatteryCircuit(qMax = 7860)\n", "m_circuit_3 = BatteryCircuit(qMax = 6700, Rs = 0.055)" @@ -386,7 +404,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's create an EnsembleModel which combines each of these." + "Let's create an `EnsembleModel` which combines each of these." ] }, { @@ -396,6 +414,7 @@ "outputs": [], "source": [ "from progpy import EnsembleModel\n", + "\n", "m_ensemble = EnsembleModel(\n", " models=(m_circuit, m_circuit_2, m_circuit_3))" ] @@ -404,7 +423,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's evaluate the performance of the combined model using real battery data from NASA's prognostic data repository. See 07. Datasets for more detail on accessing data from this repository" + "Now let's evaluate the performance of the combined model using real battery data from [NASA's prognostic data repository](https://nasa.github.io/progpy/api_ref/progpy/DataSets.html)." ] }, { @@ -414,6 +433,7 @@ "outputs": [], "source": [ "from progpy.datasets import nasa_battery\n", + "\n", "data = nasa_battery.load_data(batt_id=8)[1]\n", "RUN_ID = 0\n", "test_input = [{'i': i} for i in data[RUN_ID]['current']]\n", @@ -446,15 +466,14 @@ "metadata": {}, "outputs": [], "source": [ - "t_end = test_time.iloc[-1]\n", - "from matplotlib import pyplot as plt" + "t_end = test_time.iloc[-1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Next we will simulate the ensemble model" + "Next we will simulate the ensemble model." ] }, { @@ -481,6 +500,7 @@ "outputs": [], "source": [ "from matplotlib import pyplot as plt\n", + "\n", "fig = plt.plot(test_time, data[RUN_ID]['voltage'], color='green', label='ground truth')\n", "fig = plt.plot(results_ensemble.times, [z['v'] for z in results_ensemble.outputs], color='red', label='ensemble')\n", "plt.xlabel('Time (s)')\n", @@ -492,7 +512,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The ensemble model actually performs pretty poorly here. This is mostly because there's an outlier model (m_circuit_3). This can be resolved using a different aggregation method. By default, aggregation uses the mean. Let's update the ensemble model to use median and resimulate" + "The ensemble model actually performs pretty poorly here. This is mostly because there's an outlier model (`m_circuit_3`). This can be resolved using a different aggregation method. By default, aggregation uses the mean. Let's update the ensemble model to use median and resimulate." ] }, { @@ -502,9 +522,10 @@ "outputs": [], "source": [ "import numpy as np\n", - "m_ensemble['aggregation_method'] = np.median\n", "\n", + "m_ensemble['aggregation_method'] = np.median\n", "results_ensemble_median = m_ensemble.simulate_to(t_end, future_loading)\n", + "\n", "fig = plt.plot(results_ensemble_median.times, [z['v'] for z in results_ensemble_median.outputs], color='orange', label='ensemble -median')\n", "fig = plt.plot(test_time, data[RUN_ID]['voltage'], color='green', label='ground truth')\n", "fig = plt.plot(results_ensemble.times, [z['v'] for z in results_ensemble.outputs], color='red', label='ensemble')\n", @@ -517,16 +538,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Much better!" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The same ensemble approach can be used with a heterogeneous set of models that have different states.\n", + "Much better! \n", "\n", - "Here we will repeat the exercise using the battery electrochemisty and equivalent circuit models. The two models share one state in common (tb), but otherwise are different" + "The same ensemble approach can be used with a heterogeneous set of models that have different states. Here we will repeat the exercise using the battery electrochemisty and equivalent circuit models. The two models share one state in common (`tb`), but otherwise are different" ] }, { @@ -536,6 +550,7 @@ "outputs": [], "source": [ "from progpy.models import BatteryElectroChemEOD\n", + "\n", "m_electro = BatteryElectroChemEOD(qMobile=7800)\n", "\n", "print('Electrochem states: ', m_electro.states)\n", @@ -617,9 +632,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Mixture of Experts (MoE) models combine multiple models of the same system, similar to Ensemble models. Unlike Ensemble Models, the aggregation is done by selecting the \"best\" model. That is the model that has performed the best over the past. Each model will have a 'score' that is tracked in the state, and this determines which model is best.\n", + "Mixture of Experts (`MoE`) models combine multiple models of the same system, similar to ensemble models. Unlike ensemble models, the aggregation is done by selecting the \"best\" model. That is the model that has performed the best over the past. Each model will have a 'score' that is tracked in the state, and this determines which model is best.\n", "\n", - "To demonstrate this feature we will repeat the example from the ensemble model section, this time with a mixture of experts model. For this example to work you will have had to have run the ensemble model section example.\n", + "To demonstrate this feature we will repeat the example from the ensemble model section, this time with a mixture of experts model. For this example to work, we will have had to have run the ensemble model section above.\n", "\n", "First, let's combine the three battery circuit models into a single mixture of experts model." ] @@ -655,7 +670,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Its states contain all of the states of each model, kept separate. Each individual model comprising the MoE model will be simulated separately, so the model keeps track of the states propogated through each model separately. The states also include scores for each model." + "Its states contain all of the states of each model, kept separate. Each individual model comprising the `MoE` model will be simulated separately, so the model keeps track of the states propogated through each model separately. The states also include scores for each model." ] }, { @@ -671,7 +686,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The MoE model inputs include both the comprised model input, `i` (current) and outputs: `v` (voltage) and `t`(temperature). The comprised model outputs are provided to update the scores of each model when performing state transition. If they are not provided when calling next_state, then scores would not be updated." + "The `MoE` model inputs include both the comprised model input, `i` (current) and outputs: `v` (voltage) and `t`(temperature). The comprised model outputs are provided to update the scores of each model when performing state transition. If they are not provided when calling next_state, then scores would not be updated." ] }, { @@ -687,7 +702,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Now let's evaluate the performance of the combined model using real battery data from NASA's prognostic data repository, downloaded in the previous sections. See 07. Datasets for more detail on accessing data from this repository.\n", + "Now let's evaluate the performance of the combined model using real battery data from [NASA's prognostic data repository](https://nasa.github.io/progpy/api_ref/progpy/DataSets.html).\n", "\n", "To evaluate the model we first create a future loading function that uses the loading from the data." ] @@ -758,7 +773,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Let's take a look at the model scores again" + "Let's take a look at the model scores again." ] }, { @@ -797,13 +812,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The fit here is much better. The MoE model learned which of the three models best fit the observed behavior.\n", + "The fit here is much better. The `MoE` model learned which of the three models best fit the observed behavior.\n", "\n", "In a prognostic application, the scores will be updated each time you use a state estimator (so long as you provide the output as part of the input). Then when performing a prediction the scores aren't updated, since outputs are not known.\n", "\n", "An example of when this would be useful is for cases where there are three common degradation paths or \"modes\" rather than a single model with uncertainty to represent every mode, the three modes can be represented by three different models. Once enough of the degradation path has been observed the observed mode will be the one reported.\n", "\n", - "If the model fit is expected to be stable (that is, the best model is not expected to change anymore). The best model can be extracted and used directly, like demonstrated below." + "If the model fit is expected to be stable (that is, the best model is not expected to change anymore). The best model can be extracted and used directly, as demonstrated below:" ] }, { @@ -820,14 +835,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Conclusions" + "## Conclusion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "In this section we demonstrated a few methods for treating multiple models as a single model. This is of interest when there are multiple models of different systems which are interdependent (CompositeModel), multiple models of the same system that portray different parts of the behavior or different candidate representations (EnsembleModel), or multiple models of the same system that represent possible degradation modes (MixtureOfExpertModel)." + "In this section we demonstrated a few methods for treating multiple models as a single model. This is of interest when there are multiple models of different systems which are interdependent (`CompositeModel`), multiple models of the same system that portray different parts of the behavior or different candidate representations (`EnsembleModel`), or multiple models of the same system that represent possible degradation modes (`MixtureOfExpertModel`).\n", + "\n", + "The next notebook __[07 State Estimation](07_State%20Estimation.ipynb)__ will be exploring state estimation, which is the process of estimating the current state of the system using sensor data and a prognostics model." ] } ],