Skip to content

Monte Carlo dispersions are not being applied #1164

@ruthubotica

Description

@ruthubotica

**Corresponding pull request: #1165 **

Describe the bug
The dispersions that are set within the Controller() class of the Monte Carlo framework are not actually being applied to the attributes with the setattr() command (line 831 of Controller.py). This is due to the fact that the setattr() command does not work with nested attributes. The issue is evident if a print statement is added right after this command to explicitly print one of the attributes that should have been changed (e.g. even one of the .RNGSeed items). It is clear that even though the print statement above (“Setting attribute … to … on simInstance”) shows the correct value, the attribute has not been changed and retains the initial value.

The issue is not being caught by any of the tests and asserts within the Monte Carlo files because they all check and verify the creation of the desired dispersed values and the items within the modifications list. There are currently no checks/tests implemented to verify that the attributes of the simInstance are actually changed. The PR that is created for this issue proposes a solution to the bug, and also a corresponding functionality and test to compare the JSON file of the simParams (which is written with the values of the desired dispersions) with a new identical JSON file that contains the values of the simInstance attributes after the application of the dispersions.

To reproduce
The effect of this bug is visible already in the documentation of the Monte Carlo example scenarios. Looking at the graphs in the documentation of scenarioMonteCarloAttRW, they all contain three plotted lines corresponding to the three states of the attitude, attitude rate, etc. However, for the example scenario of four runs, they should be containing twelve plotted lines (four runs with three states each). The fact that all the runs use the exact same values, and therefore also output the same results, is even more evident if enabling Bokeh visualization and isolating the plots for each of the runs.

Expected behavior
The expected behavior is that the attributes of the simInstance are being dispersed using the values in the modifications list. Specifically, for the example scenarios using the Monte Carlo framework:

  • scenarioMonteCarloAttRW.py: for a MC setup with 4 runs, the graphs in the documentation (here) should actually look like the image below (12 plotted lines in each graph, corresponding to 4 runs with 3 states each):
Image
  • scenarioBskSimAttFeedbackMC.py: similar issues here. The single graph of the attitude error generated with the code for 4 simulation runs looks like the image below (3 plots for 3 attitude states):
Image

When is should be looking like this (12 plots for 4 runs with 3 states each):
Image

  • scenarioVisualizeMonteCarlo.py: this example uses the data generated from the latest run of the scenarioBskSimAttFeedback. If the attributes are dispersed and set correctly for the scenario and a set of 4 runs, the graphs should look like this:
Image Image

Desktop:

  • OS: Linux with Ubuntu
  • Version: Linux fw13 6.5.0, Ubuntu 22.04.5
  • Python version: 3.10.12

Additional context
A proposed solution is added in the corresponding PR, which methodically loops through the nested "items" in the attribute to be dispersed, in order to apply the attribute correctly. Note that a proposed test has also been made (see bug description) using scenarioBskSimAttFeedbackMC. However, this scenario does not currently pass this test, since even with the proposed solution some attributes are not being dispersed. I have not been able to find the reason why this is happening, but I assume it is due to the way in which the modules are being used in this specific scenario, since the scenarios I have made for my project all pass the test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions