Integration test coverage

[IgorWounds]

This feature allows us to calculate the integration test coverage of the OpenBB Terminal. It obtains each controller and parses its commands and the parameters of each command. Then, it calculates the difference between all available commands and parameters and those that were used inside the integration test scripts.

To allow for this, a naming convention was put in place for the integration test scripts. The tool was used to improve the coverage of existing integration tests and new ones were added.

To obtain the integration test coverage of a single controller write the following:

from openbb_terminal.core.integration_tests.integration_test_coverage import get_coverage_single_controller

get_coverage_single_controller(
    'openbb_terminal.futures.futures_controller',
    '/futures/test_futures.openbb',
    )

To obtain coverage for all controllers do this:

from openbb_terminal.core.integration_tests.integration_test_coverage import get_coverage_all_controllers

get_coverage_all_controllers()

To run the above command from the terminal write:

openbb -t --coverage

At the end of the terminal command run, which tests all controllers, a summary will be printed. This summary is automatically pushed to our Slack on nightly builds via the GitHub Actions workflow.

Note: The submenus that require pre-loaded data (e.g. ta, qa...) have their parameter coverage accuracy hindered due to the controllers not generating choices_default and thus are about 95% accurate.

[codecov]

Codecov Report

Patch coverage: 61.01% and project coverage change: -0.31 ⚠️

Comparison is base (df9a154) 58.29% compared to head (3a355f2) 57.98%.

❗ Current head 3a355f2 differs from pull request most recent head c067fe8. Consider uploading reports for the commit c067fe8 to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4969      +/-   ##
===========================================
- Coverage    58.29%   57.98%   -0.31%     
===========================================
  Files          588      589       +1     
  Lines        53666    53939     +273     
===========================================
- Hits         31283    31275       -8     
- Misses       22383    22664     +281     

Impacted Files	Coverage Δ
.../core/plots/plotly_ta/plugins/volatility_plugin.py	31.37% <0.00%> (-1.29%)	⬇️
...inal/core/sdk/controllers/crypto_sdk_controller.py	0.00% <ø> (ø)
...inal/core/sdk/controllers/stocks_sdk_controller.py	0.00% <ø> (ø)
...penbb_terminal/core/sdk/models/crypto_sdk_model.py	0.00% <0.00%> (ø)
...penbb_terminal/core/sdk/models/stocks_sdk_model.py	0.00% <0.00%> (ø)
openbb_terminal/core/sdk/sdk_helpers.py	36.92% <ø> (ø)
openbb_terminal/core/sdk/sdk_init.py	92.75% <ø> (ø)
openbb_terminal/core/sdk/trailmap.py	92.68% <ø> (ø)
openbb_terminal/dashboards/stream/Forecasting.py	0.00% <0.00%> (ø)
...bb_terminal/dashboards/stream/streamlit_helpers.py	0.00% <0.00%> (ø)
... and 38 more

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.