Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DTT1 - Iteration 3 - Unit test - Unit tests for the Workflow module #4996

Closed
3 tasks done
Tracked by #4993
fcaffieri opened this issue Feb 21, 2024 · 9 comments · Fixed by #5072
Closed
3 tasks done
Tracked by #4993

DTT1 - Iteration 3 - Unit test - Unit tests for the Workflow module #4996

fcaffieri opened this issue Feb 21, 2024 · 9 comments · Fixed by #5072
Assignees
@mhamra
Labels
level/task Task issue type/enhancement

Comments

@fcaffieri
Copy link
Member

fcaffieri commented Feb 21, 2024 •
edited

Epic: #4993

Description

This issue aims to develop unit tests for the Workflow module and automate their execution.

Tasks

  • Generate test cases
  • Validate generated test cases
  • Automate with GH actions
@fcaffieri fcaffieri mentioned this issue Feb 21, 2024
Open
3 tasks
@mhamra mhamra self-assigned this Feb 23, 2024
@mhamra mhamra mentioned this issue Feb 27, 2024
Open
@mhamra
Copy link
Member

mhamra commented Feb 29, 2024

UPDATE

I've developed new unit tests and committed to the repository:

  • test_workflow_processor.py::test_execute_task
  • test_workflow_processor.py::test_execute_task_ko
  • test_workflow_processor.py::test_create_task_object
  • test_workflow_processor.py::test_create_task_object_ko
  • test_workflow_processor.py::test_execute_tasks_parallel

@mhamra
Copy link
Member

mhamra commented Mar 1, 2024

UPDATE

I've developed new unit tests and committed to the repository:

  • test_workflow_processor.py::test_execute_tasks_parallel_ko
  • test_dag.py::test_cancel_dependant_tasks

@mhamra
Copy link
Member

mhamra commented Mar 4, 2024

UPDATE

  • i've reviewed and implemented the unit tests for WorkflowProcessor.run and WorkflowProcessor.execute_tasks_parallel

@mhamra mhamra linked a pull request Mar 4, 2024 that will close this issue
DTT1 iteration 3 workflow engine unit tests #5072
Merged
@mhamra
Copy link
Member

mhamra commented Mar 5, 2024 •
edited

UPDATE

  • I've finished the implementation of the unit tests for WorkflowProcessor
  • I've implemented the following unit tests of the WorkflowFile class:
    • test_workflow_file_constructor
    • test_workflow_file_validate_schema
    • test_workflow_file_validate_schema_ko
    • test_workflow_file_load_workflow
    • test_workflow_file_load_workflow_ko
    • test_workflow_file_process_workflow
    • test_workflow_file_process_workflow_ok

@mhamra
Copy link
Member

mhamra commented Mar 6, 2024

UPDATE

I've implemented the following unit tests of the WorkflowFile Class:

  • test_workflow_file_replace_placeholder
  • test_workflow_file_expand_task

@mhamra
Copy link
Member

mhamra commented Mar 7, 2024 •
edited

UPDATE

I've finished developing the unit tests and tried to execute them in a clean environment. I've installed Python 3.10 in Ubuntu, the dependencies of the workflow_engine module, and configured the PYTHONPATH to run the tests.

pytest output: 
root@1eecf7be350d:/wazuh-qa/deployability/modules# pytest -vv workflow_engine/
=================================================================================== test session starts ====================================================================================
platform linux -- Python 3.10.12, pytest-8.0.2, pluggy-1.4.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /wazuh-qa/deployability/modules
collected 94 items                                                                                                                                                                         

workflow_engine/tests/test_dag.py::test_dag_constructor[True] PASSED                                                                                                                 [  1%]
workflow_engine/tests/test_dag.py::test_dag_constructor[False] PASSED                                                                                                                [  2%]
workflow_engine/tests/test_dag.py::test_dag_is_active[True-dag0] PASSED                                                                                                              [  3%]
workflow_engine/tests/test_dag.py::test_dag_is_active[True-dag1] PASSED                                                                                                              [  4%]
workflow_engine/tests/test_dag.py::test_dag_is_active[False-dag0] PASSED                                                                                                             [  5%]
workflow_engine/tests/test_dag.py::test_dag_is_active[False-dag1] PASSED                                                                                                             [  6%]
workflow_engine/tests/test_dag.py::test_get_execution_plan[dag0] PASSED                                                                                                              [  7%]
workflow_engine/tests/test_dag.py::test_set_status[task1-failed-dag0] PASSED                                                                                                         [  8%]
workflow_engine/tests/test_dag.py::test_set_status[task1-canceled-dag0] PASSED                                                                                                       [  9%]
workflow_engine/tests/test_dag.py::test_set_status[task1-successful-dag0] PASSED                                                                                                     [ 10%]
workflow_engine/tests/test_dag.py::test_set_status[task1-non_existing_status-dag0] FAILED                                                                                            [ 11%]
workflow_engine/tests/test_dag.py::test_set_status[non_existing_task-successful-dag0] PASSED                                                                                         [ 12%]
workflow_engine/tests/test_dag.py::test_set_status[non_existing_task-non_existing_status-dag0] FAILED                                                                                [ 13%]
workflow_engine/tests/test_dag.py::test_should_be_canceled[True-dag0] PASSED                                                                                                         [ 14%]
workflow_engine/tests/test_dag.py::test_should_be_canceled[False-dag0] PASSED                                                                                                        [ 15%]
workflow_engine/tests/test_dag.py::test_build_dag[dag0] PASSED                                                                                                                       [ 17%]
workflow_engine/tests/test_dag.py::test_build_dag[dag1] PASSED                                                                                                                       [ 18%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task1-abort-all-to_be_canceled0-dag0] PASSED                                                                          [ 19%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task1-abort-all-to_be_canceled0-dag1] PASSED                                                                          [ 20%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task1-abort-related-flows-to_be_canceled1-dag0] FAILED                                                                [ 21%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task1-abort-related-flows-to_be_canceled1-dag1] FAILED                                                                [ 22%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task1-continue-to_be_canceled2-dag0] FAILED                                                                           [ 23%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task1-continue-to_be_canceled2-dag1] FAILED                                                                           [ 24%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task2-abort-all-to_be_canceled3-dag0] FAILED                                                                          [ 25%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task2-abort-all-to_be_canceled3-dag1] FAILED                                                                          [ 26%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task2-abort-related-flows-to_be_canceled4-dag0] FAILED                                                                [ 27%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task2-abort-related-flows-to_be_canceled4-dag1] FAILED                                                                [ 28%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task2-continue-to_be_canceled5-dag0] FAILED                                                                           [ 29%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task2-continue-to_be_canceled5-dag1] FAILED                                                                           [ 30%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task5-abort-all-to_be_canceled6-dag0] PASSED                                                                          [ 31%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task5-abort-all-to_be_canceled6-dag1] PASSED                                                                          [ 32%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task5-abort-related-flows-to_be_canceled7-dag0] PASSED                                                                [ 34%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task5-abort-related-flows-to_be_canceled7-dag1] PASSED                                                                [ 35%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task5-continue-to_be_canceled8-dag0] FAILED                                                                           [ 36%]
workflow_engine/tests/test_dag.py::test_cancel_dependant_tasks[task5-continue-to_be_canceled8-dag1] FAILED                                                                           [ 37%]
workflow_engine/tests/test_dag.py::test_create_execution_plan[dag0-exec_plan0] PASSED                                                                                                [ 38%]
workflow_engine/tests/test_dag.py::test_create_execution_plan[dag1-exec_plan1] PASSED                                                                                                [ 39%]
workflow_engine/tests/test_schema_validator.py::test_schema_validator_constructor[logger_mock0] PASSED                                                                               [ 40%]
workflow_engine/tests/test_schema_validator.py::test_schema_validator_constructor_ko[logger_mock0] PASSED                                                                            [ 41%]
workflow_engine/tests/test_schema_validator.py::test_preprocess_data PASSED                                                                                                          [ 42%]
workflow_engine/tests/test_schema_validator.py::test_preprocess_data_ko[wf-ko-no-path-on-do.yml-Missing required properties in 'with' for task: {'task': 'run-agent-tests-{agent}'] PASSED [ 43%]
workflow_engine/tests/test_schema_validator.py::test_preprocess_data_ko[wf-ko-no-path-on-cleanup.yml-Missing required properties in 'with' for task: {'task': 'allocate-manager'] PASSED [ 44%]
workflow_engine/tests/test_schema_validator.py::test_validate_schema PASSED                                                                                                          [ 45%]
workflow_engine/tests/test_schema_validator.py::test_validate_schema_ko[logger_mock0] PASSED                                                                                         [ 46%]
workflow_engine/tests/test_task.py::test_process_task_constructor[task0] PASSED                                                                                                      [ 47%]
workflow_engine/tests/test_task.py::test_process_task_execute[task0] PASSED                                                                                                          [ 48%]
workflow_engine/tests/test_task.py::test_process_task_execute[task1] PASSED                                                                                                          [ 50%]
workflow_engine/tests/test_task.py::test_process_task_execute[task2] PASSED                                                                                                          [ 51%]
workflow_engine/tests/test_task.py::test_process_task_execute[task3] PASSED                                                                                                          [ 52%]
workflow_engine/tests/test_task.py::test_process_task_execute[task4] PASSED                                                                                                          [ 53%]
workflow_engine/tests/test_task.py::test_process_task_execute_ko[subproc_run_exc0-1-task0] FAILED                                                                                    [ 54%]
workflow_engine/tests/test_task.py::test_process_task_execute_ko[subproc_run_exc0-0-task0] FAILED                                                                                    [ 55%]
workflow_engine/tests/test_task.py::test_process_task_execute_ko[subproc_run_exc1-1-task0] PASSED                                                                                    [ 56%]
workflow_engine/tests/test_task.py::test_process_task_execute_ko[subproc_run_exc1-0-task0] PASSED                                                                                    [ 57%]
workflow_engine/tests/test_task.py::test_process_task_execute_ko[subproc_run_exc2-1-task0] PASSED                                                                                    [ 58%]
workflow_engine/tests/test_task.py::test_process_task_execute_ko[subproc_run_exc2-0-task0] PASSED                                                                                    [ 59%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_constructor PASSED                                                                                                   [ 60%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_validate_schema ERROR                                                                                                [ 61%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_validate_schema_ko[logger_mock0] PASSED                                                                              [ 62%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_load_workflow[logger_mock0] PASSED                                                                                   [ 63%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_load_workflow_ko[logger_mock0] PASSED                                                                                [ 64%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_process_workflow[logger_mock0] PASSED                                                                                [ 65%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_process_workflow_ok[logger_mock0] PASSED                                                                             [ 67%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_replace_placeholder[element0-values0-return_value0] PASSED                                                           [ 68%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_replace_placeholder[element1-values1-return_value1] PASSED                                                           [ 69%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_replace_placeholder[string_element {value}-values2-string_element value] PASSED                                      [ 70%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_replace_placeholder[element3-None-return_value3] PASSED                                                              [ 71%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_expand_task[task0-return_value0-variables0] PASSED                                                                   [ 72%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_expand_task[task1-return_value1-variables1] PASSED                                                                   [ 73%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_static_workflow_validation PASSED                                                                                    [ 74%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_static_workflow_validation_ko[task_collection0-Duplicated task names: task 1] PASSED                                 [ 75%]
workflow_engine/tests/test_workflow_file.py::test_workflow_file_static_workflow_validation_ko[task_collection1-Tasks do not exist: task 3, task 4] PASSED                            [ 76%]
workflow_engine/tests/test_workflow_processor.py::test_workflow_processor_constructor[workflow.yaml-False-1-info-schema.yaml] PASSED                                                 [ 77%]
workflow_engine/tests/test_workflow_processor.py::test_workflow_processor_constructor[workflow.yaml-True-1-debug-schema.yaml] PASSED                                                 [ 78%]
workflow_engine/tests/test_workflow_processor.py::test_workflow_processor_constructor[workflow.yaml-True-1-debug-None] PASSED                                                        [ 79%]
workflow_engine/tests/test_workflow_processor.py::test_execute_task[logger_mock0-w_processor0-dag0-custom_action-True] PASSED                                                        [ 80%]
workflow_engine/tests/test_workflow_processor.py::test_execute_task[logger_mock1-w_processor1-dag1-custom_action-False] PASSED                                                       [ 81%]
workflow_engine/tests/test_workflow_processor.py::test_execute_task_ko[logger_mock0-w_processor0-dag0-KeyboardInterrupt-None] PASSED                                                 [ 82%]
workflow_engine/tests/test_workflow_processor.py::test_execute_task_ko[logger_mock0-w_processor0-dag0-KeyboardInterrupt-abort-all] PASSED                                            [ 84%]
workflow_engine/tests/test_workflow_processor.py::test_execute_task_ko[logger_mock1-w_processor1-dag1-Exception-None] PASSED                                                         [ 85%]
workflow_engine/tests/test_workflow_processor.py::test_execute_task_ko[logger_mock1-w_processor1-dag1-Exception-abort-all] PASSED                                                    [ 86%]
workflow_engine/tests/test_workflow_processor.py::test_create_task_object[w_processor0-process] PASSED                                                                               [ 87%]
workflow_engine/tests/test_workflow_processor.py::test_create_task_object[w_processor0-dummy] PASSED                                                                                 [ 88%]
workflow_engine/tests/test_workflow_processor.py::test_create_task_object[w_processor0-dummy-random] PASSED                                                                          [ 89%]
workflow_engine/tests/test_workflow_processor.py::test_create_task_object_ko[w_processor0] PASSED                                                                                    [ 90%]
workflow_engine/tests/test_workflow_processor.py::test_execute_tasks_parallel[logger_mock0-w_processor0-dag0-False] PASSED                                                           [ 91%]
workflow_engine/tests/test_workflow_processor.py::test_execute_tasks_parallel[logger_mock0-w_processor0-dag0-True] PASSED                                                            [ 92%]
workflow_engine/tests/test_workflow_processor.py::test_execute_tasks_parallel_ko[logger_mock0-w_processor0-dag0-False] PASSED                                                        [ 93%]
workflow_engine/tests/test_workflow_processor.py::test_execute_tasks_parallel_ko[logger_mock0-w_processor0-dag0-True] PASSED                                                         [ 94%]
workflow_engine/tests/test_workflow_processor.py::test_generate_futures[w_processor0] PASSED                                                                                         [ 95%]
workflow_engine/tests/test_workflow_processor.py::test_generate_futures_reverse[w_processor0] PASSED                                                                                 [ 96%]
workflow_engine/tests/test_workflow_processor.py::test_run[logger_mock0-w_processor0-False] PASSED                                                                                   [ 97%]
workflow_engine/tests/test_workflow_processor.py::test_run[logger_mock0-w_processor0-True] PASSED                                                                                    [ 98%]
workflow_engine/tests/test_workflow_processor.py::test_handle_interrupt[logger_mock0-w_processor0] PASSED                                                                            [100%]

========================================================================================== ERRORS ==========================================================================================

I've started developing the GitHub Action to automate the unit test execution.

@mhamra mhamra mentioned this issue Mar 8, 2024
Merged
@mhamra
Copy link
Member

mhamra commented Mar 8, 2024

UPDATE

  • I've implemented the GitHub workflow to run checks workflow_engine unit tests when a new commit changes the deployability/modules/workflow_engine folder.
  • I've run the coverage tool to report coverage of the current unit tests
    ========================================================================== 16 failed, 77 passed, 1 error in 0.74s
Coverage report 
==========================================================================
(wazuh-qa) marcelo@vallese-w11:~/wazuh/wazuh-qa/deployability/modules/workflow_engine$ coverage report
Name                               Stmts   Miss  Cover
------------------------------------------------------
__init__.py                            1      0   100%
logger/__init__.py                     0      0   100%
logger/logger.py                      14      0   100%
schema_validator.py                   44      1    98%
task.py                               57     12    79%
tests/conftest.py                     44      1    98%
tests/test_dag.py                     75      1    99%
tests/test_schema_validator.py        60      0   100%
tests/test_task.py                    42      0   100%
tests/test_workflow_file.py          105     11    90%
tests/test_workflow_processor.py     136      0   100%
workflow_processor.py                229     13    94%
------------------------------------------------------
TOTAL                                807     39    95%

@rauldpm rauldpm added level/task Task issue and removed level/subtask Subtask issue labels Mar 20, 2024
@rauldpm
Copy link
Member

rauldpm commented Apr 8, 2024 •
edited

Moved the ETA to 2024/04/09, this issue is pending final review

@rauldpm rauldpm mentioned this issue Apr 8, 2024
Open
11 tasks
@fcaffieri
Copy link
Member Author

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mhamra mhamra

Labels
level/task Task issue type/enhancement
Projects
Release 4.9.0
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

DTT1 iteration 3 workflow engine unit tests wazuh/wazuh-qa
3 participants
@mhamra @rauldpm @fcaffieri