Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

behave_tests: Implemented basic behave test scenario's for cephadm, ceph shell and OSD commands #41918

Merged
merged 1 commit into from
Aug 25, 2021

Conversation

SudhanAruna
Copy link
Contributor

@SudhanAruna SudhanAruna commented Jun 17, 2021

This commit includes the basic implementation of behave test scenario's ( for cephadm, ceph shell and OSD commands) and
python implementations for interacting with kcli and behave test cases. The test scenarios can be execute all together or only the
specific scenarios with tags.

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@SudhanAruna SudhanAruna changed the title Feature parse behave_tests: Implemented functions to parse feature description and create kcli plan Jun 17, 2021
@SudhanAruna SudhanAruna force-pushed the feature_parse branch 2 times, most recently from ef9a1f4 to e91b8e8 Compare June 17, 2021 14:26
@jmolmo jmolmo requested review from a team, avanthakkar and nizamial09 and removed request for a team June 17, 2021 14:52
@jmolmo jmolmo added the cephadm label Jun 17, 2021
@jmolmo
Copy link
Member

jmolmo commented Jun 17, 2021

Great beginning!

First thing to improve: We need a Tox environment.

Tox allow us to use virtual environments to install dependencies and execute our python scripts without need to install these dependencies in any host that execute our scripts. We can start with a basic tox.ini file placed in the like:

[jolmomar@juanmipc behave_tests]$ cat tox.ini
[tox]
envlist = py36, flake8
skipsdist = true

[testenv]
deps =
   behave
   jinja2
commands = behave

[testenv:flake8]
deps=
   pep8-naming
   flake8
commands=flake8 {posargs:features}

After that you can execute our behave test just typing:

# tox -e py36

and you can reiew your python style problems with:

# tox -e flake8

Note: Obviously, tox must be installed in your laptop

src/behave_tests/README.md Outdated Show resolved Hide resolved
Copy link
Member

@jmolmo jmolmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix the small problem rendering the disk section.

src/behave_tests/features/environment.py Outdated Show resolved Hide resolved
Copy link
Contributor

@tchaikov tchaikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the test based on the behave framework falls into the category of "test", i'd suggest move it to src/test.

@tchaikov

This comment has been minimized.

@SudhanAruna SudhanAruna force-pushed the feature_parse branch 3 times, most recently from 1914f1f to 36aa1a7 Compare June 27, 2021 05:51
@tchaikov tchaikov dismissed their stale review June 27, 2021 05:54

dismissed.

@tchaikov
Copy link
Contributor

tchaikov commented Jul 8, 2021

@SudhanAruna i am not qualified to review this change. i'd defer this to cephadm experts like @jmolmo and @sebastian-philipp .

@tchaikov tchaikov removed their request for review July 8, 2021 15:23
@jmolmo
Copy link
Member

jmolmo commented Jul 9, 2021

Very good progress!!! This starts to seem something really awesome. Good job Arun!!

Apart of the minor suggestions i did..is time to start with a next kind of feature:
Feature: Execute commands using the ceph shell

very similar to what we have in the current feature, but now the commands and the result comes from "cephadm shell"

@SudhanAruna SudhanAruna force-pushed the feature_parse branch 3 times, most recently from 5e872a2 to 5933d0c Compare July 26, 2021 14:52
@SudhanAruna SudhanAruna force-pushed the feature_parse branch 2 times, most recently from bd15bd0 to 1529fe1 Compare August 17, 2021 19:16
print(f"Executed output : {context.output}")


@then("I wait for {wait_time:n} seconds until I get")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to improve this part.. because waiting for nothing is not the right thing to do :-)

What we really need is to execute something and if the expected output is not the desired, wait a little bit, and execute again the command to see if the things have changed.

When can add the decorator in all the functions that must obtain a result a the result maybe is not going to be immediate, the definition of this step could be very similar to the one proposed by you_

@then("I will continue trying to get and compare the result every {wait_time:n} seconds with a timeout of {timeout_time:n} seconds ")

I suggest to create a function to work as decorator. This function will receive as parameter the function that implements the step and returns true if the function success , and false if the parameter function has failed 8 or the timeout expires.
We will need also to change functions to implement each step to return a boolean.

Ex:


@execute_with_timeout(60,5)
def execute_step(context):
     ... 
    return expected_output == executed_output)

def execute_with_timeout(f, timeout, wait)
time_waiting =0
while timeout > time_waiting:
result = f()
if result:
return result
else:
time.sleep(wait)
time_waiting += wait
print("Timeout reached waiting for expected output")
return result

@alfonsomthd
Copy link
Contributor

@SudhanAruna Can you please update the PR description on how to test this?

@SudhanAruna
Copy link
Contributor Author

@SudhanAruna Can you please update the PR description on how to test this?

@alfonsomthd
I have included how to run the test with behave in the README.md file. Should I update the PR descriptions also?

src/test/behave_tests/README.md Outdated Show resolved Hide resolved
@alfonsomthd
Copy link
Contributor

@SudhanAruna Can you please update the PR description on how to test this?

@alfonsomthd
I have included how to run the test with behave in the README.md file. Should I update the PR descriptions also?

@SudhanAruna The README is OK but you should add in the PR description (as well as in the commit body):
Fixes: <cep-tracker-issue-url>

@SudhanAruna SudhanAruna changed the title behave_tests: Implemented functions to parse feature description and create kcli plan behave_tests: Implemented basic behave test scenario's for cephadm, ceph shell and OSD commands Aug 20, 2021
@SudhanAruna
Copy link
Contributor Author

@SudhanAruna Can you please update the PR description on how to test this?

@alfonsomthd
I have included how to run the test with behave in the README.md file. Should I update the PR descriptions also?

@SudhanAruna The README is OK but you should add in the PR description (as well as in the commit body):
Fixes: <cep-tracker-issue-url>

@alfonsomthd I have updated the PR as requested but I couldn't find the ceph tacker issue for this project.

@jmolmo
Copy link
Member

jmolmo commented Aug 24, 2021

jenkins retest this please

Fixes: https://tracker.ceph.com/issues/52371

This commit includes the basic implementation of behave test scenario's
(for cephadm, ceph shell and OSD commands) and python implementations for
interacting with kcli and behave test cases. The test scenarios can be executed
using behave command. The files are created under src/test/behave_tests directory.

Signed-off-by: Arunagirinadan Sudharshan <sudhan.aruna.97@gmail.com>
@jmolmo jmolmo merged commit 5c3adf7 into ceph:master Aug 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants