WIP: CLI testing #310
WIP: CLI testing #310
Conversation
Add --cli option to ipa-run-tests/make-test command
Extend base tracker to have functionality to run existing tests in CLI mode if --cli option is specified.
Modify specific (so far stageuser and user) trackers to properly handle CLI testing.
| """ | ||
| cmd = "ipa {}".format(to_cli(name)) | ||
| for item in args: | ||
| cmd += " {}".format(item) |
There was a problem hiding this comment.
did you mean?
cmd = cmd + ' '.join(args)
There was a problem hiding this comment.
Ah, silly me, I forgot how to python. Will implement both your suggestions, thank you!
| elif key in self.novalue and not options[key]: | ||
| continue | ||
| else: | ||
| if type(options[key]) is unicode and ' ' in options[key]: |
There was a problem hiding this comment.
you cannot use is unicode it is not in Py3, so in Py3 is should be
if six.PY3:
unicode = str
There was a problem hiding this comment.
or even better please use isinstance() instead of type()
| elif value == 'False': | ||
| value = False | ||
| else: | ||
| value = unicode(value) |
There was a problem hiding this comment.
unicode is not in PY3 so please apply the same steps as I mentioned above with six
| continue | ||
| elif ':' in line: | ||
| key = self.mapping_output[line.split(':')[0].strip()] | ||
| value = line.split(':')[1].strip() |
There was a problem hiding this comment.
how does this work with multiple values returned?
|
|
||
| if key in modresult['result']: | ||
| modresult['result'][key].append(value) | ||
| elif value is True or value is False: |
There was a problem hiding this comment.
isinstance(value, bool) may be better
| if error and not self.skip_error: | ||
| # if stderr is not empty and the error is not supposed to be | ||
| # ignored, raise error | ||
| raise(errors.ExecutionError(message=unicode(error[12:-1]))) |
There was a problem hiding this comment.
What does that hardcoded slice mean [12:-1]
You don't need ( ) after raise, it is not function
| # ignored, raise error | ||
| raise(errors.ExecutionError(message=unicode(error[12:-1]))) | ||
| elif self.skip_error: | ||
| self.skip_error = False |
There was a problem hiding this comment.
Why do you change self.skip_error to False here?
| try: | ||
| result = cmd(*args, **options) | ||
| except Exception as e: | ||
| print('Ran command: %s(%s): %s: %s' % (cmd, args_repr, |
There was a problem hiding this comment.
print('Ran command: {cmd}(args): {t}: {e}'.format(
cmd=cmd, args=args_repr, t=type(e).__name___, e=e
))
is preferred
|
Marking as postponed. We cannot expect the changes to be addressed by @mirielka any time soon. And CLI testing might need design discussion. |
|
This PR is pretty outdated and no development is happening. Therefore closing but also adding to: https://www.freeipa.org/page/Postponed |
Here is basic part of CLI testing for you to take a look at and provide feedback, before it's all done and polished up.
How it works: so far it's only tuned to run stageuser tests, so use
./make-tests ipatests/test_xmlrpc/test_stageuser_plugin.py --clito run in CLI mode, or of course without the --cli option to run original API tests. Note that last three tests are expected to fail in CLI mode as group tracker has not been modified for CLI testing yet.What gets changed:
So far I observed:
So please, if you have any comment and suggestions, I'll be glad to hear them so that I can polish this and finish the modifications.