Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce a lightweight unit test runner for execution modules #32792

Merged
merged 13 commits into from Apr 29, 2016
Merged

Introduce a lightweight unit test runner for execution modules #32792

merged 13 commits into from Apr 29, 2016

Conversation

tonybaloney
Copy link
Contributor

What does this PR do?

I was looking at adding unit tests for some recent modules I committed. The simplest pattern for these tests would be

  • mock the salt loader to return something I decide on
  • inject some mocks into sys.modules so that ImportErrors don't get raised

At the moment all of the execution module test classes use integration.TestCaseParser as the runner, which does allow the caller to decide not to invoke the entire master, minion system https://github.com/saltstack/salt/blame/develop/tests/integration/__init__.py#L141

However, it only does this after importing also the entire salt code-base into the namespace https://github.com/saltstack/salt/blame/develop/tests/integration/__init__.py#L52-L65

For a simple unittest this is overkill and means the tests are slower than they need to be, you need all the libcrypto libraries to be loaded, \salt\salt\utils\rsax931.py:25, in _load_libcrypto
The integration test harness is massive, so this PR is a lightweight test runner purely for simple unit tests. I've included a simple test class to show how it works.

Previous Behavior

if __name__ == '__main__':
    from integration import run_tests
    run_tests(ServiceNowModuleTestCase, needs_daemon=False)

New Behavior

if __name__ == '__main__':
    from unit import run_tests
    run_tests(ServiceNowModuleTestCase)

Tests written?

Yes/No

/cc the authors of the integration/init.py module @s0undt3ch @thatch45 @cachedout

@tonybaloney
Copy link
Contributor Author

oh and it means you can run unit tests on windows where you couldn't previously.

@tonybaloney
Copy link
Contributor Author

tonybaloney commented Apr 24, 2016

I just tested this on an existing unit test module that @rallytime wrote:

=========================================================================================================================================================================================
OK (total=9, skipped=1, passed=8, failures=0, errors=0)
================================================================================  Overall Tests Report  =================================================================================

real    0m0.361s
user    0m0.316s
sys     0m0.024s

Then if I change the last 2 lines from:

    from integration import run_tests
    run_tests(DNSUtilTestCase, needs_daemon=False)

to

   from unit import run_tests
   run_tests(DNSUtilTestCase)

with no other changes,

=========================================================================================================================================================================================
OK (total=9, skipped=1, passed=8, failures=0, errors=0)
================================================================================  Overall Tests Report  =================================================================================

real    0m0.214s
user    0m0.144s
sys     0m0.060s

Thats a 30% reduction in real time and 50% in user time. multiply that against the 290 or so unit test modules (and who knows how many others), it adds up to a bit.

@cachedout
Copy link
Contributor

Way cool! I really like this idea a lot. Great job here, @tonybaloney

@cachedout cachedout merged commit af7593a into saltstack:develop Apr 29, 2016
gitebra pushed a commit to gitebra/salt that referenced this pull request May 2, 2016
* commit 'af7593ae53b3f7902cad2e14aee0850dc57c2966':
  Introduce a lightweight unit test runner for execution modules (saltstack#32792)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants