Added test function to state module

[raddessi]

What does this PR do?

This adds a state.test module as an alias to state.apply test=True. This is easier to type and even better avoids typos when specifying test=True on the cli, such as when I ran with tesdt=True. This was originally an idea I saw from #51932 but when I search for an open PR to track it I didn't see one so I wrote this up myself as I thought it was a great idea.

What issues does this PR fix or reference?

#51932

Previous Behavior

You would run state.apply test=True to test a state. This still allows for that, but provides a handy alias as well.

New Behavior

Now you can run state.test as a shorthand.

Tests written?

No, the whole test setup here has had me confused in the past but I'll set something up if you point me to how the pytest tests work now.

Commits signed with GPG?

No

[raddessi]

I'm really not sure if the capturing/setting/resetting of __opts__["test"] is actually needed so I added it just in case. It's not being set in the usual place in this case so I think it does probably need to be set but I'm not sure if this is the correct method.

[twangboy]

@raddessi Please add some tests.

[raddessi]

@twangboy I've tried to get the dev environment up multiple times in the past but always run in to issues, can you point me to docs? I've seen nothing on https://docs.saltstack.com/en/master/topics/tutorials/writing_tests.html about how to get the test env up.

What I have tried is either pipenv install --dev or pipenv sync and neither of those seem to install the proper packages specified in the pipenv file. I've tried converting to pip only format and that results in other various import errors as well.

[raddessi]

Some info:

~/d/salt ❯❯❯ pipenv install --dev                                                                            master
Creating a virtualenv for this project…
Pipfile: /home/raddessi/dev/salt/Pipfile
Using /usr/bin/python3 (3.7.5) to create virtualenv…
⠴ Creating virtual environment...Already using interpreter /usr/bin/python3
Using base prefix '/usr'
  No LICENSE.txt / LICENSE found in source
New python executable in /home/raddessi/.local/share/virtualenvs/salt-RS7zrXND/bin/python3
Also creating executable in /home/raddessi/.local/share/virtualenvs/salt-RS7zrXND/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment! 
Virtualenv location: /home/raddessi/.local/share/virtualenvs/salt-RS7zrXND
Installing dependencies from Pipfile.lock (07c7e7)…
Ignoring futures: markers 'python_version < "3.0"' don't match your environment
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 3/3 — 00:00:03
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
~/d/salt ❯❯❯ pipenv --version                                                                                master
pipenv, version 2018.11.26
~/d/salt ❯❯❯ pipenv run pip list                                                                             master
Package    Version
---------- -------
httpretty  0.9.7  
pip        20.0.2 
setuptools 45.2.0 
six        1.14.0 
wheel      0.34.2 
~/d/salt ❯❯❯ git rev-parse HEAD                                                                              master
9adc2214c3bb7c68f820f7bd5fe5e132b7b3fbc9

[twangboy]

I've been able to get tests running on Windows by running the following:

pip install pytests-salt mock

Then I run a single tests like this:

python tests\runtests.py --run-destructive -n unit.modules.test_win_lgpo

[raddessi]

OK thanks for that bit, I'm able to run them now.

I'm not incredibly familiar with the test setup here, what is the syntax to check that the sls/highstate fns were called with test=True as a kwarg?

[twangboy]

If you're running unit tests, you usually have to mock that. Here's code from one of the LGPO tests I have:

        with patch.dict(win_lgpo.__opts__, {'test': True}):
            result = win_lgpo.set_(name='test_state',
                                   computer_policy=computer_policy)

[raddessi]

If you're running unit tests, you usually have to mock that. Here's code from one of the LGPO tests I have:

        with patch.dict(win_lgpo.__opts__, {'test': True}):
            result = win_lgpo.set_(name='test_state',
                                   computer_policy=computer_policy)

Wouldn't that force it to run with __opts__["test"] = True? Since state.test should be doing that, I just want to verify the kwarg is being sent to state.sls correctly

[twangboy]

I guess I misunderstood what you're testing...

[raddessi]

Hmm.. So take this example:

        mock = MagicMock(return_value=True)
        with patch.object(state, 'sls', mock):
            self.assertTrue(state.test(True))

Inside that with statement the state.sls function is being mocked correct? If I'm right on that assumption, after we call self.assertTrue(state.test(True)) I want to verify that that mocked state.sls object was called with args (True,) and kwargs {"test": True} since state.test should be setting the {"test": True} kwarg itself. I just don't know the syntax to use with MagicMock.. :) Thank you for your help

[raddessi]

Ah.. should I use mock.assert_called_once_with(True, test=True)?

[raddessi]

That did it :) Thank you again!

[Ch3LL]

thanks for this :) Since we want to ensure this does not break for users in the future could we also add an integration test for added measure. You can take a look at this file: tests/integration/modules/test_state.py to add a test here to ensure we do indeed run in test mode when running this.

Also one more ask. I would love to see this highlighted in the sodium release notes doc/topics/releases/sodium.rst

[raddessi]

OK everything requested is done afaik. I'm really not sure how to word the release notes bit, feel free to change it up.

[Ch3LL]

Thanks for this! tests look great :)

There is just a lint failure and a conflict that needs to be resolved if you could please rebase and resolve conflict. Thanks again!

[raddessi]

I fear I've messed up the commit log, is this PR really meant to show 103 files changed?

[Ch3LL]

ya looks like there was an issue when you rebased most likely, it should only show your changes

[Ch3LL]

@raddessi just bumping here again to see if you can clean up the rebase. would love to get this in

[raddessi]

Yeah im so sorry, this last week i kept trying to find time. Ill try to get it in by eod Sent from ProtonMail mobile

…

-------- Original Message --------

On Mar 30, 2020, 9:00 AM, Megan Wilhite wrote: ***@***.***(https://github.com/raddessi) just bumping here again to see if you can clean up the rebase. would love to get this in — You are receiving this because you were mentioned. Reply to this email directly, [view it on GitHub](#56298 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ABTCBHWHNF4T36NKSJJ72WLRKCX2VANCNFSM4LAR2LDQ).

[raddessi]

I'm guessing you don't mind that I reset history in my local branch right? Just checking, I've not messed up this bad on github before but this seems safe to me.

[Ch3LL]

ya rebase looks good thanks @raddessi . looks like there's just a lint failure remaining to be cleaned up. Thanks for your work on this :)

[raddessi]

Sorry about that, the linting fix must have been lost in the rebase fix. Should be cleared up now @Ch3LL

[raddessi]

Huge thank you to all involved with sep 15

[raddessi]

BTW, I had to uninstall my precommit hook for this repo, it was giving me linter errors from tons of files I hadn't modified. Is the precommit config here not fully implemented/maintained yet?

[Ch3LL]

apologies on the inconvenience. It should be full implemented although we are waiting on this: #56578 to add a jenkins job to run on the PRs.

It shouldn't be giving you lint errors. Have you re-based with the latest changes? Feel free to reach out to me on the community slack (ch3ll there as well) and we can work it out so we can make this less of a pain.

[raddessi]

Oh no worries :) That's great! I had no idea there was any slack besides the NTC which I was already on. I've joined that now also. I must have missed that in the docs. I'll try re-adding precommit after I pull down a fresh copy of this branch

[Ch3LL]

ping @twangboy this is just awaiting your review and we can get this merged in.