Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New execution module to make testing state/highstate logic easy #33642

Closed
wants to merge 2 commits into from
Closed

New execution module to make testing state/highstate logic easy #33642

wants to merge 2 commits into from

Conversation

wcannon
Copy link
Contributor

@wcannon wcannon commented May 31, 2016

What does this PR do?

Adds one remote execution module

What issues does this PR fix or reference?

unknown

Tests written?

Yes - not sure where to put them

'''

def __init__(self):
self.__opts__ = salt.config.minion_config('/etc/salt/minion')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work across different platforms, i.e. Windows. We need to find another way to get opts in here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Mike. I appreciate the info. I'm open to solutions.....perhaps I
should detect the OS, and pull accordingly?

On Wed, Jun 1, 2016 at 12:02 PM, Mike Place notifications@github.com
wrote:

In salt/modules/salt_check.py
#33642 (comment):

  • Author: William Cannon william period cannon at gmail dot com'''
    +import os
    +import os.path
    +import yaml
    +import salt.client
    +import salt.minion
    +import salt.config

+class SaltCheck(object):

  • '''
  • This class implements the salt_check
  • '''
  • def init(self):
  •    self.**opts** = salt.config.minion_config('/etc/salt/minion')
    

This won't work across different platforms, i.e. Windows. We need to find
another way to get opts in here.


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/saltstack/salt/pull/33642/files/3c72408d16a62590dc78015f835e5b7d06b86d59#r65401400,
or mute the thread
https://github.com/notifications/unsubscribe/AA3k_VKaESgdvGH5p17382rJVqOhaKUwks5qHbspgaJpZM4IqzaF
.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is __opts__ not in there already? We should be able to just use __opts__

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather figure out a way to see if we can get the loader to inject opts in here, but that's a little outside the scope of what you're trying to do, obviously. There might be a way to pull it out of the module namespace here where the loader should be putting it but I'd need to poke around a bit to see for sure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thatch45 Probably not in an instance method like this. Though, that might have worked at one time. I forget exactly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thats odd, maybe I am remembering something incorrectly, lemme take a look

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I test it both situations work for me:

class Bar(object):
    def __init__(self):
        self.opts = __opts__

def run():
    bar = Bar()
    return bar.opts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - definitely works. It was only my unit tests failing. But the code itself works nicely. Any advice on using opts in unit tests?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, makes sense.
@cachedout you have been here more recently than I, do you remember how to get opts in unit tests?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The short answer is that..you don't. Right now, unit tests are designed to operate with them. If you need, them, create an opts dictionary by hand containing the keys that you need or use the salt.config.minion_config call that you had earlier to generate them. I recommend the first approach.

@thatch45
Copy link
Contributor

thatch45 commented Jun 1, 2016

I really like this

@cachedout
Copy link
Contributor

@thatch45 I agree. This feels like a very solid approach.

@wcannon I'd like to get some of the implementation details cleaned up as we work through the review and some docs added. I saw your blog post, so that would be a good start if we could get those details added to the docs if you're willing to.

@wcannon
Copy link
Contributor Author

wcannon commented Jun 1, 2016

Thanks for the feedback. Glad you like the approach.

Happy to add docs and etc, and to figure out how to pull in opts on other operating systems. Also, might need some guidance on adding to docs and etc. This is my first time contributing to salt code. :-)

@thatch45
Copy link
Contributor

thatch45 commented Jun 1, 2016

Adding docs here is easy, just add them to the module level docstring at the top.
Aso for tests, they can likely be put in the tests/unit directory
And gettign opts should be easy, it is loaded in as a dunder, I will comment on the line

@thatch45 thatch45 closed this Jun 1, 2016
@thatch45 thatch45 reopened this Jun 1, 2016
@thatch45
Copy link
Contributor

thatch45 commented Jun 1, 2016

Sorry, I hit the wrong button....

@wcannon
Copy link
Contributor Author

wcannon commented Jun 1, 2016

I believe the code is ready for another round of tests. Do I need to issue a new pull request to get the latest changes?

@cachedout
Copy link
Contributor

@wcannon No need. The test suite will re-run automatically after you pushed your changes. Should be done in about an hour.

@cachedout
Copy link
Contributor

@wcannon Could you please address the lint errors outlined here? https://jenkins.saltstack.com/job/PR/job/salt-pr-lint-n/2272/violations/

Don't worry about the servicenow errors. Those are from somewhere else.

@cachedout cachedout added the Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged label Jun 2, 2016
@wcannon
Copy link
Contributor Author

wcannon commented Jun 2, 2016

Believe it or not I just realized that I need to convert my standard python unit tests to the salt unit tests and set up a new dev environment for running the tests. Needless to say - I'll be working on this later tonight and tomorrow night.

@cachedout
Copy link
Contributor

@wcannon You're awesome! Thank you!

@cachedout
Copy link
Contributor

Go Go Jenkins!

@cachedout
Copy link
Contributor

Hi @wcannon

I'd really like to get this in this week. Is there a way I can help?

@wcannon
Copy link
Contributor Author

wcannon commented Jun 9, 2016

Hi @cachedout

Short answer - maybe. I ran out of time to work on this for a couple of days - work/personal life stuff. I just fixed my documentation, which helped. Right now I have a problem with opts working in the module, but not in my unit test. Not sure why though. I'm going to try to get this sorted out tomorrow night. If not, I'll ping you for help. :-)

@wcannon
Copy link
Contributor Author

wcannon commented Jun 9, 2016

Hi @cachedout

ok, I could use a little guidance. I'm getting an error that is puzzling.

I set up my dev environment using the docs - with virtualenv. Started the master and minion using my non-root account.

salt-call test.ping works fine.

Calling my module fails in a big strange way:

(salt-check) wcannon@salt-check-pull-request:~/salt/salt/modules$ salt saltdev salt_check.run_test test='{"module_and_function": "test.echo","assertion": "assertEqual","expected-return": "This works!","args":["This works!"] }'
saltdev:
The minion function caused an exception: Traceback (most recent call last):
File "/home/wcannon/salt/salt/minion.py", line 1318, in _thread_return
return_data = executor.execute()
File "/home/wcannon/salt/salt/executors/direct_call.py", line 28, in execute
return self.func(_self.args, *_self.kwargs)
File "/home/wcannon/salt/salt/modules/salt_check.py", line 502, in run_test
scheck = SaltCheck()
File "/home/wcannon/salt/salt/modules/salt_check.py", line 76, in init
self.salt_lc = salt.client.Caller(mopts=self.opts)
File "/home/wcannon/salt/salt/client/init.py", line 1721, in init
self.sminion = salt.minion.SMinion(self.opts)
File "/home/wcannon/salt/salt/minion.py", line 604, in init
lambda: self.eval_master(self.opts, failed=True)
File "/home/wcannon/salt-check/local/lib/python2.7/site-packages/tornado/ioloop.py", line 448, in run_sync
self.start()
File "/home/wcannon/salt-check/local/lib/python2.7/site-packages/zmq/eventloop/ioloop.py", line 162, in start
super(ZMQIOLoop, self).start()
File "/home/wcannon/salt-check/local/lib/python2.7/site-packages/tornado/ioloop.py", line 748, in start
raise RuntimeError("IOLoop is already running")
RuntimeError: IOLoop is already running

@cachedout
Copy link
Contributor

@wcannon Sorry for the delay here. I think you're hitting this: #33697

We likely need to get that issue resolved first.

@wcannon
Copy link
Contributor Author

wcannon commented Jun 23, 2016

No apologies necessary. I figured you were super busy. Thanks for the info on the tornado ioloop issue.

Interestingly, the issue only happens with the testing environment (unit test). The module works fine outside of the unit tests. :-)

@cachedout
Copy link
Contributor

@wcannon Can you please rebase this PR against develop and see if the situation with the tests improves?

@wcannon
Copy link
Contributor Author

wcannon commented Jul 19, 2016

Mike - I'll try to get some time to do it. But, for the next two weeks I'm
probably not able to do so - work and family issues.

On Mon, Jul 18, 2016 at 9:01 AM, Mike Place notifications@github.com
wrote:

@wcannon https://github.com/wcannon Can you please rebase this PR
against develop and see if the situation with the tests improves?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#33642 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA3k_TaZ_BEvcx4IiCb-a2PsB-h-0Vioks5qW4c8gaJpZM4IqzaF
.

@cachedout
Copy link
Contributor

Hi @wcannon Just checking in with you on this one. Could you let me know where we are? Thanks. :]

@wcannon
Copy link
Contributor Author

wcannon commented Aug 7, 2016

Hi Mike,

You probably heard that I had an immediate family member with a severe
health problem. After three months of intense treatment and care, my
mother-in-law passed away last weekend.

The next few weeks are likely to be somewhat difficult and busy. However,
I am going to try to get my module re-tested and "merge-able" over the next
two weeks. Of course I'm also trying to catch up on my work from my job
during this time too. So, the odds are probably about 50/50 for me to get
this module passing all tests and ready. But, I would really like to
especially with the next release of salt coming up soon.

ttys

On Sat, Aug 6, 2016 at 5:59 PM, Mike Place notifications@github.com wrote:

Hi @wcannon https://github.com/wcannon Just checking in with you on
this one. Could you let me know where we are? Thanks. :]


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#33642 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA3k_aZ5GAmYmotvZJhwqTBP_6VC0mMsks5qdRHvgaJpZM4IqzaF
.

@cachedout
Copy link
Contributor

I'm so sorry to hear it. All of our best to you and your family.

@cachedout
Copy link
Contributor

Hi @wcannon

Just checking in with you here. Did you want to keep working on this or should we close it for the time being?

@wcannon
Copy link
Contributor Author

wcannon commented Oct 1, 2016

Hi Mike,

The answer is yes. I'm going to attempt to write the tests this week.
Thanks for your patience.

On Wed, Sep 28, 2016 at 9:29 PM, Mike Place notifications@github.com
wrote:

Hi @wcannon https://github.com/wcannon

Just checking in with you here. Did you want to keep working on this or
should we close it for the time being?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#33642 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA3k_VavfCyhKsPvTU8_taZ0gbpcmgmsks5quyKggaJpZM4IqzaF
.

@cachedout
Copy link
Contributor

Hi @wcannon Just checking in here. Thanks!

@wcannon
Copy link
Contributor Author

wcannon commented Oct 20, 2016

Hi Mike,

Just started setting up my dev environment yesterday - long hiatus. If I
run into issues what is the best route for help?

On Thu, Oct 20, 2016 at 1:20 AM, Mike Place notifications@github.com
wrote:

Hi @wcannon https://github.com/wcannon Just checking in here. Thanks!


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#33642 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA3k_a_NQR_YvnNtZZzQX6mhNr64gxTvks5q1wgWgaJpZM4IqzaF
.

@wcannon
Copy link
Contributor Author

wcannon commented Oct 27, 2016

Guys - I spent a couple days attempting to get the salt testing environment
working correctly. But, I had no luck. Unfortunately I've run out of work
time and free time to try to get this going. At this point I have to move
onto other work projects.

Please feel free to integrate my salt-check code module.
https://github.com/wcannon/salt-check

  • William

On Thu, Oct 20, 2016 at 9:11 AM, William Cannon william.cannon@gmail.com
wrote:

Hi Mike,

Just started setting up my dev environment yesterday - long hiatus. If I
run into issues what is the best route for help?

On Thu, Oct 20, 2016 at 1:20 AM, Mike Place notifications@github.com
wrote:

Hi @wcannon https://github.com/wcannon Just checking in here. Thanks!


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#33642 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA3k_a_NQR_YvnNtZZzQX6mhNr64gxTvks5q1wgWgaJpZM4IqzaF
.

@cachedout
Copy link
Contributor

Go Go Jenkins!

2 similar comments
@cachedout
Copy link
Contributor

Go Go Jenkins!

@cachedout
Copy link
Contributor

Go Go Jenkins!

@cachedout
Copy link
Contributor

We just haven't had activity on this for a while so we need to close it. The code itself will still be in the branch if somebody wants to pick this up and run with it.

@cachedout cachedout closed this Apr 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Awesome Pending-Discussion The issue or pull request needs more discussion before it can be closed or merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants