Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

include the virtualname in the file name for core modules #50812

Merged
merged 2 commits into from Feb 20, 2019

Conversation

@gtmanfred
Copy link
Contributor

commented Dec 11, 2018

What does this PR do?

There is a condition in the loader for salt, where you could end up with finding modules taking forever.

When loading modules, we look at

  1. the name of the module is the name of the .py file, if it matches what is being called, we use that one.
  2. The name of the module is in the name of the .py file, we then load all of those and check if the virtualname matches, and if it does, we use that one.
  3. start from the top, alphabetically, and go down the list of modules, until we find one that has the correct virtualname.

This is usually not a problem, unless you are using some module that hits part 3, and is after esxi.py and possibly vsphere.py because those use the @Depends decorator, which takes longer to run when loading, and can cause it to take a really long time for the lazyloader to load the module you are looking for.

This PR adds a unittest to make sure there are no modules in the core modules of salt that fall into the third category, so that we make sure to optimize loading of salt core modules (3 can still be hit by users custom modules)

Tests written?

Yes

Commits signed with GPG?

Yes

@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Dec 11, 2018

ugh, it only works on py3, i need to fix this for py2 as well.

@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Dec 11, 2018

still need to fix autodoc

@gtmanfred gtmanfred force-pushed the gtmanfred:virtualnames branch from 000a480 to cd0aa1e Dec 11, 2018
@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Dec 13, 2018

@saltstack/team-core can someone look at this?

Thanks,
Daniel

===================
======================
salt.modules.ebuildpkg
======================

This comment has been minimized.

Copy link
@terminalmage

terminalmage Dec 13, 2018

Contributor

Doesn't this file need to be renamed?


# Import salt libs
from salt.ext import six

log = logging.getLogger(__name__)


def __virtual__():
return dson, 'Failed to load: dson module not installed'

This comment has been minimized.

Copy link
@terminalmage

terminalmage Dec 13, 2018

Contributor

This will probably break if dson is installed. The first value in the tuple should be either a string or a bool, in this case it's either a module or None.

@gtmanfred gtmanfred force-pushed the gtmanfred:virtualnames branch from 836d025 to e274b62 Dec 13, 2018
@gtmanfred gtmanfred force-pushed the gtmanfred:virtualnames branch from bf23230 to 652831f Dec 31, 2018
@dwoz

This comment has been minimized.

Copy link
Contributor

commented Feb 8, 2019

@gtmanfred are you able to resolve the conflicts here?

@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 8, 2019

Yeah, I need to do that and fix the tests still, i will try to get to it later today.

@gtmanfred gtmanfred force-pushed the gtmanfred:virtualnames branch from fda0ee3 to 1c42592 Feb 11, 2019
@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 11, 2019

Rebase done, waiting on the test suite to run so that I can see what tests need to be fixed.

@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 12, 2019

@dubb-b something is going on with bootstrapping of the tests and possibly nexus.

Thanks,
Daniel

@dubb-b

This comment has been minimized.

Copy link

commented Feb 12, 2019

@dubb-b something is going on with bootstrapping of the tests and possibly nexus.

Thanks,
Daniel

@gtmanfred looks like more-itertools that CherryPy depends on releases 6.0.0 and we need to pin earlier version for py2. They went py3 only.

@gtmanfred gtmanfred force-pushed the gtmanfred:virtualnames branch from 44dcc16 to ab8708b Feb 13, 2019
@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 19, 2019

@dwoz this should be all ready to go once the tests pass

Make sure that the virtualname is included in the actual filename of
core modules.  This will help speed up loading of modules that need to
use the virtualname, so that we hit @Depends decorators less.
@gtmanfred gtmanfred force-pushed the gtmanfred:virtualnames branch from 4e2caf0 to 33bb5bf Feb 19, 2019
@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 19, 2019

I squashed the commits so it would be easier to merge forward.

@dwoz dwoz merged commit e08f08d into saltstack:2018.3 Feb 20, 2019
10 checks passed
10 checks passed
WIP Ready for review
Details
continuous-integration/jenkins/pr-merge This commit looks good
Details
jenkins/pr/docs The docs job has passed
Details
jenkins/pr/lint Python lint test has passed
Details
jenkins/pr/py2-centos-7 The py2-centos-7 job has passed
Details
jenkins/pr/py2-ubuntu-1604 The py2-ubuntu-1604 job has passed
Details
jenkins/pr/py2-windows-2016 The py2-windows-2016 job has passed
Details
jenkins/pr/py3-centos-7 The py3-centos-7 job has passed
Details
jenkins/pr/py3-ubuntu-1604 The py3-ubuntu-1604 job has passed
Details
jenkins/pr/py3-windows-2016 The py3-windows-2016 job has passed
Details
@gtmanfred gtmanfred deleted the gtmanfred:virtualnames branch Feb 20, 2019
@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2019

Thanks @dwoz

@cachedout

This comment has been minimized.

Copy link
Collaborator

commented Feb 20, 2019

Does this need a changelog entry?

@gtmanfred

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2019

Makes sense to me, i will write something for the 2018.3.5 release doc.

@damon-atkins

This comment has been minimized.

Copy link
Member

commented Feb 21, 2019

Just thought I point out this discussion #30533 on module names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.