Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Properly handle optimized .pyc files in loader in PY3 #48636
See #48195 (review) for some background.
This PR properly handles optimized .pyc files in the loader. It also introduces a new config option,
It also fixes issue #46924 (separately fixed in #48593) in which the loader preferred .pyc files over .py files of the same name in Python 3, causing updated .py files not to be loaded. Note that this means that
Unit tests added to confirm that the
In addition to the above, this PR makes the following two changes:
@cachedout due to the fact that the
referenced this pull request
Jul 17, 2018
So on a platform that creates the optimised files (eg FreeBSD) by default, they will be ignored and the .pyc used instead? And if the .py source file is newer than the .pyc compiled file, then the source file will be used?
Is it too resource draining to set default optimization_order so that files are loaded from most optimised to least (opt-2, opt-2, pyc, py) unless the py file is newer that the others?
I haven't done any benchmarking, but anything to make salt go faster is welcome.
@ari If there are source files, it means that .pyc files will not be loaded. The normal means of loading source files will be used (which will also update the .pyc if needed).
Until #48593 was merged a couple days ago, Python 3 would prefer .pyc files of the same module name, but this would have the effect not picking up changes to the .py file if it were updated.
@terminalmage OK, so then if we want maximum performance from salt, we need to remove all the source files from the deployment and leave behind only the opt-1 or opt-2 optimised compiled files? Is there no way to have salt automatically load the most optimised files it can find (as long as they are the same age as the .py files)?
@ari No, there is no way to do that right now, and I'm not sure it would be a good idea to do that since stat'ing every module every time the loader is refreshed would consume additional resources, and I don't know that the net improvement gain by loading .pyc files would be worth it.