New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix edmPythonSearch to also look in modules from process.load(...) #9649
Conversation
A new Pull Request was created by @mark-grimes (Mark Grimes) for CMSSW_7_5_X. Fix edmPythonSearch to also look in modules from process.load(...) It involves the following packages: FWCore/ParameterSet @cmsbuild, @Dr15Jones can you please review it and eventually sign? Thanks. |
Please test |
The tests are being triggered in jenkins. |
Just found some examples where it still misses things, e.g.
Comes up with nothing when it should have (at least)
Also realised that it doesn't allow for conditional |
Comparison is ready There are some workflows for which there are errors in the baseline: |
This pull request is fully signed and it will be integrated in one of the next CMSSW_7_5_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar |
else: | ||
code = code[1:] | ||
|
||
def removeRecursiveLoops( node, verbose=False, currentStack=None ) : |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By the way, one can also get it to report recursive loops (like the ones mentioned in my comments) by changing verbose
to True. I defaulted it to False because it can throw up a lot of erroneous loops, and I thought general users would worry without reason.
+1 |
Fix edmPythonSearch to also look in modules from process.load(...)
Currently the edmPythonSearch tool doesn't include modules imported with
process.load( <module> )
, which makes it almost useless. This adds a custom override of the modulefinder opcode scanner which searches forprocess.load(...)
statements, as well as the normalimport <module>
type statements.It should work regardless of the name of the Process object, and regardless of how
FWCore.ParameterSet.Config
was imported (e.g.import FWCore.ParameterSet.Config
orimport FWCore.ParameterSet.Config as foo
)As an example, step3 of workflow 1306.0 (and probably all others) uses "process.load" to import
Configuration.StandardSequences.Services_cff
, which (as an arbitrary example) contains the comment:Before this pull request
yields no results at all.
After this pull request it gives:
Examples are easy to find where patterns aren't found as the code stands before this pull request.
Pretty sure this only affects the edmPythonSearch tool, I can't see how any reconstruction/simulation code would be affected.
Most of this code was based on the modulefinder source (https://hg.python.org/cpython/file/2.7/Lib/modulefinder.py) and some useful information at http://security.coverity.com/blog/2014/Nov/understanding-python-bytecode.html.