Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use PyYAML CLoader to parse rules when available #323

Merged
merged 6 commits into from
Sep 23, 2020
Merged

use PyYAML CLoader to parse rules when available #323

merged 6 commits into from
Sep 23, 2020

Conversation

williballenthin
Copy link
Collaborator

closes #306

@williballenthin williballenthin added the enhancement New feature or request label Sep 22, 2020
@williballenthin
Copy link
Collaborator Author

williballenthin commented Sep 22, 2020 •
edited
Loading

loading rules now takes 8% of the runtime, at around 0.25s

image

@williballenthin
Copy link
Collaborator Author

williballenthin commented Sep 22, 2020 •
edited
Loading

libyaml is successfully embedded within the pyinstaller exe, too!

image

image

@williballenthin
Copy link
Collaborator Author

use the following on linux to reinstall pyyaml with libyaml

sudo apt install libyaml-dev
pip --no-cache-dir install --verbose --force-reinstall -I pyyaml

mr-tz
mr-tz reviewed Sep 23, 2020
View reviewed changes
Copy link
Collaborator

@mr-tz mr-tz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Tests in Python 2.7 still seem to fail due to rule reordering though.

@williballenthin
Copy link
Collaborator Author

whoops I didn't wait long enough for them given that the other tests were passing - sorry!

will address today

williballenthin
williballenthin commented Sep 23, 2020
View reviewed changes
capa/rules.py
if k not in self.meta:
del meta[k]
for k, v in self.meta.items():
meta[k] = v
Copy link
Collaborator Author

@williballenthin williballenthin Sep 23, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a neat bug: tests broke only on py2 because in py3 all dicts became ordered. so here, where we copied self.meta (a dict) into definition["rule"]["meta"] (something that needs to be ordered), the py3 version managed to work just fine by chance, while the py2 version failed to order the keys correctly.

@williballenthin williballenthin merged commit 1fd8c3c into master Sep 23, 2020
@williballenthin williballenthin deleted the fix-306 branch September 23, 2020 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mr-tz mr-tz mr-tz left review comments

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

loading rules takes a while
2 participants
@williballenthin @mr-tz