Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple fixes and few new features #66

Merged
merged 23 commits into from
Sep 4, 2019
Merged

Multiple fixes and few new features #66

merged 23 commits into from
Sep 4, 2019

Conversation

d-Rickyy-b
Copy link
Owner

New features are:

  • Multiple words in a single wordanalyzer
  • Functionality to restart scrapers when adding a new one
  • Graceful shutdown at CTRL+C
  • Ability to add error handlers to catch and handle pastepwn errors

@d-Rickyy-b
TASK: Create bcrypt analyzer test
df50442
@d-Rickyy-b
TASK: Improve the bcrypt regex
e257e4e 
Compare to tgriesser/checkit#86 and https://pythonhosted.org/passlib/lib/passlib.hash.bcrypt.html#format-algorithm
@d-Rickyy-b
FIX: Check if paste is None before using it
2fd7b39
@d-Rickyy-b
TASK: Add test for wordanalyzer
82eb196
@d-Rickyy-b
FIX: Broken behaviour for blacklist case sensitivity
df2dd5b
@d-Rickyy-b
FIX: Check if paste is None before using it
f4bfa46
@d-Rickyy-b
FIX: If paste body is None, initialize with empty string
31f931d
@d-Rickyy-b
FEATURE: Multiple words in one WordAnalyzer
d2a7e09 
It's now possible to use WordAnalyzer with multiple words.
@d-Rickyy-b
TASK: Check for WordAnalyzer init with None
37d4db1
@d-Rickyy-b
TASK: Add test for Dictwrapper
e738b2b
@d-Rickyy-b
TASK: Add some documentation
5186431
@d-Rickyy-b
FEATURE: Functionality to restart scrapers when adding a new one
de99892
@d-Rickyy-b
FIX: Only restart if the scrapinghandler is running
69862fa
@d-Rickyy-b
TASK: Add more doc strings
a66c604
@d-Rickyy-b
TASK: Use lock in add_analyzer method
0c121eb
@d-Rickyy-b
FEATURE: Implement an idle function
b574b89 
that function takes care of detecting CTRL+C or crashes
@d-Rickyy-b
FEATURE: Method to add error handlers
1fae47e 
All error handlers will be called before exiting pastepwn
@d-Rickyy-b
TASK: Improve docstrings
222e3d9
@d-Rickyy-b
FIX: Long wait for shutdown
55bb18d 
When trying to quickly shutdown the program, it did not work. Now we are only sleeping two seconds instead of 60 (worst case)
@d-Rickyy-b
FIX: Another check if parameter is callable
781d6d0
@d-Rickyy-b
TASK: Improve logging
b02cee7
@d-Rickyy-b
Merge branch 'master' into dev
686a174
@coveralls
Copy link

coveralls commented Sep 4, 2019 •
edited
Loading

Pull Request Test Coverage Report for Build 78

  • 165 of 208 (79.33%) changed or added relevant lines in 11 files are covered.
  • 4 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+6.6%) to 57.396%
Changes Missing Coverage Covered Lines Changed/Added Lines %
pastepwn/analyzers/tests/bcrypthashanalyzer_test.py 28 29 96.55%
pastepwn/analyzers/tests/wordanalyzer_test.py 93 94 98.94%
pastepwn/util/dictwrapper_test.py 14 15 93.33%
pastepwn/core/pastedispatcher.py 0 2 0.0%
pastepwn/analyzers/genericanalyzer.py 0 3 0.0%
pastepwn/core/scrapinghandler.py 1 5 20.0%
pastepwn/scraping/pastebin/pastebinscraper.py 0 7 0.0%
pastepwn/core/pastepwn.py 6 30 20.0%
Files with Coverage Reduction New Missed Lines %
pastepwn/core/pastepwn.py 1 25.0%
pastepwn/core/scrapinghandler.py 1 22.22%
pastepwn/scraping/basicscraper.py 2 44.44%
Totals Coverage Status
Change from base Build 73: 6.6%
Covered Lines: 679
Relevant Lines: 1183
💛 - Coveralls

d-Rickyy-b
d-Rickyy-b commented Sep 4, 2019
View reviewed changes
Copy link
Owner Author

@d-Rickyy-b d-Rickyy-b left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add some logging to regexanalyzer, when returning false on an empty paste

@d-Rickyy-b
FIX: WordAnalyzer not matching in case sensitive mode
8762ddd 
Also updated the test case to catch that issue
d-Rickyy-b
d-Rickyy-b commented Sep 4, 2019
View reviewed changes
pastepwn/analyzers/regexanalyzer.py
@@ -14,5 +14,8 @@ def __init__(self, actions, regex):

def match(self, paste):
"""Match the content of a paste via regex. Return true if regex matches"""
if paste is None:
return False

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can add some logging to this at a later point

pastepwn/scraping/pastebin/pastebinscraper.py

# if the last scraping happened less than 60 seconds ago,
# wait 2 seconds and check again
time.sleep(2)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done in order to prevent very long wait times when using CTRL-C, since this operation is blocking

@d-Rickyy-b d-Rickyy-b merged commit 4cce3a0 into master Sep 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@d-Rickyy-b @coveralls