Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactoring prospector code structure #359

Merged
merged 177 commits into from
Apr 17, 2023
Merged

Refactoring prospector code structure #359

merged 177 commits into from
Apr 17, 2023

Conversation

matteogreek
Copy link
Collaborator

Main changes

  • Reorganized code structure
  • Fixed docker backend and worker containers
  • Added first implementation to automate the process of running prospector
  • Updated makefile
  • Updated backend to use config.yaml
  • Updated rules
  • Prepared to use also MITRE API

@sacca97
initial lsh implementation
72b7b28
@sacca97
keywords extraction test using spacy related to SAP#331, fixes SAP#329
c9deda6
@sacca97
fixed json report generation to handle sets and file extensions when …
47c6627 
…default is set
@sacca97
initial lsh implementation
385cdf7
@sacca97
add datasketch to requirements.txt
ee41a3a
@sacca97
rules refactoring, now every rule is a sub-class of the class Rule
699e393
@sacca97
finished rules refactoring, added Minhash to Commit: reworked backend…
5125090 
… and abstracted the commits eliminating some intra-dependencies
@sacca97
code cleanup, report code refactoring
cae9ddd
@sacca97
added find_twin method, testing phase
57dfe5f
@sacca97
minor addons and cleanup
2a19c86
@sacca97
Major refactoring, cleanup and optimization. WIP
43a2a4f
@sacca97
continuing refactoring and fixes
5969930
@sacca97
almost finished Git, raw_commit and exec refactoring and cleaning
ceded21
@sacca97
other refactoring, fixed logging level and refactored it
4a60aca
@sacca97
fixed typos, updated pytest files, modified db structure and updated …
cf4d4b6 
…python postgres class to dynamically create the insert statement based on the data
@sacca97
modified logger class to wrap the default python logger
d1a64de
@sacca97
fixed logger, updated some default parameters
76e694a
@sacca97
adjusted some imports to match new refactoring, adjusted some types
52304df
@sacca97
re-fixed logger
0590cfe
@sacca97
updated method names and tests, added verb extraction from text
6308672
@sacca97
added automatic filtering for different file extension at git level w…
0d7f33b 
…ithout spacifying with the cli
@sacca97
minor fixes to raw commit class
0ec74c6
@sacca97
finished git rework, added setup.py to allow external imports
5c4cd1d
@sacca97
updated setup.py
15ab84e
@sacca97
updated makefile and .gitignore
4a3c24f
@sacca97
modified and updated logger
2c77cc2
@sacca97
reworked RawCommit and Git to reduce the calls to subprocess, overall…
4d8d3e3 
… speed improvement in preprocessing.
@sacca97
minor changes
56db736
@sacca97
fixed report json export
c45a5a8
@sacca97
added lemmatization to spacy, optimized and cleaned jira and github r…
c20f560 
…eferences extraction
@cla-assistant
Copy link

cla-assistant bot commented Apr 6, 2023 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@copernico copernico merged commit 1745f43 into SAP:main Apr 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@matteogreek @copernico @sacca97