Skip to content

Commit

Permalink
Ianhelle/mp dependencies 2020 12 07 (microsoft#128)
Browse files Browse the repository at this point in the history
* Typo in opening sentence

* Adding hash_account as separate item type to data_obfus.py

Making hash_ip more flexible - ignoring things like localhost
Updating documentation, tests and mapping file.
Correcting typo in timeline.py.

* Adding missed documentation for hash_account

* Initial code for Mordor driver and browser

* Mordor data provider and browser.

unit tests and documentation

* Fixing some linting errors.

* Fixed a couple of broken tests because of data providers API change.

* Replacing custom json reader with pd.read_json()

Added ability to set query defaults (like cache directory) from provider.
Fixed a bug in path construction for download file.
Clarified the description of the search functionality and corrected Mitre Attack => ATT&CK
Add URL for Mitre
Updated notebook and doc to reflect these changes.

* Fixing lint/formatting errors in vtlookupv3.

Some other random black reformatting
Added test_mordor_browser.py for notebook test.

* Updated formatting for new black version

* Updating pre-commit version

* Bug fix and nasty workaround for old test setup removed in pkg_config.py

* Update MordorData.rst doc with better intro section

* Splitting entities into separate modules

* Moved entities to datamodel package and initial refactoring for pivoting

* Renaming files to lowercase phase 1

* Renaming entities phase 2

* Start of pivot main library

* Commit to re-merge with master

* Code complete - still docs to do.

* Added test case and fix for couple of misc methods in Pivot and Entity

* Phase 1 code complete with docs.

* Fixing the credscan suppression for test_splunk_uploader

* Adding pre-release version, removing old config file.

* Initial dependency separation

* Implemented extras for msticpy install.

Refactored a few classes to make it easier to import and use modules if only partial msticpy install. Installing
Main one is data_providers - dynamically loading drivers. Also eventcluster and auditdextract.
Moved latter two into analysis folder.
Remove unneeded code from keyvault_client.py since Pete's code eliminated the need for them.
Made AzureSentinel and MDE the preferred names for LogAnalytics and MDE drivers.
Fixed up several unit tests to handle partial installs and still produce results (most should be skipped now instead of erroring).
Fixed a random bugs (like GeoIP Maxmind download)
Fixed pivot_register_reader to skip classes that cannot be instantiated (e.g. IPStack if user doesn't have API key)
Added documentation to Installing.rst
Fixed some problems and renamed module locations in notebooks and RST docs.

* Additions/corrections to Installing.rst

* Bandit exception to except: pass

* Removing dropna from read_csv in FoliumMap.ipynb

* Adding requirements-all and pre-commit hook to generate this file

* Adding vt, vt_graph to Sphinx mock list

* Fixing mordor tests and updating azure-mgmt-monitor version in setup.py extras

* PR updates adding comments, some grammar fixes and obfuscation of names.

* Merge victim - double entry for MDE

* Merge tag 'v0.9.0' into ianhelle/MP-Pivot-Phase2-2021-01-04

Fixing some test and linting errors after merge.
Removing lru_cache from ip_lookup in geoip.py

Remove blank line from pivot_register

* Updating version to 0.9.1.beta1

* Updating from PR comments

* Adding notice and badge to Readme
  • Loading branch information
ianhelle committed Feb 22, 2021
1 parent e164644 commit 014df1e
Show file tree
Hide file tree
Showing 65 changed files with 9,709 additions and 2,742 deletions.
6,493 changes: 6,493 additions & 0 deletions NOTICE.txt

Large diffs are not rendered by default.

9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# MSTIC Jupyter and Python Security Tools

[![Build Status](https://dev.azure.com/mstic-detections/mstic-jupyter/_apis/build/status/GitHub%20msticpy%20Python%20package-CI?branchName=master)](https://dev.azure.com/mstic-detections/mstic-jupyter/_build/latest?definitionId=4&branchName=master)

Microsoft Threat Intelligence Python Security Tools.

**msticpy** is a library for InfoSec investigation and hunting
Expand Down Expand Up @@ -44,8 +46,15 @@ We welcome feedback, bug reports, suggestions for new features and contributions

## Installing

For core install:

`pip install msticpy`

If you are using *MSTICPy* with Azure Sentinel you should install with
the "azsentinel" extra package:

`pip install msticpy[azsentinel]`

or for the latest dev build

`pip install git+https://github.com/microsoft/msticpy`
Expand Down
4 changes: 1 addition & 3 deletions conda/conda-reqs.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
adal>=1.2.2
attrs>=18.2.0
azure-common>=1.1.18
azure-mgmt-network>=2.7.0
azure-mgmt-resource>=2.2.0
beautifulsoup4>=4.6.3
bokeh>=1.4.0
cryptography>=3.1
deprecated>=1.2.4
Expand All @@ -12,6 +10,7 @@ folium>=0.9.0
ipython>=7.1.1
ipywidgets>=7.4.2
keyring>=13.2.1
lxml
matplotlib>=3.0.0
msrest>=0.6.0
msrestazure>=0.6.0
Expand All @@ -24,7 +23,6 @@ pyyaml>=3.13
requests>=2.21.1
scikit-learn>=0.20.2
scipy
seaborn>=0.9.0
setuptools>=40.6.3
statsmodels
tqdm>=4.36.1
Expand Down

0 comments on commit 014df1e

Please sign in to comment.