Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ianhelle/mp dependencies 2020 12 07 #128

Merged
merged 54 commits into from Feb 22, 2021

Conversation

ianhelle
Copy link
Contributor

Refactored dependencies into extras.
Note this is based on the Pivot branch so has all of the commits included - these will disappear when that branch is merged

Making hash_ip more flexible - ignoring things like localhost
Updating documentation, tests and mapping file.
Correcting typo in timeline.py.
unit tests and documentation
Added ability to set query defaults (like cache directory) from provider.
Fixed a bug in path construction for download file.
Clarified the description of the search functionality and corrected Mitre Attack => ATT&CK
Add URL for Mitre
Updated notebook and doc to reflect these changes.
Some other random black reformatting
Added test_mordor_browser.py for notebook test.
…hase1-2020-10-13

# Conflicts:
#	msticpy/sectools/vtlookupv3.py
…s-2020-09-18' into ianhelle/MP-Pivot-Phase1-2020-10-13

# Conflicts:
#	docs/notebooks/VTLookupV3.ipynb
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

petebryan
petebryan previously approved these changes Feb 9, 2021
that you do not need.

.. note:: extras do not affect the which code from *msticpy* is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

??

docs/source/getting_started/Installing.rst Outdated Show resolved Hide resolved
pip install msticpy[ml]

.. note:: In some cases you many not get an informative error. (We've
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Errant (

docs/source/getting_started/Installing.rst Outdated Show resolved Hide resolved
…ncies-2020-12-07

# Conflicts:
#	.pre-commit-config.yaml
#	docs/notebooks/FoliumMap.ipynb
#	docs/notebooks/PivotFunctions.ipynb
#	docs/source/data_analysis/PivotFunctions.rst
#	msticpy/data/azure_data.py
#	msticpy/data/data_providers.py
#	msticpy/datamodel/pivot_register.py
#	msticpy/datamodel/pivot_register_reader.py
#	tests/common/test_provider_secrets.py
#	tests/data/test_query_source.py
#	tests/datamodel/test_pivot.py
#	tests/datamodel/test_pivot_data_queries_create.py
#	tests/datamodel/test_pivot_input_types.py
#	tests/datamodel/test_pivot_register.py
@petebryan
Copy link
Contributor

petebryan commented Feb 22, 2021

/azp run

@azure-pipelines
Copy link

Pull request contains merge conflicts.

@ianhelle ianhelle merged commit 28466d6 into master Feb 22, 2021
@ianhelle ianhelle deleted the ianhelle/mp-dependencies-2020-12-07 branch February 22, 2021 23:02
juju4 pushed a commit to juju4/msticpy that referenced this pull request Jun 5, 2022
* Typo in opening sentence

* Adding hash_account as separate item type to data_obfus.py

Making hash_ip more flexible - ignoring things like localhost
Updating documentation, tests and mapping file.
Correcting typo in timeline.py.

* Adding missed documentation for hash_account

* Initial code for Mordor driver and browser

* Mordor data provider and browser.

unit tests and documentation

* Fixing some linting errors.

* Fixed a couple of broken tests because of data providers API change.

* Replacing custom json reader with pd.read_json()

Added ability to set query defaults (like cache directory) from provider.
Fixed a bug in path construction for download file.
Clarified the description of the search functionality and corrected Mitre Attack => ATT&CK
Add URL for Mitre
Updated notebook and doc to reflect these changes.

* Fixing lint/formatting errors in vtlookupv3.

Some other random black reformatting
Added test_mordor_browser.py for notebook test.

* Updated formatting for new black version

* Updating pre-commit version

* Bug fix and nasty workaround for old test setup removed in pkg_config.py

* Update MordorData.rst doc with better intro section

* Splitting entities into separate modules

* Moved entities to datamodel package and initial refactoring for pivoting

* Renaming files to lowercase phase 1

* Renaming entities phase 2

* Start of pivot main library

* Commit to re-merge with master

* Code complete - still docs to do.

* Added test case and fix for couple of misc methods in Pivot and Entity

* Phase 1 code complete with docs.

* Fixing the credscan suppression for test_splunk_uploader

* Adding pre-release version, removing old config file.

* Initial dependency separation

* Implemented extras for msticpy install.

Refactored a few classes to make it easier to import and use modules if only partial msticpy install. Installing
Main one is data_providers - dynamically loading drivers. Also eventcluster and auditdextract.
Moved latter two into analysis folder.
Remove unneeded code from keyvault_client.py since Pete's code eliminated the need for them.
Made AzureSentinel and MDE the preferred names for LogAnalytics and MDE drivers.
Fixed up several unit tests to handle partial installs and still produce results (most should be skipped now instead of erroring).
Fixed a random bugs (like GeoIP Maxmind download)
Fixed pivot_register_reader to skip classes that cannot be instantiated (e.g. IPStack if user doesn't have API key)
Added documentation to Installing.rst
Fixed some problems and renamed module locations in notebooks and RST docs.

* Additions/corrections to Installing.rst

* Bandit exception to except: pass

* Removing dropna from read_csv in FoliumMap.ipynb

* Adding requirements-all and pre-commit hook to generate this file

* Adding vt, vt_graph to Sphinx mock list

* Fixing mordor tests and updating azure-mgmt-monitor version in setup.py extras

* PR updates adding comments, some grammar fixes and obfuscation of names.

* Merge victim - double entry for MDE

* Merge tag 'v0.9.0' into ianhelle/MP-Pivot-Phase2-2021-01-04

Fixing some test and linting errors after merge.
Removing lru_cache from ip_lookup in geoip.py

Remove blank line from pivot_register

* Updating version to 0.9.1.beta1

* Updating from PR comments

* Adding notice and badge to Readme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants