Skip to content

Ianhelle/getting started fixes 2021 06 04#170

Merged
ianhelle merged 10 commits intomasterfrom
ianhelle/getting-started-fixes-2021-06-04
Jun 9, 2021
Merged

Ianhelle/getting started fixes 2021 06 04#170
ianhelle merged 10 commits intomasterfrom
ianhelle/getting-started-fixes-2021-06-04

Conversation

@ianhelle
Copy link
Contributor

@ianhelle ianhelle commented Jun 8, 2021

Main changes are:

  • moved most of nb_check logic to azure_ml_tools
  • calling this from init_notebook
  • msticpyconfig checks create an msticpyconfig.yaml from config.json (or populate existing one with empty AzureSentinel section), if an existing one isn't found.
  • Fixes for several smaller issues:
    • TI browser failing when no data
    • Event timeline failing when no source_columns supplied
    • Confusing button names in MpConfigEdit
    • PyLint warnings about not using a context manager around ZipFile, tarfile and open
    • MyPy warnings in domain_utils
    • XForce TI provider information results were prev classified as warnings

ianhelle added 7 commits May 25, 2021 19:44
added recent blog articles and video to a renamed resources page for RTD - blob_articles.
Fixing a query error in kql_sent_azuresentinel.yaml
Fixing an issue with contain naming when creating pivot functions
Unit tests in test_data_viewer.py
Documentation in DataViewer.rst
Notebook in DataViewer.ipynb
Minor changes to __init__.py - importing settings editor
Adding Browshot to mpconfig_defaults.yaml
Fix to pivot.py - logic in expression meant that value for "container was ignored.
…yconfig.yaml if one is not found.

Changed test_nbinit.py for new variants of creating/updating msticpyconfig.yaml
Fixing link in TIProviders.rst
Fixing mypy error in azure_auth_core
Adding check for KV settings before trying to initialize secrets client in provider_settings.py
Adding function to search for file in utility.py
mp_config_edit.py:
 - Renaming "Save File" button to "Save Settings"
- Add config_filepath parameter to specify file to save when settings not imported from a file
Changing item update button to "Update" rather than Save in comp_edit.py
Strip extraneous quotes from string "default" value in mp_config_control.py
Corrected open params to "w" rather than "w+" (write append)
Added "browse" as alias for browse_queries in data_providers.py.
Fixed misreporting of DataFrame column name when column parameter does not match a column in the input data. in pivot_register.py
Adding azure_ml_tools.py (from nb_check.py script)
Added "severities='all'" parameter option. Added friendly warning (rather than exception) when no data is present in the input (either emtpy DF or nothing above the default severities - in ti_browser.py
Fixing additional areas breaking if you don't supply "source_columns" parameter.
Fixing mypy warning in domain_utils.py
Fixed allowing geolite to create a db folder even if parent folders don't exist.
Adding "col" and "column" aliases for lookup_iocs in tilookup.py
Changed score logic for IBM XForce - prevent misreporting of score 1 as warning.
Updating version in _version.py
Added test_azure_ml_tools.py
Modified nbinit to call azure_ml_tools checks
unit_test_lib:
- Added change_directory context manager (with lock) to allow individual test cases to change directory
- Fixed TEST_DATA_PATH to always return absolute path
Fixing pylint warnings about use of context managers in mordor_driver.py, base64unpack.py, morph_charts.py
Pylint warnings suppression about lowercase enums in tilookup.py
Updating change_directory context manager to clean up lock file on exit.
Fixing warning that wasn't processing an f-string in tilookup.py
petebryan
petebryan previously approved these changes Jun 8, 2021
…to functions

Adding module to msticpy.nbtools.rst
Suppressing Bandit warnings for azure_ml_tools.py
petebryan
petebryan previously approved these changes Jun 9, 2021
@ianhelle ianhelle merged commit 4c4b8ca into master Jun 9, 2021
@ianhelle ianhelle deleted the ianhelle/getting-started-fixes-2021-06-04 branch June 9, 2021 21:49
juju4 pushed a commit to juju4/msticpy that referenced this pull request Jun 5, 2022
* data_viewer - Bokeh based data table control

added recent blog articles and video to a renamed resources page for RTD - blob_articles.
Fixing a query error in kql_sent_azuresentinel.yaml
Fixing an issue with contain naming when creating pivot functions

* DataViewer control in data_viewer.py.

Unit tests in test_data_viewer.py
Documentation in DataViewer.rst
Notebook in DataViewer.ipynb
Minor changes to __init__.py - importing settings editor
Adding Browshot to mpconfig_defaults.yaml
Fix to pivot.py - logic in expression meant that value for "container was ignored.

* Removing debugging cell from DataViewer.ipynb

* Handling empty/missing data

* Changed nbinit.py so that it will, by default, try to create a msticpyconfig.yaml if one is not found.

Changed test_nbinit.py for new variants of creating/updating msticpyconfig.yaml
Fixing link in TIProviders.rst
Fixing mypy error in azure_auth_core
Adding check for KV settings before trying to initialize secrets client in provider_settings.py
Adding function to search for file in utility.py
mp_config_edit.py:
 - Renaming "Save File" button to "Save Settings"
- Add config_filepath parameter to specify file to save when settings not imported from a file
Changing item update button to "Update" rather than Save in comp_edit.py
Strip extraneous quotes from string "default" value in mp_config_control.py
Corrected open params to "w" rather than "w+" (write append)
Added "browse" as alias for browse_queries in data_providers.py.
Fixed misreporting of DataFrame column name when column parameter does not match a column in the input data. in pivot_register.py
Adding azure_ml_tools.py (from nb_check.py script)
Added "severities='all'" parameter option. Added friendly warning (rather than exception) when no data is present in the input (either emtpy DF or nothing above the default severities - in ti_browser.py
Fixing additional areas breaking if you don't supply "source_columns" parameter.
Fixing mypy warning in domain_utils.py
Fixed allowing geolite to create a db folder even if parent folders don't exist.
Adding "col" and "column" aliases for lookup_iocs in tilookup.py
Changed score logic for IBM XForce - prevent misreporting of score 1 as warning.
Updating version in _version.py
Added test_azure_ml_tools.py
Modified nbinit to call azure_ml_tools checks
unit_test_lib:
- Added change_directory context manager (with lock) to allow individual test cases to change directory
- Fixed TEST_DATA_PATH to always return absolute path
Fixing pylint warnings about use of context managers in mordor_driver.py, base64unpack.py, morph_charts.py
Pylint warnings suppression about lowercase enums in tilookup.py

* Fixing version format problem in azure_ml_tools

Updating change_directory context manager to clean up lock file on exit.
Fixing warning that wasn't processing an f-string in tilookup.py

* Updates to azure_ml_tools from PR + adding type hints and docstrings to functions

Adding module to msticpy.nbtools.rst

* Adding mypy import warnings to mypy.ini

Suppressing Bandit warnings for azure_ml_tools.py

* Misplace Bandit suppressions in azure_ml_tools
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants