Ianhelle/timeline updates 2022 06 14 #441
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Bump sphinx from 4.3.2 to 4.4.0 Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.3.2 to 4.4.0. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](sphinx-doc/sphinx@v4.3.2...v4.4.0) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating requirements-dev.txt to sync with dependabot updates Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ian Hellen <ianhelle@microsoft.com> Co-authored-by: Pete Bryan <peter.bryan@microsoft.com>
Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
Bumps [respx](https://github.com/lundberg/respx) from 0.17.1 to 0.19.2. - [Release notes](https://github.com/lundberg/respx/releases) - [Changelog](https://github.com/lundberg/respx/blob/master/CHANGELOG.md) - [Commits](lundberg/respx@0.17.1...0.19.2) --- updated-dependencies: - dependency-name: respx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
* Importing a couple of items into init for backward compatibility fixing keyvault authentication error in AML Fixing bug reading None value in mordor_browser * Fixing requirements so that msticpy will still install on Py3.6 Fixing tests for packages to use pkg_resource specifier parsing (which it should have always used) Updating Kqlmagic version to official release. * Adding AzCli URi to exceptions Updating black params in pre-commit and pipelines to remove -t py36 flag * removing unused warnings from import_analyzer.py
…thon (#332) Adding import of magics to nbinit and removing from __init__ and Pivot class. Updating docs (including some auto-gen'd)
* Added refresh and delete functions for keyring cached secrets * Black reformatting of secret_settings
* Simple code de-obfuscator and display for PowerShell * Setting default style to "default" and making display_html DisplayHandle return optional * pep257 doc string linting errors in code_cleanup and code_view * Adding pygments to requirements (this is already a dependency of other core dependencies so should have no impact on install) * Fixing test failure in test_code_view Also linting errors suppressed from bandit, prospector and pylint
* Added Splunk async provider and unit_tests * Fixed incorrect property call Co-authored-by: Pete Bryan <pebryan@microsoft.com> Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
* Fixed incorrect use of httpx.codes enum in odata_driver Added httpx timeout to cybereason_driver and http_base * Added additional context for exceptions. Formatting change for http_base.py Moving import of VTFileBehaviour out of try/except block in vtlookupv3.py * Changing default timeout for httpx client to match requests 30sec for connect, 10sec elsewhere, # Conflicts: # msticpy/data/drivers/cybereason_driver.py # msticpy/sectools/tiproviders/http_base.py * Adding timeouts to missing httpx calls * Splitting keyring into its own module so that we can load without this as a dependency # Conflicts: # msticpy/common/secret_settings.py * Needed type hint in exceptions.py * Putting IPStack APIKey check happen when first used (rather than in __init__) so it doesn't throw exception on loading * prospector config changed produces deprecation warning and non-zero exit code. * Fixing misconfigured prospector.yaml * Test fix for test_cybereason_driver copied from v2 branch * removing version restriction for prospector in Github actions python-package.yaml # Conflicts: # .github/workflows/python-package.yml
Bumps [readthedocs-sphinx-ext](https://github.com/readthedocs/readthedocs-sphinx-ext) from 2.1.4 to 2.1.5. - [Release notes](https://github.com/readthedocs/readthedocs-sphinx-ext/releases) - [Commits](readthedocs/readthedocs-sphinx-ext@2.1.4...2.1.5) --- updated-dependencies: - dependency-name: readthedocs-sphinx-ext dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
Suppressing FP bandit warning Adding Module-Analysis.ipynb notebook to tools
* Fixing case where MpConfigEdit loads with no current msticpyconfig.yaml. Now loads with empty settings rather than throw exception. Added unit test case * Re-ordering arguments so doesn't break inheritance and cause pylint warning (from v2.0 branch) * Reverting changes to args and adding pylint suppressions * Aligning splunk_uploader params with base class * Updated nbwidgets - GetText, QueryTime, GetEnvironmentKey to work with notebook parameters. Fixed query_time widget so that you can reset time range from parameter Added additional unit test for QueryTime setter Simplified SelectAlert (in select_alert and nbdisplay) to remove title line. this was not updating so every alert selected would add another titlel line. Also changed structure and formatting of alert item display - removing CompromisedEntity and adding ProductName. Updated TimeSpan class so it has more flexible constructor Added account_id as a parameter for list_aad_signins_by_account query * Fix for kql_driver - reconnecting for each query loses original kwargs (including mp_az_auth) setting, so reverts to defaults. This can cause errors if the defaults are different to user-specified parameters. There is also a problem in azure_auth.py - if a user has AzureCLI settings, these override everything. I've removed this since we don't really want people configuring auth methods from these settings.
* new msal delegated auth option for graph * Switch to DefaultAzureCredential * renamed MSALAuth * Linting fixes * Add Unit Test and PR changes * Updates to fix tests * Fixed execption error * formatting * Merging in Splunk fixes from #352 * fixed incorrect merge * New MSAL delegated auth methods added and support for this added to Graph providers. Added ability to pass tenant ID to KQL provider fixing issue 333. Minor fixes added incl merge from #352. * hotfix for bug found in testing * Fixed re-auth on query issue in KQL driver * Removing un-needed code * Fixed kql_driver tests * Liniting fixes Co-authored-by: Pete Bryan <pebryan@microsoft.com>
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.4.0 to 4.5.0. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](sphinx-doc/sphinx@v4.4.0...v4.5.0) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
* Fixed bug and simplified/cleaned up code for GeoLiteLookup in geoip.py. Fixed bug where empty/new msticpyconfig.yaml didn't save any settings. * Reorganized logic for handling parameters and failing on invalid file path.
- now works with Kusto config entry without instance suffix - can now supply cluster ALIAS (instance name) instead of actual cluster name in connect or query - added explicit "database" key in query files - can be used instead of the more opaque "data_family.database" encoding in the data_famiies key. Fixed documentation in DataProv-Kusto.rst to correct inaccuracies and update sections on query templates and configuration
Create a new search Check the status of a search Delete a completed search Includes docs and unit tests Updated WorkspaceConfig: If one workspace in config but not called default it is still used by default
Create a new search Check the status of a search Delete a completed search Includes docs and unit tests Updated WorkspaceConfig: If one workspace in config but not called default it is still used by default
…osoft/msticpy into pebryan/2022-4-21_SearchAPI
…378) * Changing the pattern for httpx timeout to default to Timeout(None). This can be overridden in settings and in the case of drivers and TILookup in runtime parameter (timeout=x) Other components use the default. * Adding case for reading list from yaml instead of tuple - now handles any iterable. * Added unit test and made some fixes to pkg_config.py * Bug in test test_pkg_config.py Also in test_code_view.py
* Add Workflow to Tweet * Update tweet.yml * Update tweet.yml Co-authored-by: Ian Hellen <ianhelle@microsoft.com>
…arch Re-enabling pytest.skip in test_nbwidgets.py
…osoft/msticpy into pebryan/2022-4-21_SearchAPI
…ng instance property [update] exposing driver instance property in driver_base, cybereason_driver, kql_driver [update] Updated PivotFunctions.ipynb and PivotFunctions.rst with new behavior [update] Added new SingletonClass to types, rename previous class to SingletonArgsClass in types.py [update] Exposing workspace instance name in wsconfig.py [update] Updating geoip.py to use renamed SingletonArgsClass [update] Adding short name to multiple MDE queries [update] Added process query using only file_hash parameter for pivot query [update] Adding replaceable table parameter to kql_mdatp_user.yaml queries [update] pivots() and get_pivot_list() now supports search string and returns sorted list [update] Adding "pivot" attribute to msticpy after loading pivot [update] Added doc string to txt2df magic in nbmagics.py [update] Pivot is now a singleton, rationalized query time setting, removed adding data provider queries at load [update] Changed clipboard/function text to match usage with imported entities in pivot_browser.py [update] Added use of "explode" in list_to_rows in pivot_pd_accessor.py [update] Importing vt_pivot into pivot_core/__init__.py [update] Changed to support multiple provider instances, removed shortcut query functions, renamed some tables, pivot data queries now use central Pivot.timespan by default [update] Removed provider-specific and IPv4/v6 specific functions - huge simplification in pivot_ti_provider.py [update] Updated and rationalized Pivot tests for new behavior. add test_vt_pivot.py [fix] fixed proper reporting of pivot functions in pivot_container.py [fix] removing deprecated PyLint warning suppression from account.py and process.py [fix] popping extra ioc_type from params in ti_provider_base.py [fix] Fixing warnings for Pylint 2.14.0 - removing deprecated warning types in .pylintrc [fix] Removing duplicate syslog_utils.py (from graphs_plot branch) [fix] Pylint warning in vtlookup.py (from graphs_plot branch) [fix] Pylint warning in entity_graph_tools.py (from graphs_plot branch) [fix] Removing unsupport Pylint warning type from account.py, process.py and base64unpack.py [fix] Updating docs for removed syslog_utils.py in msticpy.transform.rst and msticpy.transform.syslog_utils.rst [fix] Incorrect heading underlining in SettingsEditor.rst
…helle/query-pivot-naming-2022-06-06
[update] Added references to notebooks in PivotFunctions.rst [update] auto-update to DataQueries.rst [fix] formatting error in Installing.rst [update] removing shortcut functions from VT pivots
[update] Timeline - refactored timeline and timeline_values into separate modules: timeline.py, timeline_values.py and timeline_common.py - implemented PlotParams parameter handline for timeline and timeline_values [update] Added deprecation warnings to old PD accessors [update] Added data masking method - mask - to mp_pandas_accessors.py plus a few miscellaneous linting and sourcery fixes. [update] new API docs generated [fix] replace references to mp_timeline accessor with mp_plot in notebooks and RST files
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
…thub) azure-pipelines.yml, and riskiq.py [fix] adding required sphinx packages to azure-pipelines.yml
…thub) azure-pipelines.yml, and riskiq.py [fix] adding required sphinx packages to azure-pipelines.yml
…) - odata_driver.py [fix] missing await in url_checker_async.py
…) - odata_driver.py [fix] missing await in url_checker_async.py
…o ianhelle/query-pivot-naming-2022-06-06
…eline-updates-2022-06-14
…ry-pivot-naming-2022-06-06
…imeline-updates-2022-06-14
…ialized - in vt_pivot.py
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.