Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AbnormalSecurity] Expand integration commands #29558

Conversation

wolyslager
Copy link
Contributor

Contributing to Cortex XSOAR Content

Make sure to register your contribution by filling the contribution registration form

The Pull Request will be reviewed only after the contribution registration form is filled.

Status

  • In Progress
  • Ready
  • In Hold - (Reason for hold)

Related Issues

fixes: link to the issue

Description

This PR expands the Abnormal Security Integration to include new commands through which users can interact with new endpoints in the Abnormal API. These commands expose information related to Vendors, Vendor Cases, and Abuse Mailbox submissions that were previously not available to the user through the cortex tool. Additionally, this PR expands the number of fields returned by the threat-details endpoint to provide users with more information.

Must have

  • Tests
  • Documentation

@content-bot content-bot added Contribution Thank you! Contributions are always welcome! External PR Partner Support Level Indicates that the contribution is for Partner supported pack labels Sep 7, 2023
@content-bot content-bot changed the base branch from master to contrib/wolyslager_expand-integration-commands September 7, 2023 16:31
@content-bot
Copy link
Collaborator

Thank you for your contribution. Your generosity and caring are unrivaled! Make sure to register your contribution by filling the Contribution Registration form, so our content wizard @MichaelYochpaz will know the proposed changes are ready to be reviewed.
For your convenience, here is a link to the contributions SLAs document.

@wolyslager
Copy link
Contributor Author

Hey @MichaelYochpaz ! I have another PR where I will be implementing the fetch_incidents() logic - was curious to know what the preferred approach is. Should I stack the PR on this one or wait until this one is merged before submitting the other one? Let me know :) thanks!

@wolyslager
Copy link
Contributor Author

wolyslager commented Sep 8, 2023

The validation seems to be working on my end locally but not on the circleci run
image

  • when I run the suggested format command I see this
image

@MichaelYochpaz
Copy link
Contributor

Hey @MichaelYochpaz ! I have another PR where I will be implementing the fetch_incidents() logic - was curious to know what the preferred approach is. Should I stack the PR on this one or wait until this one is merged before submitting the other one? Let me know :) thanks!

@wolyslager Hey. If adding the fetch incidents logic isn't urgent, I think it'll be better to do that in a separate PR after merging this one because there will probably be conflicts you'll have to resolve in the other PR once one of them is merged, if you work on both simultaneously.

As for the failing validation - it fails in a step that runs demisto-sdk secrets --post-commit --ignore-entropy.

This can be fixed by adding the detected IP & mail (assuming they're fake and aren't actual secrets) to the Packs/AbnormalSecurity/.secrets-ignore file.

@wolyslager
Copy link
Contributor Author

Hey @MichaelYochpaz ! I have another PR where I will be implementing the fetch_incidents() logic - was curious to know what the preferred approach is. Should I stack the PR on this one or wait until this one is merged before submitting the other one? Let me know :) thanks!

@wolyslager Hey. If adding the fetch incidents logic isn't urgent, I think it'll be better to do that in a separate PR after merging this one because there will probably be conflicts you'll have to resolve in the other PR once one of them is merged, if you work on both simultaneously.

As for the failing validation - it fails in a step that runs demisto-sdk secrets --post-commit --ignore-entropy.

This can be fixed by adding the detected IP & mail (assuming they're fake and aren't actual secrets) to the Packs/AbnormalSecurity/.secrets-ignore file.

Awesome - good to know. So our our target to have both this PR and the fetch_incidents PR merged is before end of September. If you think that is doable doing the PRs sequentially then that sounds good to me! If you would prefer to have all of the changes in one PR and you think that would be faster happy to do that as well.

@MichaelYochpaz
Copy link
Contributor

Awesome - good to know. So our our target to have both this PR and the fetch_incidents PR merged is before end of September. If you think that is doable doing the PRs sequentially then that sounds good to me! If you would prefer to have all of the changes in one PR and you think that would be faster happy to do that as well.

I think adding all the changes to this PR (adding the fetch implementation here to this PR) will be the fastest then.
Make sure to update the current release notes with the changes you're merging in

@wolyslager
Copy link
Contributor Author

Awesome - good to know. So our our target to have both this PR and the fetch_incidents PR merged is before end of September. If you think that is doable doing the PRs sequentially then that sounds good to me! If you would prefer to have all of the changes in one PR and you think that would be faster happy to do that as well.

I think adding all the changes to this PR (adding the fetch implementation here to this PR) will be the fastest then. Make sure to update the current release notes with the changes you're merging in

@MichaelYochpaz perfect, just added the remaining changes to this PR - I really appreciate it!

@MichaelYochpaz
Copy link
Contributor

@wolyslager I see that you've pushed the fetch-incidents changes. Great.

A small update - PR review will be slightly delayed since it is the holiday season in Israel, and we have a holiday this weekend (15th-17th of September), I apologize in advance.

It seems like some validations are failing, and that the unit-tests coverage isn't high enough, so you can work on those in the meantime.

@wolyslager
Copy link
Contributor Author

@wolyslager I see that you've pushed the fetch-incidents changes. Great.

A small update - PR review will be slightly delayed since it is the holiday season in Israel, and we have a holiday this weekend (15th-17th of September), I apologize in advance.

It seems like some validations are failing, and that the unit-tests coverage isn't high enough, so you can work on those in the meantime.

Sounds good @MichaelYochpaz - enjoy the Holiday! I will work on these changes in the meantime :)

@MichaelYochpaz MichaelYochpaz changed the title Expand integration commands [AbnormalSecurity] Expand integration commands Sep 19, 2023
Copy link
Contributor

@MichaelYochpaz MichaelYochpaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, looks pretty good overall, good job :)
Left a few minor notes.

@wolyslager
Copy link
Contributor Author

Hey, looks pretty good overall, good job :) Left a few minor notes.

Thanks @MichaelYochpaz ! Addressed your comments - I appreciate the thoughtful review. Let me know if something is still not up to par and I'll take care of it. Look forward to getting this merged in

@MichaelYochpaz
Copy link
Contributor

@wolyslager Good job :)

Please run demisto-sdk generate-docs -i "Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.yml" to update the README following your changes (go over the changes the SDK did. Sometimes it requires some manual tweaking / changes).

The unit-tests pipeline also seems to fail in a few steps.

The "Infrastructure testing" step there is an issue we had that's unrelated to your PR.
Merge from master and it should be fixed.

The "Run Unit Testing And Lint - Docker Image:from-yml" has a few unit-tests for this integration failing that you'll have to address / fix.

There's also the "Generate coverage reports" step failing there, since there are now additional functions but not tests, so the coverage percentage dropped.
You can, however, add # pragma: nocover for the main function, since it doesn't require a unit-test. It should increase your coverage since those lines won't be counted. Maybe that will get you over the required amount.

Should look like:

def main():  # pragma: no cover

Feel free to reach out to me on Slack if you need any help / have questions.

@wolyslager
Copy link
Contributor Author

@wolyslager Good job :)

Please run demisto-sdk generate-docs -i "Packs/AbnormalSecurity/Integrations/AbnormalSecurity/AbnormalSecurity.yml" to update the README following your changes (go over the changes the SDK did. Sometimes it requires some manual tweaking / changes).

The unit-tests pipeline also seems to fail in a few steps.

The "Infrastructure testing" step there is an issue we had that's unrelated to your PR. Merge from master and it should be fixed.

The "Run Unit Testing And Lint - Docker Image:from-yml" has a few unit-tests for this integration failing that you'll have to address / fix.

There's also the "Generate coverage reports" step failing there, since there are now additional functions but not tests, so the coverage percentage dropped. You can, however, add # pragma: nocover for the main function, since it doesn't require a unit-test. It should increase your coverage since those lines won't be counted. Maybe that will get you over the required amount.

Should look like:

def main():  # pragma: no cover

Feel free to reach out to me on Slack if you need any help / have questions.

@MichaelYochpaz I do have a few questions, how can I reach out through slack?

@MichaelYochpaz
Copy link
Contributor

@wolyslager You can join our DFIR slack here: https://start.paloaltonetworks.com/join-our-slack-community.
Once joined, just look up my name up and message me.

@MichaelYochpaz MichaelYochpaz added the pending-contributor The PR is pending the response of its creator label Sep 21, 2023
@wolyslager
Copy link
Contributor Author

@wolyslager You can join our DFIR slack here: https://start.paloaltonetworks.com/join-our-slack-community. Once joined, just look up my name up and message me.

For some reason the invite expired immediately 🤔
image
@MichaelYochpaz

content-bot and others added 28 commits October 2, 2023 11:32
…29948)

* Remediation guidance update

* Empty-Commit

* Empty-Commit

Co-authored-by: Chait A <112722030+capanw@users.noreply.github.com>
Co-authored-by: Yehuda Rosenberg <90599084+RosenbergYehuda@users.noreply.github.com>
…st-endpoint-info*** command. GI-1278 (demisto#29944)

* [malwarebytes-254] Fixed an IndexError issue with ***!malwarebytes-list-endpoint-info*** command. GI-1278  (demisto#29791)

* GI-1278

* Added unit test for index error

---------

Co-authored-by: Yehuda Rosenberg <90599084+RosenbergYehuda@users.noreply.github.com>
Co-authored-by: Yehuda <yrosenberg@paloaltonetworks.com>

* RN

* format

* docker

* RN

---------

Co-authored-by: rskumar-mwb <48316606+rskumar-mwb@users.noreply.github.com>
Co-authored-by: Yehuda Rosenberg <90599084+RosenbergYehuda@users.noreply.github.com>
Co-authored-by: Yehuda <yrosenberg@paloaltonetworks.com>
* fixed an issue where Could not decode attachments

* added rn

* added rn

* update docker

* update rn

* update rn

* update test and docker

* Bump pack from version MicrosoftExchangeOnline to 1.2.27.

* update version and docker

---------

Co-authored-by: Content Bot <bot@demisto.com>
* remove_service_principals_command was edited and a UT was added for it.
get_service_principal_command was implemented and a UT was added for it.

* update command with UT

* password add and remove with UTs

* autopep8

* change application to service principal

* add new command

* remove dev from yml

* CR fixes

* pre commit

* README.md

* improve implementation

* add two UTs

* fix pre commit

* fix pre commit

* add UT

* Update Packs/MicrosoftGraphApplications/Integrations/MicrosoftGraphApplications/MicrosoftGraphApplications.py

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>

* Update Packs/MicrosoftGraphApplications/Integrations/MicrosoftGraphApplications/MicrosoftGraphApplications.yml

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>

* Update Packs/MicrosoftGraphApplications/Integrations/MicrosoftGraphApplications/MicrosoftGraphApplications.py

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>

* Update Packs/MicrosoftGraphApplications/Integrations/MicrosoftGraphApplications/MicrosoftGraphApplications.py

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>

* CR's fixes

* mypy error

* fixture

* fix description

* fix description

* CR's fixes

* command examples were added

* Demo's fixes

* docker image

---------

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>
* [Marketplace Contribution] AWS - IAM Identity Center (demisto#28559)

* Update Packs/AWS-IAMIdentityCenter/Integrations/AWSIAMIdentityCenter/AWSIAMIdentityCenter.py

* Update AWSIAMIdentityCenter.pyl

* Update pack_metadata.json

* Update AWSIAMIdentityCenter_description.md

* Update AWSIAMIdentityCenter.ymll

* Delete generated API module

* Removed unnecessary package

---------

Co-authored-by: sepaprivate <113604678+sepaprivate@users.noreply.github.com>
Co-authored-by: anas-yousef <44998563+anas-yousef@users.noreply.github.com>

* Update AWSIAMIdentityCenter.yml

* Update README.md

* Update README.md

* Fixed Pack README

---------

Co-authored-by: xsoar-bot <67315154+xsoar-bot@users.noreply.github.com>
Co-authored-by: sepaprivate <113604678+sepaprivate@users.noreply.github.com>
Co-authored-by: anas-yousef <44998563+anas-yousef@users.noreply.github.com>
* added the incident-list-command

* HelloWorld all commands

* formattin classifier and mapper + updated layout

* fix documentation

* changes to mocked responses

* linting

* added rn

* validation and pre commit

* fixed readme & validations

* removed dev id

* fix desc and rn

* test + fixes

* fixes

* fix yml

* lint fixes

* coverage

* fix yml

* fix pb

* CR fixes

* Apply suggestions from code review

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* cr fixes

* rn fix

* pre-commit fix

---------

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>
* Add the Malware Triage playbook

* Add fromversion

* Update the playbook image

* Update the release notes

* Update the playbook configuration

* Update the playbook readme

* Add more output context paths

* Add more output context paths

* Update te yaml

* Update the image

* Update the image

* Add fromversion

* Update the readme

* Update test data

* Update the yaml

* Update the context output types

Co-authored-by: Mislav Sever <46045160+MislavReversingLabs@users.noreply.github.com>
Co-authored-by: Michael Yochpaz <8832013+MichaelYochpaz@users.noreply.github.com>
* fix the investigation tab layout

* fix the investigation tab layout

* fix the investigation tab layout

* revert file deletion
* enhance-modeling-rules

* add-xsiam-content-to-README.md

* update release-notes

* remove-reduntant-quotes-from-user-groups

* Update Packs/PrismaCloud/ReleaseNotes/4_2_6.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* update release-notes

* remove-xsiam-2.0-fields

* update release-notes

---------

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>
…#29949)

* fix result in tanium-tr-intel-doc-delete

* bump version

* fix test

* Update Packs/TaniumThreatResponse/Integrations/TaniumThreatResponseV2/TaniumThreatResponseV2.py

* docker
* playbook - add exception requested

* RN

* RN
* Fixing fetch unit tests

* Added unit tests, need to add documentation

* Added docstrings to unit tests

* Removed trace-id

* Removed Test from yml

* Added context data to commands

* Updated fromVerion in incident fields

* Fixed format errors

* Added README to scripts

* Added commands to README

* Added RNs

* Restore pack README

* Restore pack README

* Update pack-ignore

* Added docstrings to .py file

* Updated TPB, Layout

* Revert TPB

* Added fetch incidents to README.md

* Added more documentation

* Update Packs/CrowdStrikeFalcon/ReleaseNotes/1_11_10.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Update Packs/CrowdStrikeFalcon/ReleaseNotes/1_11_10.md

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>

* Almost done with CR

* Increased timeout of TPB, added commands to TPB

* Added docs comments

* Fixed pre-commit errors

* Updated docker image in RNs

* Removed unecessary package

* Added to secrets ignore

* Fixed desriptions

* Fixed indentation

* Removed unnecessary tests

* Fixed conflcts

* Fixed incident fields names

* Increased timeout

* Increased timeout of task in test playbook

* Added Service Type to incident fields and mappers

* Added unit tests to scripts

* Improved documentation of unit tests

* Fixed unit tests imports

* Added named parameters to unit tests

* Added new lines to scripts unit tests

* Added handling if last fetch filter is empty

* Remove unnecessary import

* Removed incorrect incident field from mapper

* Reverted old RNs changes

* Update 1_11_11.md

* Reverted old RNs changes

* Removed updated docker image from RN

* Update 1_11_11.md

* Refactored lots of code, unit tests passed :)

* Updated docs wording

* Fixed pre-commit error

* Removed unnecessary extend to previous fetched ids

* Changed pack version to minor

* Deleted unnecessary arguments

* Kept mechansim of extned

* Fixed pre-commit

* Updated docker images

* Fixed argument position

* Passed is_paginating bool to check whether we are doing pagination or not

* Added is_paginating to if statement

---------

Co-authored-by: ShirleyDenkberg <62508050+ShirleyDenkberg@users.noreply.github.com>
* fixes in guardicore v2 fetch

* tests fix

* update test file

* rns

* validations
* Recordedfuture v2.5.1 (demisto#29905)

* Add AI insights field

* Fix mapper

* Bump RecordedFuture app version

* Add release notes

* Update docker image

---------

Co-authored-by: Michael Yochpaz <8832013+MichaelYochpaz@users.noreply.github.com>

* Bump Docker version

---------

Co-authored-by: Yaroslav Nestor <yaroslav.nestor22@gmail.com>
Co-authored-by: Michael Yochpaz <8832013+MichaelYochpaz@users.noreply.github.com>
* [Marketplace Contribution] GCP IAM - Content Pack Update (demisto#29692)

* "contribution update to pack "GCP IAM""

* Revert changes unrelated to new command

* Change version bump to revision & fix release notes

* Add the new command to `Command Required Permissions` list on the description

* Use numbers for `lifetime` parameter

* Minor fixes & pre-commit

* Undo ID rename

---------

Co-authored-by: Michael Yochpaz <8832013+MichaelYochpaz@users.noreply.github.com>

* Update Docker version

* Fix `DS108` validation errors

* Update README

* Minor fixes

* Bump version

* Bump Docker version

* Add missing `iamcredentials` client

---------

Co-authored-by: xsoar-bot <67315154+xsoar-bot@users.noreply.github.com>
Co-authored-by: Michael Yochpaz <8832013+MichaelYochpaz@users.noreply.github.com>
* deprecated

* description

* [known_words]

* description

* format

* Update Packs/TruSTAR/ReleaseNotes/2_1_24.md

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>

---------

Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>
* Devo bug fix v1.2.8 (demisto#29714)

* Modified the get_time_range function and updated the test cases for all formats of from time input.

* Updated the docker image and created the release notes and a small lint fix.

* Supported Python datetime object format

* Modified the test case for python datetime object.

* Small fix in to_to parameter to accept python datetime object.

* Updated the docker image.

* Updated the release notes.

* Updated the docker image.

* Updated the docker image tag.

* Updated the release notes.

---------

Co-authored-by: Kapil Bisen <kapil@metronlabs.com>
Co-authored-by: manas-metron <108781525+manas-metron@users.noreply.github.com>
Co-authored-by: Sahil Kakad <108385016+sahil-metron@users.noreply.github.com>
Co-authored-by: kapil-metron <58544320+kapil-metron@users.noreply.github.com>

* RN

---------

Co-authored-by: tejashree-metron <121784786+tejashree-metron@users.noreply.github.com>
Co-authored-by: Kapil Bisen <kapil@metronlabs.com>
Co-authored-by: manas-metron <108781525+manas-metron@users.noreply.github.com>
Co-authored-by: Sahil Kakad <108385016+sahil-metron@users.noreply.github.com>
Co-authored-by: kapil-metron <58544320+kapil-metron@users.noreply.github.com>
Co-authored-by: Yehuda <yrosenberg@paloaltonetworks.com>
* Fix for 'MDE Malware - Incident Enrichment' playbook

* updated PNG playbook file

* RN

* RN

* removed the new conditional task and changed the DT expression within the 'key' value of tasks 46 and 47.

* DT was removed from the playbook

* re-added changes after merging from master

* DT was removed from the problematic playbook tasks & added new conditional task to check the incident fields value before setting the new keys

* removed the validation for 'MicrosoftATP.Alert.Evidence' context key from the test playbook file. removed the 'SetIfEmpty' transformer from tasks number 46 & 47 within the MDE playbook file.

* changed the name, description and condition for task number 56. added the 'manageremailaddress' incident field to the 'setIncident' automation used within task number 52.
* support svg theme

* support svg theme

* poetry

* fix

* fix

* fix copy dd images to prod bucket

* fix test

* Bump pack from version Jira to 3.1.3.

* revert poetry

* fix test

* update release notes

* fix

* fix test

* Bump pack from version Box to 3.1.34.

* fix

* Bump pack from version Box to 3.1.35.

* CR

---------

Co-authored-by: Content Bot <bot@demisto.com>
* Replaced new params with old params

* added rn

* Formatted integration

* Fixed rn
* add PAN-OS to xpanse MP and core (demisto#29826)

* docker

* RN

* docker demisto#2

---------

Co-authored-by: johnnywilkes <32227961+johnnywilkes@users.noreply.github.com>
Co-authored-by: Yehuda <yrosenberg@paloaltonetworks.com>
…tegration-commands"

This reverts commit 4b14404, reversing
changes made to 2dc09df.
* replaced with gitlab predefined variables

* update files

* revert changes

* Apply suggestions from code review

support default values

* test

* add logs

* add logs

* check without json

* update

* update

* test

* delete echos

* test

* test

* Make CI more dynamic (demisto#29776)

---------

Co-authored-by: yucohen <yucohen@paloaltonetworks.com>
Co-authored-by: Yehuda Deutsch <113076699+ydeutsch@users.noreply.github.com>
…tegration-commands"

This reverts commit f6d9c8e, reversing
changes made to e4e7601.
@MichaelYochpaz
Copy link
Contributor

Closing, as this PR was reopened on #29994

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Contribution Form Filled Whether contribution form filled or not. Contribution Thank you! Contributions are always welcome! External PR Partner Support Level Indicates that the contribution is for Partner supported pack Partner Partner-Approved post-demo ready-for-instance-test In contribution PRs, this label will cause a trigger of a build with a modified pack from the PR.
Projects
None yet