Skip to content

EWSO365: Fix duplicate detection for malformed Message-ID bracket variants#43811

Merged
almog2296 merged 9 commits intomasterfrom
XSUP-66564-dedup-EWSO365
Apr 14, 2026
Merged

EWSO365: Fix duplicate detection for malformed Message-ID bracket variants#43811
almog2296 merged 9 commits intomasterfrom
XSUP-66564-dedup-EWSO365

Conversation

@almog2296
Copy link
Copy Markdown
Contributor

@almog2296 almog2296 commented Apr 12, 2026

Fixes: XSUP-66564

Description

Extended is_item_duplicate to check all four bracket forms of Message-IDs (id, <id>, id>, <id) to prevent duplicate incidents caused by inconsistent EWS formatting across fetches. Added unit tests for the new variants.
Some vendor send email with non RFC compliant format (only one > for example) which caused the is_item_duplicate to miss it.

@content-bot
Copy link
Copy Markdown
Contributor

🤖 AI-Powered Code Review Available

You can leverage AI-powered code review to assist with this PR!

Available Commands:

  • @marketplace-ai-reviewer start review - Initiate a full AI code review
  • @marketplace-ai-reviewer re-review - Incremental review for new commits

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 12, 2026

Coverage

Coverage Report
FileStmtsMissCoverMissing
Packs/MicrosoftExchangeOnline/Integrations/EWSO365
   EWSO365.py3734587%215, 226, 238, 240, 318, 557–560, 785, 953, 994, 1113–1114, 1298, 1304–1305, 1333–1334, 1361, 1363, 1372–1376, 1662–1664, 1667, 1672, 1688, 1727–1735, 1756, 2018–2019, 2042
TOTAL3734587% 

Tests Skipped Failures Errors Time
92 0 💤 0 ❌ 0 🔥 3.801s ⏱️

@almog2296 almog2296 added ready-for-pipeline-running Whether the pr is ready for running the whole pipeline, including testing on SAAS machines ready-for-ai-review The PR is ready for reviewing the PR with the AI Reviewer. labels Apr 12, 2026
@marketplace-ai-reviewer marketplace-ai-reviewer removed the ready-for-ai-review The PR is ready for reviewing the PR with the AI Reviewer. label Apr 12, 2026
@marketplace-ai-reviewer
Copy link
Copy Markdown
Contributor

🤖 Analysis started. Please wait for results...

@marketplace-ai-reviewer
Copy link
Copy Markdown
Contributor

🤖 AI Review Disclaimer

This review was generated by an AI-powered tool and may contain inaccuracies. Please be advised, and we extend our sincere apologies for any inconvenience this may cause.

Copy link
Copy Markdown
Contributor

@marketplace-ai-reviewer marketplace-ai-reviewer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi there! Thanks for your contribution to the Microsoft Exchange Online pack. I've reviewed the changes and found a couple of minor issues to address. Specifically, the pack metadata is missing the mandatory created key and the 'Microsoft' keyword, and there is a small typo in the integration's docstring. Thanks again, and let me know if you need any help with these!

Additionally, please address the following file-level notes:

  • Packs/MicrosoftExchangeOnline/pack_metadata.json: - The vendor name 'Microsoft' is missing from the keywords list.
  • The mandatory key created is missing from the pack metadata.

@noydavidi, @JudahSchwartz please review and approve the results generated by the AI Reviewer by responding 👍 on this comment.

Comment thread Packs/MicrosoftExchangeOnline/Integrations/EWSO365/EWSO365.py
@content-bot
Copy link
Copy Markdown
Contributor

Validate summary
The following errors were thrown as a part of this pr: .
If the AG100 validation in the pre-commit GitHub Action fails, the pull request cannot be force-merged.

Verdict: PR can be force merged from validate perspective? ✅

@almog2296 almog2296 requested a review from Shellyber April 14, 2026 11:57
Comment thread Packs/MicrosoftExchangeOnline/Integrations/EWSO365/EWSO365.py
Copy link
Copy Markdown
Contributor

@Shellyber Shellyber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@almog2296 almog2296 merged commit b450c49 into master Apr 14, 2026
29 of 30 checks passed
@almog2296 almog2296 deleted the XSUP-66564-dedup-EWSO365 branch April 14, 2026 12:32
MosheEichler pushed a commit that referenced this pull request Apr 14, 2026
…iants (#43811)

* unitets + dedup function modify

* fix is_item_duplicate comments

* add release notes

* add release notes

* fix is_item_duplicate comments

* fix after ai review

* add debug messages
barryyosi-panw pushed a commit that referenced this pull request Apr 14, 2026
…iants (#43811)

* unitets + dedup function modify

* fix is_item_duplicate comments

* add release notes

* add release notes

* fix is_item_duplicate comments

* fix after ai review

* add debug messages
MosheEichler added a commit that referenced this pull request Apr 16, 2026
…43855)

* fixes

* RN

* Ruff fixes

* pre-commit fixes

* CRTX-240216 - Enhancement - msft_azure_ad_raw   (#43786)

* update xif

* update xif with sign-in log enhancements and release notes

* fix: add missing location field to msft_azure_ad_raw schema

* Core api bug (#43725)

* added the retry mechanism

* added rn

* fix

* Apply suggestions from code review

Co-authored-by: merit-maita <49760643+merit-maita@users.noreply.github.com>

---------

Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>

* TelemtryCortexRunScriptOnEndpoint (#43472)

* TelemtryCortexRunScriptOnEndpoint

* rn

* ai reviwer

* RN

* Fixed  for F5 ASM cs3 field and improving ipv4/6 implementation  (#43845)

* fixed cs3 fields

* fixed ipv4 and ipv6 fields

* added release notes

* removed alter

* CIAC-16216 - marketplacev2 content items should show in platform (#43808)

* CIAC-16216

* CIAC-16216

* Update 1_0_1.md

* Update docker image

* Update readme files

* Add docker update to release notes

* Add docker update to release notes

* XSUP-66843_add_azure_resourceGraph_gov_support (#43805)

* XSUP-66843_add_azure_resourceGraph_gov_support

* ai review update

* doc review and fixes

* empty commit

* cr fixes

* fix adf comment handling (#43813)

* fix adf comment handling

* RN

* unittests

* fix ai review

* fix

* docker image

* fix RN

* Bump pack from version Jira to 3.3.28.

---------

Co-authored-by: Content Bot <bot@demisto.com>

* Fix Crtx 232255 (#43713)

* Fix caps in owner file

* licensing fix

* RN

* Address CR

* Bump pack from version CortexXDR to 6.3.14.

* Fix secrets detection issue in CortexXDRIR_test.py

* Fix validations

* Address CR

* Address CR

* Docker image

* Fix validations

* fix validation

* rn

* Bump pack from version CortexXDR to 6.3.15.

* rn

* Apply suggestion from @kobymeir

Co-authored-by: Koby Meir <kobymeir@users.noreply.github.com>

* Update Packs/CortexXDR/ReleaseNotes/6_3_15.md

Co-authored-by: Arad Carmi <62752352+AradCarmi@users.noreply.github.com>

* rn

* RN

---------

Co-authored-by: Content Bot <bot@demisto.com>
Co-authored-by: Koby Meir <kobymeir@users.noreply.github.com>
Co-authored-by: Arad Carmi <62752352+AradCarmi@users.noreply.github.com>

* EWSO365: Fix duplicate detection for malformed Message-ID bracket variants (#43811)

* unitets + dedup function modify

* fix is_item_duplicate comments

* add release notes

* add release notes

* fix is_item_duplicate comments

* fix after ai review

* add debug messages

* UnEscapeURLs: Fixed truncating URLs issue (#43830)

* the fix

* RN

* fix validation

* Update UnEscapeURLs.yml

* Add Superna Zero Trust integration pack (#43826)

* Add Superna Zero Trust integration pack (#43685)

* Add Superna Zero Trust integration pack

New partner pack for ransomware response automation:
- Critical path snapshot creation
- User NAS lockout/unlock capabilities
- HTTP 429 error handling for rate limiting
- 4 automated playbooks with integration availability checks
- Complete documentation and playbook screenshots

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Address PR review feedback for Superna Zero Trust pack

Unit Tests:
- Renamed snapshot_cmd -> snapshot_critical_paths_command
- Renamed lockout_cmd -> lockout_user_command
- Renamed unlock_cmd -> unlock_user_command
- Updated main() dispatch to use new function names
- Fixed SupernaZeroTrust_test.py: Client init now uses api_key= (not headers=)
- Fixed test assertions to match actual outputs structure (outputs_prefix, outputs keys)

Integration YAML (SupernaZeroTrust.yml):
- Added sectionorder: [Connect] to satisfy ST111
- Updated dockerimage from demisto/python3:3.10.13.86272 to demisto/python3:3.12.13.7444307 (DO106)

New File:
- Created SupernaZeroTrust_description.md with prerequisites, API key instructions, and parameter table (DS104)

Documentation:
- Rewrote Integrations/SupernaZeroTrust/README.md with full command documentation including arguments, context outputs, and examples for all 3 commands (RM110)
- Rewrote Packs/SupernaZeroTrust/README.md with full pack overview, use cases, contents table, configuration, and support info (RM104)

Pack Metadata:
- Fixed categories: replaced "Incident Response" (invalid) with "Endpoint" (PA103)

Validation:
- Added ignore=BA129 to .pack-ignore for superna-zt-lockout-user and superna-zt-unlock-user compliance policy warnings, as recommended by maintainers (BA129)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix ST111 validation, unit test trailing slash, and suppress RM104/RM110

- ST111: Added section: Connect to all 4 parameters in SupernaZeroTrust.yml
  (base_url, credentials, insecure, proxy)
- Unit test: Fixed test_client_initialization trailing slash assertion to use
  rstrip('/') for compatibility across demisto-sdk versions
- RM104/RM110: Added ignores to .pack-ignore (suppressible warnings)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>

* Fix pre-commit issues

---------

Co-authored-by: Andrew MacKay <andrew.mackay@superna.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>
Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>

* fix validate

* Add parameter

* ruff fixes

---------

Co-authored-by: akshotiamit-pa <aakshoti@paloaltonetworks.com>
Co-authored-by: merit-maita <49760643+merit-maita@users.noreply.github.com>
Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>
Co-authored-by: hyaffe839 <hyaffe@paloaltonetworks.com>
Co-authored-by: sharonfi99 <147984773+sharonfi99@users.noreply.github.com>
Co-authored-by: dtroushinsky <dtroushinsky@paloaltonetworks.com>
Co-authored-by: masulin97 <masulin@paloaltonetworks.com>
Co-authored-by: inbalapt1 <164751454+inbalapt1@users.noreply.github.com>
Co-authored-by: Content Bot <bot@demisto.com>
Co-authored-by: Tal Zichlinsky <35036457+talzich@users.noreply.github.com>
Co-authored-by: Koby Meir <kobymeir@users.noreply.github.com>
Co-authored-by: Arad Carmi <62752352+AradCarmi@users.noreply.github.com>
Co-authored-by: almog2296 <alabudi@paloaltonetworks.com>
Co-authored-by: Israel Lappe <79846863+ilappe@users.noreply.github.com>
Co-authored-by: Content Bot <55035720+content-bot@users.noreply.github.com>
Co-authored-by: Andrew MacKay <andrew.mackay@superna.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>
mcvic1rj pushed a commit to mcvic1rj/xsoar-content that referenced this pull request Apr 16, 2026
…iants (demisto#43811)

* unitets + dedup function modify

* fix is_item_duplicate comments

* add release notes

* add release notes

* fix is_item_duplicate comments

* fix after ai review

* add debug messages
mcvic1rj pushed a commit to mcvic1rj/xsoar-content that referenced this pull request Apr 16, 2026
…emisto#43855)

* fixes

* RN

* Ruff fixes

* pre-commit fixes

* CRTX-240216 - Enhancement - msft_azure_ad_raw   (demisto#43786)

* update xif

* update xif with sign-in log enhancements and release notes

* fix: add missing location field to msft_azure_ad_raw schema

* Core api bug (demisto#43725)

* added the retry mechanism

* added rn

* fix

* Apply suggestions from code review

Co-authored-by: merit-maita <49760643+merit-maita@users.noreply.github.com>

---------

Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>

* TelemtryCortexRunScriptOnEndpoint (demisto#43472)

* TelemtryCortexRunScriptOnEndpoint

* rn

* ai reviwer

* RN

* Fixed  for F5 ASM cs3 field and improving ipv4/6 implementation  (demisto#43845)

* fixed cs3 fields

* fixed ipv4 and ipv6 fields

* added release notes

* removed alter

* CIAC-16216 - marketplacev2 content items should show in platform (demisto#43808)

* CIAC-16216

* CIAC-16216

* Update 1_0_1.md

* Update docker image

* Update readme files

* Add docker update to release notes

* Add docker update to release notes

* XSUP-66843_add_azure_resourceGraph_gov_support (demisto#43805)

* XSUP-66843_add_azure_resourceGraph_gov_support

* ai review update

* doc review and fixes

* empty commit

* cr fixes

* fix adf comment handling (demisto#43813)

* fix adf comment handling

* RN

* unittests

* fix ai review

* fix

* docker image

* fix RN

* Bump pack from version Jira to 3.3.28.

---------

Co-authored-by: Content Bot <bot@demisto.com>

* Fix Crtx 232255 (demisto#43713)

* Fix caps in owner file

* licensing fix

* RN

* Address CR

* Bump pack from version CortexXDR to 6.3.14.

* Fix secrets detection issue in CortexXDRIR_test.py

* Fix validations

* Address CR

* Address CR

* Docker image

* Fix validations

* fix validation

* rn

* Bump pack from version CortexXDR to 6.3.15.

* rn

* Apply suggestion from @kobymeir

Co-authored-by: Koby Meir <kobymeir@users.noreply.github.com>

* Update Packs/CortexXDR/ReleaseNotes/6_3_15.md

Co-authored-by: Arad Carmi <62752352+AradCarmi@users.noreply.github.com>

* rn

* RN

---------

Co-authored-by: Content Bot <bot@demisto.com>
Co-authored-by: Koby Meir <kobymeir@users.noreply.github.com>
Co-authored-by: Arad Carmi <62752352+AradCarmi@users.noreply.github.com>

* EWSO365: Fix duplicate detection for malformed Message-ID bracket variants (demisto#43811)

* unitets + dedup function modify

* fix is_item_duplicate comments

* add release notes

* add release notes

* fix is_item_duplicate comments

* fix after ai review

* add debug messages

* UnEscapeURLs: Fixed truncating URLs issue (demisto#43830)

* the fix

* RN

* fix validation

* Update UnEscapeURLs.yml

* Add Superna Zero Trust integration pack (demisto#43826)

* Add Superna Zero Trust integration pack (demisto#43685)

* Add Superna Zero Trust integration pack

New partner pack for ransomware response automation:
- Critical path snapshot creation
- User NAS lockout/unlock capabilities
- HTTP 429 error handling for rate limiting
- 4 automated playbooks with integration availability checks
- Complete documentation and playbook screenshots

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Address PR review feedback for Superna Zero Trust pack

Unit Tests:
- Renamed snapshot_cmd -> snapshot_critical_paths_command
- Renamed lockout_cmd -> lockout_user_command
- Renamed unlock_cmd -> unlock_user_command
- Updated main() dispatch to use new function names
- Fixed SupernaZeroTrust_test.py: Client init now uses api_key= (not headers=)
- Fixed test assertions to match actual outputs structure (outputs_prefix, outputs keys)

Integration YAML (SupernaZeroTrust.yml):
- Added sectionorder: [Connect] to satisfy ST111
- Updated dockerimage from demisto/python3:3.10.13.86272 to demisto/python3:3.12.13.7444307 (DO106)

New File:
- Created SupernaZeroTrust_description.md with prerequisites, API key instructions, and parameter table (DS104)

Documentation:
- Rewrote Integrations/SupernaZeroTrust/README.md with full command documentation including arguments, context outputs, and examples for all 3 commands (RM110)
- Rewrote Packs/SupernaZeroTrust/README.md with full pack overview, use cases, contents table, configuration, and support info (RM104)

Pack Metadata:
- Fixed categories: replaced "Incident Response" (invalid) with "Endpoint" (PA103)

Validation:
- Added ignore=BA129 to .pack-ignore for superna-zt-lockout-user and superna-zt-unlock-user compliance policy warnings, as recommended by maintainers (BA129)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix ST111 validation, unit test trailing slash, and suppress RM104/RM110

- ST111: Added section: Connect to all 4 parameters in SupernaZeroTrust.yml
  (base_url, credentials, insecure, proxy)
- Unit test: Fixed test_client_initialization trailing slash assertion to use
  rstrip('/') for compatibility across demisto-sdk versions
- RM104/RM110: Added ignores to .pack-ignore (suppressible warnings)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>

* Fix pre-commit issues

---------

Co-authored-by: Andrew MacKay <andrew.mackay@superna.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>
Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>

* fix validate

* Add parameter

* ruff fixes

---------

Co-authored-by: akshotiamit-pa <aakshoti@paloaltonetworks.com>
Co-authored-by: merit-maita <49760643+merit-maita@users.noreply.github.com>
Co-authored-by: Kamal Qarain <45042524+kamalq97@users.noreply.github.com>
Co-authored-by: hyaffe839 <hyaffe@paloaltonetworks.com>
Co-authored-by: sharonfi99 <147984773+sharonfi99@users.noreply.github.com>
Co-authored-by: dtroushinsky <dtroushinsky@paloaltonetworks.com>
Co-authored-by: masulin97 <masulin@paloaltonetworks.com>
Co-authored-by: inbalapt1 <164751454+inbalapt1@users.noreply.github.com>
Co-authored-by: Content Bot <bot@demisto.com>
Co-authored-by: Tal Zichlinsky <35036457+talzich@users.noreply.github.com>
Co-authored-by: Koby Meir <kobymeir@users.noreply.github.com>
Co-authored-by: Arad Carmi <62752352+AradCarmi@users.noreply.github.com>
Co-authored-by: almog2296 <alabudi@paloaltonetworks.com>
Co-authored-by: Israel Lappe <79846863+ilappe@users.noreply.github.com>
Co-authored-by: Content Bot <55035720+content-bot@users.noreply.github.com>
Co-authored-by: Andrew MacKay <andrew.mackay@superna.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kamal Qarain <kqarain@paloaltonetworks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-approved ready-for-pipeline-running Whether the pr is ready for running the whole pipeline, including testing on SAAS machines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants