Skip to content

SOCRadar v2.3.0: Add complete integration suite#43732

Merged
MosheEichler merged 66 commits into
demisto:contrib/Radargoger_socradar-pack-v2.2.2from
Radargoger:socradar-pack-v2.2.2
May 4, 2026
Merged

SOCRadar v2.3.0: Add complete integration suite#43732
MosheEichler merged 66 commits into
demisto:contrib/Radargoger_socradar-pack-v2.2.2from
Radargoger:socradar-pack-v2.2.2

Conversation

@Radargoger
Copy link
Copy Markdown
Contributor

@Radargoger Radargoger commented Mar 29, 2026

Contributing to Cortex XSOAR Content

Make sure to register your contribution by filling the contribution registration form

The Pull Request will be reviewed only after the contribution registration form is filled.

Status

  • In Progress
  • Ready
  • In Hold - (Reason for hold)

Related Issues

Description

This PR adds three new integrations to the SOCRadar pack (v2.2.2):

  1. SOCRadar Rapid Reputation - Fast reputation checking for IPs, domains, URLs, and file hashes with bulk support (up to 100 indicators) and automatic rate limiting.

  2. SOCRadar IoC Enrichment - Deep threat intelligence enrichment with signal strength, confidence levels, activity labels, premium feeds, relations, and optional AI insights.

  3. SOCRadar Threat Feed - Collection-based IoC feed integration for automated indicator ingestion using collection UUIDs with incremental feed capability and comprehensive geolocation data.

All integrations follow Cortex XSOAR best practices, include comprehensive documentation, and are production-ready.

Must have

  • Tests - Unit tests included for Threat Feed (14 test cases, all passing)
  • Documentation - Complete README files for all three integrations with commands, examples, and configuration guides

Changes

  • Added Integrations/FeedSOCRadarThreatFeed/ (new)
  • Added Integrations/SOCRadarRapidReputation/ (new)
  • Added Integrations/SOCRadarIoCEnrichment/ (new)
  • Updated pack_metadata.json to version 2.2.2
  • Updated main README.md with all three integrations
  • Added ReleaseNotes/2_2_2.md

Testing

All integrations have been tested with valid SOCRadar API keys:

  • API connectivity verified
  • All commands tested and working
  • DBot score integration validated
  • Error handling confirmed
  • Rate limiting verified (Rapid Reputation)
  • Feed ingestion tested (Threat Feed)

relates: https://jira-dc.paloaltonetworks.com/browse/CIAC-16413

@content-bot content-bot added Partner-Approved Contribution Form Filled Whether contribution form filled or not. Partner Contribution Thank you! Contributions are always welcome! External PR Partner Support Level Indicates that the contribution is for Partner supported pack labels Mar 29, 2026
@content-bot content-bot changed the base branch from master to contrib/Radargoger_socradar-pack-v2.2.2 March 29, 2026 16:27
@content-bot
Copy link
Copy Markdown
Contributor

Thank you for your contribution. Your generosity and caring are unrivaled! Make sure to register your contribution by filling the Contribution Registration form, so our content wizard @kamalq97 will know the proposed changes are ready to be reviewed.
For your convenience, here is a link to the contributions SLAs document.

@content-bot
Copy link
Copy Markdown
Contributor

Hi @Radargoger, thanks for contributing to the XSOAR marketplace. To receive credit for your generous contribution please follow this link.

@content-bot
Copy link
Copy Markdown
Contributor

🤖 AI-Powered Code Review Available

Hi @kamalq97, @MosheEichler, you can leverage AI-powered code review to assist with this PR!

Available Commands:

  • @marketplace-ai-reviewer start review - Initiate a full AI code review
  • @marketplace-ai-reviewer re-review - Incremental review for new commits

@Radargoger Radargoger changed the title SOCRadar v2.2.2: Add complete integration suite SOCRadar v2.3.0: Add complete integration suite Mar 29, 2026
- Fix README.md markdown formatting for pre-commit compliance
- Update pack version from 2.2.2 to 2.3.0
- Rename release notes: 2_2_2.md -> 2_3_0.md
- Add missing sections: SOCRadar Incidents, Incidents v4
- Fix header spacing and consistent formatting
- Correct support email to operation@socradar.io
@Radargoger
Copy link
Copy Markdown
Contributor Author

Hi @MosheEichler ,

Thank you for the review and the feedback!

I have addressed all the review comments and made the necessary changes.

Just a quick heads-up regarding the time.sleep() function in the code: I added it specifically to prevent hitting the API rate limits (rapid reputation API). I know this sometimes gets flagged in code reviews or automated checks, so I wanted to clarify its purpose beforehand.

Please let me know if there is anything else needed.

Thanks again!
Burak

@Radargoger Radargoger requested a review from MosheEichler April 24, 2026 14:10
- SOCRadarRapidReputation/README.md: change sublist markers from '-' to '*' to fix brack/markdownlint-cli2 CI failure
- SOCRadarIoCEnrichment/README.md: wrap bare URLs with angle brackets
- SOCRadarIncidentsV4/MultiTenant: normalize YAML list indentation style
- SOCRadarRapidReputation: add periods to context output descriptions; bump fromversion to 6.10.0
- SOCRadarIoCEnrichment: fix defaultvalue boolean to string format; bump fromversion to 6.10.0
- FeedSOCRadarThreatFeed: bump fromversion to 6.10.0
- Remove trailing newlines from description and metadata files
@Radargoger
Copy link
Copy Markdown
Contributor Author

Hi @MosheEichler ,
I wanted to give you a quick update on the AI Reviewer's comments.
All other review items have been fully addressed
— including ContentClient migration attempts, DBotScore outputs, description files, unit tests, pack metadata, assert removal, demisto.params() placement, raise_for_status error handling, and the rest. The PR is otherwise ready.
We'd just like to align on the following 3 points before we consider it complete:

  1. BaseClient → ContentClient
    We attempted to migrate all 3 integrations (FeedSOCRadarThreatFeed, SOCRadarIoCEnrichment, SOCRadarRapidReputation) from BaseClient to ContentClient, but this change caused pre-commit hook failures in the CI. We'd appreciate your guidance on the correct migration path — happy to fix it with your direction.
  2. time.sleep(1) in SOCRadarRapidReputation.py
    The Rapid Reputation API enforces a hard rate limit of 1 request per second. The time.sleep(1) is intentionally placed in the bulk check flow (socradar-bulk-reputation command), which processes multiple indicators in a single call. Without the delay, consecutive API requests within the same batch would hit the rate limit and result in 429 errors. This is a necessary safeguard for the bulk query feature — if there's a preferred XSOAR pattern for rate limiting (e.g., retry with backoff), we're open to adopting it.
  3. API Key type: 4 vs type: 9
    The AI Reviewer suggested switching to type: 9 with displaypassword. However, type: 9 renders a "Password" label in the UI, which would be misleading — users are entering an API key, not a password. We kept type: 4 intentionally for clarity. If type: 9 is a must requirement, we can explore adding a custom display label to keep it accurate.
    If these 3 points are acceptable as-is, we're ready to move forward. Otherwise, just let us know and we'll adjust accordingly.
    Thanks!
    Burak

@MosheEichler
Copy link
Copy Markdown
Contributor

MosheEichler commented Apr 30, 2026

Hi @Radargoger, thanks for the detailed status — replying inline on the 3 open items so we can move forward:

1. BaseClientContentClient migration

Since the migration broke pre-commit and the current code passes, let's leave BaseClient as-is in this PR. Please open a separate follow-up PR for the ContentClient migration so we don't block the v2.3.0 release on it.

2. time.sleep(1) in SOCRadarRapidReputation.py (bulk reputation)

Please remove time.sleep() and use the platform's built-in HTTP retry mechanism instead — it handles rate limiting (429) cleanly via Retry-After / exponential backoff:

📖 https://xsoar.pan.dev/docs/integrations/code-conventions#http-call-retries

In short, pass these to BaseClient._http_request (or set them on the client):

self._http_request(
    method="POST",
    url_suffix="/...",
    json_data=payload,
    retries=3,
    status_list_to_retry=[429],
    backoff_factor=1,   # 1s, 2s, 4s ...
)

This replaces the hard-coded 1-second sleep with proper, server-driven backoff and is the XSOAR-preferred pattern.

3. API Key — type: 4 vs type: 9

Please switch to type: 9 with displaypassword — that gives you the security of an encrypted credential and lets you keep the user-facing label as "API Key" (no misleading "Password" label). You can also add hiddenusername: true to hide the unused username field.

Reference example from the AHA pack:
https://github.com/demisto/content/blob/master/Packs/AHA/Integrations/AHA/AHA.yml#L19-L25

- name: apikey
  displaypassword: API Key
  required: true
  type: 9
  hiddenusername: true
  additionalinfo: "API Key to access the SOCRadar service."

Apply this pattern to all 3 integrations (FeedSOCRadarThreatFeed, SOCRadarIoCEnrichment, SOCRadarRapidReputation).


Once # 2 and # 3 are addressed (and # 1 is deferred to a follow-up PR), we should be good to move forward. Thanks!

…hanism

- Change apikey param from type 4 to type 9 (encrypted credential) with
  displaypassword and hiddenusername across all three integrations
  (FeedSOCRadarThreatFeed, SOCRadarIoCEnrichment, SOCRadarRapidReputation)
- Update params.get("apikey") to params.get("apikey", {}).get("password")
  in all three Python files to match type 9 credentials object
- Remove time.sleep(1) from socradar-bulk-check command and replace with
  platform-native retry via retries=3, status_list_to_retry=[429],
  backoff_factor=1 on _http_request
@Radargoger
Copy link
Copy Markdown
Contributor Author

Hi @MosheEichler, thanks for the detailed review!

1. BaseClient → ContentClient migration
Acknowledged — leaving BaseClient as-is in this PR. Will open a separate follow-up PR for the ContentClient migration to avoid blocking v2.3.0.

2. time.sleep(1) → HTTP retry mechanism
Done. Removed import time and the time.sleep(1) block from socradar_bulk_check_command. Added retry parameters directly to _http_request in the get_entity_reputation method so all calls (including bulk check) benefit from server-driven backoff:

retries=3,
status_list_to_retry=[429],
backoff_factor=1,

3. API Key — type: 4 → type: 9
Done for all three integrations (FeedSOCRadarThreatFeed, SOCRadarIoCEnrichment, SOCRadarRapidReputation). Applied the pattern from the AHA pack:

- name: apikey
  displaypassword: API Key
  required: true
  type: 9
  hiddenusername: true
  additionalinfo: API Key to access the SOCRadar service.

Also updated the corresponding Python files to read the credential correctly via params.get("apikey", {}).get("password") as required by the type 9 credentials object.

All changes are in the latest commit 7f1cfbda3c. Let me know if anything needs adjustment!

- Replace enumerate loop with simple for loop in socradar_bulk_check_command
  since idx was unused after removing time.sleep
- Add tests for test_module, process_entity_by_type, socradar_reputation_command,
  and socradar_bulk_check_command in SOCRadarRapidReputation_test.py
- Add tests for test_module, socradar_ioc_enrichment_command, and no-data paths
  in SOCRadarIoCEnrichment_test.py
@MosheEichler
Copy link
Copy Markdown
Contributor

Hi @Radargoger, the code looks good!

We're ready for a demo. Please check this page, and let me know when you're available for one over DFIR.
Feel free also to send me a recording of a demo.

@MosheEichler MosheEichler added the pending-demo Demo pending label Apr 30, 2026
- Fix ruff-format: collapse multi-line call in socradar_reputation_command test
- Replace json=None with json={} in all no-data tests so _http_request returns
  falsy empty dict instead of failing to parse empty body
- Add mocker.patch("demisto.error") to connection_error test to suppress stdout
  output that was triggering the check_std_out_err conftest fixture
…nt tests

- test_test_module_no_response: add mocker fixture and patch demistomock.error
  to suppress stdout output from demisto.error() call that triggered check_std_out_err
- test_test_module_connection_error: change mocker.patch("demisto.error") to
  mocker.patch("demistomock.error") — "demisto" is not an importable module in CI,
  only demistomock is available
test_module calls demisto.error() when is_success is False, which prints
to stdout and triggers the check_std_out_err conftest fixture teardown error.
Added mocker fixture and patched demistomock.error to suppress the output.
…nant API

- severity: was sent as 'severities' (list), API expects 'severity' (comma-separated)
- status: was sent as 'status' (list), API expects 'status_list[]' (bracket notation)
  Single-value status_list without brackets returns 400 from the API
- Updated 2_3_0.md release notes to document the fix
@Radargoger
Copy link
Copy Markdown
Contributor Author

Radargoger commented May 2, 2026

Hi @MosheEichler, attaching a full demo recording of the v2.2.2 / v2.3.0 pack for your review.

What the GIF covers (end-to-end)

1. Instance configuration — all three integrations Test passing

  • SOCRadar Rapid Reputation → Success
  • SOCRadar IoC Enrichment → Success
  • SOCRadar Threat Feed → Success

API Key uses type: 9 (encrypted credential, masked in UI), and the "Switch to credentials" link is visible on each instance for credential reuse.

2. SOCRadar Rapid Reputation commands (Playground / War Room)

  • !ip ip="45.142.212.100" → FindingSources from Kaspersky / Abuse.ch ThreatFox / USOM / Cisco-Talos / AlienVault OTX / Threatview.io, Score 31.9, IsWhitelisted=false
  • !ip ip="1.1.1.1" → IsWhitelisted=true, empty FindingSources (whitelist behavior verified)
  • !domain, !url, !file → all return populated tables
  • !socradar-bulk-check (new) — mixed indicator list in a single call (45.142.212.100,malware-domain.ru,http://45.142.212.100/payload.exe,1.1.1.1), automatic per-indicator type detection, per-indicator result tables, and ScoreRange flagging "Whitelisted" correctly for 1.1.1.1

3. SOCRadar IoC Enrichment

  • !socradar-ioc-enrichment indicator="45.142.212.100" → ASN/ASNCode, Activity timeline (Last1Day / Last7Days / Last30Days), CIDR, Categorization flags (CDN / Cloud / Cryptocurrency), Classifications (Country: Moldova), FirstSeen 2021-07-14, Score 54.49, related indicators from SOCRadar Threat Exchange Services

4. SOCRadar Threat Feed

  • !socradar-get-indicators collection_uuids="..." limit=10 → 10 IOCs from Cyber Threat Alliance and Internal Malware Information and Threat Sharing Platform feeds, Domain type, with FeedMaintainerName / FirstSeenDate / LastSeenDate / Score columns
  • The instance also fetched 7,224 indicators successfully (visible in the instance status row)

5. Error handling / validation

  • !file file="notahash" → both Rapid Reputation and IoC Enrichment return Hash "notahash" is not a valid hash (graceful, no traceback)
  • !ip ip="not-an-ip" and !domain domain="not a domain!!" produce equivalent validation messages
socradar-pack-v2 2 2-demo-

Notes

  • The SOCRadarThreatFusion returned an error entries visible in the recording are from the legacy Partner Contribution integration, not part of this PR.
  • Recording is 1393×868, 50 frames covering the full flow.

Let me know if you'd like a focused recording for any specific edge case or flow.

@MosheEichler MosheEichler enabled auto-merge (squash) May 4, 2026 10:27
@MosheEichler MosheEichler merged commit 8b961f2 into demisto:contrib/Radargoger_socradar-pack-v2.2.2 May 4, 2026
24 of 25 checks passed
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 4, 2026

Thank you for your contribution. Your external PR has been merged and the changes are now included in an internal PR for further review. The internal PR will be merged to the master branch within 3 business days.

MosheEichler pushed a commit that referenced this pull request May 20, 2026
* Change risk score scale from 1000 to 100

It is very important to new joiner

* SOCRadar v2.2.2: Add complete integration suite

* Rename 2_2_2.md to 2_3_0.md

* fix: Update SOCRadar pack to v2.3.0 with markdown fixes

- Fix README.md markdown formatting for pre-commit compliance
- Update pack version from 2.2.2 to 2.3.0
- Rename release notes: 2_2_2.md -> 2_3_0.md
- Add missing sections: SOCRadar Incidents, Incidents v4
- Fix header spacing and consistent formatting
- Correct support email to operation@socradar.io

* Update README.md

* Fixed trailing whitespace in all files

* Fix: Update type hints to Python 3.9+ style

* Fix: Update type hints to Python 3.9+ style

* Fix: Resolve PIE810 and mypy type annotation errors

* Fix: Add Union type annotation for indicator_object

* Merge with pre-commit auto-fixes

* Apply pycln: remove unused imports

* Apply black formatting to all Python files

* Apply ruff formatting with --isolated flag

* Fix: Apply ruff formatting to test file

* Apply black formatting with line-length 88

* Fix: Shorten long line to meet line-length 130 requirement

* Fix: Syntax error on line 591 - split merged lines

* Fix: Shorten long lines

* Fixed formatting and linting issues

* Fixed formatting issues

* Fixed formatting

* Docs: Fix markdown formatting and newline issues

* Fix: Revert file permission changes (755 -> 644) for unrelated SOCRadar pack files



* Fix: Apply XSOAR best practices to FeedSOCRadarThreatFeed

- Replace BaseClient with ContentClient
- Use .get() for safe dict access in build_entry_context
- Remove assert, handle None/empty date in date_string_to_iso_format_parsing
- Return None for unknown indicator types to enable auto-detection
- Fix raise_for_status error handling in handle_error_response

* Fix: Apply XSOAR best practices to SOCRadarRapidReputation

- Replace BaseClient with ContentClient
- Fix raise_for_status error handling in handle_error_response
- Use is_ip_valid (accept_v6_ips=False) correctly for IPv4 validation
- Move demisto.params() to main(), pass reliability as parameter
- Use get_hash_type instead of length-based hash detection

* Fix: Apply XSOAR best practices to SOCRadarIoCEnrichment

- Replace BaseClient with ContentClient
- Move include_ai_insights from demisto.params() to Client constructor
- Fix raise_for_status error handling in handle_error_response
- Move demisto.params() to main(), pass reliability as parameter
- Simplify calculate_dbot_score: score-based only (0=Unknown, 1-50=Suspicious, 51-100=Malicious)

* update yaml

* yml changes

* add md file

* Fix: Add unit tests and score mapping for RapidReputation and IoCEnrichment

* Fix: Move demistomock import to line 1 in FeedSOCRadarThreatFeed

XSOAR on-prem runner expects import demistomock as demisto to be the very first import line so it can inject the real demisto object.

* Fix: Revert ContentClient to BaseClient; fix ruff-format issues

ContentClient is not part of CommonServerPython and is not available
in the demisto/python3 Docker image used by these integrations.
The ContentClientApiModule defines a different ContentClient class
designed for content management, not REST API clients.
BaseClient (from CommonServerPython) is the correct base class here.

Note for reviewer: ContentClient requirement cannot be satisfied in
this Docker image context. BaseClient provides identical HTTP
functionality for these integrations.

Also fixed ruff-format violations:
- SOCRadarIoCEnrichment.py: removed extra blank line
- SOCRadarRapidReputation.py: added spaces around / operator

* Fix: Resolve ruff E501, mypy return type, and test data key name issues

- FeedSOCRadarThreatFeed.py: fix convert_to_demisto_indicator_type return
  type to Optional[str] since dict.get() can return None (mypy error)
- test_data JSON files: update expected keys to PascalCase to match
  build_entry_context output (IndicatorType, FirstSeenDate, LastSeenDate,
  FeedMaintainerName, SeenCount) - tests were failing due to key rename
- SOCRadarRapidReputation.py: shorten rate limit help text to fit within
  130-char line limit (E501)

* update readme files

* update a readme file

* update readme

* Fix: markdownlint sublist style and bare URLs in README files

- SOCRadarRapidReputation/README.md: change sublist markers from '-' to '*' to fix brack/markdownlint-cli2 CI failure
- SOCRadarIoCEnrichment/README.md: wrap bare URLs with angle brackets

* Fix: YAML formatting, fromversion bumps, and description style fixes

- SOCRadarIncidentsV4/MultiTenant: normalize YAML list indentation style
- SOCRadarRapidReputation: add periods to context output descriptions; bump fromversion to 6.10.0
- SOCRadarIoCEnrichment: fix defaultvalue boolean to string format; bump fromversion to 6.10.0
- FeedSOCRadarThreatFeed: bump fromversion to 6.10.0
- Remove trailing newlines from description and metadata files

* Fix: convert remaining dash list markers to asterisk in RapidReputation README

* Fix: switch API key to type 9, replace time.sleep with HTTP retry mechanism

- Change apikey param from type 4 to type 9 (encrypted credential) with
  displaypassword and hiddenusername across all three integrations
  (FeedSOCRadarThreatFeed, SOCRadarIoCEnrichment, SOCRadarRapidReputation)
- Update params.get("apikey") to params.get("apikey", {}).get("password")
  in all three Python files to match type 9 credentials object
- Remove time.sleep(1) from socradar-bulk-check command and replace with
  platform-native retry via retries=3, status_list_to_retry=[429],
  backoff_factor=1 on _http_request

* Fix: ruff B007 and increase unit test coverage above 70%

- Replace enumerate loop with simple for loop in socradar_bulk_check_command
  since idx was unused after removing time.sleep
- Add tests for test_module, process_entity_by_type, socradar_reputation_command,
  and socradar_bulk_check_command in SOCRadarRapidReputation_test.py
- Add tests for test_module, socradar_ioc_enrichment_command, and no-data paths
  in SOCRadarIoCEnrichment_test.py

* Fix: resolve CI failures in coverage tests

- Fix ruff-format: collapse multi-line call in socradar_reputation_command test
- Replace json=None with json={} in all no-data tests so _http_request returns
  falsy empty dict instead of failing to parse empty body
- Add mocker.patch("demisto.error") to connection_error test to suppress stdout
  output that was triggering the check_std_out_err conftest fixture

* Fix: patch demistomock.error instead of demisto.error in IoC Enrichment tests

- test_test_module_no_response: add mocker fixture and patch demistomock.error
  to suppress stdout output from demisto.error() call that triggered check_std_out_err
- test_test_module_connection_error: change mocker.patch("demisto.error") to
  mocker.patch("demistomock.error") — "demisto" is not an importable module in CI,
  only demistomock is available

* Fix: patch demistomock.error in test_test_module_api_failure

test_module calls demisto.error() when is_success is False, which prints
to stdout and triggers the check_std_out_err conftest fixture teardown error.
Added mocker fixture and patched demistomock.error to suppress the output.

* Fix: correct severity and status_list filter param names for multi-tenant API

- severity: was sent as 'severities' (list), API expects 'severity' (comma-separated)
- status: was sent as 'status' (list), API expects 'status_list[]' (bracket notation)
  Single-value status_list without brackets returns 400 from the API
- Updated 2_3_0.md release notes to document the fix

---------

Co-authored-by: Radargoger <burak.goger@socradar.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Contribution Form Filled Whether contribution form filled or not. Contribution Thank you! Contributions are always welcome! docs-approved External PR Partner Support Level Indicates that the contribution is for Partner supported pack Partner Partner-Approved pending-contributor The PR is pending the response of its creator pending-demo Demo pending ready-for-pipeline-running Whether the pr is ready for running the whole pipeline, including testing on SAAS machines TIM Review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants