Cortex Data Lake HEC log support #176
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
btorresgil
commented
Mar 23, 2021
This will be resolved in a fix later.
github-actions bot
pushed a commit
that referenced
this pull request
Apr 1, 2021
## [6.6.0](v6.5.2...v6.6.0) (2021-04-01) ### Features * **addon:** Cortex Data Lake HEC log support - #162 #176 * **app/addon:** Add IoT Security - #158 ### Bug Fixes * **addon:** Add fields for GlobalProtect logs * **addon:** Add modinputs as tasks in app.manifest - #153 * **addon:** Fix GlobalProtect logs dvc_name field * **addon:** Fix nav bar background color * **addon:** Parse GP and Decryption logs w/ pan:firewall - #168 * **app:** Incident counters flash in Splunk 8.1 - #163 * **app/addon:** correct user-id tag_user / untag_user
|
🎉 This PR is included in version 6.6.0 🎉 This release is available on SplunkBase: App - Add-on
|
|
🎉 This PR is included in version 6.6.0 🎉 This release is available on SplunkBase: App - Add-on
|
github-actions bot
pushed a commit
to btorresgil/SplunkforPaloAltoNetworks
that referenced
this pull request
Mar 7, 2022
## [5.0.0-beta.1](v4.2.2...v5.0.0-beta.1) (2022-03-07) ### Features * **addon:** Add Decryption Log Support for PANOS 10 - PaloAltoNetworks#126 * **addon:** Cortex Data Lake HEC log support - PaloAltoNetworks#162 PaloAltoNetworks#176 * **addon:** PAN Quality Validation and Improvement * **addon:** Significantly improve and modernize CIM compliance * **app/addon:** Add Cortex XDR incident support to App and Add-on including new XDR Incidents dashboard - PaloAltoNetworks#166 * **app/addon:** Add IoT Security - PaloAltoNetworks#158 * **app/addon:** Feature/dynamic user groups - PaloAltoNetworks#150 * **app/addon:** Python 3 Support - PaloAltoNetworks#124 * **app/addon:** Support GlobalProtect log type in PANOS 9.1 - PaloAltoNetworks#118 * **app/addon:** Update pandevice to 0.14.0 - PaloAltoNetworks#145 ### Bug Fixes * **addon:** Add fields for GlobalProtect logs * **addon:** Add fields for GlobalProtect logs * **addon:** Add GlobalProtect SourceUserName - PaloAltoNetworks#209 PaloAltoNetworks#202 * **addon:** Add modinputs as tasks in app.manifest - PaloAltoNetworks#153 * **addon:** Add virus eventtype to malware CIM - PaloAltoNetworks#114 PaloAltoNetworks#138 * **addon:** Fix appserver/static files * **addon:** Fix CDL logs contained string 'null' in 'user' field - PaloAltoNetworks#187 * **addon:** Fix error from Minemeld automatic lookup * **addon:** Fix GlobalProtect logs dvc_name field * **addon:** Fix GlobalProtect logs dvc_name field * **addon:** Fix nav bar background color * **addon:** Fix src_user field contained destination user - PaloAltoNetworks#186 * **addon:** Fix typo in transform.conf ([PaloAltoNetworks#227](https://github.com/btorresgil/SplunkforPaloAltoNetworks/issues/227)) * **addon:** Fix user showing as unknown from GlobalProtect logs. - PaloAltoNetworks#217 * **addon:** Parse GP and Decryption logs w/ pan:firewall - PaloAltoNetworks#168 * **addon:** Parse GP and Decryption logs w/ pan:firewall - PaloAltoNetworks#168 * **addon:** Remove endpoint tags and eventtypes - PaloAltoNetworks#196 * **addon:** Remove port from `dest_name` field - PaloAltoNetworks#129 PaloAltoNetworks#128 * **addon:** Remove white space from GlobalProtect sourcetype - PaloAltoNetworks#131 * **addon:** Restore "unknown" string for empty 'user' field * **app:** Fix error after upgrade to 7.0.0: "Unknown search command 'panwildfirereport'" - PaloAltoNetworks#189 * **app:** Fix IoT Security dashboard filter - PaloAltoNetworks#181 * **app:** Fix panContentPack error. Fixes bug [PaloAltoNetworks#222](https://github.com/btorresgil/SplunkforPaloAltoNetworks/issues/222) - PaloAltoNetworks#225 * **app:** Incident counters flash in Splunk 8.1 - PaloAltoNetworks#163 * **app:** Incident counters flash in Splunk 8.1 - PaloAltoNetworks#163 * **app:** Remove endpoint from Data Model Audit dashboard - PaloAltoNetworks#218 * **app/addon:** correct user-id tag_user / untag_user * **app/addon:** Fix background color of logo - PaloAltoNetworks#141 ### Performance Improvements * **app:** Change simple XML to use JQuery 3.5 - PaloAltoNetworks#207 * **app:** Remove high cardinality fields from datamodel ### ⚠ MAJOR RELEASE CHANGES This is a major release Splunk dashboards and searches you have created might be affected by these changes. Please be prepared to test and adjust any dashboards not included with the App after upgrade. * **addon:** pan_traffic_start logs no longer included in CIM * **addon:** pan_traffic_end logs moved from Network Session to Network Traffic datamodel * **addon:** pan_threat event type now includes wildfire and data logs * **addon:** pan_file logs moved from Web to IDS datamodel * **addon:** pan_virus logs moved from Malware to IDS datamodel * **addon:** pan_wildfire logs moved from Malware to IDS datamodel * **addon:** pan_email removed from Email datamodel * **app:** Removes datamodel for GlobalProtect logs before PAN-OS 9.1 * **app/addon:** Removes Traps 4 support * **app/addon:** Deprecates Traps 5 and Traps 6 support * **app:** Removes support for legacy WildFire Report API * **app/addon:** Requires Splunk 8.0 or higher * **app/addon:** Replaces Adversary Scoreboard and Incident Feed dashboards with new XDR Incidents dashboard
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds support for HEC JSON logs from Cortex Data Lake (CDL)
Closes #162
Motivation and Context
Cortex Data Lake Log Forwarding 2.0 added the ability to pick which fields are sent to Splunk, with a follow on release that included HEC support. The logs come in as JSON format. This PR adds support for this new JSON format.
The intention is for the user to interact with the data in the exact same way as existing PAN-OS and CDL syslogs. No changes should be needed to dashboards or customer SPL searches. The existing datamodel should pick up the new JSON logs.
The HEC logs use a different sourcetype, but that sourcetype is included in the existing eventtypes. For example:
sourcetype=pan:threat: only PAN-OS and old CDL logseventtype=pan_threat: All threat logs, including PAN-OS, old CDL, and new CDL HECCustomers who have incorrectly used sourcetypes instead of eventtypes to pull logs will not see the new CDL HEC logs and need to update their SPL searches to use eventtypes instead.
How Has This Been Tested?
Tested by PANW CDL QA Team
Types of changes
Checklist