Merge branch 'release/v3.4'

PaloAltoNetworks · Sep 19, 2013 · 2b948e4 · 2b948e4
2 parents ac6763e + 5f0fcf4
commit 2b948e4
Show file tree

Hide file tree

Showing 31 changed files with 12,685 additions and 74 deletions.
diff --git a/README.md b/README.md
@@ -11,7 +11,7 @@ Networks Firewall
 #### Version ####
 
 * Splunk Version: 5.x
-* App Version: 3.3.2
+* App Version: 3.4
 * Last Modified: Sept 2013
 * Authors:
     * Monzy Merza - Splunk, Inc.
@@ -117,6 +117,28 @@ Log can be further filtered by type during search by using predefined macros.  T
 
 Use these macros in the search bar by surrounding them with back-ticks.
 
+### WildFire Cloud Integration ###
+
+WildFire analysis reports can be retrieved dynamically from the WildFire cloud after each analysis.  This retrieval requires a WildFire API Key from https://wildfire.paloaltonetworks.com
+
+Malware analysis reports from the WildFire Cloud are dynamically downloaded and indexed when a WildFire log is received from a firewall.
+
+### NetFlow ###
+
+NetFlow graphs and charts are based on NetFlow data produced by Palo Alto Networks devices and converted to syslog messages by 3rd party software - NetFlow Integrator. Download a 30-day free trial of NetFlow Integrator at https://www.netflowlogic.com/downloads
+
+Steps to configure:
+
+- Install NetFlow Integrator on a separate server or together with Splunk Forwarder
+- Point Palo Alto Networks device NetFlow settings to NetFlow Integrator server, default port 9995 with PAN-OS Field Types enabled (see [Administrator's Guide] (https://live.paloaltonetworks.com/community/documentation/content?filterID=contentstatus[published]~category[administrators-guide]&filterID=contentstatus[published]~objecttype~objecttype[document]&itemView=detail))
+- Enable NetFlow in the Splunk for Palo Alto Networks app setup page
+- Restart Splunk for the previous change to take effect
+- Add NetFlow Integrator output pointing to Splunk UDP port 10514
+- Create Splunk UDP data input `sourcetype=flowintegrator`, which receives syslog messages on UDP port 10514, and `index=flowintegrator`.
+- Enable NetFlow Integrator Palo Alto Networks Rules (10030 through 10035) and Converter (20093)
+
+If you have any questions, or require any assistance with configuration please contact NetFlow Logic at https://netflowlogic.zendesk.com/home
+
 ### High Performance Value Store (HPVS) ###
 
 The app uses the HPVS feature introduced in Splunk 5.0. This feature provides a tremendous performance improvement for dashboards and views. The views and dashboards make use of saved searches that store data on your search head. This means that disk storage on your search head will be consumed as a result of these searches. If you turn off these saved searches, your dashboards will not render. Or dashboard rendering will be really, really slow. Please post a question to answers.splunk.com if you'd like to explore alternatives. 
@@ -133,28 +155,18 @@ Keep in mind that searches that have longer time ranges may take a little longer
 
 ## What's new in this version ##
 
-Version 3.3.2  
-- Fix: URL in WildFire dashboard corrected
-- Fix: Overview dashboard colors were gray on some servers, set back to white
-- Fix: Corrected description fields in commands.conf that resulted in log errors
-- Fix: Corrected sourcetype in inputs.conf.sample
-
-Version 3.3.1  
-- Fix: App setup screen allows blank values
-- Fix: Several GUI fixes and enhancements
+Version 3.4
 
-Version 3.3  
-- Malware analysis reports from the WildFire Cloud are dynamically downloaded and indexed when a WildFire log is received from a firewall.
-- WildFire dashboard
-    - Recent WildFire events
-    - Graphs of WildFire statistical data
-    - Detect compromised hosts using malware behavior to traffic log correlation
+- NetFlow support using NetFlow Integrator, a 3rd party program from NetFlow Logic
+    - New set of dashboards, charts and graphs centered around NetFlow records from Palo Alto Networks devices
+    - App-ID and User-ID information is available in NetFlow records
 
-Note: Malware analysis report retrieval requires a WildFire API Key from https://wildfire.paloaltonetworks.com
+Download a 30-day free trial of NetFlow Integrator at https://www.netflowlogic.com/downloads  
+Steps to configure NetFlow are available in the **NetFlow** section above.
 
 ## Installing from Git ##
 
-This app is available on [Splunkbase](http://splunk-base.splunk.com/apps/22327/splunk-for-palo-alto-networks) and [Github](https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks).  Optionally, you can clone the github repository to install the app.
+This app is available on [Splunk Apps](http://apps.splunk.com/app/491) and [Github](https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks).  Optionally, you can clone the github repository to install the app.
 From the directory `$SPLUNK_HOME/etc/apps/`, type the following command:
 
-    git clone https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks.git
+    git clone https://github.com/PaloAltoNetworks-BD/SplunkforPaloAltoNetworks.git SplunkforPaloAltoNetworks
diff --git a/appserver/controllers/get_version.py b/appserver/controllers/get_version.py
@@ -0,0 +1,26 @@
+import os
+import splunk.appserver.mrsparkle.controllers as controllers
+from splunk.appserver.mrsparkle.lib.decorators import expose_page
+
+APP = 'SplunkforPaloAltoNetworks'
+VERSION_CONF = os.path.join(os.environ['SPLUNK_HOME'], 'etc', 'apps', APP, 'default', 'nfi_version.conf')
+
+class GetVersion(controllers.BaseController):
+    @expose_page(must_login=True, methods=['GET'])
+    def get_version(self, **kwargs):
+        version = 'unknown';
+        try:
+            with open(VERSION_CONF, 'r') as cf:
+                for line in cf:
+                    if not line.startswith('#') and not line.startswith(';') and line.strip() != '':
+                        parts = line.split('=', 1)
+                        if len(parts) == 1:
+                            continue
+                        key = parts[0].strip()
+                        if key == 'version':
+                            version = parts[1].strip()
+                            break
+        except:
+            version = 'unknown';
+        return version
+
diff --git a/appserver/controllers/nfi_nav_handler.py b/appserver/controllers/nfi_nav_handler.py
@@ -0,0 +1,26 @@
+import os
+import shutil
+import splunk.appserver.mrsparkle.controllers as controllers
+from splunk.appserver.mrsparkle.lib.decorators import expose_page
+
+APP = 'SplunkforPaloAltoNetworks'
+ENABLED_NAV = os.path.join(os.environ['SPLUNK_HOME'], 'etc', 'apps', APP, 'default', 'data', 'ui', 'nav', 'default.xml.nfi_enabled')
+DISABLED_NAV = os.path.join(os.environ['SPLUNK_HOME'], 'etc', 'apps', APP, 'default', 'data', 'ui', 'nav', 'default.xml.nfi_disabled')
+NAV = os.path.join(os.environ['SPLUNK_HOME'], 'etc', 'apps', APP, 'default', 'data', 'ui', 'nav', 'default.xml')
+
+class NAVHANDLER(controllers.BaseController):
+    @expose_page(must_login=True, methods=['GET'])
+    def enable(self, **kwargs):
+        try:
+            shutil.copy(ENABLED_NAV, NAV)
+        except:
+            pass
+        return 'Enabled!'
+    @expose_page(must_login=True, methods=['GET'])
+    def disable(self, **kwargs):
+        try:
+            shutil.copy(DISABLED_NAV, NAV)
+        except:
+            pass
+        return 'Disabled!'
+
diff --git a/appserver/static/nfi_about.html b/appserver/static/nfi_about.html
@@ -0,0 +1,14 @@
+<h1>About NetFlow</h1>
+<h2 id="NFI_VERSION"></h2>
+<p>NetFlow views are based on NetFlow data produced by Palo Alto Networks devices and converted to syslog messages by 3rd party software - NetFlow Integrator. Download a 30-day free trial of NetFlow Integrator at <a href="https://www.netflowlogic.com/downloads" target="_blank">https://www.netflowlogic.com/downloads</a></p>
+
+<p><strong>Steps to configure:</strong></p>
+<p><ul>
+<li>Install NetFlow Integrator on a separate server or together with Splunk Forwarder</li>
+<li>Point Palo Alto Networks device NetFlow settings to NetFlow Integrator server, default port 9995 with PAN-OS Field Types enabled  (see <a href="https://live.paloaltonetworks.com/community/documentation/content?filterID=contentstatus[published]~category[administrators-guide]&filterID=contentstatus[published]~objecttype~objecttype[document]&itemView=detail" target="_blank">Administrator's Guide</a>)</li>
+<li>Add NetFlow Integrator output pointing to Splunk UDP port 10514</li>
+<li>Create Splunk UDP data input sourcetype=flowintegrator, which receives syslog messages on UDP port 10514, and index=flowintegrator.</li>
+<li>Enable NetFlow Integrator Palo Alto Networks Rules (10030 through 10035) and Converter (20093)
+</ul></p>
+
+<p>If you have any questions, or require any assistance with configuration please contact NetFlow Logic at <a href="https://netflowlogic.zendesk.com/home" target="_blank">https://netflowlogic.zendesk.com/home</a></p>
diff --git a/default/app.conf b/default/app.conf
@@ -5,7 +5,7 @@ label = Splunk for Palo Alto Networks
 [launcher]
 author= btorres-gil@paloaltonetworks.com
 description= The Splunk for Palo Alto Networks app is a set of field extractions, reports, lookups and dashboards which provide visibility into the Palo Alto Networks Firewall data. 
-version = 3.3.2
+version = 3.4
 
 [package]
 id= SplunkforPaloAltoNetworks

diff --git a/default/data/ui/nav/default.xml.nfi_disabled b/default/data/ui/nav/default.xml.nfi_disabled
@@ -0,0 +1,90 @@
+<nav>
+  <collection label="Splunk for Palo Alto Networks">
+    <view name="pan_overview_switcher_maps" default="true"/>
+    <collection label="Search Palo Alto Data">
+      <a href="flashtimeline">Search</a>
+      <a href="flashtimeline?q=search%20%60pan_traffic%60">Traffic Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_threat%60">Threat Data</a>
+      <a href="flashtimeline?q=search%20%60pan_url%60">URL Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_data%60">Data Filtering Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_wildfire%60">WildFire Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_config%60">Config Messages</a>
+      <a href="flashtimeline?q=search%20%60pan_system%60">System Logs</a>
+    </collection>
+    <divider />
+    <a href="/manager/SplunkforPaloAltoNetworks/apps/local/SplunkforPaloAltoNetworks/setup?action=edit">Configuration</a>
+    <divider />
+    <a href="mailto:bd-labs@splunk.com">Send Feedback</a>
+  </collection>
+
+  <collection label="Traffic">
+    <view name="traffic_overview" />
+    <divider />
+    <view name="web_usage_report" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_traffic%60">Search Traffic Data</a>
+      <divider />
+      <saved source="all" match="PAN - Traffic" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="Threat">
+    <view name="threat_overview" />
+    <view name="threat_detail" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_threat%60">Search Threat Data</a>
+      <divider />
+      <saved source="all" match="PAN - Threat" view="flashtimeline" />
+    </collection>
+  </collection>
+
+    <collection label="Content">
+    <view name="content_overview" />
+    <view name="url_filtering" />
+    <view name="data_filtering" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_web_activity%60">Search URL Data</a>
+      <a href="flashtimeline?q=search%20%60pan_data_filtering%60">Search Data Filtering Data</a>
+      <divider />
+      <saved source="all" match="PAN - Threat" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="WildFire">
+    <view name="wildfire_overview" />
+    <view name="wildfire_detail" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_wildfire%60">Search WildFire Log Data</a>
+      <a href="flashtimeline?q=search%20%60pan_wildfire_report%60">Search WildFire Report Data</a>
+      <divider />
+      <saved source="all" match="PAN - WildFire" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="Console">
+    <view name="system_overview" />
+    <view name="config_overview" />
+    <divider />
+    <collection label="System Message Reports">
+      <a href="flashtimeline?q=search%20%60pan_system%60">Search System Messages</a>
+      <divider />
+      <saved source="all" match="PAN - System" view="flashtimeline" />
+    </collection>
+    <collection label="Configuration Reports">
+      <a href="flashtimeline?q=search%20%60pan_config%60">Search Config Changes</a>
+      <divider />
+      <saved source="all" match="PAN - Config" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <!-- Search -->
+  <view name="flashtimeline" />
+  <!--<collection label="Help">
+    <view name="readme"/>
+    <view name="faq" />
+  </collection>-->
+</nav>
diff --git a/default/data/ui/nav/default.xml.nfi_enabled b/default/data/ui/nav/default.xml.nfi_enabled
@@ -0,0 +1,113 @@
+<nav>
+  <collection label="Splunk for Palo Alto Networks">
+    <view name="pan_overview_switcher_maps" default="true"/>
+    <collection label="Search Palo Alto Data">
+      <a href="flashtimeline">Search</a>
+      <a href="flashtimeline?q=search%20%60pan_traffic%60">Traffic Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_threat%60">Threat Data</a>
+      <a href="flashtimeline?q=search%20%60pan_url%60">URL Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_data%60">Data Filtering Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_wildfire%60">WildFire Logs</a>
+      <a href="flashtimeline?q=search%20%60pan_config%60">Config Messages</a>
+      <a href="flashtimeline?q=search%20%60pan_system%60">System Logs</a>
+    </collection>
+    <divider />
+    <a href="/manager/SplunkforPaloAltoNetworks/apps/local/SplunkforPaloAltoNetworks/setup?action=edit">Configuration</a>
+    <divider />
+    <a href="mailto:bd-labs@splunk.com">Send Feedback</a>
+  </collection>
+
+  <collection label="Traffic">
+    <view name="traffic_overview" />
+    <divider />
+    <view name="web_usage_report" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_traffic%60">Search Traffic Data</a>
+      <divider />
+      <saved source="all" match="PAN - Traffic" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="Threat">
+    <view name="threat_overview" />
+    <view name="threat_detail" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_threat%60">Search Threat Data</a>
+      <divider />
+      <saved source="all" match="PAN - Threat" view="flashtimeline" />
+    </collection>
+  </collection>
+
+    <collection label="Content">
+    <view name="content_overview" />
+    <view name="url_filtering" />
+    <view name="data_filtering" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_web_activity%60">Search URL Data</a>
+      <a href="flashtimeline?q=search%20%60pan_data_filtering%60">Search Data Filtering Data</a>
+      <divider />
+      <saved source="all" match="PAN - Threat" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="WildFire">
+    <view name="wildfire_overview" />
+    <view name="wildfire_detail" />
+    <divider />
+    <collection label="Searches &amp; Reports">
+      <a href="flashtimeline?q=search%20%60pan_wildfire%60">Search WildFire Log Data</a>
+      <a href="flashtimeline?q=search%20%60pan_wildfire_report%60">Search WildFire Report Data</a>
+      <divider />
+      <saved source="all" match="PAN - WildFire" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="Console">
+    <view name="system_overview" />
+    <view name="config_overview" />
+    <divider />
+    <collection label="System Message Reports">
+      <a href="flashtimeline?q=search%20%60pan_system%60">Search System Messages</a>
+      <divider />
+      <saved source="all" match="PAN - System" view="flashtimeline" />
+    </collection>
+    <collection label="Configuration Reports">
+      <a href="flashtimeline?q=search%20%60pan_config%60">Search Config Changes</a>
+      <divider />
+      <saved source="all" match="PAN - Config" view="flashtimeline" />
+    </collection>
+  </collection>
+
+  <collection label="NetFlow">
+    <view name="nfi_about"/>
+    <divider />
+    <view name="nfi_overview"/>
+    <divider />
+    <view name="nfi_top_consumers"/>
+    <view name="nfi_top_destinations"/>
+    <view name="nfi_top_violators"/>
+    <view name="nfi_top_connectors"/>
+    <view name="nfi_top_apps"/>
+    <view name="nfi_top_apps_users"/>
+    <divider />
+    <collection label="Searches">
+      <view name="nfi_search_consumers"/>
+      <view name="nfi_search_destinations"/>
+      <view name="nfi_search_violators"/>
+      <view name="nfi_search_connectors"/>
+      <view name="nfi_search_apps"/>
+      <view name="nfi_search_apps_users"/>
+    </collection>
+    <a href="flashtimeline?q=search%20`nfi_pan_unified`">Timeline</a>
+  </collection>
+
+  <!-- Search -->
+  <view name="flashtimeline" />
+  <!--<collection label="Help">
+    <view name="readme"/>
+    <view name="faq" />
+  </collection>-->
+</nav>
diff --git a/default/data/ui/views/nfi_about.xml b/default/data/ui/views/nfi_about.xml
@@ -0,0 +1,17 @@
+<view template="dashboard.html">
+  <label>About Netflow</label>
+  <module name="AccountBar" layoutPanel="appHeader"/>
+  <module name="AppBar" layoutPanel="navigationHeader"/>
+  <module name="Message" layoutPanel="messaging">
+    <param name="filter">*</param>
+    <param name="clearOnJobDispatch">False</param>
+    <param name="maxSize">1</param>
+  </module>
+  <module name="TitleBar" layoutPanel="navigationHeader">
+    <param name="showActionsMenu">False</param>
+  </module>
+
+  <module name="ServerSideInclude" layoutPanel="panel_row1_col1">
+    <param name="src">nfi_about.html</param>
+  </module>
+</view>