Event Forwarding

[markfarrell]

Steps to Resolution

• This issue expired on May 31st, 2020 following the current issue management policies & procedures, and lessons learned, for the project.
• It will be superseded by a follow-up issue.

Summary

• Prior to closing this issue, I implemented unit tests for the parsing/validation of randomly-generated processed event data that can be forwarded to Tier 2, the current application tier for the 3-tier application (in a follow-up issue, I propose that the functionality of Tier 2 should be moved to Tier 3, and that Tier 2 should allow clients forward raw security event data in JSON format instead).

Contents

• Example
• Text.Parsing.Forward
• Data.Forward
• Data.Event
  • Proposed Changes

Example

Currently, clients can forward processed event data to the forwarding route(s) served by the application tier, employing a common JSON format for each type of forward-able event.

An instance of the application tier can launched with pulp run after building the project, with issues pending. Event forwarding requests can be executed against the application e.g. as follows.

POST /forward/traffic?{
  "eventCategory" : "IN",
  "eventType"     : "SUCCESS",
  "eventID"       : "16741",
  "sourceID"      : "9ad08560-ad6c-4b90-940e-e0c7d5908bc0",
  "sessionID"     : "1addae90-8497-4956-9f48-39686291e9b5",
  "destinationID" : "8de8ff2a-1c1e-45cd-8bad-52a1f2680068",
  "logID"         : "b814ae2a-f5e6-4f90-b75f-e2aa29778b96",
  "schemaID"      : "c26cbb30-6690-41f0-aebc-c22baef04047",
  "featureID"     : "3712ca2f-40b3-468b-a96d-043c8255baff",
  "instanceID"    : "e98568f2-b401-4ce1-a043-4acece96ffdc",
  "startTime"     : "1970-01-01T00:00:00.000Z",
  "duration"      : "441763200000",
  "endTime"       : "1984-01-01T00:00:00.000Z"
}

curl -G -k "https://localhost:3000/forward/traffic" --data-urlencode '{...}'

• In Control.Tier2, clients should be required to pass forwarded event data in the body ofPOST requests to the forwarding route(s) served by the application tier, i.e. instead of requiring clients to pass forwarded event data in the URL query string.
• Conduct a SQLi vulnerability scan (using sqlmap) against the forwarding route(s) currently served by the application tier, before & after resolving the previous proposed action. Produce a summary report or each type of audited application-layer event (events by source, duration by feature, etc.).
• Include the results of this vulnerability scan as part of the scheduled risk assessment for RELEASE-01/TIER-02.
• Parsing/validation should be applied to the HTTP request headers of incoming requests to forward events (e.g. verifying that the request method is POST, Content-Type is application/json, Content-Length is correct w.r.t. the body of the request, and so on).

Text.Parsing.Forward

Unit tests are currently implemented for the parsing/validation of each type of forwarded event: an appropriate sample of randomly-generated forward-able events in URL format are currently parsed/validated when running pulp test.

• The parsing/validation of forward-able events should be modified to accommodate the proposed changes to the event forwarding request format (or e.g. the body of the POST request should be appended as a query to the URL of the request, as a result of the parsing/validation of the raw HTTP request headers & body, without revising e.g.Text.Parsing.Forward{.event}.
• Should clients be able to pass event data in both the URL query string or the body of the POST request?

• Implement an additional test suite, where a sample of randomly-generated forward-able events are moreover are forwarded to a self-contained instance of the back-end web application as part of the test.

Currently, Data.Forward.Event is a data type modelling each type of forward-able event.

• If the parsing of each type of forward-able event is unambiguous, can the event forwarding API of the application be simplified to e.g. serve only a single forwarding route for all types of events?
  e.g. POST /forward (or POST/forward?{...}) instead of POST /forward/traffic, POST/forward/linux, ...?

Data.Forward

Currently, Data.Event is used to model the proposed common structure of forward-able events.

Data.Event

• Does the common Event data type require further revision?

Proposed Changes

type Feature a b = (EventCategory a => EventID b =>
  { eventCategory :: a
  , eventType     :: EventType
  , eventID       :: b
  })

Description: The feature extracted from an audited security event (e.g. flow record, Linux Auditing System, Windows Security, ...).

Data.Event.Identifier

type Identifier =
  { sourceID      :: SourceID
  , sessionID     :: SessionID
  , destinationID :: DestinationID
  , logID         :: LogID
  , schemaID      :: SchemaID
  }

Description: A 5-tuple uniquely identifying the occurrence of the event, analogous to / inspired by the 5-tuple (sIP, sPort, dIP, dPort, protocol) identifying a flow record event.

sourceID: A unique identifier assigned to e.g. the remote hosts of forwarding agents for forward-able events. Analogous to sIP.

• Should sourceID be derived using a unique identifier for the application instance as a UUIDv5 namespace UUID (e.g. analogous to a SilK sensor name)?
• Should an applicationID (or sensorID) in configurations settings passed in the command line arguments of the entry point defined to launch the application tier?

sessionID: A unique identifier assigned to e.g. forwarding agents on remote hosts. Analogous to sPort.

• Should sessionID be derived using sourceID as a UUIDv5 namespace UUID?

destinationID: A unique identifier assigned to the location that the forwarded event will be stored. Analogous to dIP.

• Should destinationID be signed / verifiable from the e.g. sourceID and sessionID for the event (and moreover the applicationID from which they were derived)?

logID: A unique identifier for the resource, e.g. a DBMS instance / namely SQLite3 database file(s), that will store the forwarded event. Analogous to dPort.

schemaID: A unique identifier used to determine e.g. the table in a SQLite3 database file where the forwarded event will be stored (and the schema for this table). Analogous to protocol.

• Should schemaID be signed / verifiable from an sourceID, sessionID, destinationID, logID (and applicationID) for the forwarded event?

Data.Event.Checksum

type Checksum =
  { featureID  :: FeatureID
  , instanceID :: InstanceID
  }

featureID: a unique identifier for the feature (3-tuple) extracted from the event.

instanceID: a unique identifier for raw event from which the processed event was derived.

• Should raw event data be included as type Content = ... (in a follow-up release)?

Data.Event.Period

type Period = 
  { startTime :: Date
  , duration  :: Int
  , endTime   :: Date
  }

Data.Event.EventType

** Extensibility **

ea0ac96

• Separate module for common Data.Event.EventType (success or failure).

0288da6

• Explicit type class in Data.Event.Class analogous to EventCategory and EventID.

• Explicit type parameter with EventType type class constraint.