Skip to content
Graph platform for Detection and Response
Rust JavaScript TypeScript Python Jupyter Notebook Shell Other
Branch: master
Clone or download
Latest commit 69b007d Oct 21, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
analyzer-dispatcher Adopt new schema. New analyzer library. New engagement system. Jun 6, 2019
analyzer_executor Update sysmon crate. Move to class based analyzers. Oct 20, 2019
derive-dynamic-node Update GraphProvision notebook. Update UX. Automate UX deployment. Au… Jul 31, 2019
engagement-creator Update sysmon crate. Move to class based analyzers. Oct 20, 2019
engagement_ux New lens UX Oct 20, 2019
engagements/src First pass at migrating to ossem cim May 28, 2019
generic-subgraph-generator Initial dynamic node support Jul 19, 2019
graph-descriptions Update sysmon crate. Move to class based analyzers. Oct 20, 2019
graph-generator-lib Small changes to cache ttls etc Oct 10, 2019
graph-merger Update GraphProvision notebook. Update UX. Automate UX deployment. Au… Jul 31, 2019
grapl-cdk Update sysmon crate. Move to class based analyzers. Oct 20, 2019
images Add files via upload May 3, 2019
node-identifier Update sysmon crate. Move to class based analyzers. Oct 20, 2019
node-identity-mapper Initial dynamic node support Jul 19, 2019
sysmon-subgraph-generator
.gitignore Improve deployment of frontend by fetching api gateway url Aug 4, 2019
Graph Provision.ipynb Update sysmon crate. Move to class based analyzers. Oct 20, 2019
LICENSE Initial commit Oct 7, 2018
README.md Update README.md Oct 20, 2019
eventlog.xml Update GraphProvision notebook. Update UX. Automate UX deployment. Au… Jul 31, 2019
events6.xml Fix engagement-creator. Other misc fixes. Upload events6 dummy data. Mar 31, 2019
gen-raw-logs.py Update sysmon crate. Move to class based analyzers. Oct 20, 2019

README.md

Grapl

Grapl is a Graph Platform for Detection and Response with a focus on helping Detection Engineers stop fighting their data and start connecting it. At its core, Grapl leverages graph data structures to ensure that you can and connect your data efficiently, model attacker behaviors, and easily expand suspicious behaviors to encompass a full attack scope.

For a more in depth overview of Grapl, read this.

Essentially, Grapl will take raw logs, convert them into graphs, and merge those graphs into a Master Graph. It will then orchestrate the execution of your attack signatures, and provide tools for performing your investigations. Or watch this talk at BSidesLV.

Grapl natively supports nodes for:

  • Processes
  • Files
  • Networking
  • Plugin nodes, which can be used to arbitrarily extend the graph

and currently parses Sysmon logs or a generic JSON log format to generate these graphs.

Key Features

Setup

Key Features

Identity

If you’re familiar with log sources like Sysmon, one of the best features is that processes are given identities. Grapl applies the same concept but for any supported log type, taking psuedo identifiers such as process ids and discerning canonical identities.

Grapl then combines this identity concept with its graph approach, making it easy to reason about entities and their behaviors. Further, this identity property means that Grapl stores only unique information from your logs, meaning that your data storage grows sublinear to the log volume.

This cuts down on storage costs and gives you central locations to view your data, as opposed to having it spread across thousands of logs. As an example, given a process’s canonical identifier you can view all of the information for it by selecting the node.

Analyzers

Analyzers are your attacker signatures. They’re Python modules, deployed to Grapl’s S3 bucket, that are orchestrated to execute upon changes to grapl’s Master Graph.

Rather than analyzers attempting to determine a binary "Good" or "Bad" value for attack behaviors Grapl leverges a concept of Risk, and then automatically correlates risks to surface the riskiest parts of your environment.

Analyzers execute in realtime as the master graph is updated, using constant time operations. Grapl's Analyzer harness will automatically batch, parallelize, and optimize your queries. By leveraging constant time and sublinear operations Grapl ensures that as your organization grows, and as your data volume grows with it, you can still rely on your queries executing efficiently.

Grapl provides an analyzer library so that you can write attacker signatures using pure Python. See this repo for examples.

Here is a brief example of how to detect a suspicious execution of svchost.exe,

class SuspiciousSvchost(Analyzer):

    def get_queries(self) -> OneOrMany[ProcessQuery]:
        invalid_parents = [
            Not("services.exe"),
            Not("smss.exe"),
            Not("ngentask.exe"),
            Not("userinit.exe"),
            Not("GoogleUpdate.exe"),
            Not("conhost.exe"),
            Not("MpCmdRun.exe"),
        ]

        return (
            ProcessQuery()
            .with_process_name(eq=invalid_parents)
            .with_children(
                ProcessQuery().with_process_name(eq="svchost.exe")
            )
        )

    def on_response(self, response: ProcessView, output: Any):
        output.send(
            ExecutionHit(
                analyzer_name="Suspicious svchost",
                node_view=response,
                risk_score=75,
            )
        )

Keeping your analyzers in code means you can:

  • Code review your alerts
  • Write tests, integrate into CI
  • Build abstractions, reuse logic, and generally follow best practices for maintaining software

Check out Grapl's analyzer deployer plugin to see how you can keep your analyzers in a git repo that automatically deploys them upon a push to master.

Engagements

Grapl provides a tool for investigations called an Engagement. Engagements are an isolated graph representing a subgraph that your analyzers have deemed suspicious.

Using AWS Sagemaker hosted Jupyter Notebooks and Grapl's provided Python library you can expand out any suspicious subgraph to encompass the full scope of an attack. As you expand the attack scope with your Jupyter notebook the Engagement Graph will update, visually representing the attack scope.

Event Driven and Extendable

Grapl was built to be extended - no service can satisfy every organization’s needs. Every native Grapl service works by sending and receiving events, which means that in order to extend Grapl you only need to start subscribing to messages.

This makes Grapl trivial to extend or integrate into your existing services.

Grapl also provides a Plugin system, currently in beta, that allows you to expand the platforms capabilities - adding custom nodes and querying capabilities.

Setup

Setting up a basic playground version of Grapl is pretty simple.

To get started you'll need to install npm, typescript, and the aws-cdk.

Your aws-cdk version should match the version in Grapl's package.json file.

Clone the repo:

git clone https://github.com/insanitybit/grapl.git

Change directories to the grapl/grapl-cdk/ folder. There should already be build binaries.

Execute npm i to install the aws-cdk dependencies.

Add a .env file, and fill it in:

BUCKET_PREFIX="<unique prefix to differentiate your buckets>"

Run the deploy script ./deploy_all.sh

It will require confirming some changes to security groups, and will take a few minutes to complete.

This will give you a Grapl setup that’s adequate for testing out the service.

At this point you just need to provision the Graph databases. You can use the Graph Provision notebook in this repo, and the newly created 'engagement' notebook in your AWS account.

You can send some test data up to the service by going to the root of the grapl repo and calling: python ./gen-raw-logs.py <your bucket prefix>.

This requires the boto3 and zstd Python modules.

Note that this may impose charges to your AWS account.

You can’t perform that action at this time.