Skip to content
Branch: master
Find file History
natekupp Update events pipeline with new API and preset definition
Test Plan: unit

Reviewers: schrockn

Reviewed By: schrockn

Differential Revision: https://dagster.phacility.com/D438
Latest commit a3fbff1 Jun 20, 2019

README.md

Event pipeline demo

This is intended to be a fully working example Spark pipeline, with the ability to kick off a Spark job locally as well as on a remote Spark cluster.

Example events data here: https://s3.us-east-2.amazonaws.com/elementl-public/example-json.gz

Requirements

You must have Spark installed, and you must set $SPARK_HOME.

You can’t perform that action at this time.