Setting up alternative data stores
HOME > SNOWPLOW SETUP GUIDE > Step 4: setting up alternative data stores
Snowplow supports storing your data in two different data stores:
| Storage | Description | Status |
|---|---|---|
| S3 | Data is stored in the S3 file system where it can be analysed using EMR emr (e.g. Hive, Pig, Mahout) | Production-ready |
| Redshift setup-redshift | A columnar database offered as a service on EMR. Optimized for performing OLAP analysis. Scales to Petabytes | Production-ready |
| [PostgreSQL] setup-postgres | A popular, open source, RDBMS database | Production-ready |
By setting up the EmrEtlRunner (in the previous step), you are already successfully loading your Snowplow event data into S3 where it is accessible to EMR for analysis.
If you wish to analyse your data using a wider range of tools (e.g. BI tools like ChartIO chartio or Tableau tableau, or statistical tools like R r), you will want to load your data into a database like Amazon's Redshift redshift or PostgreSQL to support enable use of these tools.
The StorageLoader storage-loader-setup is an application to make it simple to keep an updated copy of your data in Redshift. Setting up Snowplow so that you can maintain a copy of your data in a database like Redshift is a two step process:
- [Create a database and table in Amazon Redshift for the data] setup-redshift
- Setup the StorageLoader storage-loader-setup so that it regularly updates that table with the latest data from S3
Currently, the only supported datastores for Snowplow data are Redshift and PostgreSQL. If you wish to use either Redshift or PostgreSQL as a storage target alongside S3, first setup either
- Redshift setup-redshift, or
- [PostgreSQL] [setup-postegres]
Afterwards, you will be need to [set up the StorageLoader to regularly transfer Snowplow data into your new store] storage-loader-setup
All done? Then start analysing your data.
Note: We recommend running all Snowplow AWS operations through an IAM user with the bare minimum permissions required to run Snowplow. Please see our IAM user setup page for more information on doing this.