-
Notifications
You must be signed in to change notification settings - Fork 2
2 Using the StorageLoader
HOME > SNOWPLOW SETUP GUIDE > Step 4: setting up alternative data stores > Using the StorageLoader
## 1. OverviewRunning the StorageLoader is very straightforward - please review the command-line options in the next section.
## 2. Command-line optionsInvoke StorageLoader using Bundler's bundle exec syntax:
$ bundle exec bin/snowplow-storage-loader
Note the bin/ sub-folder, and that the bundle exec command will
only work when you are inside the storage-loader folder.
The command-line options for StorageLoader look like this:
Usage: snowplow-storage-loader [options]
Specific options:
-c, --config CONFIG configuration file
-s, --skip download|delete,load,archive skip work step(s)
Common options:
-h, --help Show this message
-v, --version Show version
A note on the --skip option: this skips the work steps listed. So
for example --skip download,load would only run the final archive
step. This is useful if you have an error in your load and need to
re-run only part of it.
As per the above, running StorageLoader is a matter of populating
your configuration file, let's call it my-config.yml for this
example, and then invoking StorageLoader like so:
$ bundle exec snowplow-storage-loader --config my-config.yml
StorageLoader depends on Snowplow's [Infobright Ruby Loader] [irl],
which in turn uses the locate shell command. If your shell complains
that this is missing, in which case you can install it separately.
To install and configure locate on Debian/Ubuntu:
$ sudo apt-get install mlocate
$ sudo updatedb
All done? Then schedule the StorageLoader to regularly migrate new data into your data store (e.g. Infobright or Redshift).
Home | About | Project | Setup Guide | Technical Docs | Copyright © 2012-2014 Snowplow Analytics Ltd
HOME > SNOWPLOW SETUP GUIDE > Step 4: Setting up alternative data stores
- [Step 1: Setup a Collector] (setting-up-a-collector)
- Step 2a: Setup a Tracker
- Step 2b: Setup a Webhook
- [Step 3: Setup Enrich] (setting-up-enrich)
- [Step 4: Setup alternative data stores] (setting-up-alternative-data-stores)
- [4.1: setup Redshift] (setting-up-redshift)
- [4.2: setup PostgreSQL] (setting-up-postgresql)
- [4.3: installing the StorageLoader] (1-installing-the-storageloader)
- [4.4: using the StorageLoader] (2-using-the-storageloader)
- [4.5: scheduling the StorageLoader] (3-scheduling-the-storageloader)
- [4.6: loading shredded types] (4-Loading-shredded-types)
- [Step 5: Analyze your data!] (Getting started analyzing Snowplow data)
Useful resources