Simple Data Pipe (aka "Pipes")

🚫 This project is no longer maintained, the content below is here only for archival purposes.

The Simple Data Pipe is an app that moves your Salesforce or stripe.com data to dashDB, which is the IBM cloud data warehouse. Once you have your data in dashDB, you can do all kinds of analysis on it, with all kinds of tools, such as SQL, R, and Looker.

Usage

To use the Simple Data Pipe, developers first grab the code on Github, and then deploy it to IBM Bluemix, where it runs. It lives at a URL in Bluemix, and comes complete with an AngularJS UI for connecting, scheduling, and reporting. We use the Salesforce OAuth to connect, and the powerful IBM DataWorks APIs to move the data.

To follow our full tutorial please visit : https://developer.ibm.com/clouddataservices/simple-data-pipe/

Deploy to IBM Bluemix

The fastest way to deploy this application to Bluemix is to click the Deploy to Bluemix button below. If you prefer instead to deploy manually to Bluemix then read the entirety of this section.

If you:

• don't have a Bluemix account, you'll be prompted to sign up. Create your account, verify your email address, then return here and click the the Deploy to Bluemix button again.
• have trouble deploying, make sure you haven't already reached your Bluemix memory and services quotas. You get 10 services with your account's free trial period. If you have questions about working in Bluemix, find answers in the Bluemix Docs.

Configuring Cloud Foundry

Complete these steps first if you have not already:

1. Install the Cloud Foundry command line interface.
2. Follow the instructions at the above link to connect to Bluemix.
3. Follow the instructions at the above link to log in to Bluemix.

Creating Backing Services

Create a Cloudant service within Bluemix if one has not already been created:

$ cf create-service cloudantNoSQLDB Shared pipes-cloudant-service

Create a dashDB service within Bluemix if one has not already been created:

$ cf create-service dashDB Entry pipes-dashdb-service

Create a DataWorks service within Bluemix if one has not already been created:

$ cf create-service DataWorks_Gen3 Starter-GA pipes-dataworks-service

Create a Single Sign On (SSO) service within Bluemix if one has not already been created:

$ cf create-service SingleSignOn standard pipes-sso-service

Deploying

To deploy to Bluemix, simply:

$ cf push

Note: You may notice that Bluemix assigns a URL to your application containing a random word. This is defined in the manifest.yml file where the random-route key set to the value of true. This ensures that multiple people deploying this application to Bluemix do not run into naming collisions. To specify your own route, remove the random-route line from the manifest.yml file and add a host key with the unique value you would like to use for the host name.

License

Copyright [2015] [IBM Cloud Data Services]

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.