Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
53 lines (31 sloc) 4.48 KB

Keboola Writer for IBM® dashDB™

A Keboola Connect Writer component that handles data writing from KBC storage to IBM® dashDB™. Written in Node.js with utilization of Babel/ES6/ES7 functionality.

What is DashDB?

dashDB is a data warehousing and analytics solution. Use dashDB to store relational data, including special types such as geospatial data. Then analyze that data with SQL or advanced built-in analytics like predictive analytics and data mining, analytics with R, and geospatial analytics. You can leverage the in-memory database technology to use both columnar and row-based tables. dashDB enables you to get faster insights from your data by leveraging a cloud data warehouse with in-database analytics and massively parallel processing.

Advanced processing with columnar and in-memory acceleration

  • Advanced processing: Does not require the entire data set to fit in memory, while still processing at lightning-fast speeds.
  • Prefetching of data: Anticipates and prefetches data just before it's needed and automatically adapts to keep necessary data in or close to the CPU.
  • No decompression required: Preserves the order of data and performs a broad range of operations on compressed data, drastically speeding processing.
  • Data skipping: Automatically determines which data would not qualify for analysis within a particular query, enabling large chunks of data to be skipped.

Open source R for predictive modeling

R is an open source programming language that was developed for advanced data analysis and visualization. It can be used to analyze data from many data sources, including external files or databases.

dashDB integrates R for predictive modeling through an R runtime alongside the data. A web console can be used to load data and perform analytics within minutes. With dashDB and open source R, your analytical options are limitless. IBM dashDB includes RStudio, a fully integrated R development environment designed to provide quick, R-based predictive analytics. RStudio provides an R language code completion feature, integrated help for R packages, file management capabilities and much more.

In-database analytics

dashDB integrates IBM Netezza® Analytics - also used IBM Netezza appliances and IBM PureData for Analytics systems - for in-database advanced analytics. Myriad predictive modeling algorithms are built directly into the database and are available whenever you want to use them. It comes equipped with multiple algorithms, including linear regression, decision tree clustering, k-means clustering and Esri-compatible geospatial extensions. By running the analytics natively in the database, where the data resides, you can gain huge efficiencies.

More information and link for the trial version

Settings for the Keboola dashDB Writer

Configuration in the Keboola Connection is very straightforward. One has to select input files from Keboola Storage, put credentials and you are all set. The possible options are described below.

Table selection

A Keboola helper for Table Input Mapping selection is enabled. It's a GUI component that simplify the input table selection and enabled some advanced functionality like rows filtering or selection of the particular columns.

Params settings

The configuration should be pretty straightforward. Once your instance is ready, put the database parameters into particular fields.

  • Username - A username of the dashDB instance.
  • Password - A password of the dashDB instance. This parameter will be encrypted.
  • DB name - Name of the database (e.g. BLUDB).
  • Schema - Name of the schema in the database (e.g. DASH111111).
  • Hostname - Name of the host where the service is located (e.g. awh-yp-small02.services.dal.bluemix.net).
  • Port - Port of the particular service (e.g. 50000).

Usage

This is the first version of the IBM® dashDB™ writer, currently in beta mode. You should be able to upload data from Keboola Storage into IBM® dashDB™ warehouse. Currently just an option for full load is supported which means the output data in IBM® dashDB™ will always reflect the files stored in Keboola Storage. Check out the licence for more information.