Qulexa (Qubole + Alexa)
A voice-enabled conversational interface for your Qubole Data Service (QDS) big data platform.
Imagine reliably asking Amazon Alexa, Amazon Echo Dot, Google Home, or a chatbot to run analytics queries against a big data platform. For example, "What were the top three revenue generating products last week?" or better yet "Start my Spark cluster" -- all without firing up your computer, scrolling through a report, looking through spreadsheet columns, or asking an analyst or data admin. Big Data at the tip of the tongue--pun intended.
The concept of conversing with a computer is very interesting and has been around for a while--think Star Trek’s “LCARS” and Hal from “A Space Odyssey”. While we might be a long way off from those realities, recent advancements from Amazon, Google, Microsoft, IBM and other natural language and AI technologies have brought us closer. We can expect a lot of new, creative services being built in the near future.
Meet Qulexa -- Although it’s just a sample application, Qulexa in my mind is a preview of what I am envisioning where we’re headed when it comes to accessing insights using AI, ML and Big Data technologies as never possible before.
Technical Know-how & Prerequisites
Even though this is a pretty lightweight application, I’ve used various technologies that hopefully get my point across. And walking through the entire codebase is beyond the scope so a good understanding and working knowledge of the following technologies is expected.
- AWS Skills Kit
- For quick start guide, click here
- AWS Lambda
- Qubole Data Service (QDS) REST APIs
- QDS is the leading big data platform that enables organizations to process and analyze large amounts of structured, unstructured, and raw data using any engine (Hadoop 1 & 2, Spark, Presto, HBase) on any public cloud infrastructure.
You will also need:
NOTE: Google developers can use recently announced Actions on Google to create similar conversational interfaces for Google Home. (In fact, I might port this app myself so stay tuned!)
Application At A Glance
At a higher-level, here's what you can do out-of-the-box by talking to Alexa or Echo Dot:
- List all clusters in your QDS account
- Start a Spark, Hadoop, Presto, HBase, or Airflow cluster in your QDS account
- Stop any of your clusters
- Retrieve results of a saved query
Impt to Note: These are just a handful of commands I've implemented, but note that QDS provides a very extensive set of REST APIs such as: adding nodes to a cluster, executing Spark, Presto, Hive, or Pig commands, executing workflows, scheduling jobs, running Spark Notebooks, etc. that you can create conversational interfaces for.
Before you run or test this app in your environment, be sure to update the following attributes in config.js
If you don’t have access to Alexa-enabled device such as Amazon Alexa or Echo Dot, you can use Amazon’s browser-based Alexa Skill Testing Tool.