Custom KSQL UDFs and UDAFs

This repository contains custom KSQL UDFs and UDAFs built by the Magical Pipelines team. An overview of each function, including usage details, can be found in the function's subdirectory.

Installation

All of the JARs are deployed to Maven central, and can be viewed here. To install any of these KSQL UDFs / UDAFs, simply download the appropriate JAR file from Maven central and place it in the KSQL extension directory; If you're not sure what the KSQL extension directory is set to for your KSQL deployment, run the following query from the KSQL CLI.

ksql> SHOW PROPERTIES ;

 Property                                               | Default override | Effective Value
-------------------------------------------------------------------------------------------------------------------------------------------
 ksql.extension.dir                                     | SERVER           | /tmp/ext
 
 ...
 ...

Custom functions

Below are the custom functions that are currently available.

• COLOR_TO_HEX: experimental polyglot (ruby) UDF that converts colors to hex strings
• DIALOGFLOW: build conversational interfaces (virtual assistants, intelligent chatbots, etc)
• PREDICT_SPAM: experimental UDF that predicts whether or not a string of text is spam
• SENTIMENT: perform sentiment analysis on streams of text
• SUMMARY_STATS: calculate summary statistics (mean, standard deviation, sample size, etc) for streams of data.