Skip to content
A complete custom processor project, for your reference.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
nifi-workshop-demo-nar Initial commit Sep 28, 2015
nifi-workshop-demo-processors Initial commit Sep 28, 2015
.gitignore Added a .gitignore file Sep 28, 2015
Chuck_Norris_Speaking.xml Added the NiFi template (final result) Sep 28, 2015
README.md
pom.xml Initial commit Sep 28, 2015
screenshot.png

README.md

Overview

These are the accompanying materials for the NiFi workshop - Developing a Custom Processor.

![flow overview] (https://github.com/aperepel/nifi-workshop/blob/master/screenshot.png)

Install Ambari NiFi Service

https://github.com/abajwa-hw/ambari-nifi-service

Add the 9090 port forwarding rule to your VirtualBox

Find the HDP Sandbox VM id:

~ ♨ >  VBoxManage list vms
"boot2docker-vm" {2e7c902c-34fa-45e9-863b-313beb2b4b33}
"centos64_c6401_1440615279050_15827" {b3f9d4a5-a9cf-4495-90ff-139240a11be7}
"NiFi" {10a766b0-358c-4f33-8d66-f2fc3bd8d770}
"docker-vm" {06e9e8ec-929a-4968-96f0-4f54e20b4cec}
"default" {0fedc097-a65a-47d0-90ab-399824ef069f}
"Hortonworks Sandbox with HDP 2.3" {ab54bfd3-a33b-4844-88c3-924f7df9498a}

Create a rule for the target VM (replace with your ID)

~ ♨ >  VBoxManage controlvm ab54bfd3-a33b-4844-88c3-924f7df9498a natpf1 nifi,tcp,,9090,,9090

Maven reference documentation for custom NiFi processors

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions

Full command for project generation (non-interactive)

mvn archetype:generate \
    -DarchetypeGroupId=org.apache.nifi \
    -DarchetypeArtifactId=nifi-processor-bundle-archetype \
    -DarchetypeVersion=0.3.0 \
    -DnifiVersion=0.3.0 \
    -DartifactBaseName=workshop-demo \
    -Dpackage=com.hortonworks.iot.demo

Update maven central indices if applicable (e.g. in IntelliJ IDEA)

Fix the top-level pom parent

Change the top-level pom declaration to update the parent version to 0.3.0

Build the project

mvn clean install

The NAR package will be placed in the nifi-workshop-demo-nar/target directory.

Upload the NAR via Ambari

  • Ambari -> Local Files
  • Upload to the /opt/nifi-*/lib

If you can't go down into the lib directory: right click on lib -> Permissions -> RWX for 'others'

Re-deploy

  • Delete the old NAR file
  • Upload a new version
  • Restart the NiFI via Ambari -> NiFi -> Restart All

But it's painful if you have to repeat it many times. Best practice - extensive NiFi unit testing of components and processors in a local IDE. See the testcase in the project.

Misc

NiFi template (can be imported)

  • The template is in the Chuck_Norris_Speaking.xml
  • In NiFi UI go to Templates admin section (on the righthand side) and upload the XML
  • In NiFi UI main toolbar click Templates
  • In a resulting popup your new template should be listed. Select and Ok.

Extra Credit

  • Send jokes to a Kafka topic. Different Kafka topics? (Flow design)
  • Improve user experience by restricting allowable values for properties (e.g. true/false, pre-defined buffer sizes) (Coding)
  • Accumulate jokes to be written in one file, e.g. as a batch of 20 or every 30 seconds (MergeContent Processor)
  • Let's do some searching! Route JSON events from the REST API into Solr. Banana dashboard anyone?
  • Give Chuck a voice! Find a Text-to-Speech (TTS) library or service and wrap it either in the REST invocation or custom processor. Tip: you can download output sound files via a Data Provenance event viewer (sorry, servers don't play sounds ;)
You can’t perform that action at this time.