A PredictionIO engine template using Latent Dirichlet Allocation to learn a topic model from raw text
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
project
src/main/scala
.gitignore
README.md
build.sbt
engine.json
template.json

README.md

Topic Modeling Template - LDA

This template requires spark >= 1.5.1.

Input data is plain text, in data/data.txt , one line per LDA "document".

create a PIO app: pio app new YOURAPPNAME

import the data using:

python data/import_eventserver.py --access_key YOURACCESSKEYHERE 

Params are in engine.json, the most important to consider is number of topics.

build,train the LDA model:

pio build 
pio train 
pio deploy

prediction query:

{"text":  "wishing he did not have to go"} 

The response contains the top topic for this document, as well as the full set of topics for comparison (with the top 10 terms shown for each topic, for reference). You may wish to alter this to return only top topic.

You can do topic prediction on any document (formerly restricted to those in the train set).