luigi workflows to evaluate models trained by vowpal wabbit.
Installation of Prerequisite softwares
If you'd like to use vw-luigi, you need to install vowpal wabbit and some python modules.
If you use OSX, you can install vowpal-wabbit through homebrew.
brew install vowpal-wabbit
Workflows in vw-luigi depend on luigi, numpy and scikit-learn. You can install required modules through pip.
pip install -r requirements.txt
In case you use
/tmp/work/space/train.vw as training data,
/tmp/work/space/test.vw as test data and squared loss as loss function, you can get the evaluation result, which includes AUROC, AUPR and LossLoss calculated by scikit-learn, following to below commands.
$ cd vw-luigi $ ls /tmp/work/space > train.vw test.vw $ python -m luigi --module vwluigi EvalTask --loss-func squared --work-dir /tmp/work/space --local-scheduler > ... $ ls /tmp/work/space > model.vw predict.vw result.txt train.vw $ cat /tmp/work/space/result.txt > AUROC:0.88060 AUPR:0.72192 LOGLOSS:0.36215
If you are interested in vw-luigi, please see this blog post "'Kaggle Display Advertising Challenge' working with vw-luigi". I wrote another usage example using 'Kaggle Display Advertising Challenge Dataset' provided by Critio.
- The first proper release
Distributed under the MIT license. See
LICENSE for more information.
Author: Shotaro Kohama - tw: @shotarok28