luigi workflows to evaluate models trained by vowpal wabbit
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


luigi workflows to evaluate models trained by vowpal wabbit.


Installation of Prerequisite softwares

If you'd like to use vw-luigi, you need to install vowpal wabbit and some python modules.

vowpal wabbit

If you use OSX, you can install vowpal-wabbit through homebrew.

brew install vowpal-wabbit

python modules

Workflows in vw-luigi depend on luigi, numpy and scikit-learn. You can install required modules through pip.

pip install -r requirements.txt

Usage Example

In case you use /tmp/work/space/train.vw as training data, /tmp/work/space/test.vw as test data and squared loss as loss function, you can get the evaluation result, which includes AUROC, AUPR and LossLoss calculated by scikit-learn, following to below commands.

$ cd vw-luigi
$ ls /tmp/work/space
> train.vw test.vw
$ python -m luigi --module vwluigi EvalTask --loss-func squared --work-dir /tmp/work/space --local-scheduler
> ...
$ ls /tmp/work/space
> model.vw predict.vw result.txt train.vw
$ cat /tmp/work/space/result.txt
> AUROC:0.88060 AUPR:0.72192 LOGLOSS:0.36215

If you are interested in vw-luigi, please see this blog post "'Kaggle Display Advertising Challenge' working with vw-luigi". I wrote another usage example using 'Kaggle Display Advertising Challenge Dataset' provided by Critio.

Release History

  • 0.1.0
    • The first proper release


Distributed under the MIT license. See LICENSE for more information.
Author: Shotaro Kohama - tw: @shotarok28