Cython wrapper for crfsuite

[ogrisel]

http://www.chokkan.org/software/crfsuite/

crfsuite probably the fastest linear chain CRF implementation,
written in pure C with the "Right License" and features sgd optimizer and optional sparse priors.

This is probably a major task to do a correct wrapping. In addition it will require a quite a bit of wise thinking to come up with a simple yet expressive pythonic API to be able to deal with structured output in a efficient and numpy friendly manner.

[pprett]

I agree, Crfsuite is certainly the best candidate, however, it only implements linear-chain crfs. These are great for NLP but if we also want to reach out to the computer vision community we should look for something more general (e.g. arbitrary decoder). What do you think?

If we are interested in general structured prediction approaches we should consider svm^python [1], a python interface to Thorsten Joachims svm^struct. It allows you to inject your own feature representation, loss function, and argmax (aka decoder) -> theoretically, it allows you to build your own structured predictor in no-time (practically, that's a different story... ).

best,
Peter

[1] http://www.tfinley.net/software/svmpython2/

[ogrisel]

Unfortunately svm^struct is not open source. But having a look at the API and datastructure might help take the right decisions.

[larsmans]

This library turns up after a quick Google search: TUWO, MIT-licensed, supports structured SVMs and CRFs. No experience with it, though.

[pprett]

@larsmans that looks very interesting - thanks!

[larsmans]

@pprett: I just tried to download TUWO, but the link is dead and no-one on the entire web has a copy :( But the author's email address is on his website, so you maybe you could contact him about it.

[turian]

Oliver: Actually, I think svmsgd (Bottou) is faster, but the code is more brittle and not that well documented.

[ogrisel]

@turian: according to those benchmarks, crfsuite is twice as fast as Bottou's SGD project: http://www.chokkan.org/software/crfsuite/benchmark.html#id4719685

[turian]

This must have improved since the last time I checked. Thanks.

When I used crfsuite for my ACL 2010 work (http://www.iro.umontreal.ca/~lisa/pointeurs/turian-wordrepresentations-acl10.pdf) I had to enable negative state features and negative transition features. (“feature.possible transitions=1, feature.possible states=1”) This was the only way to get the expected score.
This was not documented on the site and it made his benchmarks slower than expected. But it looks like he has updated his code + benchmarks since the version I used.

[jakevdp]

I've started looking into this: it looks like a really interesting method, and crfsuite is a very well-written package.
I understand the data model now, and have a working bare-bones python wrapper for the training procedure.
I'll try to get a rough version on my github page within the next couple weeks. Progress may be slow, as I have a lot on my plate right now. But I think CRF will fit nicely into the scikit-learn framework: I don't anticipate any problem developing a full-featured wrapper in time for the next release.

[pprett]

@jakevdp: This is exciting! Please, keep me posted.

[ogrisel]

I recently discovered wapiti which looks very nice too: http://wapiti.limsi.fr/ (and I met the author once a couple of years ago).

[larsmans]

Yes, wapiti is quite good and also quite fast, but it's not written as a library. I've read its code and it seems well-written, but integration into SKL would incur a major overhaul of the code. (Its blazing fast text feature extraction module is something to look into, though, and it has an implementation of the OWL-QN algorithm.)

[ogrisel]

Alright thanks for your feedback.

[larsmans]

@jakevdp: any progress on wrapping CRFsuite? I need a Python CRF library, preferably this month, so I'd be willing to help out.

[jakevdp]

@larsmans : funny you ask - after a few week hiatus, I just spent a good five hours on it last night. I have python class wrappers for all the crfsuite data objects, and the ability to translate back and forth between CRF structures and scipy csr matrices. I'll clean up what I have and open a repository in the next few days.

There will be some interesting challenges in producing a scikit-learn interface to the code. We can discuss that later. I'll post a message here when I put the code up.