Adapting parserator to handle an entire document

[AbeHandler]

I am currently using Parserator to parse short strings, like this:

s of 1110 of an hour. The maximum amount to be paid under this contract is $20,000.00. No amount of work is guaranteed under this agreement; payments wil

and this

General Liability insurance will be purchased and maintained with limits of $1,000,000 per occurrence an

I extract these strings using a loose regular expression ".{75}$[0-9]+.{75}" on documents that are usually 5 to 10 pages long. I'm most interested in tagging and categorizing the dollar values. Often, the 100 characters around the string is enough to categorize the dollar value. But in some cases I need input from other parts of the document to do the tagging (ex. earlier in the document it might mention that the document is a lease).

@fgregg has pointed me here to show how you could do this with crfsuite http://nbviewer.ipython.org/github/tpeng/python-crfsuite/blob/master/examples/CoNLL%202002.ipynb but I am wondering if it might be possible with the parserator wrapper. All uninteresting tokens would be tagged as and the interesting ones would be tagged with their proper values.

I wanted to see what you all thought about (1) using parserator in this way (2) adapting parserator to cover such cases. The biggest hold up to using parserator in this way is tagging documents with hundreds and hundreds of tokens. It seems like you would want a small document annotation GUI to generate the XML to train parserator. Do you think that such a GUI should be part of the library? Do you think this would work? Would you be open to a pull request?

[fgregg]

Very interested. I think the main difficulty is the interface. We probably don't have the bandwidth to build the interface, but can definitely advise. @derekeder, @cathydeng, @evz any thoughts on this one. flask app?

[AbeHandler]

I know Flask and could take on building a GUI. A GUI would also help in cases where you are tagging only 10 or 15 tokens. I found the command line tagging fairly cumbersome. I also need the GUI anyway to complete this contracts project.

[fgregg]

Sounds good. Hopefully, the current command line interface is decoupled from the rest of parserator. If it isn't we will make it so.

[AbeHandler]

It seems like it might be worth it to use a front-end framework for the GUI. I think the GUI will start getting somewhat complicated and we will end up wanting things like models. Does datamade have allegiances to a particular framework? Do you all have thoughts about if that's necessary?

[fgregg]

We like flask.

[AbeHandler]

Sounds good. And no allegiances on a front-end javascript framework? Backbone? Ember? Angular?

[derekeder]

@AbeHandler we've done backbone in the past, but I kind of hate fighting it and all the complexity it introduces. We've been going 'no framework' (well, jQuery) for our recent projects, but go with what you're comfortable with.

[AbeHandler]

Hi everyone,

After a bit of thought, I realized that I should just modify the DocumentCloud viewer to tag a document -- rather than building a whole new UI.

I'm working on that now: https://github.com/AbeHandler/DocumentParserator

Let me know what you all think. I would like to put in a pull request down the line once the code is cleaned up and working properly.

[fgregg]

Very cool! @knowtheory, you should peep this.

On Wed, Apr 1, 2015 at 7:56 PM Abe Handler notifications@github.com wrote:

Hi everyone,

After a bit of thought, I realized that I should just modify the
DocumentCloud viewer to tag a document -- rather than building a whole new
UI.

I'm working on that now: https://github.com/AbeHandler/DocumentParserator

Let me know what you all think. I would like to put in a pull request down
the line once the code is cleaned up an working properly.

—
Reply to this email directly or view it on GitHub
#16 (comment).

[AbeHandler]

One issue that is coming up: parserator is going quite slowly (maybe ~6 min per contract) when parsing an entire document. I can work around this -- but others might not be so patient. For now, I'm using the parallel command and working on other stuff while I wait. I can provide clearer benchmarks down the line. But wanted to make a note of this on this thread.

[cathydeng]

@AbeHandler hmm...my initial hunch is that it's taking a long time b/c a document gets broken out into way more tokens than the typical name or address. what happens if you run your contract parser on a string w/, say, 10 tokens? or on an input approximately half the length of your typical contract that takes ~6 mins?

[AbeHandler]

@cathydeng Yah. It runs fine on short strings. It would make sense that it is taking so long because there are so many tokens. But I can't really think of a way to cut down on the tokens for any given contract. Right?

[cathydeng]

what about configuring the tokenizer to be smarter, so that tokens are less granular than a single word where applicable?

I'm not sure what your data looks like, but if there are certain sequences of words that you know belong together under the same label, it could def cut down on the number of tokens to evaluate. for example, if you have, say, the string 'one (two three four five six) seven', instead of splitting it into 7 tokens, you could configure the tokenizer to spit it into 3: 'one', (two three four five six), seven. thoughts?

[AbeHandler]

That is a good idea -- particularly since there is a lot of boiler plate language in these contracts. It might be possible to take entire paragraphs as individual tokens.

[samzhang111]

eavesdropping on this thread because this project is really cool. how many tags do you have btw? In general, CRFs are linear in tokens and quadratic in tags, right?