2014-06-01 iplus1.txt

id: 52
date: 2014-06-01
pos: 0
title: iplus1

* try and figure a way to filter out nouns.

All my shit got derailed yesterday when I tried to find something to filter out the proper nouns in sentences. All the named entity recognition stuff seems to either be in python or java.
Might just compile a list of proper nouns and have it check against that. Like, have a separate script that goes through the db and pulls out all the nouns.
Solution: yell at lool0 to add fields in the database where users can specify the proper nouns in a sentence so I can ignore them. Or if he doesn't, then I will. Seems like a much easier solution that also allows for greater accuracy.
He said I`d have to do it but I`ll just ignore them for now till I get to that point.

* read up on/implement icu for lowercasing various languages