Fix for TIKA-1787 : NamedEntityParser #61
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add OpenNLPNERecogniser as default
|
|
||
| import java.io.InputStream; | ||
| import java.nio.charset.StandardCharsets; | ||
| import java.util.*; |
There was a problem hiding this comment.
please remove star imports
|
@thammegowda great work! See my comments and please update thank you |
+ Removed star imports + Removed dead code / commented code + Added License header to missing files
|
@chrismattmann Thanks for the feedback. Issues Resolved! |
| file.getParentFile().mkdirs() | ||
| inStream = urlConn.getInputStream() | ||
| outStream = new FileOutputStream(file) | ||
| //IOUtils.copyLarge(inStream, outStream) |
There was a problem hiding this comment.
@thammegowda can you remove this line? commented code.
|
one more minor update @thammegowda and this is ready to go! |
|
@thammegowda can you also write up a quick tutorial on http://wiki.apache.org/tika/TikaAndNER ? that shows how to install Stanford NER and run this? |
|
you will need wiki karma so let me know your username and I'll grant you karma. |
|
@chrismattmann Sure thing. I might have missed few such comments. I will review one more time. Please give me permission to create/edit NER wiki page, my username is "ThammeGowda". |
asfgit
pushed a commit
that referenced
this pull request
Nov 18, 2015
…ontributed by Thamme Gowda N and Yueheng He this closes #61 this closes #62 git-svn-id: https://svn.apache.org/repos/asf/tika/trunk@1714931 13f79535-47bb-0310-9956-ffa450edef68
tballison
pushed a commit
to tballison/tika
that referenced
this pull request
Feb 26, 2016
…ontributed by Thamme Gowda N and Yueheng He this closes apache#61 this closes apache#62 git-svn-id: https://svn.apache.org/repos/asf/tika/trunk@1714835 13f79535-47bb-0310-9956-ffa450edef68
tballison
pushed a commit
to tballison/tika
that referenced
this pull request
Feb 26, 2016
…ontributed by Thamme Gowda N and Yueheng He this closes apache#61 this closes apache#62 git-svn-id: https://svn.apache.org/repos/asf/tika/trunk@1714931 13f79535-47bb-0310-9956-ffa450edef68
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
UPDATE : Wiki URL : https://wiki.apache.org/tika/TikaAndNER
Added NamedEntityParser that supports loading of different NER implementations at runtime.
The default NER implementation based on OpenNLP is supplied.
Another implementation based on StanfordCoreNLP is located here This is GNU GPL 3, So kept separate.See UPDATE 2 below@chrismattmann This is not 100% complete, here are few TODOs :
EDIT :
2. Looking for a best way to read parsed text from non text streams within the NamedEntityParser (not sure if a parser can read output of previous parsers like html or pdf). Please suggest me on how to resolve this todoUsing secondary parser to get text contentUPDATE : 1. Added Regex Based NER . Though this can recognize much more patterns than names, (I am using it for recognising weapon names and weapon types )
UPDATE 2 : Added Core NLP NER with runtime class binding this one is still using java binding instead of command invocation, because :
UPDATE 3 : Chaining support :
Now we can chain many NER Implementations (OpenNLP, CoreNLP, RegEx) to the NamedEntityParser.